[rsnapshot-discuss] feature suggestion

[Gandalf Corvotempesta]

I would like to suggest a feature.
With huge backups servers having to backup huge clients (millions of
files), the delete phase is very very slow. In my case, it takes 4
hours on some machines.

During these 4 hours the rotation is stuck, even using "lazy_delete"

I would like to suggest a feature where the deletion is made by an
external program (called, in example, by cron, during the middle of
the day, if backup are run in the night). With this option set to 1,
rsnapshot should only move the older snapshot to a temporary directory
(like now with lazy_delete) but never try to delete it. Just the move.

Something like "skip_delete" that set with "lazy_delete" will just do
the "mv" phase skipping the "rm" phase. The "rm" could be easily run
by cron with a one-liner during the day.

In my case this would help very much.

------------------------------------------------------------------------------

[Gandalf Corvotempesta]

2016-09-05 10:48 GMT+02:00 Gandalf Corvotempesta

Post by Gandalf Corvotempesta
Something like "skip_delete" that set with "lazy_delete" will just do
the "mv" phase skipping the "rm" phase. The "rm" could be easily run
by cron with a one-liner during the day.

I did a very simple patch to my local version of rsnapshot an I'm
testing it right now.
On next backup schedule (a couple of hours, for a test machine) I
would be able to see if this improves performance or not. If
everything is ok, the _delete.$$ directory should be reteined by using
"use_lazy_deletes 1" and "use_external_deletes 1"

------------------------------------------------------------------------------

[Gandalf Corvotempesta]

2016-09-05 11:06 GMT+02:00 Gandalf Corvotempesta

Post by Gandalf Corvotempesta
I did a very simple patch to my local version of rsnapshot an I'm
testing it right now.
On next backup schedule (a couple of hours, for a test machine) I
would be able to see if this improves performance or not. If
everything is ok, the _delete.$$ directory should be reteined by using
"use_lazy_deletes 1" and "use_external_deletes 1"

Working as expected on a test machine.
This night i'll be able to test deeply on huge servers.

Any change to get this merged if I'll post a pull-request on github ?

------------------------------------------------------------------------------

[Gandalf Corvotempesta]

2016-09-05 13:10 GMT+02:00 Gandalf Corvotempesta

Post by Gandalf Corvotempesta
Working as expected on a test machine.
This night i'll be able to test deeply on huge servers.

Worked as expected also on huge servers. Backup time cut down by 3-4
hours (the "rm" time)

------------------------------------------------------------------------------

[Gandalf Corvotempesta]

Is there a reason you added yet another parameter to rsnapshot
(which some would say already has too many) rather than using
use_lazy_deletes 2
to signify this new mode?

No apparent reason, i can use lazy_deletes as 2 if would get merged
upstream