[rsnapshot-discuss] Moving to a delete dir rather than rm

Discussion:

Thierry Lavallee

2015-09-29 14:02:22 UTC

hi,
Is there a way for Rsnapshot to MV rather than RM here?
[27/Sep/2015:04:00:48] /bin/rm -rf /media/backupServer/home/daily.6/

I would prefer Rsnapshot to act quickly by simply moving files to a
/to_delete directory. I would then run a regular cron to empty that up.
Because at the present time sometimes my monthly or weekly are unable to
run, because the daily is still running.

Thanks

Scott Hess

2015-09-29 15:19:51 UTC

Permalink

Post by Thierry Lavallee
hi,
Is there a way for Rsnapshot to MV rather than RM here?
[27/Sep/2015:04:00:48] /bin/rm -rf /media/backupServer/home/daily.6/
I would prefer Rsnapshot to act quickly by simply moving files to a
/to_delete directory. I would then run a regular cron to empty that up.
use_lazy_deletes 1
Changes default behavior of rsnapshot and does not initially
remove the oldest snapshot. Instead it moves that directory to
_delete.[processid] and continues as normal. Once the backup
has been completed, the lockfile will be removed before
rsnapshot starts deleting the directory.
Enabling this means that snapshots get taken sooner (since the
delete doesn't come first), and any other rsnapshot processes
are allowed to start while the final delete is happening. This
benefit comes at the cost of using more disk space. The default
is 0 (off).
The details of how this works have changed in rsnapshot version
1.3.1. Originally you could only ever have one .delete
directory per backup level. Now you can have many, so if your
next (eg) hourly backup kicks off while the previous one is
still doing a lazy delete you may temporarily have extra
_delete directories hanging around.

Example log output:
[29/Sep/2015:03:45:01] /usr/bin/rsnapshot daily: started
[29/Sep/2015:03:45:01] echo 21655 > /var/run/rsnapshot.pid
[29/Sep/2015:03:45:01] mv /.snapshots/daily.6/ /.snapshots/_delete.21655/
[29/Sep/2015:03:45:01] mv /.snapshots/daily.5/ /.snapshots/daily.6/
[29/Sep/2015:03:45:01] mv /.snapshots/daily.4/ /.snapshots/daily.5/
[29/Sep/2015:03:45:01] mv /.snapshots/daily.3/ /.snapshots/daily.4/
[29/Sep/2015:03:45:01] mv /.snapshots/daily.2/ /.snapshots/daily.3/
[29/Sep/2015:03:45:01] mv /.snapshots/daily.1/ /.snapshots/daily.2/
[29/Sep/2015:03:45:01] mv /.snapshots/daily.0/ /.snapshots/daily.1/
[29/Sep/2015:03:45:01] mv /.snapshots/hourly.3/ /.snapshots/daily.0/
[29/Sep/2015:03:45:01] rm -f /var/run/rsnapshot.pid
[29/Sep/2015:03:45:01] /bin/rm -rf /.snapshots/_delete.21655
[29/Sep/2015:03:48:47] /usr/bin/rsnapshot daily: completed successfully

-scott

Thierry Lavallee

2015-09-29 16:46:30 UTC

Permalink

Thanks Scott!
And this will work the same if I have sync_first enabled?

Post by Thierry Lavallee
use_lazy_deletes 1
Changes default behavior of rsnapshot and does not initially
remove the oldest snapshot. Instead it moves that directory to
_delete.[processid] and continues as normal. Once the backup
has been completed, the lockfile will be removed before
rsnapshot starts deleting the directory.
Enabling this means that snapshots get taken sooner (since the
delete doesn't come first), and any other rsnapshot processes
are allowed to start while the final delete is happening. This
benefit comes at the cost of using more disk space. The default
is 0 (off).
The details of how this works have changed in rsnapshot version
1.3.1. Originally you could only ever have one .delete
directory per backup level. Now you can have many, so if your
next (eg) hourly backup kicks off while the previous one is
still doing a lazy delete you may temporarily have extra
_delete directories hanging around.

[29/Sep/2015:03:45:01] /usr/bin/rsnapshot daily: started
[29/Sep/2015:03:45:01] echo 21655 > /var/run/rsnapshot.pid
[29/Sep/2015:03:45:01] mv /.snapshots/daily.6/ /.snapshots/_delete.21655/
[29/Sep/2015:03:45:01] mv /.snapshots/daily.5/ /.snapshots/daily.6/
[29/Sep/2015:03:45:01] mv /.snapshots/daily.4/ /.snapshots/daily.5/
[29/Sep/2015:03:45:01] mv /.snapshots/daily.3/ /.snapshots/daily.4/
[29/Sep/2015:03:45:01] mv /.snapshots/daily.2/ /.snapshots/daily.3/
[29/Sep/2015:03:45:01] mv /.snapshots/daily.1/ /.snapshots/daily.2/
[29/Sep/2015:03:45:01] mv /.snapshots/daily.0/ /.snapshots/daily.1/
[29/Sep/2015:03:45:01] mv /.snapshots/hourly.3/ /.snapshots/daily.0/
[29/Sep/2015:03:45:01] rm -f /var/run/rsnapshot.pid
[29/Sep/2015:03:45:01] /bin/rm -rf /.snapshots/_delete.21655
[29/Sep/2015:03:48:47] /usr/bin/rsnapshot daily: completed successfully
-scott

Scott Hess

2015-09-29 17:06:30 UTC

Permalink

Basically sync_first pulls the rsync outside of the periodic lock. So it's
orthogonal to this.

More practically, I run with sync_first and lazy deletes, seems to work
fine.

BTW, my runs are scheduled in this temporal order:
00:00 sync && hourly
03:00 quarterly on first day of every third month
03:15 monthly on first day of the month
03:30 weekly on day 0 of the week
03:00 daily

I run this order so that when the oldest backup does fire, it steals the
oldest previous backup, and then that backup only has to move directories
around. It's only half a theory, though, since the periods break sync
(weekly is only sync'ed with monthly 1 in 7 times). On the other hand, a
long-running monthly has a half hour before daily kicks in. Lazy deletes
pretty much make that moot, though.

-scott

Post by Thierry Lavallee
Thanks Scott!
And this will work the same if I have sync_first enabled?
hi,

Post by Thierry Lavallee
Is there a way for Rsnapshot to MV rather than RM here?
[27/Sep/2015:04:00:48] /bin/rm -rf /media/backupServer/home/daily.6/
I would prefer Rsnapshot to act quickly by simply moving files to a
/to_delete directory. I would then run a regular cron to empty that up.
use_lazy_deletes 1
Changes default behavior of rsnapshot and does not initially
remove the oldest snapshot. Instead it moves that directory to
_delete.[processid] and continues as normal. Once the backup
has been completed, the lockfile will be removed before
rsnapshot starts deleting the directory.
Enabling this means that snapshots get taken sooner (since the
delete doesn't come first), and any other rsnapshot processes
are allowed to start while the final delete is happening. This
benefit comes at the cost of using more disk space. The default
is 0 (off).
The details of how this works have changed in rsnapshot version
1.3.1. Originally you could only ever have one .delete
directory per backup level. Now you can have many, so if your
next (eg) hourly backup kicks off while the previous one is
still doing a lazy delete you may temporarily have extra
_delete directories hanging around.