[rsnapshot-discuss] diff from compressed files

[Thierry Lavallee via rsnapshot-discuss]

Hi,

Currently the cpanel backups on my server creates an uncompressed backup
directory per account with all files from the account. Takes a lot of space.

There is an option to COMPRESS to a tar.gz file for each account. Which
CONSIDERABLY reduce the space.

Is there any way for rsnapshot to sync only the changes _within_
compressed files?

Thanks

[Patrick O'Callaghan]

On 12 June 2017 at 16:46, Thierry Lavallee via rsnapshot-discuss <

Post by Thierry Lavallee via rsnapshot-discuss
Currently the cpanel backups on my server creates an uncompressed backup
directory per account with all files from the account. Takes a lot of space.
There is an option to COMPRESS to a tar.gz file for each account. Which
CONSIDERABLY reduce the space.
Is there any way for rsnapshot to sync only the changes *within*
compressed files?

Not directly. Rsnapshot uses rsync, which only syncs whole files. There
might be a way to get what you want using rdiff instead of rsync, but I
suspect it would require quite a bit of work to adapt rsnaphot to use it.

poc

[Thierry Lavallee via rsnapshot-discuss]

I was maybe thinking about the --rsyncable option in gzip
https://beeznest.wordpress.com/2005/02/03/rsyncable-gzip/

Post by Patrick O'Callaghan
On 12 June 2017 at 16:46, Thierry Lavallee via rsnapshot-discuss
Currently the cpanel backups on my server creates an uncompressed
backup directory per account with all files from the account.
Takes a lot of space.
There is an option to COMPRESS to a tar.gz file for each account.
Which CONSIDERABLY reduce the space.
Is there any way for rsnapshot to sync only the changes _within_
compressed files?
Not directly. Rsnapshot uses rsync, which only syncs whole files.
There might be a way to get what you want using rdiff instead of
rsync, but I suspect it would require quite a bit of work to adapt
rsnaphot to use it.
poc

[Patrick O'Callaghan]

Post by Thierry Lavallee via rsnapshot-discuss
I was maybe thinking about the --rsyncable option in gzip
https://beeznest.wordpress.com/2005/02/03/rsyncable-gzip/

That would get you is a reduction in data traffic (because rsync only
transmits the changed blocks of the file), but the resulting file is still
just as large at the destination, i.e. if you have a 100MB file with a
single block changed, only that block will be sent, but at the other end
you will still get a full additional 100MB being stored.

If the filesystem on the backup host uses deduplication or Copy On Write at
the block level (e.g. ZFS, BTRFS etc.), that could still be a win, but
rsnapshot doesn't know about that level of detail.

poc

[David Cantrell]

Post by Thierry Lavallee via rsnapshot-discuss
I was maybe thinking about the --rsyncable option in gzip
https://beeznest.wordpress.com/2005/02/03/rsyncable-gzip/

For files that rsync copies across a network I believe that its default
behaviour is to only transfer the chunks of the file that have changed,
so if you use that option to create your compressed files it should Just
Work. rsnapshot will still, however, *store* complete copies of every
version of the file.

[Thierry Lavallee via rsnapshot-discuss]

The new cpanel backup creates an uncompressed tar file for every account.
The legacy cpanel backup used to create an open directory of all the
files associated with an account, making the rsync in rsnapshot work as
expected.

Is there a way to use the NEW cpanel uncompressed tar, and retreive only
the diff?

Thanks