[rsnapshot-discuss] rsnapshot destination an LTO3 drive

[Balogh László]

Hi,

In the near future i'll buy an LTO3 drive (LTO3 because i got a lot of
LTO3 tapes). And i'm thinking about to archive my backup regularry (once
a month) also to LTO3 tapes. So few questions about it:
1. Is rsnapshot able to make copy to tape devices?
2. If yes, can it handle more tapes? I mean the total archive is about
4-6TB, and my tapes are 400GB, so i'll need to change the tapes. I think
i'll make the tape backup manually, not started from cron, because while
backup i have to be there to change the tapes. But is it possible, that
rsnapshot is automatically asking for the next tape, so what i should do
is to swap the tape, and than it continous backuping automatically?
Or if not, what should be the best sollution (software) to do this job?

Thank
Br
Laszlo


---
Ezt az e-mailt az Avast víruskeresõ szoftver átvizsgálta.
https://www.avast.com/antivirus

[Ken Woods]

I don't understand why you are trying to do the simplest tasks in the most
difficult manner possible. A 4tb hard drive is 36000 forint ($130usd)

With a machine with only 8GB of ram, you likely don't have that much data.
Wouldn't a drive make this a lot easier? I don't know about you, but it
doesn't take very long for my time to add up to $130.


kw

Post by Balogh László
Hi,
In the near future i'll buy an LTO3 drive (LTO3 because i got a lot of
LTO3 tapes). And i'm thinking about to archive my backup regularry (once a
1. Is rsnapshot able to make copy to tape devices?
2. If yes, can it handle more tapes? I mean the total archive is about
4-6TB, and my tapes are 400GB, so i'll need to change the tapes. I think
i'll make the tape backup manually, not started from cron, because while
backup i have to be there to change the tapes. But is it possible, that
rsnapshot is automatically asking for the next tape, so what i should do is
to swap the tape, and than it continous backuping automatically?
Or if not, what should be the best sollution (software) to do this job?
Thank
Br
Laszlo
------------------------------
[image: Avast logo] <https://www.avast.com/antivirus>
Ezt az e-mailt az Avast víruskereső szoftver átvizsgálta.
www.avast.com <https://www.avast.com/antivirus>
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
rsnapshot-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

[Balogh László]

A 4TB drive is about 40-50000 HUF. I have totally 9TB in that system
(3x2TB, and 1x3TB). Data to be secured is about 6TB at the time. All of
my internal SATA ports (4) are in use. And i have here a lot of new LTO3
cardridges, so the only one i have to buy is a used drive. :)

Br
Laszlo

Post by Ken Woods
I don't understand why you are trying to do the simplest tasks in the
most difficult manner possible. A 4tb hard drive is 36000 forint
($130usd)
With a machine with only 8GB of ram, you likely don't have that much
data. Wouldn't a drive make this a lot easier? I don't know about
you, but it doesn't take very long for my time to add up to $130.
kw
Hi,
In the near future i'll buy an LTO3 drive (LTO3 because i got a
lot of LTO3 tapes). And i'm thinking about to archive my backup
regularry (once a month) also to LTO3 tapes. So few questions
1. Is rsnapshot able to make copy to tape devices?
2. If yes, can it handle more tapes? I mean the total archive is
about 4-6TB, and my tapes are 400GB, so i'll need to change the
tapes. I think i'll make the tape backup manually, not started
from cron, because while backup i have to be there to change the
tapes. But is it possible, that rsnapshot is automatically asking
for the next tape, so what i should do is to swap the tape, and
than it continous backuping automatically?
Or if not, what should be the best sollution (software) to do this job?
Thank
Br
Laszlo
------------------------------------------------------------------------
Avast logo <https://www.avast.com/antivirus>
Ezt az e-mailt az Avast víruskereső szoftver átvizsgálta.
www.avast.com <https://www.avast.com/antivirus>
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
rsnapshot-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

---
Ezt az e-mailt az Avast víruskereső szoftver átvizsgálta.
https://www.avast.com/antivirus

[Ken Woods]

Next, you are going to tell us you are using a FUSE version of ZFS with dedup turned on.

A 4TB drive is about 40-50000 HUF. I have totally 9TB in that system (3x2TB, and 1x3TB). Data to be secured is about 6TB at the time. All of my internal SATA ports (4) are in use. And i have here a lot of new LTO3 cardridges, so the only one i have to buy is a used drive. :)
Br
Laszlo

I don't understand why you are trying to do the simplest tasks in the most difficult manner possible. A 4tb hard drive is 36000 forint ($130usd)
With a machine with only 8GB of ram, you likely don't have that much data. Wouldn't a drive make this a lot easier? I don't know about you, but it doesn't take very long for my time to add up to $130.
kw

Post by Balogh László
Hi,
1. Is rsnapshot able to make copy to tape devices?
2. If yes, can it handle more tapes? I mean the total archive is about 4-6TB, and my tapes are 400GB, so i'll need to change the tapes. I think i'll make the tape backup manually, not started from cron, because while backup i have to be there to change the tapes. But is it possible, that rsnapshot is automatically asking for the next tape, so what i should do is to swap the tape, and than it continous backuping automatically?
Or if not, what should be the best sollution (software) to do this job?
Thank
Br
Laszlo
Ezt az e-mailt az Avast víruskereső szoftver átvizsgálta.
www.avast.com
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
rsnapshot-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Ezt az e-mailt az Avast víruskereső szoftver átvizsgálta.
www.avast.com

[Chris Tebb]

Underwater.

Post by Ken Woods
Next, you are going to tell us you are using a FUSE version of ZFS with dedup turned on.

A 4TB drive is about 40-50000 HUF. I have totally 9TB in that system (3x2TB, and 1x3TB). Data to be secured is about 6TB at the time. All of my internal SATA ports (4) are in use. And i have here a lot of new LTO3 cardridges, so the only one i have to buy is a used drive. :)
Br
Laszlo

I don't understand why you are trying to do the simplest tasks in the most difficult manner possible. A 4tb hard drive is 36000 forint ($130usd)
With a machine with only 8GB of ram, you likely don't have that much data. Wouldn't a drive make this a lot easier? I don't know about you, but it doesn't take very long for my time to add up to $130.
kw

Post by Balogh László
Hi,
1. Is rsnapshot able to make copy to tape devices?
2. If yes, can it handle more tapes? I mean the total archive is about 4-6TB, and my tapes are 400GB, so i'll need to change the tapes. I think i'll make the tape backup manually, not started from cron, because while backup i have to be there to change the tapes. But is it possible, that rsnapshot is automatically asking for the next tape, so what i should do is to swap the tape, and than it continous backuping automatically?
Or if not, what should be the best sollution (software) to do this job?
Thank
Br
Laszlo
Ezt az e-mailt az Avast víruskereső szoftver átvizsgálta.
www.avast.com(https://www.avast.com/antivirus)
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
rsnapshot-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Ezt az e-mailt az Avast víruskereső szoftver átvizsgálta.
www.avast.com(https://www.avast.com/antivirus)

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
rsnapshot-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

[Patrick O'Callaghan]

Post by Balogh László
In the near future i'll buy an LTO3 drive (LTO3 because i got a lot of
LTO3 tapes). And i'm thinking about to archive my backup regularry (once a
1. Is rsnapshot able to make copy to tape devices?

I doubt it. Rsnapshot just runs rsync, which can only work if copying to a
"reasonable" filesystem, i.e. one that can be mounted on Linux and have
hard links and random seeking ability. Neither of these are typical of tape
systems, though I don't know anything about LTO3 specifically.

Post by Balogh László
2. If yes, can it handle more tapes? I mean the total archive is about
4-6TB, and my tapes are 400GB, so i'll need to change the tapes. I think
i'll make the tape backup manually, not started from cron, because while
backup i have to be there to change the tapes. But is it possible, that
rsnapshot is automatically asking for the next tape, so what i should do is
to swap the tape, and than it continous backuping automatically?

Almost 0 chance of that happening. Rsync itself knows nothing about
changing media and will just stop if it thinks the target system is full.
It would require serious amounts of hacking to get that to work IMHO.

Post by Balogh László
Or if not, what should be the best sollution (software) to do this job?

There are lots of backup systems available on Linux. A venerable one that
might work in your case is Amanda, but I have no experience of it. Other
than that, you may have to get something commercial.

poc

[Balogh László]

Ok thanks for info.
I think i have to write a script which uses tar (it can handle tape
devices, and changing media should work also).
So with tar than i can easily copy everything to tape, independently
from rsnapshot.

Br
Laszlo

Post by Balogh László
In the near future i'll buy an LTO3 drive (LTO3 because i got a
lot of LTO3 tapes). And i'm thinking about to archive my backup
regularry (once a month) also to LTO3 tapes. So few questions
1. Is rsnapshot able to make copy to tape devices?
I doubt it. Rsnapshot just runs rsync, which can only work if copying
to a "reasonable" filesystem, i.e. one that can be mounted on Linux
and have hard links and random seeking ability. Neither of these are
typical of tape systems, though I don't know anything about LTO3
specifically.
2. If yes, can it handle more tapes? I mean the total archive is
about 4-6TB, and my tapes are 400GB, so i'll need to change the
tapes. I think i'll make the tape backup manually, not started
from cron, because while backup i have to be there to change the
tapes. But is it possible, that rsnapshot is automatically asking
for the next tape, so what i should do is to swap the tape, and
than it continous backuping automatically?
Almost 0 chance of that happening. Rsync itself knows nothing about
changing media and will just stop if it thinks the target system is
full. It would require serious amounts of hacking to get that to work
IMHO.
Or if not, what should be the best sollution (software) to do this job?
There are lots of backup systems available on Linux. A venerable one
that might work in your case is Amanda, but I have no experience of
it. Other than that, you may have to get something commercial.
poc

---
Ezt az e-mailt az Avast víruskereső szoftver átvizsgálta.
https://www.avast.com/antivirus

[Patrick O'Callaghan]

Post by Balogh László
Ok thanks for info.
I think i have to write a script which uses tar (it can handle tape
devices, and changing media should work also).
So with tar than i can easily copy everything to tape, independently from
rsnapshot.

Well tar means 'tape archiver', so yes.

poc

[Balogh László]

Exactly. ;)

Post by Balogh László
Ok thanks for info.
I think i have to write a script which uses tar (it can handle
tape devices, and changing media should work also).
So with tar than i can easily copy everything to tape,
independently from rsnapshot.
Well tar means 'tape archiver', so yes.
poc

---
Ezt az e-mailt az Avast víruskereső szoftver átvizsgálta.
https://www.avast.com/antivirus

[Nico Kadel-Garcia]

Post by Balogh László
Ok thanks for info.
I think i have to write a script which uses tar (it can handle tape
devices, and changing media should work also).
So with tar than i can easily copy everything to tape, independently from
rsnapshot.
Br
Laszlo

I think you've got two things to do:

1) Look at "AMANDA", the old freeware tape archive system. It's stable,
reasonably flexible, and I did work with it way back when Exabytes were
common. Under *no* circumstances try to invent your own tape backup
inventory and scheduling system, I've dealt with so many horrible ones in
my career that I twitch like a junkie when I hear of yet another engineer
who thinks they can do a good job from scratch.

2) You only really need to back up one or two hardlinked images, such as
"daily.0" and "weekly.0" and "monthly.0" for a normal tape backup scheme.

3) "daily.0" doesn't mean anything if they tape backup occurs in the midst
of the ranspshot process, you'll have a mixed backup. Until and unless
someone successfully publishes an update for rsnapshot that uses something
like timestamps instead of sequential numbers, such as
"daily.2016-01-01_12-30-04" to segregate the backups consistently, you're
dependent on the timestamps of the top directory rsnapshot backups to tell
when it was made. That information has to be recorded somewhere ideally in
the very beginning or in the label on your backup tape.

[Balogh László]

Thanks for info, i'll check it.

Br
Laszlo

Post by Balogh László
Ok thanks for info.
I think i have to write a script which uses tar (it can handle
tape devices, and changing media should work also).
So with tar than i can easily copy everything to tape,
independently from rsnapshot.
Br
Laszlo
1) Look at "AMANDA", the old freeware tape archive system. It's
stable, reasonably flexible, and I did work with it way back when
Exabytes were common. Under *no* circumstances try to invent your own
tape backup inventory and scheduling system, I've dealt with so many
horrible ones in my career that I twitch like a junkie when I hear of
yet another engineer who thinks they can do a good job from scratch.
2) You only really need to back up one or two hardlinked images, such
as "daily.0" and "weekly.0" and "monthly.0" for a normal tape backup
scheme.
3) "daily.0" doesn't mean anything if they tape backup occurs in the
midst of the ranspshot process, you'll have a mixed backup. Until and
unless someone successfully publishes an update for rsnapshot that
uses something like timestamps instead of sequential numbers, such as
"daily.2016-01-01_12-30-04" to segregate the backups consistently,
you're dependent on the timestamps of the top directory rsnapshot
backups to tell when it was made. That information has to be recorded
somewhere ideally in the very beginning or in the label on your backup
tape.

---
Ezt az e-mailt az Avast víruskereső szoftver átvizsgálta.
https://www.avast.com/antivirus

[Scott Zahn]

Hi László,

Sorry for the late reply in this thread. There is a backup_exec function in
rsnapshot for coordinating actions in between individual backup tasks. It
could also run something at the end of your backup. Here is an example of
how I use it:

backup_exec ssh root@<server> "/usr/local/bin/mysqlsnap.pl -s 8G
/dev/VolGroup00/LogVol00 /.mysqlsnap"
backup <server>:/.mysqlsnap/mysql <backup_dir>/
backup_exec ssh root@<server> "umount /dev/VolGroup00/.mysqlsnap &&
lvremove --force /dev/VolGroup00/.mysqlsnap"

In my example, I make an LVM snapshot of mysql, back it up, then delete the
snapshot. Doing something like tarring everything up to tape at the end of
your backup would be really straightforward. The syntax is simply
"backup_exec <command to be run>".

I hope this helps.

cheers,
Scott

Post by Balogh László
Thanks for info, i'll check it.
Br
Laszlo

Post by Balogh László
Ok thanks for info.
I think i have to write a script which uses tar (it can handle tape
devices, and changing media should work also).
So with tar than i can easily copy everything to tape, independently from
rsnapshot.
Br
Laszlo

1) Look at "AMANDA", the old freeware tape archive system. It's stable,
reasonably flexible, and I did work with it way back when Exabytes were
common. Under *no* circumstances try to invent your own tape backup
inventory and scheduling system, I've dealt with so many horrible ones in
my career that I twitch like a junkie when I hear of yet another engineer
who thinks they can do a good job from scratch.
2) You only really need to back up one or two hardlinked images, such as
"daily.0" and "weekly.0" and "monthly.0" for a normal tape backup scheme.
3) "daily.0" doesn't mean anything if they tape backup occurs in the midst
of the ranspshot process, you'll have a mixed backup. Until and unless
someone successfully publishes an update for rsnapshot that uses something
like timestamps instead of sequential numbers, such as
"daily.2016-01-01_12-30-04" to segregate the backups consistently, you're
dependent on the timestamps of the top directory rsnapshot backups to tell
when it was made. That information has to be recorded somewhere ideally in
the very beginning or in the label on your backup tape.
------------------------------
[image: Avast logo] <https://www.avast.com/antivirus>
Ezt az e-mailt az Avast víruskereső szoftver átvizsgálta.
www.avast.com <https://www.avast.com/antivirus>
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
rsnapshot-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

[Nico Kadel-Garcia]

Post by Scott Zahn
Hi László,
Sorry for the late reply in this thread. There is a backup_exec function
in rsnapshot for coordinating actions in between individual backup tasks.
It could also run something at the end of your backup. Here is an example
/dev/VolGroup00/LogVol00 /.mysqlsnap"
backup <server>:/.mysqlsnap/mysql <backup_dir>/
lvremove --force /dev/VolGroup00/.mysqlsnap"
In my example, I make an LVM snapshot of mysql, back it up, then delete
the snapshot. Doing something like tarring everything up to tape at the end
of your backup would be really straightforward. The syntax is simply
"backup_exec <command to be run>".
I hope this helps.
cheers,
Scott

Which, without extra thought and cautious record keeping, leaves the ape
unlabeled and with no inventory of what was on the tape. That kind of
problem is why I pointed to AMANDA.

[Scott Zahn]

Post by Nico Kadel-Garcia

Post by Scott Zahn
Hi László,
Sorry for the late reply in this thread. There is a backup_exec function
in rsnapshot for coordinating actions in between individual backup tasks.
It could also run something at the end of your backup. Here is an example
/dev/VolGroup00/LogVol00 /.mysqlsnap"
backup <server>:/.mysqlsnap/mysql <backup_dir>/
lvremove --force /dev/VolGroup00/.mysqlsnap"
In my example, I make an LVM snapshot of mysql, back it up, then delete
the snapshot. Doing something like tarring everything up to tape at the end
of your backup would be really straightforward. The syntax is simply
"backup_exec <command to be run>".
I hope this helps.
cheers,
Scott

Which, without extra thought and cautious record keeping, leaves the ape
unlabeled and with no inventory of what was on the tape. That kind of
problem is why I pointed to AMANDA.

I guess it depends on how simple or complex the solution needs to be.