An updated version of the script can be found here.
I recently bought a NAS, a Synology DiskStation DS211j and stuffed two 1TB disks in it. I configured the disks to be in RAID 1 (mirrored) in case one of them decides to die. I then brought the NAS to a family member’s house and installed it there. Now she uses it to back up her important files (and as a storage tank for music and videos).
The good thing for me is that I can now make off-site backups of my home directories. I configured the DS211j to accept SSH connections so that I can log into it (as user admin or root). I used the web interface to create a directory for my backups (which appeared to be /volume1/BackupLennart after logging in with SSH).
After making a hole in her firewall that allowed me to connect to the DS211j, I created a backup script in /etc/cron.daily with the following contents:
#!/bin/bash # # This script makes a backup of my home dirs to a Synology DiskStation at # another location. I use LVM for my /home, so I make a snapshot first and # backup from there. # # Time-stamp: <2011-02-06 21:30:14 (lennart)> ############################### # Some settings ############################### # LVM options VG=raidvg01 LV=home MNTDIR=/mnt/home_rsync_snapshot/ # rsync options DEST=root@remote-machine.example.com:/volume1/BackupLennart/ SRC=${MNTDIR}/* OPTIONS="-e ssh --delete --progress -azvhHS --numeric-ids --delete-excluded " EXCLUSIONS="--exclude lost+found --exclude .thumbnails --exclude .gvfs --exclude .cache --exclude Cache" ############################### # The real work ############################### # Create the LVM snapshot if [ -d $MNTDIR ]; then # If the snapshot directory exists, another backup process may be # running echo "$MNTDIR already exists! Another backup still running?" exit -1 else # Let's make snapshots mkdir -p $MNTDIR lvcreate -L5G -s -n snap$LV /dev/$VG/$LV mount /dev/$VG/snap$LV $MNTDIR fi # Do the actual backup rsync $OPTIONS $EXCLUSIONS $SRC $DEST # Remove the LVM snapshot if [ -d $MNTDIR ]; then umount /dev/$VG/snap$LV lvremove -f /dev/$VG/snap$LV rmdir $MNTDIR else echo "$MNTDIR does not exist!" exit -1 fi |
Let’s walk through it: in the first section I configure several variables. Since I use LVM on my server, I can use it to make a snapshot of my /home partition. The LVM volume group I use is called ‘raidvg01’. Withing that VG my /home partition resides in a logical volume called ‘home’. The variable MNTDIR is the place where I mount the LVM snapshot of ‘home’.
The rsync options are quite straight forward. Check the rsync man page to find out what they mean. Note that I used the --numeric-ids option because the DS211j doesn’t have the same users as my server and this way all ownerships will still be correct if I ever need to restore from this backup.
In the section called “The real work” I first create the MNTDIR directory. Subsequently I create the LVM snapshot and mount it. After this the rsync backup can be run and finally I unmount the snapshot and remove it, followed by the removal of the MNTDIR.
Since the script is placed in /etc/cron.daily it will be executed every day. Since we use SSH to connect to the remote DS211j I set up SSH key access without a password. This Debian howto will tell you how to set that up.
The only thing missing in this setup is that the backups are not stored in an encrypted form on the remote NAS, but for now this is good enough. I can’t wait until the network bandwidth on both sides of this backup connection get so fast (and affordable) that I can easily sync my music as well. Right now uploads are so slow that I hardly dare to include those. I know that I shouldn’t complain since the Netherlands has one of the highest broadband penetrations in the world, but, hey, don’t you just always want a little more, just like Oliver Twist?
Thank you so much for this script, I have a DS210j configured in RAID1 and a LVM setup on my Arch Linux, I was looking for reliable backup solution based on rsync and LVM snapshot, so your script came just handy and perfectly fit my setup.
Dear Lennart, that is really a great script. I will try to implement it myself. However, there is one thing I don’t quite understand:
How does rsync handle interruptions of the internet connection between the two Diskstations?
At least in Germany, your router is forced to disconnect from the internet once every 24 h. So when I backup a bigger chunk of data, rsync won’t be finished with the job before a forced disconnection takes place. Moreover, the router has to refresh its domain as its IP changes after each reconnection (via DynDNS in my case). Does this lead to additional trouble?
Can rsync handel this issue itself or do I have to add a loop into the script that restarts rsync after a specific error code is reported?
I would be very grateful for any comment on that!
Dear Summertea,
Both my provider and the provider of my sister (Where the NAS is located) don’t force reconneting, so I don’t have any real experiences with that. My guess would be that if an rsync transmission is disconnected it will end the transfer. If the disconnection and the reconnection happen fast enough (I’d say on the order of a few seconds) everything might be OK, but I wouldn’t count on that for my backups.
Since rsync (with the -P option) can continue transferring partially transferred files, you could use that to run the interupted backup again so that it can complete. I guess that the return value of rsync will be non-zero it the transfer is interrupted (or if anything else happens). You could test for that return value in the script as you propose and keep rerunning the rsync command (with -P) until it finishes without problems.
This is more or less what I propose (I didn’t test it), replace the rsync line in the script with:
Good luck!
Lennart, thanks for sharing this and your knowledge. I just set up a Linux Mint 12 PC and just got the DS211J NAS as well. What I was hoping for was to create a DropBox-like solution between the Linux Mint PC (client) and the DS211J NAS (server). Another brilliant individual named Phil has figured out how to create your own DropBox-like syncing solution. I just have not been able to figure out how to make it work with the DS211J NAS. You’re obviously smarter than I am and thought you may have an interest in what Phil has done (using rsync and lsyncd) and you may be able to relate his work to the DS211J. If you have an interest and are able to figure it out, please post and share with the rest of us.
The two Phil links that would be of interest are:
1. His Blog that describes “HOW TO build your own open source Dropbox clone”
http://fak3r.com/howto-build-your-own-open-source-dropbox-clone/
2. His related lipsync open source project that creates an easy to use installer (here’s where I don’t know what values are relevant to DS211J when it is set up as an rsync server).
https://github.com/philcryer/lipsync
Thanks again for sharing your skill with the world.
From the U.S.,
Nick
Dear Nick,
Thanks for the comments! The link to Phil’s blog was an interesting read. I hadn’t heard of lsyncd before and it offers some interesting prospects, like syncing some directories of my PC at work with my server at home. At the moment I’m using unison for that, but since the computer and my server are always connected lsyncd seems easier as it does everything in the background (I guess I’ll keep using unison on my laptop, because it’s not always connected).
Using lsyncd/lipsync for my backups to the NAS won’t be a good idea, however, because that would mean that every change (including erroneous deletes etc.) will immediately be mirrored to the NAS. No real problem for a dropbox-like situation, but the idea of backups is that they change less fast than the data you’re working on, so you can undo any mistakes you made.
Going back to your problem of installing lipsync on your computer, I took a quick look at the install script and it seems to ask only a few questions:
After getting the answers to these questions lipsync will set up SSH key, copy it to the NAS and finish the configuration of lsyncd, a first synchronisation, etc.
If it isn’t working for you, could you tell me where things go wrong (error messages, etc.)?
Good luck and thanks a lot!
@ NICK : DSM4 and CloudStation (BETA) bring a easier way to remotely sync data between computers !
but am still lookin’ for user friendly rsync backup from computers to synolgy’s DS !
any idea ?
Sebastien
Very good article. I definitely love this site.
Keep it up!