There are several ways to avoid that your photo collection will be eaten by time, all based on a backup that you can store on your local network or somewhere else. This article describes how to make a very efficient local backup system with two standard tools: rsync and cron. Both are available on Linux and Mac OSX, I’ll use the former here.
Let’s start at the end. To make an incremental backup of your photo directory called photos to another hard disk in the same pc twice a day at 11:50 and 17:50, every day a week and the whole year long and for the rest of your life and for free, you must add the following line somewhere to your Linux system:
50 11,17 * * * rsync -a photos/ /media/backup/
Read that line twice. It doesn’t only look magic, it is! What you are looking at is nothing less than a complete and very reliable backup program. The backup command (backup what to where) starts at rsync, the when part is at that strange looking beginning of the line.
This line is a so-called cronjob. A cronjob does something at a certain time.
You can also use rsync to backup your photo directory to another pc or removable hard drive in your local network or to a pc somewhere else, as explained below.
Rsync – remote synchronization – is a very well known and widely used program to make backups (and restores if needed); it is run from the command line. Rsync works this way: the first time it copies all the content you specified to another hard drive you specified. Depending on the size of the source folder and the speed of the (network) connection, this can take some time. But the next time only the differences (or deltas, ie. new photos) are transmitted, which is usually very fast.
The general form of the rsync command is like this:
rsync -a <source> <destination>
rsync -av <src1> <src2> <src3> <destination>
The -a switch combines several options, like: keep the original time stamp (-t, the date you took the photo instead of the date that the photo was copied to another drive), work recursively (-r, include subdirectories), and more. The -v means ‘be verbose’, tell what you do. You can add as many source directories as you like, the last directory is always the destination.
$ rsync -a photos/ /media/backup/
to synchronize the content of the folder photos in the home directory to a second hard drive called backup that is mounted in /media. After this action the location /media/backup contains a new folder named photos. Both directories contain exactly the same number of bytes.
This example shows that rsync always starts to look in the home directory. If your photos are not stored in /home but on a disk called drive1 that is mounted under /media, then use this form:
$ rsync -av /media/drive1/photos/ /media/backup/
If you want to write the backup to a hard drive in another pc in your Linux home network, say something like this:
$ rsync -av photos/ <user>@<local IP address>:/home/<user>
$ rsync -av photos/ firstname.lastname@example.org:/home/joe/
This command copies the directory photos in the home directory to the account of user joe on a network pc with address 192.168.0.105, so you’ll find it back at /home/joe/photos. Mind the colon (:) after the address.
Check the browser interface of your router to find that address, usually by typing something like 192.168.0.1 in the address bar, then search for ‘network settings’ or something like that. If your router has been set up to use DHCP, you might want to give that backup pc a static IP address. Ask you local Linux or Mac forum in case you need help with this.
If you want to make the backup to a Linux machine elsewhere on the internet using ssh, do this:
$ rsync -e ssh -av photos/ <user>@<IP address>:/home/<user>
Some more useful options are:
-n: do a dry run and show what will happen, but don’t transfer anything
-h: make output better readable for human beings (sent 10.68M bytes instead of sent 10,677,755 bytes)
– -exclude=”*.tiff” – -exclude=”*.xml”: transfer all content except tiff files and xml files
– -delete: if something is deleted on the source, delete at destination as well (for security reasons I don’t use this).
Of course rsync can be also used to synchronize directories between Windows, Mac and Linux/Unix machines – and other machines as well, without doubt.
Type ‘man rsync’ in a terminal to learn more about the dozens (hundreds?) other options as well.
The word ‘cron’ comes from the Greek ‘chronos‘, time. The software utility cron on a Linux or Unix system is simply a job scheduler: execute this and that command at that and that time. To instruct the system to do something we must make a cron table file, abbreviated to crontab.
To make a crontab, open a terminal and type
$ crontab -e
(that ‘-e’ means edit). Note: this will open the crontab file in the editor that is set by your system. If you want to use another one, say geany, type this first:
$ export EDITOR=geany
In that window you can see the instructions in English how to make a crontab, all preceded by a # to make them comments. On the last line you see
# m h dom mon dow command
That respectively stands for:
m – minute
h – hour
dom – day of month
mon – month
dow – day of week
command – the command to be executed.
The minute field accepts values from 0 to 59, the hour field wants values between 0 and 23 (so no am/pm notation) and the day of the month is notated with values between 1 and 31. For the month field you must use the English short form: Jan, Feb, Mar, etc.. Same for the day of the week: Mon, Tue, Wed. When you use * in a field, that means ‘all of them’. In the ‘day of month’ field this translates to ‘every day of the month’ for example. If you want to make your backup at 17h50, the minute and hour fields are like so:
50 17 * * *
To make the backup twice a day at 11h50 and 17h50, say this:
50 11,17 * * *
To make the backup every hour between 9h50 and 17h50, say:
50 9-17 * * *
To make the backup once a week on Friday at 17h50, say:
50 17 * * Fri
I make twice a day a backup of my photo directory located in /home, every day of the week, the whole year long, for free, to a second hard disk in my main pc. To realize that, I write the following line in the crontab file:
50 11,17 * * * rsync -a photos/ /media/backup/
Save the crontab file. People often say ‘we have made a cronjob’. Type
$ crontab -e
to edit the crontab file. To list your cronjobs, type
$ crontab -l
You can add more cronjobs, each on a new line.
Some final notes
Never trust computers. Don’t think that your backup is there just because you put it into a cronjob. Check that periodically with your very own eyes. Or add an option to the rsync command to write all its output to a log file, like so:
rsync -a --log-file=photos.rsync.log photos/ /media/backup/
Don’t forget to check that log from time to time…
Discipline! Backing up a photo directory over the years (I talk 20, 30, 40 years…) is quite difficult but do-able. But it requires some discipline. Never mess with the synchronized backup. It’s even best not to touch it at all! Do not rearrange the directory structure of the backup, only work on the original source directory. I don’t say this to teach you a lesson, I say this because I have learned those lessons myself – the hard way, so to say.
Changing hard disks every 5 years is quite common in production environments. Sooner or later the thing dies. I do that as well (changing I mean, for now… 😉 ). The price of a hard disk divided by 5 is a very reasonable cost per year to protect your photos.
None of the commands in this article are destructive. That means that by using any of them, you can not lose any single photo in your collection. But when you start to add extra options to rsync to fine-tune the process, you must know what you are doing. If you don’t know what you are doing, you can lose photos. So be careful.
It is also a very good idea to learn how to use rsync and cronjobs with only two small folders (name them source and destination for example) with some subdirs and only a couple of photos in them.
A relative new way to protect your photos is to use cloud storage. That gets less and less expensive. Perhaps for another article…
Thanks for reading.