Automated Backup via Rsync, Tar and SCP
In the previous article, I have discussed about synchronizing data with rync. In this article I will continue the topic to a new discussion: how to use rsync for automated backup. Here are the scenarios that I use in this article:
There are 3 servers (Server1, Server2, Server3) that have data in it and need to be backed up daily to a backup server (Backupsvr) in directory /backups. I will use a user account named backupusr on backupsvr. The backupusr have full control access to directory /backups. I will prepare the public/private key pair for backupusr and use it for key-based authentication. For the backup process, I will use these three steps:
1) Use rsync to back up data in each server in to a separate directory in the each local server. By doing this, each server will have backup copy of its data in separate directory that can be restored quickly and easily when needed.
2) Create a compressed archive file of backed-up directory for offsite transfer. This process will create a compact file that can be easily transfer offsite.
3) Transfer the compressed archive file to the backupsvr using SCP. In this step, I will use key-based authentication for SCP.
Here are the detail steps :
Preparing the public/private dsa key pair and place the private key in backupsvr and distribute the public key the server-server.
In Server1
1. Create user account backupusr. I will use this user account to generate public/private dsa key pair.
[root@server1 ~]# useradd backupusr
[root@server1 ~]# passwd backupusr
Changing password for user backupusr.
New UNIX password:
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
[root@server1 ~]#
2. Generate public/private dsa key pair for user backupusr. I will create key pair called identity (identity and identity.pub) in backupusr home directory with null-passphrase.
[root@server1 ~]# su - backupusr
[backupusr@server1 ~]$ ssh-keygen -f identity-backupusr -t dsa -N ""
Generating public/private dsa key pair.
Your identification has been saved in identity-backupusr.
Your public key has been saved in identity-backupusr.pub.
The key fingerprint is:
26:e3:55:7a:fd:0f:e8:a6:5c:52:bf:a4:eb:bc:69:cd [email protected]
[backupusr@server1 ~]$
3. Copy the identity file to root home directory. Copying this private key file is to make it easier for root to access the file.
[backupusr@server1 ~]$ exit
[root@server1 ~]# cp /home/backupusr/identity-backupusr .
In Backupsvr
1. Create user account backupusr.
[root@backupsvr ~]# useradd backupusr
[root@backupsvr ~]# passwd backupusr
Changing password for user backupusr.
New UNIX password:
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
2. Create directory /backups and give permissions only to the backupusr
[root@backupsvr ~]# mkdir /backups
[root@backupsvr ~]# chown backupusr.backupusr /backups
[root@backupsvr ~]# chmod 700 /backups
3. Log on as backupusr in Backupsvr, create .ssh directory and copy the identity-backupusr.pub (public key) from Server1, copy the identity-backupuser.pub file to .ssh/authorized_keys file and change the permissions.
[backupusr@backupsvr ~]$ scp [email protected]:identity-backupusr.pub .
The authenticity of host 'server1.mydomain.com (1.2.3.2)' can't be established.
RSA key fingerprint is 8d:35:1e:e4:1a:87:26:9a:b0:96:c3:5d:cd:1b:c1:ed.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'server1.mydomain.com' (RSA) to the list of known hosts.
backupusr@backupsvr's password:
identity-backupusr.pub 100% 615 0.6KB/s 00:00
[backupusr@backupsvr ~]$ cat identity-backupusr.pub >> .ssh/authorized_keys
[backupusr@backupsvr ~]$ chmod 700 .ssh
[backupusr@backupsvr ~]$ chmod 700 .ssh/authorized_keys
In Server2
1. Copy the identity file (private key file) from Server1 to root home directory in Server2.
[root@server2 ~]# scp [email protected]:identity-backupusr .
The authenticity of host 'server1.mydomain.com (1.2.3.2)' can't be established.
RSA key fingerprint is 8d:35:1e:e4:1a:87:26:9a:b0:96:c3:5d:cd:1b:c1:ed.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'server1.mydomain.com' (RSA) to the list of known hosts.
backupusr@server1's password:
identity-backupusr.pub 100% 615 0.6KB/s 00:00
[root@server2 ~]#
2. Test the connection from Server2 to Backupsvr using key-based SSH authentication. If login no longer requires a password, then the key-based ssh authentication is working.
[root@server2 ~]# ssh –i identity-backupusr backupsvr.mydomain.com
[backupusr@backupsvr ~]$
In Server3
1. Copy the identity file (private key file) from Server1 to root home directory in Server3.
[root@server3 ~]# scp [email protected]:identity-backupusr .
The authenticity of host 'server1.mydomain.com (1.2.3.2)' can't be established.
RSA key fingerprint is 8d:35:1e:e4:1a:87:26:9a:b0:96:c3:5d:cd:1b:c1:ed.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'server1.mydomain.com' (RSA) to the list of known hosts.
backupusr@server1's password:
identity-backupusr.pub 100% 615 0.6KB/s 00:00
[root@server3 ~]#
2. Test the connection from Server3 to Backupsvr using key-based SSH authentication. If login no longer requires a password, then the key-based ssh authentication is working.
[root@server3 ~]# ssh –i identity-backupusr backupsvr.mydomain.com
[backupusr@backupsvr ~]$
In Server1
1. Test the connection from Server1 to Backupsvr using key-based SSH authentication. If login no longer requires a password, then the key-based ssh authentication is working.
[root@server1 ~]# ssh –i identity-backupusr backupsvr.mydomain.com
[backupusr@backupsvr ~]$
The next step is to create the backup script file and schedule it.
In Server1
Prepare the backup script
[root@server1 ~]# vi /backups/backup.sh
#!/bin/bash
EXT_BACKUP_PATH=”[email protected]:/backups”
INT_BACKUP_PATH=”/backups”
DIRS=”/data /home”
weekday=`date +%a`
shorthostname=`hostname -s`
# Backup to local directory
rsync -avHK –delete $DIRS $INT_BACKUP_PATH/$weekday
# Create archive of backed-up directory for offsite transfer
tar -zcvf $INT_BACKUP_PATH/$shorthostname.$weekday.backup.gz $INT_BACKUP_PATH/$weekday
#Transfer archives file to the backup server
scp -i /root/identity-backupusr $INT_BACKUP_PATH/$shorthostname.$weekday.backup.gz $EXT_BACKUP_PATH
Then make the file executable and scheduled the script. In this scenario, I scheduled the script to run every night using cron.
[root@server1 ~]# chmod +x /backups/backup.sh
[root@server1 ~]# crontab –e
@daily sh /backups/backup.sh
Then create the same script for server2 and server3 then scheduled the script.
Note: you can refer to an article about crontab if you want it to run at other times.
In Server2 and Server3
Create the backup script (or copy it from server1) then schedule it
When the scheduled time comes, the server will backup the data via rsync, archive the backed up copy then transfer the archive file to the backup server.