Setting up Duplicity and Duply on a CentOS 6/7 cPanel box for S3 backups
Updated August 12 2016: Changed conf file configuration as required by Duply 1.11.2.
I was pleasantly surprised at how easy it was to install Duply and Duplicity to enable S3 backups on my CentOS cPanel box. This is a bit rushed, so instructions are brief. If you notice anything wrong, please let me know in the comments and I fix it up.
Note: This how-to assumes you already have an AWS account with S3 activated and a bucket ready to go. If you don't, do that first. There's lots of good tutorials on how to set all that up. The hardest part for me was ensuring that the policies were properly setup to allow Duply to backup to S3. If you get connection or authentication errors, chances are your policy is not right.
Let's do this!
cd /usr/local/src wget http://dl.fedoraproject.org/pub/epel/6/i386/python-lockfile-0.8-3.el6.noarch.rpm rpm -Uvh python-lockfile-0.8-3.el6.noarch.rpm wget http://downloads.sourceforge.net/project/librsync/librsync/0.9.7/librsync-0.9.7.tar.gz tar -zxvf librsync-0.9.7.tar.gz cd librsync-0.9.7 ./configure make AM_CFLAGS=-fPIC make install
Install Boto:
cd /usr/local/src git clone git://github.com/boto/boto.git cd boto python setup.py install
Install Duply:
cd /usr/local/src wget http://code.launchpad.net/duplicity/0.6-series/0.6.24/+download/duplicity-0.6.24.tar.gz tar -zxvf duplicity-0.6.24.tar.gz cd duplicity-0.6.24 python setup.py install # Test install duplicity --version # duplicity 0.6.24 cd /usr/local/src wget http://downloads.sourceforge.net/project/ftplicity/duply%20%28simple%20duplicity%29/1.7.x/duply_1.7.3.tgz tar -zxvf duply_1.7.3.tgz cp duply_1.7.3/duply /usr/local/bin/duply
Test installation
duply --version # duply version 1.7.3
The next step is to get our first backup running. To do this we need to create a profile for each directory you want to backup. Duply stores it's profiles in /root/.duply
Enter this to create your first backup:
duply first_backup create
You then need to edit the two files within /root/.duply/first_backup, which conf and exclude.
In conf, there are many options, but these are the ones you will need to get going:
GPG_PW='my_super_secret_password_which_cannot_lose!' TARGET='s3://s3.amazonaws.com//my_backup_folder' export AWS_ACCESS_KEY_ID='AWS_ACCESS_ID' export AWS_SECRET_ACCESS_KEY='AWS_ACCESS_SECRET' SOURCE='/' MAX_AGE=1M TEMP_DIR=/tmp
In exclude, you will enter the directories to backup. For this example let's backup /home/myhome
+ /home/myhome **
Contrary to the instructions in the exlude file, do not enter '- **'as the delimiter line, only enter '**'. If you wish to exclude a directory within the backup directory you can do so like this:
- /home/myhome/useless_stuff + /home/myhome **
To start your backup, simply run from the command line:
duply first_backup backup
If you did everything correctly, you should see the backups starting to run. Once complete, you can check the S3 bucket you created for these backups to verify everything is there. Be sure to try and restore a file or folder. The docs for Duply are very clear and the built in help is sufficient to get you doing most things with ease.
Enjoy!
Update: October 2020
If you see this error:
NoAuthHandlerFound: No handler was ready to authenticate
You need to upgrade your conf file from:
TARGER_USER='AWS_ACCESS_ID' TARGET_PASS='AWS_ACCESS_SECRET'
to:
export AWS_ACCESS_KEY_ID='AWS_ACCESS_ID' export AWS_SECRET_ACCESS_KEY='AWS_ACCESS_SECRET'