Encrypted backup with duplicity and Backblaze

February 2018 ยท 4 minute read

Motivation

Beside the reliability and integrity of a backup I’d want to ensure that all backed up data are not acessible by third parties.

Requirements

Setup

Preparation

I presume that there’s a running minimal Ubuntu Xenial installation with a directory /mnt/backup which should be backed up to Backblaze.

Installation

Install necessary software: apt-get install duplicity gnupg gnupg-agent

Then create a new GPG key which is dedicated to backup encryption. For details about key generation check the Ubuntu GPG HowTo out.

Create a backup script which is triggered regularly by cron or something else:

!/usr/bin/env bash

ACCOUNTID="XXXXXXXXXXXX"
APPLICATION_KEY="000000000000000000000000000000000000000000"
KEYID="FFFFFFFFFFFFFFFF"
FULL_AGE="90D"
FULL_BACKUPS="2"
LOCKFILE="/tmp/duplicity.lock"
GPGBINARY="/usr/bin/gpg"
OPTIONS="-v0 --no-print-statistics" # because there is no --quiet

## ^ config ends here ^ ##

OPTIONS="${OPTIONS} --gpg-binary=${GPGBINARY} --tempdir=/mnt/backup/tmp --archive-dir=/mnt/backup/.duplicity --use-agent"

if [ "$1" = "unlock" ]; then
  gpg-connect-agent /bye
  ${GPGBINARY} -sanu ${KEYID} "$0"
  exit $?
fi

if [ -f ${LOCKFILE:?} ]; then
  echo >&2 "Other instance active. Abort"
  exit 1
fi

function finish {
  rm -rf ${LOCKFILE:?}
}
trap finish EXIT
echo $$ > ${LOCKFILE:?}

## Transfer to b2
## There's a bug in duplicity which prevents us from adding more then one backup into one bucket.
## Therefore each backup has it's own bucket. See https://bugs.launchpad.net/duplicity/+bug/1523498
cd /mnt/backup/
for BACKUPPATH in dom0 www static vms/*; do
  BUCKET=your-prefix-${BACKUPPATH#vms/}
  BUCKET=$(tr . - <<< ${BUCKET})
  duplicity ${OPTIONS} --name=${BUCKET:?} --encrypt-key=${KEYID} --asynchronous-upload --full-if-older-than=${FULL_AGE:?} --num-retries=10 ${BACKUPPATH:?} b2://${ACCOUNTID}:${APPLICATION_KEY}@${BUCKET} &&
  duplicity ${OPTIONS} --name=${BUCKET:?} --force remove-all-but-n-full ${FULL_BACKUPS:?} b2://${ACCOUNTID}:${APPLICATION_KEY}@${BUCKET}
done

Duplicity uses your GPG key to en- and decrypt all data, even the meta data. But keep in mind, that it keeps a copy of the decrypted meta data on the local disk (defaults to ~/.cache/duplicity) and utilizes a temporary directory. If just a part of your disk is encrypted (instead of FDE) adapt the script to point those locations (option --tempdir and --archive-dir) to the encrypted part.

Duplicity needs access to your private key to decrypt those metadata, therefore the script utilizes gpg-agent to unlock the key and keep it in an agent session. So after rebooting the machine or killing the agent ensure that you call the script with unlock as parameter and provide the GPG key passphrase to load it into the agent session. Afterwards duplicity uses the agent to access the key for all operations which depend on it.

I’d like to separate the different logical parts of my backup into different sets. Therefore the script above creates a new backup (and bucket) for each folder in /mnt/backup.
All my VMs create automatic backups in subfolders of /mnt/backup/vms and the script applies some “Voodoo” to map the VM names to valid B2 bucket names. You can ignore this part if you don’t need it.

If the target bucket is not created yet, it will be created automatically. New B2 buckets are per default private, that’s fine but they are also configured to keep all versions of all files in the bucket. I’m pretty sure you don’t want to keep all backup files even if they are already deleted by duplicity. So after adding new folders (aka creating new buckets) login to yout Backblaze account and update the settings of the new bucket.

B2 bucket list B2 Lifecycle Settings

Configuration

  • ACCOUNTID and APPLICATION_KEY must be changed to your personal B2 account ID and application key: B2 bucket list B2 application key
  • KEYID must be changed to the ID of your generated GPG key
  • FULL_AGE defines after which time a new full backup will be triggered. For valid values check the duplicity man page (section TIME FORMATS)
  • FULL_BACKUPS defines how many full backups should be keept before deleting old ones

Per default duplicity is quite noisy and logs some stuff like statistics, etc. It has no --quiet option but can be silenced with -v0 --no-print-statistics (see OPTIONS variable)

Restore a backup

To restore a previously created backup just adapt the following command to your needs and keep your GPG key passphrase on hand:
duplicity --tempdir=/mnt/backup/tmp --archive-dir=/mnt/backup/.duplicity restore b2://${ACCOUNTID}:${APPLICATION_KEY}@${BUCKET} /mnt/backup/restore

If you just need some files from the backup add the option --file-to-restore <RELATIVE_PATH> to the restore command above.