Ubuntu Xenial FDE on Hetzner

January 24, 2018 ยท 4 minutes read


Instead of encrypting only the sensitive parts of hosted applications (e.g. mail storage, database partitions, etc.) it was an appealing idea to try full disk encryption (FDE).



Most of this is also covered by muellis blog but instead of btrfs I preferred ext4 and utilized a modified version of the Hetzner installation script.


Boot into linux rescue mode and make a local copy of the installation script cp -a /root/.oldroot/nfs/install /root/.oldroot/nfs/images/Ubuntu-1604-xenial-64-minimal.tar.gz /root/.
Then modify the copied script(s) so they setup also LUKS (Linux Unified Key Setup):

install.sh: Add before # LVM

status_busy "Creating LUKS container"
status_donefailed $?


functions.sh: Add before make_lvm() {

make_luks() {
  echo -n "CHANGEME" | cryptsetup luksFormat /dev/md1 -d -
  echo -n "CHANGEME" | cryptsetup luksOpen /dev/md1 md1 -d -
  return 0

functions.sh: Edit pv variable definitions in make_lvm() { to match

    # create PVs
    for i in $(seq 1 $LVM_VG_COUNT) ; do
      debug "# Creating PV $pv"
      wipefs -af $pv |& debugoutput
      pvcreate -ff $pv 2>&1 | debugoutput

    # create VGs
    for i in $(seq 1 $LVM_VG_COUNT) ; do

This changes just add another step which encrypts md1 with LUKS and password CHANGEME and adapts the physical volume paths which are used for LVM.
Of course this is just are very hacky “implementation” because it uses a hard coded password (CHANGEME) and hard coded mapper names/paths. But for my singular use case everything else would be impractical.


Now we utilize the installation magic from Hetzner:
/root/install/installimage -n my.hostname.tld -r yes -l 1 -p /boot:ext4:512M,lvm:vg0:all -v vg0:swap:swap:swap:4G,vg0:root:/:ext4:20G -i /root/Ubuntu-1604-xenial-64-minimal.tar.gz -K /root/.ssh/robot_user_keys
As you can see I’d only used 20G for logical volume root. That’s just because this is a XEN server where each DomU has it’s own logical volume and has nothing to do with the FDE.

To be able to decrypt md1 after the reboot (and without KVM) it’s necessary to finalize the installation in chroot.

change root:

mount /dev/vg0/root /mnt
rm -r /mnt/run/lock/
mount /dev/md0 /mnt/boot/
chroot-prepare /mnt
chroot /mnt

Configure crypttab and avoid bug 1256730:

echo "# https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1256730" > /usr/share/initramfs-tools/conf-hooks.d/forcecryptsetup
echo "export CRYPTSETUP=y" >> /usr/share/initramfs-tools/conf-hooks.d/forcecryptsetup
echo "md1 /dev/md1 none luks" > /etc/crypttab

Install necessary packages:

apt-get update
apt-get install -y cryptsetup busybox dropbear

Configure initramfs and dropbear:
Replace {IP}, {GATEWAY}, {NETMASK}, {HOSTNAME} with appropriate values

mkdir -p /etc/initramfs-tools/root/.ssh/
chmod ug=rwX,o= /etc/initramfs-tools/root/.ssh/
cp /root/.ssh/authorized_keys /etc/initramfs-tools/root/.ssh/authorized_keys
echo "ifconfig eth0" >> /usr/share/initramfs-tools/scripts/init-bottom/dropbear
echo "DEVICE={IP}::{GATEWAY}:{NETMASK}:{HOSTNAME}:eth0" >> /etc/initramfs-tools/initramfs.conf

(Optional) Add a short unlock script (borrowed by stinkyparkia.wordpress.com) for convenience:

cat <<"OUTEREOF" > /etc/initramfs-tools/hooks/crypt_unlock.sh

if [ "$1" == "prereqs" ]; then
  echo dropbear
  exit 0

. "${CONFDIR}/initramfs.conf"
. /usr/share/initramfs-tools/hook-functions

if [ "${DROPBEAR}" == "n" ] || [ ! -r "/etc/crypttab" ]; then
  exit 0

cat > "${DESTDIR}/bin/unlock" <<"EOF"
if PATH=/lib/unlock:/bin:/sbin /scripts/local-top/cryptroot; then
  kill `ps | grep '[c]ryptroot' | awk '{ print $1 }'`
  # following line kill the remote shell right after the passphrase has
  # been entered.
  kill -9 `ps | grep "\-[s]h" | awk '{ print $1 }'`
  exit 0
exit 1

chmod 755 "${DESTDIR}/bin/unlock"
mkdir -p "${DESTDIR}/lib/unlock"
cat > "${DESTDIR}/lib/unlock/plymouth" <<"EOF"
[ "$1" == "--ping" ] && exit 1
/bin/plymouth "$@"

chmod 755 "${DESTDIR}/lib/unlock/plymouth"
echo "To unlock root-partition run 'unlock'" >> ${DESTDIR}/etc/motd

And finaly update initramfs and grub

update-initramfs -u -k all

Now everything should be fine and you can reboot into your fresh installed Ubuntu with full disk encryption.

Client side

Because SSH key errors are annoying and normally you don’t login as root to your server, just add the following to your local SSH client configuration (~/.ssh/config) and you can SSH into busybox with ssh unlock:

Host unlock
  User root
  HostName {HOSTNAME}
  UserKnownHostsFile ~/.ssh/dropbear.known


If there are any problems (e.g. you are not able to SSH into busybox) just boot into the rescue system and chroot into the installed Ubuntu:

echo -n "CHANGEME" | cryptsetup luksOpen /dev/md1 md1 -d -
vgchange -a y
mount /dev/vg0/root /mnt
rm -r /mnt/run/lock/
mount /dev/md0 /mnt/boot/
chroot-prepare /mnt
chroot /mnt

After fixing any errors (e.g. DEVICE config in /etc/initramfs-tools/initramfs.conf) exit from chroot and try again:

umount /mnt/boot
umount /mnt/proc
umount /mnt/dev
umount /mnt/sys
umount /mnt
# ^ all optional

Hetzner KVM can be very handy while debuging any boot issues. I’d much fun with a wrong DEVICE string.

Change passphrase

LUKS provides up to 8 key slots (0 to 7) to store different passwords. To change your current password add a new one cryptsetup luksAddKey /dev/md/1 (asks for a valid password), verify it with cryptsetup luksOpen --test-passphrase --key-slot 1 /dev/md/1 && echo OK and then remove the old password with cryptsetup luksRemoveKey /dev/md/1 (asks for a valid password).

Before removing the old passphrase I wait until next reboot to verify that everything works just fine with the new password.