Tag Archives: sparse file

Auto mount an encrypted IMG file stored on NFS share

Yes, here we are again.
Now that I have a NAS at home, it’s about time to get rid of all these single USB disks connected to my Raspberry PIs.

I have a share called nfsshare available from my NAS (IP: 192.168.1.10). The full share path is 192.168.1.10:/volume1/nfsshare. My NAS handles NFS version 4.

So, here what I’ve done, to setup my Banana Pro Pi with Armbian based on Debian 10 (buster).

Configure NFS client

First of all, we need to create the mount point where we’re going to access the nfs share (let’s use /nfs) and install the packages for NFS.

mkdir /nfs
apt-get install nfs-common

Once done, a minimal tuning of idmapd.conf, if you have defined a domain/workgroup within your network. In this example I’m using mydomain.loc.

sed -i 's/#Domain = local.domain.edu/Domain = mydomain.loc/' /etc/idmapd.conf

Update our /etc/fstab file, to make sure it mounts at boot, and test if all works as expected:

192.168.1.10:/volume1/nfsshare /nfs nfs4 auto,_netdev,nofail,noatime,nolock 0 0

I have used _netdev to make sure that the system understands that this needs the network up before trying to mount, and, if something goes wrong, the boot continues (nofail). This is very handy on systems without a proper monitor where you rely on ssh connections.

Now, with a simple mount /nfs command, you should be able to get the share mounted. df -Th or mount commands are what I would you to verify.

Cool, we have now the share mounted. Issue a quick shutdown -r now to see if all works as expected. Once your device is back online, ssh into it and check with df -Th or mount commands again. Hopefully, you can see your nfs share mounted to /nfs.

Create and configure your Encrypted “space”

I have already discussed something about encrypted devices in another post. This will be a revised version of the previous post, without custom scripts, but simply using what Debian offers out of the box.

Create an empty IMG file to host our encrypted space

I have decided to create 500GB of encrypted space to store my data. To do so, I did the following:

  • install the required software for encryption
  • create a sparsefile (on my /nfs share)
  • encrypt it
  • format it (ext4)
  • setup the auto mount
apt-get install cryptsetup

dd of=/nfs/file_container.img bs=1 count=0 seek=500G

cryptsetup -y luksFormat /nfs/file_container.img
cryptsetup luksOpen /nfs/file_container.img cryptcontainer

mkfs.ext4 -L cryptarchive /dev/mapper/cryptcontainer

During the above steps, you will be asked to set a passphrase, and use it to open the IMG file. Pretty straight forward, isn’t it?

Cool. Now we have 500GB sparsefile called file_container.img store on our share /nfs ready to be mounted somewhere and utilised.

To make sure we can mount at boot, we need a secret key that we are going to use to decrypt the IMG file without typing any passphrase.

Let’s create this key stored under /root (in this example). You can store wherever you want, as long as it’s accessible before the decryption start. Another good place is /boot.

dd if=/dev/urandom of=/root/keyfile bs=1024 count=4
chmod 0400 /root/keyfile

Now we need to add this key within the IMG file

cryptsetup luksAddKey /nfs/file_container.img /root/keyfile

Next step, is to instruct /etc/crypttab, with the details about our encrypted file and its key.
Just add the following line at the end of /etc/crypttab file.

cryptcontainer /nfs/file_container.img /root/keyfile luks

Now, there is a problem. Your OS needs to know that the IMG file isn’t stored locally and has a dependency on the NFS share. In fact, if the OS tries to decrypt the IMG file before mounting the NFS share, it will fail, and you might get stuck in a no-end booting, forcing sometimes to get your mini monitor for troubleshooting, a spare keyboard and anti-human positions to reach your small Pi etc etc… basically, a nightmare!

So, here a trick that seems working.
In Debian, there is a file called /etc/default/cryptdisks
Within this file, we are going to make sure that CRYPTDISKS_ENABLE is set to yes and CRYPTDISKS_MOUNT is set to our NFS mount (/nfs). In this way, the service that handles the encryption/decription will wait for /nfs mounted before starting.
IMPORTANT: this must be a mountpoint within /etc/fstab

Here the content of my /etc/default/cryptdisks file

# Run cryptdisks initscripts at startup? Default is Yes.
CRYPTDISKS_ENABLE=Yes

# Mountpoints to mount, before cryptsetup is invoked at initscripts. Takes
# mountpoins which are configured in /etc/fstab as arguments. Separate
# mountpoints by space.
# This is useful for keyfiles on removable media. Default is unset.
CRYPTDISKS_MOUNT="/nfs"

# Default check script. Takes effect, if the 'check' option is set in crypttab
# without a value.
CRYPTDISKS_CHECK=blkid

Amazing! Now, just the last bit: update /etc/fstab with the reference of our device. Because now we have setup all the necessary to open the encrypted IMG file and associate it to a mountable device. But we haven’t mounted it yet!

Create the mount point:

mkdir /cryptoarchive

Update /etc/fstab, appending this line:

/dev/mapper/cryptcontainer /cryptoarchive ext4 defaults,nofail 0 2

Again, the nofail, as for the NFS share, to avoid the boot process to get stuck in case of errors, and allow you to ssh into the device and troubleshoot.

Now we’re ready to try a mount /cryptoarchive, a df -Th and mount checks, and also a shutdown -r now, to verify that the NFS share gets mounted and the IMG encrypted disk mounted and available too.

Happy playing! 😉

Create an encrypted LUKS sparse file

For an updated version of this article, check this post!

I’m going to create a 2.7TB of file (sparse file) on my 3TB disk.
This file is a ‘thin provisionig‘ file, which means that it will expand only when/if used. FYI – it won’t shrink after usage. So a 2.7TB sparse file, once created, will be seen as 2.7T but it’s actually using a very little space (almost not noticeable). It will grow using it.
I will also encrypt it using cryptosetup LUKS format, to protect the content.

Why a sparse file?
Well, it’s practical, and it’s very handy if you want to move your files from a disk which is quite full already. In fact, I have my backups on this USB drive, EXT4 formatted. A 3TB drive half full.
A sparse file can be actually bigger than the available space on the disk. Of course, if you fill it up, it will error out. But while you want to move files, it’s… your saviour! 🙂

Using my example, I have 1.2TB used on this 3TB disk. I want to move all this data in an encrypted container that can potentially use the whole disk. So? Sparse file is the solution!
Theoretically I could also have had like… 2.2/2.5TB of data (on a 2.7TB available disk). As long as you free up space moving the files, things should work 🙂

First of all… if you haven’t done it yet, just install the package:

# apt-get install cryptsetup

Create the sparsefile

Let’s create a 2.7TB sparse file in /3TB.
[/3TB is the mount point of my /dev/sdb1 USB device]

# dd of=/3TB/file_container.img bs=1 count=0 seek=2700G

Create an encrypted LUKS container

We mount the sparse file just created and set the encrypted password. NOTE: you can change/add/remove this password (key) [later on explained how 😉 ]

# losetup /dev/loop0 /3TB/file_container.img 
# cryptsetup -y luksFormat /dev/loop0 
WARNING! ======== This will overwrite data on /dev/loop0 irrevocably. 
Are you sure? (Type uppercase yes): YES 
Enter passphrase: 
Verify passphrase: 
#

Open the Vault

Now, it’s time to ‘turn on’ this encrypted volume device and check that all went well

# cryptsetup luksOpen /dev/loop0 myarchive
Enter passphrase for /3TB/file_container.img:

# cryptsetup status myarchive
/dev/mapper/myarchive is active.
type: LUKS1
cipher: aes-xts-plain64
keysize: 256 bits
device: /dev/loop0
loop: /file_container.img
offset: 4096 sectors
size: 5662306304 sectors
mode: read/write

And of course, we need to format the device. I’ll use EXT4.

# mkfs.ext4 -L cryptarchive /dev/mapper/myarchive
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 707788288 4k blocks and 176947200 inodes
[...]
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

At this point, you can mount this device /dev/mapper/myarchive as usual:

# mkdir /cryptoarchive
# mount -t ext4 /dev/mapper/myarchive /cryptoarchive

You are now able to use your encrypted sparse file! All files can be now copied/moved into/cryptoarchive

Close the Vault

To unmount the vault, you need to follow these steps. This is important when you turn off your server!

# umount /cryptoarchive
# cryptsetup luksClose /dev/mapper/myarchive
# losetup -d /dev/loop0

Add key to the container

LUKS allows for up to 8 passwords to each partition/vault.
You can add other 7 basically and use all of these 8 passwords to access your vault.

To add keys, vault needs to be ‘open’

# losetup /dev/loop0 /3TB/file_container.img
# cryptsetup luksOpen /dev/loop0 myarchive
# cryptsetup luksAddKey /dev/loop0
< enter any current phrase - and add new phrase >

Change the key of the container

To change the key, the vault/container needs to be ‘close’

# cryptsetup luksClose /dev/mapper/myarchive
# cryptsetup luksChangeKey /dev/mapper/myarchive 
< verify and change your passphrase here>

Setup auto mount

This will generate a new file as a key, added to the vault and set to be used to auto mount at boot:

# dd if=/dev/urandom of=/root/keyfile bs=1024 count=4
# chmod 0400 /root/keyfile
# cryptsetup luksAddKey /dev/loop0 /root/keyfile
Enter any passphrase:
~#

Than we need to setup /etc/crypttab and /etc/fstab

# cat /etc/crypttab
# <target name>	<source device>		<key file>	<options>
myarchive /dev/loop0 /root/keyfile luks

# tail -n1 /etc/fstab
/dev/mapper/myarchive /cryptoarchive ext4 defaults,noauto 0 2

After that, I have personally created a specific init script. I couldn’t find the right way to run losetup /dev/loop0 /3TB/file_container.img before activating the archive, so…
First of all, I’ve disables cryptdisks via /etc/default/cryptdisks

CRYPTDISKS_ENABLE=No

Than, I’ve created this init script, saved as /etc/init.d/crypt-myvault

#! /bin/sh
### BEGIN INIT INFO
# Provides:          crypt-myvault
# Required-Start:    $remote_fs $syslog mountall netatalk
# Required-Stop:     $remote_fs $syslog mountall netatalk
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Manage encrypted vault
# Description:
### END INIT INFO

CONFFILE=/etc/default/crypt-myvault

. $CONFFILE > /dev/null 2>&1

if [ -z ${SPARSEFILE} ] || [ -z ${VAULT} ]; then
    echo "Required parameters missing.
	  Please configure $CONFFILE

	  # Example #
	  ENABLE=true
	  SPARSEFILE=/<path>/<file.img>
	  VAULT=<vault_name>
          "
    exit 1
fi


if [ ! $ENABLE ]; then
   echo "$0 disabled in $CONFFILE. Please set ENABLE=true and retry"
   exit 1
fi

checkconfig () {
if grep -q "^/dev/mapper/$VAULT" /etc/fstab && grep -q "^$VAULT" /etc/crypttab ; then
    CHK=0
else
    CHK=1
    echo "$VAULT doesn't seem configured in /etc/fstab or /etc/crypttab.\nUnble to continue."
fi
}

mount_myarchive () {
 /sbin/losetup /dev/loop0 $SPARSEFILE
 /sbin/cryptdisks_start $VAULT
 /bin/mount /dev/mapper/$VAULT
}

umount_myarchive () {
 /bin/umount /dev/mapper/$VAULT
 /sbin/cryptdisks_stop $VAULT
 /sbin/losetup -d /dev/loop0
}



case "$1" in
    start)
        echo "Enabling and mounting $VAULT"
	checkconfig
        mount_myarchive
        ;;
    stop)
        echo "Umounting and disabling $VAULT"
	checkconfig
        umount_myarchive
        ;;
    check)
	checkconfig
	[ $CHK -eq 0 ] && echo "Check OK" || echo "Check FAILED"
	;;
    *)
        echo "Usage: $0 start|stop|check"
        exit 1
        ;;
esac

exit 0

This script requires also a /etc/defaults/crypt-myvault configuration file:

# Conf file for /etc/init.d/crypt-myvault
ENABLE=true
SPARSEFILE=/3TB/file_container.img
VAULT=myarchive
VAULT_MOUNT_POINT=/cryptoarchive

To finish, we need to enable the script:

# update-rc.d crypt-myvault defaults

We need to be sure that this script runs AFTER the USB drive is mounted. So… I’ve added ‘netatalk’ as required-start in the header of the init script, as I’ve noticed that once netatalk starts, the USB disk is already mounted.
I’m sure there is a better/nicer way, but this seems to work well for me 🙂

Sources:

http://serverfault.com/questions/696554/creating-a-grow-on-demand-encrypted-volume-with-luks

LUKS passphrases: Changing, adding, removing

https://www.howtoforge.com/automatically-unlock-luks-encrypted-drives-with-a-keyfile