A couple weeks ago I was coming into work and wanted to quickly finish up some
experimentation I was doing with regards to setting up automatic disk decryption
at boot using TPM 2.0 and LUKS. I have been using full disk encryption (FDE) for
a while now but I wanted to try out the different ways that
supports unlocking the disk. Of the various methods, it seems that using a TPM
2.0 device to unlock the disk, given the validity of the system firmware and
secure boot state, was most interesting. The most desireable approach is
actually to use TPM 2.0 along with a passphrase or PIN, but this
functionality is not yet
available as it will be released in systemd v251.
Anyways, I had previously added a slot to the TPM consisting of PCRs 0,2,4,7 and found that the disk was not automatically unlocking. I could verify that I had the slot added with:
# systemd-cryptenroll /dev/nvme0n1p2
I wanted to remove the old slot and try again, so I issued this command:
# systemd-cryptenroll /dev/nvme0n1p2 --wipe-slot=0,2,4,7
Then, I went to add a new slot but with different PCRs, so I did:
# systemd-cryptenroll /dev/nvme0n1p2 --tpm2-device=auto --tpm2-pcrs=0,7
This command prompted me for a passphrase like usual, but for some reason it wouldn’t accept my passphrase. When I reissued a command to view the slots, this is what I saw:
# systemd-cryptenroll /dev/nvme0n1p2 SLOT TYPE 1 tpm2
My passphrase was gone! 😱
At this point I was in a bit of a panic. I tried a few more times out of insanity to try and add in the passphrase again so that I could work on my encrypted disk, but to no avail. I started to collect myself and had a couple realizations. First, I couldn’t think of any way to restore the passphrase into the LUKS header of the disk without just reformatting it, and second, my disk was still decrypted at this point in time.
So, while continuously being paranoid of a kernel panic or largely improbable cosmic event knocking out my laptop and costing all my data, I continued to work until lunch came around, at which point I went back to the house and grabbed my SATA to USB hard drive adapter and a spare 8TB hard disk drive I had laying around. (Okay, the drive was meant to go into my desktop as a replacement but I have been too lazy to do it as of late.)
I brought the two devices back to the office and continued to work until the
closing of the day. I immediately hooked up the drive to my laptop and began
copying over all my data with an
rsync command. I left my laptop at the office
that day (locked to the desk, mind you) while the backup operation ran
The next morning I came in to see that it had thankfully finished. With my
fingers crossed, I rebooted the laptop back into a Gentoo LiveDVD environment
and reformatted the LUKS partition with my old passphrase. Then, I began
recreating my LVM volumes on top of it and adjusting my
accordingly. After the volumes were all mounted, I ran another
rsync to begin
the restoration process and found something else to do at work while this was
running. Needless to say this also took basically the entire day, and I left my
laptop at the office for a second night while this completed.
At the dawn of the final day, I returned to my laptop to see it had finished.
After verifying that my
/etc/fstab was correct and making the same adjustments
to the kernel commandline, I rebooted the machine and was greeted with a
pleasant TTY asking for my credentials. 👏
The last thing I needed to do after rebooting was fix the SELinux contexts on
the machine with a simple
rlpkg -ar and manually fix the contexts on the
various mountpoints for the root inode of the volumes (sit tight for a minor
writeup on this as well).
After all was done it was like I had never screwed up in the first place. What I
didn’t mention was that I was also researching on the side what exactly I did
wrong while the recovery was taking place. You may have already guessed it, but
my error was in thinking that the
--wipe-slot argument took PCRs to wipe and
not the slot itself. Normally when you encrypt a disk with LUKS and a
passphrase, the passphrase goes into slot 0. When issuing the above wipe
command, I ended up removing the passphrase slot entirely. If the TPM unlocking
had worked, I could in theory have not worried so much about it, however I would
not be able to update the kernel or initramfs without breaking the disk
decryption process. Additionally, I never got it to work in the first place with
I am writing this from the same laptop a couple weeks since this happened with a fully working disk decryption with TPM 2.0 setup. It turns out that in addition to the errors I made above, dracut currently has a bug that omits some of the required libraries for systemd-cryptsetup to do its job. I had to manually include these in my dracut config for the time being until a new version with the fix is released. Lastly, I opted to stick with PCRs 0,7 for the time being as this allows me to update my kernel and initramfs together without needing to add a new slot for them every time.
Going forward, I intend to add a recovery key to the disk in the event that something like this happens again. That way, I should still be able to manipulate the LUKS header in case I do something similarly silly in my experimentation.
Hopefully this was an interesting read and a lesson to carefully read the corresponding documentation when working on particularly sensitive projects.
’til next time, see ya! 👋