Move unneeded directories and files from dom0 root filesystem

[tlaurion]

@osresearch @marmarek :
When comparing two root snapshots per

[user@dom0 ~]$ cat /lib/systemd/system-shutdown/root-autosnap 
#!/bin/sh

#This permits wyng-backup to backup root-autosnap and root-autosnap-back, taken at each system shutdowns like any other QubesOS LVMs.
#This also permits to restore to different states of dom0 from Heads and compare dom0 between reboots

#TODO: backup /boot content into a LVM and apply same logic, corresponding to each dom0 snapshots
#https://github.com/tasket/wyng-backup/issues/63

#We delete the backup of last shutdown snapshot (last last shutdown)
/usr/sbin/lvremove --noudevsync --force -An qubes_dom0/root-autosnap-back || true
#We take a snapshot of root-autosnap into root-autosnap-back
/usr/sbin/lvcreate --noudevsync --ignoremonitoring -An -pr -s qubes_dom0/root-autosnap -n root-autosnap-back
#We remove root-autosnap
/usr/sbin/lvremove --noudevsync --force -An qubes_dom0/root-autosnap || true
#We create root-autosnap from root
/usr/sbin/lvcreate --noudevsync --ignoremonitoring -An -pr -s qubes_dom0/root -n root-autosnap

And then we compare the content of the filesystems, we see that:

• /etc/libvirt/libxl
• /etc/lvm
• /etc/xdg/adjtime
• /home
• /root
• /var

Would need to be out of root fs to be able to have a RO QubesOS dom0 with dmverity

Originally posted by @tlaurion in #4371 (comment)

[tlaurion]

@DemiMarie approves moving those out at #4371 (comment)

@marmarek @osresearch input on #4371 (comment) ?

Personally I would be fine with moving these off of the root filesystem.

[tlaurion]

Also was #5777 (comment)

[ben-grande]

@tlaurion Can you please adapt the title? I don't think you mean to move from (out of) dom0, but move to another directory so dm-verity works?

[tlaurion]

@tlaurion Can you please adapt the title? I don't think you mean to move from (out of) dom0, but move to another directory so dm-verity works?

Title is good. To enable dm-verity, moving volatile/dynamic content out of dom0 filesystem would allow the it to to be verified with dm-verity hashing.

Experiment comparing snapshots between reboot would give a fresher list of states needing to be kept out to have read only dom0 between updates and verification.

[tlaurion]

Is this GSoC material?

Most needed.

[ben-grande]

@tlaurion Can you please adapt the title? I don't think you mean to move from (out of) dom0, but move to another directory so dm-verity works?

Title is good. To enable dm-verity, moving volatile/dynamic content out of dom0 filesystem would allow the it to to be verified with dm-verity hashing.

But it is not moving from dom0 to something out of dom0 right? It is moving "in" dom0, from root to another place. This is why I asked for the title clarification. Note, I do not fully understand dm-verity, this is just my bare understanding of it.

[tlaurion]

@tlaurion Can you please adapt the title? I don't think you mean to move from (out of) dom0, but move to another directory so dm-verity works?

Title is good. To enable dm-verity, moving volatile/dynamic content out of dom0 filesystem would allow the it to to be verified with dm-verity hashing.

But it is not moving from dom0 to something out of dom0 right? It is moving "in" dom0, from root to another place. This is why I asked for the title clarification. Note, I do not fully understand dm-verity, this is just my bare understanding of it.

From referred now old safeboot

Protecting the runtime system integrity (by optionaly booting from a read-only root with dm-verity and signed root hash)

As per op

And then we compare the content of the filesystems, we see that:

/etc/libvirt/libxl
/etc/lvm
/etc/xdg/adjtime
/home
/root
/var

This was a long time ago, didn't reassess lvm changes between snapshots, but that would give complete picture. Change QubesOS configs, snapshot, assess.

Heads recently adapted its root hashing feature tu support dom0 lvm hashing in standard installation, which can be simulated per commands in op.
linuxboot/heads#2067. This feature aims to hash everything in standard OS libs and binaries path to act as a HIDS Ala tripwire, and can be activated to check root hashes on boot, without expecting dm-verity to be possible (dm-verity is a fast hashing mechanism for volumes expected to not change between boot).

Files in dom0 changes without OS base changing, this is related but not limited to dom0 configs, logs, and other artifacts that should not be in dom0 in the goal of having a dom0 that is RO, prerequisite of dm-verity.

To me title is good and references are sound. It's about moving irrelevant runtime changes out of dom0, this issue gives guidelines to investigate between lvm snapshots to see what needs to be done.

[marmarek]

It's about moving irrelevant runtime changes out of dom0, this issue gives guidelines to investigate between lvm snapshots to see what needs to be done.

No, it isn't moving them out of dom0, it about possibly moving them outside of dom0 root partition, into another dom0 partition, but still dom0. There is definitely no desire to move system configuration out of dom0.
The above list kinda looks like it isn't even about much software change, but partition layout. /var, /root, /home and even /etc are supposed to be mutable, and persistent. Making them completely read-only is undesirable, but moving to a separate partition may be a good idea. Technically, large part of /etc also doesn't change much, but the changing parts are not limited to what you listed - for example when user change their password, /etc/shadow will be modified (and such modification must survive). Similarly, enabling/disabling repositories (both for dom0 updates and for templates) will modify some files in /etc. There are more examples like this.

[marmarek]

As for /etc, AFAIK openSUSE is attempting an interesting thing: move default configuration as much as possible into /usr (/usr/etc, usr/lib etc) - so, /etc contains only configuration specific to the current installation. I've seen somebody telling they want to go as far as the fresh installation should have almost empty /etc (except files explicitly created by the installer, like /etc/fstab). It would be interesting in this context, but I'm not sure how far they are on this way right now, last time I checked fresh openSUSE installation had still quite full /etc.