How to install Nix in a Qubes OS AppVM

Author: Solène

Date: 15 May 2023

Tags: qubes qubesos nix nixos

Comment on Mastodon

Intro

I'm still playing with Qubes OS, today I had to figure how to install Nix because I rely on it for some tasks. It turned out to be a rather difficult task for a Qubes beginner like me when not using a fully persistent VM.

Here is how to install Nix in an AppVm (only /home/ is persistent) and some links to the documentation about `bind-dirs`, an important component of Qubes OS that I didn't know about.

Qubes OS documentation: How to make any file persistent (bind-dirs)

Nix project website

bind-dirs

Behind this unfriendly name is a smart framework to customize templates or AppVM. It allows running commands upon VM start, but also make directories explicitly persistent.

The configuration can be done at the local or template level, in our case, we want to create `/nix` and make it persistent in a single VM, so that when we install nix packages, they will be stay after a reboot.

The implementation is rather simple, the persistent directory is under the `/rw` partition in ext4, which allows mounting subdirectories. So, if the script finds `/rw/bind-dirs/nix` it will mount this directory on `/nix` on the root filesystem, making it persistent and without having to copy at start and sync on stop.

Setup

A limitation for this setup is that we need to install nix in single user mode, without the daemon. I suppose it should be possible to install Nix with the daemon, but it should be done at the template level as it requires adding users, groups and systemd units (service and socket).

In your AppVM, run the following commands as root:

This creates an empty directory `nix` owned by the regular Qubes user named `user`, and we tell bind-dirs that this directory is persistent.

/!\ It's not clear if it's a bug or a documentation issue, but the creation of `/rw/bind-dirs/nix` wasn't obvious. Someone already filled a bug about this, and funny enough, they reported it using Nix installation as an example.

GitHub issue: clarify bind-dirs documentation

Now, reboot your VM, you should have a `/nix` directory that is owned by your user. This mean it's persistent, and you can confirm that by looking at `mount | grep /nix` output which should have a line.

Finally, install nix in single user mode, using the official method:

Now, we need to fix the bash code to load Nix into your environment. The installer modified `~/.bash_profile`, but it isn't used when you start a terminal from dom0, it's only used when using a full shell login with `bash -l`, which doesn't happen on Qubes OS.

Copy the last line of `~/.bash_profile` in `~/.bashrc`, this should look like that:

Now, open a new shell, you have a working Nix in your environment \o/

You can try it using `nix-shell -p hello` and run `hello`. If you reboot, the same command should work immediately without need to download packages again.

Configuration

In your Qube settings, you should increase the disk space for the "Private storage" which is 2 GB by default.

Conclusion

Installing Nix in a Qubes OS AppVM is really easy, but you need to know about some advanced features like bind-dirs. This is a powerful feature that will allow me to make lot of fun stuff with Qubes now, and using nix is one of them!

Going further

If you plan to use Nix like this in multiple AppVM, you may want to set up a local substituter cache in a dedicated VM, this will make your bandwidth usage a lot more efficient.

How to make a local NixOS cache server