Docking pains

Enabling

According to the NixOS manual and the the Wiki article on Docker there's really not much to it: To enable Docker, all you need to do is update your configuration.nix to include

virtualisation.docker.enable = true;

As pure and simple as Nix should be.

According to the manual: "/This option enables docker, a daemon that manages linux containers. Users in the "docker" group can interact with the daemon (e.g. to start or stop containers) using the docker command line tool./"

Read that last line carefully: "/Users in the "docker" group can interact with the daemon [...]/". Yup. That means we need to make sure our user is in the correct group:

# replace thomas with the name of your user
users.extraUsers.thomas = {
  extraGroups = [
    "docker"
    # ... other groups
  ];
# ... remaining configuration
};

What isn't immediately obvious is this: You must log out and back in before this setting change takes effect. Let's repeat that to make sure we understand:

You *must* log out and back in before this setting change takes effect.

From what I can tell, this goes for any change to a user's groups, but it isn't particularly well documented anywhere. (Psst: I am not the only one to have run into this.)

But wait; there's more! This is only vaguely referenced in the manual ("/using the docker command line tool/"), but to have access to the Docker CLI, you're going to have to install Docker (pkgs.docker) for your user, either by putting it in configuration.nix's systemPackages or by using a solution such as this.

And that's it. If all you wanted to do was set up docker to run with the default configuration, you're done now. Congrats! Have a donut. You've earned it.

The space race

Ah, yes, disk space ... We've got Docker in place now, and, assuming the group change setting has taken effect, we can start playing with it. That's what I did. For a day or so. As per the usual NixOS song and dance, I wanted to change some configuration settings, so I tried to rebuild my system and got this fateful message:

No space left on device

Now, this isn't anything new. I've realized since setting up the OS, that I should have probably allocated more space for the root partition (someone once told me that NixOS "trades disk space for sanity"). "Oh, well," I thought. "Guess I have to delete some old generations again." So I ran the garbage collector. This usually frees up about 7--10GB of space, but now it was hardly removing two! I tried all the tricks that I knew of, but nothing seemed to make a difference. And then the thought struck me: "Docker is installed as a system service. That means it probably stores images system-wide too!". And indeed, after looking through the 'docks' (har har), I found that the default place Docker stores data is in /var/lib/docker.

So I killed all of my containers, deleted all of my images, and lo and behold: My root partition had suddenly lost nearly 10GB! Superb!

The next step, then, would be to figure out how to store the data somewhere else. Luckily, the docs (NixOS and Docker) are quite clear on this point: For your Docker configuration, you can specify an option, --data-root, and have the data stored there instead. In general, I prefer not to mess around with where things are stored too much, but in some cases it makes life easier (until I get around to repartitioning my drive, anyway), so I decided I'd put it under /home/docker for now. This is easily done like this:

virtualisation.docker.extraOptions = "--data-root /home/docker";

This setting means that my /home partition carries some extra data, but it's got more than enough space to deal with it.

Putting it into practice

Now, having experienced first-hand how space-hungry Docker can be, and having read through the documentation, I found that there are other options that might come in handy. For now, I decided to have the system aggressively auto-prune on a weekly basis. This should keep me from running into space issues any time soon, and if it gets annoying I can always change the settings.

At the end of this little adventure, the resulting configuration.nix should look something like this:

# Docker CLI (either put this here or in your user config)
environment.systemPackages = with pkgs; [ docker ];

# Put your user in the correct group
users.extraUsers.thomas.extraGroups = [ "docker" ];

# Set up the Docker daemon
virtualisation.docker = {
  enable = true;
  autoPrune = {
    enable = true;
    flags = ["--all"];
  };
  extraOptions = "--data-root /home/docker";
};

In summary, these are the steps needed:

1. Enable virtualisation.docker
2. Make sure your user is in the ~"docker"~ group. (Log out and back in!)
3. Install Docker for your user
4. (Optional) If, like me, you have issues with space, change the data-root to somewhere else, such as a different partition or an external drive.

So there you have it, folks! It really is quite simple ... once you figure out all the tricky parts.