Nix: override packages with overlays

A brief summary of the problem

To make this easier to follow, I'll make the problem more concrete. One of my goals for myself when working with Kubernetes, is to have absolutely all configuration stored as code. Regardless of how you feel about YAML, there's no denying that it can easily get tedious and repetitive with no templating or programming language features baked in.

For a while now, I've been wanting to look into the Dhall language for configuration, and this seemed a perfect opportunity, considering it also has Kubernetes bindings. But here's the problem: the Dhall version in nixpkgs is 1.24.0, but the Kubernetes bindings require at least 1.27.0 to work.

What are overlays?

For a more thorough explanation, consult the NixOS wiki or watch /Nixpkgs Overlays --- A place for all excluded packages/ by Nicolas Pierron. In short, overlays are functions that transform package sets by adding or overriding keys. Notably, Mozilla has their own set of Nix packages, Mozilla nixpkgs, which contains a Rust overlay, allowing Nix users to stay up to date with the quick cadence of Rust releases. In our case, it allows us to replace the packages that are out of date with newer updated versions.

There's quite a bit of content out there on how to use overlays: the aforementioned resources are a great place to start, and blog posts like the DOs and DON'Ts of overlays by Flying Circus offer further insights. That said, I couldn't find anything that really fit my use case. Thankfully, the helpful people over at the NixOS Discourse forums pointed me in the right direction.

The solution

Now that we know what they are and how they work, we can take a stab at creating our own overlays.

Let's assume our folder structure looks like the following:

├── shell.nix
└── nix-files
    └── overlay.nix

We can then write a very simple shell.nix file that just sets us up with the dhall package.

{ pkgs ? import <nixpkgs> { overlays = [ (import ./nix-files/overlay.nix) ]; }
}:
stdenv.mkDerivation {
  name = "dhall";
  buildInputs = [ pkgs.dhall ];
}

There's nothing terribly exciting going on here, but notice that when importing nixpkgs, we specify a list of overlays. In our case, it's only one, but we can use as many as we want.

With this all set up, let's have a look at the overlay:

self: super: {
  dhall = builtins.fetchTarball
    "https://github.com/dhall-lang/dhall-haskell/releases/download/1.30.0/dhall-1.30.0-x86_64-linux.tar.bz2";
}

The first line lists the expression's parameters. By convention, they're called self and super. In our case, we needn't concern ourselves with them, because we're replacing one of the keys entirely. The actual value of the expression is just a set with a single key, dhall, mapped to the result of fetching a tarball. This replaces that value in the original package set.

Some additional notes on the overlay: We're using the plain version of the builtins.fetchTarball function in this example. There's also a version that takes an attribute set with a URL and a hash, which will make sure that the you get the same version every time. Furthermore, we're specifying which exact version we want (1.30.0). That's perfectly fine for a little example like this, and will probably be good enough for a little dev environment. It would be ideal, though, if we could make sure we're always up to date with the latest release. The aforementioned forum help thread has a suggestion for how to do this, but it seemed to have some unexpected issues, so I’ll leave that as an exercise for the reader.

Now, when loading the shell.nix file, we should see the following:

$ dhall version
1.30.0

Boom.