Pinning nix-shell package versions for reproducibility

Introduction

I rely on nix-shell to manage dependencies for the projects I work on, and when combined with direnv (more on that in a future post!), it automatically configures my shell. All the necessary dependencies are installed with the correct versions for the project, making them readily available in my path.

If you’re new to nix-shell, it acts as an environment manager designed to simplify project dependency management, providing a straightforward and reproducible approach to setting up development environments. It ensures that the required dependencies, libraries, and tools are configured consistently, offering a hassle-free development experience. Nix-shell’s key advantage lies in its ability to create self-contained environments, particularly valuable for projects with intricate dependencies.

You can create a shell with only the dependencies and versions you need, and nothing more.

Practical example

For example, consider a simple project written in TypeScript/Node.js/JavaScript. Typically, you’d have a file named shell.nix in the project’s root directory. To enter the shell, navigate to the directory containing the file and run nix-shell.

{ pkgs ? import <nixpkgs> {}
}:
pkgs.mkShell {
  name = "projects.my-project-name";
  buildInputs = [
    pkgs.bashInteractive
    pkgs.nodejs
  ];
}

It will include bash and nodejs from latest nixpkgs when the user enters that shell on the command line.

Latest major version

Now what if we need a particular version of Node? For a major version that is pretty easy - we can just append a major version number to the package reference. In the following example we pin to Node.js version 18.* in semantic versioning terms, but you could also indicate 14, 16, 19 or 20 for example. Be aware though that not all major versions are available in the latest nixpkgs repository so you should check using the website I’ll describe next.

{ pkgs ? import <nixpkgs> {}
}:
pkgs.mkShell {
  name = "projects.my-project-name";
  buildInputs = [
    pkgs.bashInteractive
    pkgs.nodejs-18_x
  ];
}

Using `with` to simplify `buildInputs`

As an aside we can use little trick to simplify the references in buildInputs so we don’t have to prefix each dependency with pkgs..

{ pkgs ? import <nixpkgs> {}
}:
pkgs.mkShell {
  name = "projects.my-project-name";
  buildInputs = with pkgs; [
    bashInteractive
    nodejs-18_x
  ];
}

Now each reference will be prefixed with pkgs automatically because we included with pkgs; after buildInputs =.

Pinning to a specific version

To pin to a specific version of Node.js though we need to add a little more code to the nix-shell file. This is because we need to pull the package from the nixpkgs repository at the git commit that references that particular version. Firstly, we can use the Nix package versions website to find a SHA hash of the dependency.

The author of that handy page also has an interesting blog post that goes into more detail on the problem that you might find interesting as an aside. He also goes into more detail on how this style of pinning can be imperfect.

Enter the name of the package you want to find and you’ll get back a table of available versions. I searched nodejs and got back a list of versions and found the version 18.14.0 that I was looking for. Copy the SHA hash from the Revision column of the table because you’ll need this up next - in my case this was 55070e598e0e03d1d116c49b9eff322ef07c6ac6.

As an example of how this kind of pinning can be imperfect there is no Node.js version 18.13.0 available because a derivation was never committed for it!

Now we can use the following sample nix-shell file to pull down a particular version of the nodejs package. We will still pull the latest version of Bash, but we are specifying a particular git commit hash of nixpkgs to pull nodejs from.

{ pkgs ? import <nixpkgs> {}
}:
let
  # node 18.14.0
  nodePkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/55070e598e0e03d1d116c49b9eff322ef07c6ac6.tar.gz") { };
in
pkgs.mkShell {
  name = "projects.my-project-name";
  buildInputs = with pkgs; [
    bashInteractive
    nodePkgs.nodejs-18_x
  ];
}

Note the new nodePkgs variable that imports a tar file of the commit in question from github and is then used to prefix the reference to nodejs-18_x. This is how we can now be sure that we will always get node 18.4.0 when we instantiate this nix-shell.

Using yarn as the package manager

Now as a bonus let’s see how we can use yarn with our project and ensure it is referencing the correct version of node. Normally you would just add the package reference yarn into the buildInputs list, but we need to tell it to use the exact version of node that our project specifies.

{ pkgs ? import <nixpkgs> {}
}:
let
  # node 18.14.0
  nodePkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/55070e598e0e03d1d116c49b9eff322ef07c6ac6.tar.gz") { };
in
pkgs.mkShell {
  name = "projects.my-project-name";
  buildInputs = with pkgs; [
    bashInteractive
    nodePkgs.nodejs-18_x
    (yarn.override { nodejs = nodePkgs.nodejs-18_x; })
  ];
}

This installs yarn and passes an override through to it that specifies the correct version of node for yarn to reference.

Putting .bin from node_modules into $PATH

Speaking of node dependencies; we can also put the node_modules/.bin directory on the path to make it easier to run and reference scripts installed by our npm/yarn dependencies. This can be achieved by adding a shellHook to the nix-shell file that is written in bash/shell and will be executed right before the new shell is handed to the user.

{ pkgs ? import <nixpkgs> {}
}:
let
  # node 18.14.0
  nodePkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/55070e598e0e03d1d116c49b9eff322ef07c6ac6.tar.gz") { };
in
pkgs.mkShell {
  name = "projects.my-project-name";
  buildInputs = with pkgs; [
    bashInteractive
    nodePkgs.nodejs-18_x
    (yarn.override { nodejs = nodePkgs.nodejs-18_x; })
  ];

  shellHook = ''
    export PATH="$PWD/node_modules/.bin/:$PATH"
  '';
}

Of course you could run any code here or add any path to your $PATH.

Pinning all the things

You can also pin the overall packages import so that you always get the same version of bash or any other package that are being imported from pkgs. This is done by replacing the <nixpkgs> token with a fetchTarball that takes a SHA hash from nixpkgs just like the nodePkgs pinning we did earlier.

{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/e0629618b4b419a47e2c8a3cab223e2a7f3a8f97.tar.gz") {}
}:
let
  # node 18.14.0
  nodePkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/55070e598e0e03d1d116c49b9eff322ef07c6ac6.tar.gz") { };
in
pkgs.mkShell {
  name = "projects.my-project-name";
  buildInputs = with pkgs; [
    bashInteractive
    nodePkgs.nodejs-18_x
  ];
}

A good place to find the latest safe hash for the various branches of nixpkgs a good place to start is status.nixos.org that shows the build status for each of them. Generally, you will want to either grab the latests stable version, which is nixpkgs-23.11 at the time of writing, or the latest unstable with nixpkgs-stable.

Conclusion

So, there are a few tricks to make using a nix-shell a little easier and more specific. In another post I will describe how I combine this with direnv so that the shell is automatically instantiate for me when I change directories.