Speeding up NixOS package search on the terminal
For those who are using the new command-line interface of Nix, you’ve most likely went through to pain of searching packages through nix search
.
For reference, here’s what the experience looks like searching for a package from nixpkgs except it always like that every time you run the command. [1]
nix search nixpkgs
experience which took 8 minutes in total to complete the whole searchIt’s painful to use to the point where people tend to recommend search.nixos.org or MyNixOS for searching packages. [2] Ideally, searching packages for your system shouldn’t rely on online services. Luckily for us, there are ways how to mitigate against that.
I’m just going to say it. That package search result looks ugly.
That might be another reason why nix search
is not much used.
We’re using Nix v2.18 for this post. Everything that is featured here is considered to be experimental: from flakes, the command-line interface, and even the referenced manual may change over time.
In order to enable them in the first place, you’ll have to add |
What’s happening with nix search
But first, let’s take a closer inspection as to why nix search
is behaving like that.
Let’s look at the following command which is one of the examples from the manual.
nix search nixpkgs blender
At first glance, most people expect nix search nixpkgs blender
searches for package blender
from their nixpkgs instance.
It does work as intended which is searching through packages from nixpkgs containing the word blender
.
Just not our own nixpkgs instance.
It just doesn’t behave to how most people expect as they overlook one thing about flakes: the registry.
Per the manual:
Flake registries are a convenience feature that allows you to refer to flakes using symbolic identifiers such as
nixpkgs
, rather than full URLs such asgit://github.com/NixOS/nixpkgs
. You can use these identifiers on the command line (e.g. when you donix run nixpkgs#hello
) or in flake input specifications inflake.nix
files. The latter are automatically resolved to full URLs and recorded in the flake’sflake.lock
file.
In short, nixpkgs
from nix search nixpkgs blender
points to a flake from the registry.
We can view what nixpkgs
points to by running nix registry list
.
nix registry list | grep flake:nixpkgs
global flake:nixpkgs github:NixOS/nixpkgs/nixpkgs-unstable
As stated from the manual, the format from the nix registry list
output comes in the form of <type> <from> <to>
where…
-
type
denotes which registry it came from (which we’ll discuss shortly later in this post). -
from
typically has the flake identifier. -
to
refers to the flake reference it points to (e.g.,github:NixOS/nixpkgs
,github:nixos-community/home-manager
).
We can see that nixpkgs points to the nixpkgs-unstable branch.
That is, nix search nixpkgs blender
is pretty much the same as the following command.
nix search github:NixOS/nixpkgs/nixpkgs-unstable blender
This seems fine except the flake reference isn’t pointing to a fixed point like a specific commit, it points to a branch of a remote Git repo which can change over time.
So every time you run the command, nixpkgs
resolves to the most recent version of nixpkgs-unstable.
This is why it downloads and evaluates a new nixpkgs instance every time it runs (or between every few hours, which is the pace for nixpkgs-unstable updates).
Not exactly a good experience for a system package search compared to other operating systems like Arch Linux, Fedora, or OpenSUSE.
Seem like flake registry is not a convenience feature at all. More like a hindrance, really.
It is a CONVENIENCE FEATURE! I use it to quickly test and run packages from my nixpkgs instance as well as other projects with flake.
It’s just that most NixOS systems are not properly configured with it by default. You’ll see what I mean.
Pinning nixpkgs to the registry
Most would expect to search packages in their nixpkgs instance of their NixOS system. As hinted from the previous section, we can make use of multiple registries. As stated right after the previous excerpt from the manual:
There are multiple registries. These are, in order from lowest to highest precedence:
The global registry, which is a file downloaded from the URL specified by the setting
flake-registry
. It is cached locally and updated automatically when it’s older thantarball-ttl
seconds. The default global registry is kept in a GitHub repository.The system registry, which is shared by all users. The default location is
/etc/nix/registry.json
. On NixOS, the system registry can be specified using the NixOS optionnix.registry
.The user registry
~/.config/nix/registry.json
. This registry can be modified by commands such asnix registry pin
.Overrides specified on the command line using the option
--override-flake
.
A quick way to show this would be pinning nixpkgs
from the global registry to the user registry with nix registry pin nixpkgs
.
nix registry list | grep 'flake:nixpkgs'
after pinninguser flake:nixpkgs github:NixOS/nixpkgs/ec750fd01963ab6b20ee1f0cb488754e8036d89d
global flake:nixpkgs github:NixOS/nixpkgs/nixpkgs-unstable
Since we’re going to show configuring the system registry with NixOS, don’t forget to remove the entry from the user registry with |
Both the global and system registry can be configured in the NixOS system. In this case, we’ll be focusing on adding our flake inputs of our NixOS system into the system registry.
nix.registry
expects an attribute set of flake inputs.
And the outputs
attribute from our flake can be a function that expects an attribute set of flake inputs returning an attribute set.
We can basically set the system registry like so.
nix.registry
{
description = "Very simple sample of a Nix flake with NixOS and home-manager";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
nixpkgs-stable.url = "github:NixOS/nixpkgs/nixos-23.05";
nixpkgs-unstable.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
};
outputs = inputs@{ nixpkgs, ... }: {
nixosConfigurations.desktop = nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
modules = [
({ config, lib, ... }: {
nix.registry = lib.mapAttrs (_: flake: { inherit flake; }) inputs;
})
./hosts/desktop
];
};
};
}
Once we rebuild our NixOS system with nixos-rebuild
, we should now see our flake inputs included in the system registry.
nix registry list | grep '^system'
system flake:nixpkgs path:/nix/store/kcmipm57ph9bpzz8bs80iiijiwbyzwy3-source?lastModified=1699099776&narHash=sha256-X09iKJ27mGsGambGfkKzqvw5esP1L/Rf8H3u3fCqIiU%3D&rev=85f1ba3e51676fa8cc604a3d863d729026a6b8eb
system flake:nixpkgs-stable path:/nix/store/3s69yxbbl116zwga3i6cy7prplywq0bn-source?lastModified=1699291058&narHash=sha256-5ggduoaAMPHUy4riL%2BOrlAZE14Kh7JWX4oLEs22ZqfU%3D&rev=41de143fda10e33be0f47eab2bfe08a50f234267
system flake:nixpkgs-unstable path:/nix/store/mj0hy52z22q5gpsf33akndxiclxd8ray-source?lastModified=1699343069&narHash=sha256-s7BBhyLA6MI6FuJgs4F/SgpntHBzz40/qV0xLPW6A1Q%3D&rev=ec750fd01963ab6b20ee1f0cb488754e8036d89d
system flake:self path:/nix/store/v68idbapq3m8sz0fds66vzgg7agg10g9-source?lastModified=1699449415&narHash=sha256-Whc5OQzHTJtyBbnFsDAzdjICWK2BnCDCBP6s%2Bk/oLGQ%3D&rev=e1a0efea49b2e22055d2454e76f3d01ae42fde07&revCount=3
If we run nix search nixpkgs blender
once again, it should still evaluate but it should be done only once since:
-
We have
nixpkgs
pinned to our nixpkgs instance. -
We’re fully taking advantage of flake evaluation caching meaning subsequent package searches should be (almost) instantaneous.
By configuring our registry through NixOS, we’re only evaluating nixpkgs once until we update our flake inputs with nix flake update
.
So instead of suffering every time you run the command like in the video at the beginning, you suffer at least once every update.
As another bonus, we can now take advantage of other nix
subcommands such as nix build
, nix shell
, or nix develop
.
# These commands should now use our nixpkgs instance. Hoorah!
nix run nixpkgs#neofetch
nix develop nixpkgs#gcc
nix build nixpkgs#vale
nix search nixpkgs asciidoctor
# From the previous example, we can quickly run packages from other nixpkgs
# branch easily.
nix shell nixpkgs-stable#hugo
nix shell nixpkgs-unstable#hugo