Using nix based docker images in gitea actions

2023-06-22

Gitea recently added an integrated CI/CD setup that's (mostly) compatible with GitHub Actions based around act.

Using out of the box components this almost makes a great nix-based workflow running, but it requires a bit of work to get up and running.

Gitea actions and Docker

At it's core this relies on spawning a docker container that will run the shell code or pre-made actions. Act advertises a number of images with various amount of pre-installed software. Designed to be close enough to the default GitHub environment to allow re-using existing workflows with no or minimal changes.

As someone who has been leaning more and more into the NixOS ecosystem, all these pre-installed software isn't required. As most of my CI pipelines can run with just the nix package manager installed.

The problem

There exists a pre-made nixos/nix docker image that would suit my needs, if not for one small thing. The gitea actions runner spawns the container with /bin/sleep as the entrypoint, and this path doesn't exist in the pre-made nix image.

Luckily this is straight forward to fix, and we can take the opportunity to customize the image a bit to further optimize it for our specific needs.

Using nix to build our nix image

Being fully nix-pilled, we of course build our docker images with nix.

Below is the full flake.nix for building the image, some further explanation will follow after.

{
  inputs = {
    nix.url = "github:/nixos/nix?ref=2.16.1"; # using nix 2.16.1
    nix.inputs.nixpkgs.follows = "nixpkgs";
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.05"; # and nixos 23.05 for our packages
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = {
    self,
    flake-utils,
    nix,
    nixpkgs,
    ...
  }:
    flake-utils.lib.eachDefaultSystem (system: let
      pkgs = (import nixpkgs) {
        inherit system;
      };
      lib = pkgs.lib;
    in rec {
      packages = rec {
        # a modified version of the nixos/nix image
        # re-using the upstream nix docker image generation code
        base = import (nix + "/docker.nix") {
          inherit pkgs;
          name = "nix-ci-base";
          maxLayers = 10;
          extraPkgs = with pkgs; [
            nodejs_20 # nodejs is needed for running most 3rdparty actions
            # add any other pre-installed packages here
          ];
          # change this is you want 
          channelURL = "https://nixos.org/channels/nixpkgs-23.05";
          nixConf = {
            substituters = [
              "https://cache.nixos.org/"
              "https://nix-community.cachix.org"
              # insert any other binary caches here
            ];
            trusted-public-keys = [
              "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
              "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
              # insert the public keys for those binary caches here
            ];
            # allow using the new flake commands in our workflows
            experimental-features = ["nix-command" "flakes"]; 
          };
        };
        # make /bin/sleep available on the image
        runner = pkgs.dockerTools.buildImage {
          name = "nix-runner";
          tag = "latest";
        
          fromImage = base;
          fromImageName = null;
          fromImageTag = "latest";
        
          copyToRoot = pkgs.buildEnv {
            name = "image-root";
            paths = [pkgs.coreutils-full];
            pathsToLink = ["/bin"]; # add coreutuls (which includes sleep) to /bin
          };
        };
      };
    });
}

For people less familiar with nix, assuming you have your nix setup, you can build the image from the above file with the following steps

create a new folder
in the folder initialize a git repository with git init .
add flake.nix as shown above
add flake.nix to git using git add flake.nix (nix will ignore any file not in git)
run nix build .#runner

This will result in a .tar.gz image symlinked to result, which can be imported into docker using docker load -i result.

Once loaded into docker, you can push it to your container registry of choice where the gitea runner can use it from.

Customizing the image

Luckily for use, the upstream docker nix package has most of our needs covered with its configuration options. We can use that to add any package to the base image such as nodejs (which is needed to run most 3rdparty pre-made actions) or any other package we expect to commonly need in our workflows. Additionally, we can customize the nix.conf that ends up in the docker image to add extra binary caches and enable the flake sub-command.

Even with the base image customized we still need to layer another image on top of it to ensure /bin/sleep exists. This image is a thin layer that adds symlinks to the coreutils binaries to /bin and nothing else.

Using the image

Once you have the image in a place where your action runners can pull it, you can configure it for your gitea runner under the nix label.

services.gitea-actions-runner.instances.nix-runner = {
  enable = true;
  name = "nix-runner";
  # take the git root url from the gitea config
  # only possible if you've also configured your gitea though the same nix config
  # otherwise you need to set it manually
  url = config.service.gitea.settings.server.ROOT_URL; 
  # use your favaorite nix secret manager to get a path for this
  tokenFile = "/path/to/your/secret"; 
  labels = [
    "nix:docker://icewind1991/nix-runner"
  ];
}

And then configure your workflow to use the nix image