4o1x5.dev/content/post/package-workspace-projects-into-container-with-nix.md

---
title: Package rust workspace project into containers with nix
description: A little technique I built up with actions and nix
date: 2024-09-03 00:00:00+0000
draft: false
toc: true
---

One of my projects that I have been working on for months is based on microservices. Meaning I have a bunch of programs that need to be containerized in order to run it in a kubernetes cluster.
I have tried building the images with the generic `dockerfile` but it resulted in unbelievably large images. One microservice was a 35MB binary when I compiled it, but in the other hand the docker image I made was almost 10x as much (340MB). Not only was it inefficient but I could have wasted time optimizing the images. But then I realized that I've been using nix for a few months, it can make containers right? Hell yeah it can! And It does it neatly packed. The 35MB binary packaged in a container comes out to be 37MB and I had to do 0 optimization. Also It neatly integrated my already existing flake and modules.
I'll showcase the process in [one of my existing projects](https://git.4o1x5.dev/4o1x5/producer-consumer/). It's a basic rust workspace with 2 services and a common package including some classes. But for the sake of eliminating confusion I'll also write down how to make a `cargo-hakari` workspace with [crane](https://crane.dev)

## Create the project

Crane has a template that generates all the code needed to start off.

```bash
nix flake init -t github:ipetkov/crane#quick-start-workspace
```

Upon running this command in a directory you are greeted with a `cargo-hakari` workspace setup.

```
├── Cargo.lock
├── Cargo.toml
├── deny.toml
├── flake.nix
├── my-cli
│   ├── Cargo.toml
│   └── src
│       └── main.rs
├── my-common
│   ├── Cargo.toml
│   └── src
│       └── lib.rs
├── my-server
│   ├── Cargo.toml
│   └── src
│       └── main.rs
└── my-workspace-hack
    ├── Cargo.toml
    ├── build.rs
    └── src
        └── lib.rs
```

## Adjusting the flake.nix to our needs

Since your usecase will most likely differ from mine it's important to keep in mind what to change during the process of development.
Whenever you add a new crate to the project sadly you will need to manually sync that up with your `flake.nix` file so during compilation `hakari` knows where to find the packages. If you forget about this you will get errors stating that cargo cannot find the crates...

```nix
fileSetForCrate = crate: lib.fileset.toSource {
  root = ./.;
  fileset = lib.fileset.unions [
    ./Cargo.toml
    ./Cargo.lock
    ./producer
    ./consumer
    ./common
    crate
  ];
};

```

### Don't forget essential packages!

If you application uses any imported crates you will 100% require `pkg-config`. If you need to connect to the internet, or make any connections with HTTP you will need `openssl`. I still haven't figured out why `libiconv` is needed but after searching for it a bit it most likely does some encoding. I'm not sure if your project will need it but it can't hurt much for leaving it in.

```nix
# Common arguments can be set here to avoid repeating them later
commonArgs = {
	inherit src;
	strictDeps = true;

	buildInputs = with pkgs; [
		openssl
		pkg-config
		libiconv
	] ++ lib.optionals pkgs.stdenv.isDarwin [
		pkgs.libiconv
		pkgs.openssl
		pkgs.pkg-config
	];
	nativeBuildInputs = with pkgs;[
		openssl
		pkg-config
		libiconv
	];
};

```

### Define packages

Another example from the project, at the bottom of the variable definitions (`let`-`in`) you will need to copy paste the code below and just replace the details, like for example:
If you have a `consumer-two` you will need to define that package as a variable then use it in the crane configs.

```nix
producer = craneLib.buildPackage (individualCrateArgs // {
	pname = "producer";
	cargoExtraArgs = "-p producer";
	src = fileSetForCrate ./producer;
});
consumer = craneLib.buildPackage (individualCrateArgs // {
	pname = "consumer";
	cargoExtraArgs = "-p consumer";
	src = fileSetForCrate ./consumer;
});
```

```nix
packages = {
	inherit consumer producer;
}
```

## Compile a service into a container

After you have defined you packages you can just go ahead and use `buildLayeredImage` from `dockerTools` to wrap the package in a container.
I also included some additional packages here for safety.

```nix
consumer-container = pkgs.dockerTools.buildLayeredImage {
	name = "consumer";
	tag = "latest";
	contents = with pkgs; [
		cacert
		openssl
		pkg-config
		libiconv
	];

	config = {
		WorkingDir = "/app";
		Volumes = { "/app" = { }; };
		Entrypoint = [ "${consumer}/bin/consumer" ];
	};
};

```

After this running the nix build command will make a syslink called `result`.

```bash
nix build .#consumer-container
```

## Automate with ~~github~~ forgejo actions

After building the image with nix, you get a `result` syslink file, it's a tar.gz linux container image file. We can use docker to import that tar.gz file and tag the image, then upload it to some repository.

```bash
nix build .#consumer-container
docker image load --input result
docker image tag consumer:latest git.4o1x5.dev/4o1x5/consumer:latest
docker image push git.4o1x5.dev/4o1x5/consumer:latest
```

We can use these commands in an workflow by using _&&_ to run them sequentially.

```yml
name: CD

on:
  push:
    branches: ["master"]

jobs:
  docker:
    runs-on: ubuntu-latest
    steps:
      -
        name: Checkout repo
        uses: https://github.com/actions/checkout@v4
        with:
          repository: '4o1x5/producer-consumer'
          ref: 'master'
          token: '${{ secrets.GIT_TOKEN }}'
      -
        name: Set up QEMU for docker
        uses: https://github.com/docker/setup-qemu-action@v3
      -
        name: Set up Docker Buildx
        uses: https://github.com/docker/setup-buildx-action@v3

      -
        name: Set up nix cachix
        uses: https://github.com/DeterminateSystems/magic-nix-cache-action@main
      -
        name: Login to git.4o1x5.dev container registry
        uses: docker/login-action@v3
        with:
          registry: git.4o1x5.dev
          username: ${{ secrets.GIT_USERNAME }}
          password: ${{ secrets.GIT_TOKEN }}

      -
        name: Setup nix for building
        uses: https://github.com/cachix/install-nix-action@v27
        with:
            #  add kvm support, else nix won't be able to build containers
            extra_nix_config: |
              system-features = nixos-test benchmark big-parallel kvm
      -
        name: Build, import, tag and push consumer container
        run: |
        	nix build .#consumer-container && \
        	docker image load --input result && \
            docker image tag consumer:latest git.4o1x5.dev/4o1x5/consumer:latest && \
            docker image push git.4o1x5.dev/4o1x5/consumer:latest
```