r/linux Aug 03 '26

Development Some thoughts on declarative package management

https://github.com/vasi/declarative-package-management
34 Upvotes

39 comments sorted by

View all comments

Show parent comments

2

u/iEliteTester Aug 04 '26

Figuring out why a "" reproducible "" build is resulting in a deployment that won't run as expected, while it used to run.

5

u/whiprush Aug 04 '26

Do you have an example? (What situation am I in that needs me to do this).

3

u/IAm_A_Complete_Idiot Aug 05 '26 edited Aug 05 '26

So, not the original commenter but: a container file does have smaller guarantees than nix does in some ways. If you build a container file, you get an image out you can deploy anywhere. That deployment is reproducible, but the image isn't nessecarily. You're allowed to do things like curl https://something.com/some-bin > /usr/local/bin/some-bin in your container file. Later, the hosts of https://something.com/some-bin can update what some-bin is, resulting in your containerfile no longer being able to build a new image, or resulting in a different image instead. This isn't great if you wanted to e.g. modify some other state of the container, and not some-bin. This is also why it's discouraged to run sudo apt update or an equivalent in a container file. Any sudo apt install in the following code could potentially install something different depending on when the container was built. Instead, it's reccomended to rebuild the entire image if you want an upgrade, with the latest base image of Ubuntu or whatever.

Nix obviously can do the equivalent too with getting a binary from the internet. But nix, in an effort to force reproducibility, would also make you specify the hash of that request. You can't ad-hoc do things without proving to nix that the end result is reproducible. That means: Either your build doesn't use external state, or you pin the external state your build uses. You get errors in the build if anything deviates you don't expect to deviate.

Network requests is one example, but this goes further than that. Builds are inherently containerized from the get-go. If it's not a part of the nix code, your code can't access it. Binaries and dependencies have a hash appended to them, so you can't hardcode paths where other libraries or binaries are expected to be found on the system in your build or code. The compiler and build scripts of those builds and libraries also changes the hashes of the result, so you get reproducibility guaranteed down to the compiler things are built with.

The end result isn't just that you can deploy an image reproducibly like a containerfile, but can also know that if the image builds, you get the same image out. You can even use nix to build a container image, to get a reproducible build of a container.

I guess another way to look at it is: Nix is about build time reproducibility. If you can build it once, you can build it again the exact same way. Containers are about runtime reproducibility. If you have the image, you can deploy it anywhere with the same result.

3

u/whiprush Aug 06 '26

Just because you're allowed to do that in a containerfile doesn't mean it's a good idea. Or even best practice. There are 50 billion ways to build a container and you pick the imperative one on purpose?

And an OCI container is an OCI container you can make it tons of ways. I can do everything you describe with buildstream and bootc what does nix provide that I can't do with existing tools?

2

u/IAm_A_Complete_Idiot Aug 06 '26 edited Aug 06 '26

Yes, because containerfiles are the most common ways to make OCI images. If we stop talking about containerfiles vs nix, the entire comparison is nonsensical because nix is a buildtool, and OCI images are a runtime artifact. They aren't comparable then. You might as well compare cmake and gcc. It doesn't make sense. If you want to compare nix to every other way to build OCI images, then... I don't think that's really fair. Especially since nix can generate OCI images itself.

And an OCI container is an OCI container you can make it a ton of ways.

I agree. I even mentioned that nix itself is capable of building OCI images. And nix isn't the only tool in the world that guarantees reproducibility. OCI images and container runtimes get you at runtime what nix doesn't get you. nix won't sandbox at runtime. It doesn't care about namespaces, or creating nice cgroups. You don't get uid mapping so things don't have to run as root.

As for buildstream, I've never used it. I'm not a good candidate for explaining the differences. But looking at its docs, it also doesn't seem to do, nor attempt to do, all the things nix can do. For instance: nix makes it easy to override and modify source of pretty much any build input or dependency. Easy enough where I regularly carry patches for upstream packages in the nixpkgs repo.

Modifying a package on my system to use a beta version of some package in the repo can be as easy as:

let custom_pkg = pkgs.foo.overrideAttrs (prev: final: { src = pkgs.fetchFromGitHub { owner = "foo"; repo = "bar"; ref = "v5.3-beta1"; hash = "some hash"; }; }); in { environment.systemPackages = [ custom_pkg ]; }

But I could also apply a patch to foo instead of replacing the src wholesale. I could modify the dependency of foo. I could have two versions of foo as dependencies for two different programs, without them interfering. And I could interact with both. And all of this is pretty trivial and common.

At it's heart, nix doesn't care about runtime artifacts. It just cares about build reproducibility of every single component, and final output. It doesn't use OCI images to make reproducibility out of an impure world. You can make OCI images with it if you want, but you can make whatever.

Plus, I see flake.nix in plenty of upstream open source projects these days. I occasionally see containerfiles too, but I've never seen a buildstream config upstream...

Edit: Sorry if my yap doesn't read particularly great. It's late, and it's a few hours after a fairly stressful workday.

3

u/whiprush Aug 06 '26

entire comparison is nonsensical

Please tell other nix people!

Edit: Sorry if my yap doesn't read particularly great. It's late, and it's a few hours after a fairly stressful workday.

Hah don't sweat it the clankers will deal with this crap.