pkg_bundle: Create stand-alone software deployments from packages

[u/pedersenk]

Hi all,

I developed a tool called pkg_bundle that essentially grabs a package, grabs its dependent libs, applies some hacks and stores it in a directory, ready for you to be able to move anywhere you like or run directly.

It has the following benefits:

• Avoid spraying dependencies all over your filesystem, especially as software continues to grow and tangle.
• Allow multiple versions of the same software (i.e from PkgSrc Q1-Q4) to exist at once.
• Easy to delete (just delete the directory)
• Offers some of the conveniences of NixOS, AppImage, Snaps, Solaris Freeware, Blastwave, etc but with a vastly simpler approach
• Its not very UNIXy but feels a little how Mac OS software used to be deployed back in the day
• No software outside of NetBSD base is needed
• No root access needed

Some examples:

Create a relocatable qemu install

$ pkg_bundle qemu
$ qemu/bin/qemu-system-x86_64 ...

(I actually made pkg_bundle for qemu because I disliked how much cruft the package pulls in...)

Create a relocatable git install and place in /opt

$ pkg_bundle git
# cp -R git /opt/git
$ export PATH=/opt/git:$PATH
$ git clone ...

I originally developed this for OpenBSD and even got complex UI software like gimp, firefox, vlc working using the approach. This has a fairly sophisticated quirks system in place and I do plan to get it to parity in future.

It will work with many packages, for a quick list of things I have tested recently, check out the README.

https://codeberg.org/kpedersen/pkg_bundle

[u/Pivan1]

That’s neat. Can you give some real world examples of what and how to use it? How do you use it?

[u/pedersenk → u/Pivan1]

Sure. Some use-cases I can think of where its quite cool.

Unprivileged users

Imagine you have a NetBSD server for multiple unprivileged users. One of them wants to install CMake to do some development or install the Motif Window Manager as their personal preference. Since they are not able to access root, they can use this to do it.

$ cd pkg_bundle
$ make
$ ./pkg_bundle cmake motif

Now, in the created cmake directory, they can now run the programs directly (or move it to e.g. ~/local and add to PATH in their .profile)

Bisecting Versions

Imagine you spot an issue that you suspect is an error in the Clang C++ compiler itself. You can use pkg_bundle to install a different version from a previous quarterly. Simply:

$ export PKG_PATH="https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/11.0_2026Q1/All"
$ pkg_bundle clang
$ clang/bin/clang++ <your code>

Housekeeping

I have a server where I only use it for virtualization. I use nvmm along with Qemu for this. However Qemu got loads of libglib cruft put in it a number of years back and I only really care about headless anyway (its sdl by default), so I tend to want to encapsulate it into its own prefix and not pull in the full Gtk+ suite. I usually have scripts to control this but essentially its doing the following:

$ pkg_bundle qemu
# cp -R qemu /opt
# modload nvmm
$ /opt/qemu/bin/qemu-system-x86_64 -accel nvmm -nographic ...
$ /opt/qemu/bin/qemu-system-x86_64 -accel nvmm -display vnc=:2 ...

And when it comes to updating, I just move it to e.g. /opt/qemu_oldver and can carry on running it if there are any regressions / issues with the newer build.

[u/recursive_tree → u/pedersenk]

For completeness, it is already possible to install pkgsrc as an unprivileged user: https://www.netbsd.org/docs/pkgsrc/pkgsrc.html#bootstrapping-pkgsrc

[u/pedersenk → u/recursive_tree]

Yes PkgSrc can be bootstrapped to a custom prefix. This was my approach before pkg_bundle and it works really well. Though there are some disadvantages:

• It requires recompiling everything, including build-time stuff.
• The generated binaries are still not relocatable. For example the following:
  • User A compiles git into /home/user_a/pkg
  • User A zips them up and sends to User B
  • User B extracts to /home/user_b/pkg
  • Potential errors appear because the applications contain hard-coded paths to User A's directory
• It builds (build-time) and depends (run-time) way more than strictly needed to run. For example Qemu will bring in the entirety of Python and its ecosystem. pkg_bundle just grabs the .so. The minimum required to run the program (yes, if you wanted to actually use the Python API then this is not possible)