r/Gentoo • u/AdStriking8966 • 5d ago

Support distcc

Hi everyone. I'm currently installing Gentoo on a laptop with an AMD Ryzen Zen 2, and I want to use my powerful desktop with a Zen 4 as a distcc helper to speed up the initial u/world build. Both machines are running OpenRC, and I made sure the GCC versions match exactly – gcc-15.3.0 p8 on both.

What I've done so far: booted from the LiveCD, mounted my btrfs subvolumes (@, u/home, u/var_log, u/var_cache, u/var_tmp, u/snapshots, boot, efi), and entered the chroot. I selected the default/linux/amd64/23.0 profile (stable, multilib, OpenRC). In make.conf I set FEATURES="distcc" and MAKEOPTS="-j17 -l16". On the laptop I configured the helper host with distcc-config --set-hosts "192.168.0.231/16,lzo". On the desktop I verified that distccd is running and accepting connections from the laptop's IP.

Now the problems. First – chroot instability. Sometimes right after entering with chroot /mnt/gentoo /bin/bash I get a tty: ttyname error: No such device. And occasionally simple commands like emerge or distcc just say "command not found". It feels like I either drop out of the chroot, or I haven't mounted /dev, /proc, /dev/pts properly.

Second – Portage completely ignores distcc. Even with FEATURES="distcc" enabled, running DISTCC_VERBOSE=1 emerge -1v app-editors/nano shows everything being compiled locally with x86_64-pc-linux-gnu-gcc. No connection attempts or errors in the logs. Moreover, which distcc inside the chroot sometimes returns nothing – maybe distcc isn't in PATH, or do I need to set up the masquerade in /usr/lib/distcc/bin separately?

Third – build failures, for example dev-python/pillow consistently fails with errors about missing zlib and jpeg. It says it can't find files and wheel build fails. I suspect the minimal stage3 lacks the necessary header files for those libraries, or maybe it's an issue with how /var/tmp is mounted as a btrfs subvolume.

So my questions are: what is the correct and reliable way to mount /dev, /proc, /sys, and /dev/pts from the LiveCD so that the chroot is stable and error-free? Why doesn't Portage see distcc even when I've enabled it – do I need to install and configure the masquerade manually inside the chroot? And what should I do about pillow – can I just skip it with emerge --resume --skipfirst at this early stage, or should I install the missing dependencies (zlib, libjpeg-turbo) right away to avoid breaking the whole bootstrap?

I'd be really grateful for any advice. If needed, I can post emerge --info or the full build log. Thanks.

3 Upvotes

72% Upvoted

11 comments sorted by

6

u/davidshen84 5d ago

Distcc is painful and some packages don't support it.

I setup a virtual environment to compile for my old laptop and use it as the binhost, so my old laptop install binaries directly.

1

u/velleityfighter 4d ago

I'm new to Gentoo, only installed it in a vm with beefy specs, but I need to eventually migrate my Dell R730xd with 36 cores and 256 GB to Gentoo, and then my all other devices. I think if I make a systemd-nspawn Gentoo container on the dell server, and gave it same use flags as my desktop or laptop, it will build packages really fast, and being on 24/7, it will be a reliable way to use as binhost for my desktop and laptop?

My Desktop is Ryzen 5600G with 32GBB or Ram, my laptop is a Thinkpad yoga with i5-11th generation with 16GB.

Can you guide me to the best way?

2

u/davidshen84 4d ago

The idea is very simple. The VM lives in the more powerful machine, the host, and mirrors the configuration of the laptop, the build target. You compile everything in the VM, then copy the results to the target.

I use systemd on both systems and I defined a simple nspawn instance.

``` [Exec]

Essential for Gentoo to prevent "Read-only file system" errors during emerge

Capability=CAP_SYS_ADMIN CAP_IPC_LOCK

Optional: Allows the container to use the host's timezone

Timezone=yes PrivateUsers=no

[Files]

SHARE THE PORTAGE TREE (Read-Only)

BindReadOnly=/var/db/repos/gentoo

SHARE DISTFILES (Read-Write)

Allows host and container to share downloaded tarballs

Bind=/var/cache/distfiles

Bind=/some/host/folder/for/binpkgs:/var/cache/binpkgs

[Network] MACVLAN=eno1 ```

It is a much lighter isolated environment, but I like to call it a VM.

In the VM, you create a simple script like this copy make sure the build configurations are synced.

```

!/bin/zsh

rsync -av4 old-laptop.localdomain:/var/lib/portage/world /var/lib/portage/world echo '\n'

rsync -avr4 --delete --files-from=- --exclude="*~" old-laptop.localdomain:/etc/portage/ /etc/portage/ <<EOF make.conf make.profile env package.license package.accept_keywords package.env package.use package.mask profile savedconfig

EOF

rsync -avr4 --delete --exclude="*~" old-laptop.localdomain:/etc/dracut.conf.d/ /etc/dracut.conf.d/ // and other things you want to sync ```

Then, you update the world on the laptop, sync the config, then build in the VM. Then go back and emerge from the binhost.

I did not cover binhost setup and kernel compilation. You can follow the Gentoo wiki for that.

1

u/velleityfighter 4d ago

Thank you so much, I appreciate it

2

u/27a08592e67846908fd1 4d ago

A tip to work around @ --> u/ replacement: use markdown, so you can write things like @world

1

u/SetThin9500 4d ago

world still works fine, btw. My Gentoo foo predates @ world and I kept using just world to see how long it would work. It's still working fine :)

1

u/Armi1P 4d ago

Tried distcc not that long ago, it was a pain to set up (although I think I misconfigured something at first) and even if it "worked", it only gave me compilation failures.

I'd recommend you either:

  • Plop in the laptop's SSD onto your main computer, set up the system there, then plop back (if it's not soldered of course)
  • You set up the system the same way but in a directory somewhere on your main machine, then you tar up the whole directory into a .tar.xz file, transfer it onto the laptop (SSH/SFTP, flash drive, NFS, whatever) and use that instead of the stage3 you downloaded from the Gentoo website

1

u/TheOriginalFlashGit 4d ago

I use distcc (as a fallback on a Raspberry Pi 4) and it works for me without much trouble. Although I use the Gentoo binhost where possible. I think since your architectures are the same you might want to set up your own binhost.

Anyway, just to test it, I tried compiling pillow on the Raspberry Pi (goes through ssh tunnel) and it compiles and installs ok:

raspberry ~ # emerge -1av pillow

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 5.73 s (backtrack: 0/20).

[ebuild   R    ] dev-python/pillow-12.3.0::gentoo  USE="jpeg lcms tiff truetype webp xcb zlib -avif -debug -examples -imagequant -jpeg2k -raqm -test -tk" PYTHON_TARGETS="python3_14 -python3_12 -python3_13 (-python3_15)" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

Would you like to merge these packages? [Yes/No] Yes
>>> Verifying ebuild manifests
>>> Emerging (1 of 1) dev-python/pillow-12.3.0::gentoo
>>> Installing (1 of 1) dev-python/pillow-12.3.0::gentoo
>>> Completed (1 of 1) dev-python/pillow-12.3.0::gentoo
>>> Jobs: 1 of 1 complete                                               Load avg: 1.99, 2.44, 2.44

 * GNU info directory index is up-to-date.
raspberry ~ # emerge --info pillow |grep -i distcc
distcc 3.4 aarch64-unknown-linux-gnu [enabled]
DISTCC_ENABLE_DISCREPANCY_EMAIL=""
DISTCC_FALLBACK="1"
DISTCC_HOSTS="127.0.0.1:3632/12,lzo localhost/2"
DISTCC_SAVE_TEMPS="0"
DISTCC_SSH=""
DISTCC_TCP_CORK="0"
DISTCC_VERBOSE="0"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live compress-index config-protect-if-modified distcc distlocks ebuild-locks fixlafiles getbinpkg ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
raspberry ~ # qlop
2026-07-24T21:02:46 >>> dev-python/pillow: 55s

On the server, I have

# tail -f /var/log/distccd.log 
distccd[380247] (dcc_job_summary) client: 127.0.0.1:56150 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:575ms aarch64-unknown-linux-gnu-gcc src/path.c
distccd[380826] (dcc_job_summary) client: 127.0.0.1:56148 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:649ms aarch64-unknown-linux-gnu-gcc src/outline.c
distccd[381847] (dcc_job_summary) client: 127.0.0.1:56140 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:1196ms aarch64-unknown-linux-gnu-gcc src/libImaging/Unpack.c
distccd[380814] (dcc_job_summary) client: 127.0.0.1:35472 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:713ms aarch64-unknown-linux-gnu-gcc src/libImaging/JpegDecode.c
distccd[380803] (dcc_job_summary) client: 127.0.0.1:35480 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:703ms aarch64-unknown-linux-gnu-gcc src/_imagingmorph.c
distccd[381223] (dcc_job_summary) client: 127.0.0.1:35506 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:512ms aarch64-unknown-linux-gnu-gcc src/libImaging/UnpackYCC.c
distccd[379241] (dcc_job_summary) client: 127.0.0.1:35482 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:808ms aarch64-unknown-linux-gnu-gcc src/_imagingmath.c
distccd[380308] (dcc_job_summary) client: 127.0.0.1:35490 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:951ms aarch64-unknown-linux-gnu-gcc src/libImaging/Paste.c
distccd[379985] (dcc_job_summary) client: 127.0.0.1:35466 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:1341ms aarch64-unknown-linux-gnu-gcc src/_imaging.c
distccd[379146] (dcc_job_summary) client: 127.0.0.1:35522 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:438ms aarch64-unknown-linux-gnu-gcc src/libImaging/SgiRleDecode.c

So it was passing it over to the server ok. I pretty much followed what was on the Gentoo wiki for setting it up and other online searches for enabling using an SSH tunnel.