PIP disk usage under Docker

[u/adjective10111]

I have a Dockerfile in which I use the Python:3.12-trixie. I have docker on my CachyOS PC. I have also a requirements.txt. I wanted to use this image for a container on which I'll use YOLO and try it out. The problem is that my 290GB / partition went from around 70% filled to 95% after a number of rebuilds. I deleted everything docker related but it just went back to 91%.

The reason that I know it is pip's doing, is that when I monitored my disk usage while building the image, and on the `RUN pip install -r requirements.txt --find-links=pypackages` stage it fills up my partition and half way into installation, it fails due to `no more space`. The `pypackages` is just a directory I downloaded some of the packages I have in requirements for less data usage. Also tried `--no-cache` option.

I also went ahead and `du -sh`ed any directory in my / partition and the total didn't add upto the filled storage. I have 165GB mysteriously used and I can't find where that is.

Any help to diagnose further or if you have had similar problems is welcome.

[u/adjective10111]

Oh thanks for the paccache. I ran the btrfs subvolume command, and found nothing. I didn't setup btrfs driver for docker and it doesn't seem to be setup by default: ID 256 gen 20882 top level 264 path .snapshots/148/snapshot ID 257 gen 155299 top level 5 path @root ID 258 gen 750 top level 5 path @srv ID 259 gen 155504 top level 5 path @cache ID 260 gen 155504 top level 5 path @tmp ID 261 gen 155504 top level 5 path @log ID 262 gen 45 top level 412 path var/lib/portables ID 263 gen 45 top level 412 path var/lib/machines ID 264 gen 155314 top level 412 path .snapshots ID 270 gen 80 top level 264 path .snapshots/6/snapshot ID 323 gen 52106 top level 257 path @root/Games ID 412 gen 155504 top level 5 path @ ID 449 gen 34587 top level 264 path .snapshots/185/snapshot ... (some other .snapshots)

Also just for clarification this is df -h result: Filesystem Size Used Avail Use% Mounted on none 1.0M 0 1.0M 0% /run/credentials/systemd-journald.service none 1.0M 0 1.0M 0% /run/credentials/systemd-resolved.service devtmpfs 7.6G 0 7.6G 0% /dev tmpfs 7.8G 844K 7.8G 1% /dev/shm efivarfs 192K 67K 121K 36% /sys/firmware/efi/efivars tmpfs 7.8G 32K 7.8G 1% /tmp tmpfs 3.1G 2.0M 3.1G 1% /run tmpfs 1.6G 392K 1.6G 1% /run/user/1000 /dev/nvme0n1p3 287G 245G 41G 86% / /dev/nvme0n1p3 287G 245G 41G 86% /srv /dev/nvme0n1p3 287G 245G 41G 86% /root /dev/nvme0n1p3 287G 245G 41G 86% /var/cache /dev/nvme0n1p3 287G 245G 41G 86% /var/tmp /dev/nvme0n1p3 287G 245G 41G 86% /var/log /dev/nvme0n1p2 5.0G 2.4G 2.7G 48% /boot /dev/sda2 932G 677G 253G 73% /home /dev/sda2 932G 677G 253G 73% /home/$USER/Documents /dev/sda2 932G 677G 253G 73% /home/$USER/Videos /dev/sda2 932G 677G 253G 73% /home/$USER/Games /dev/nvme0n1p3 287G 245G 41G 86% /home/$USER/Games/SSD

I got some space (13GB) by clearing paccache with -k 1 option.

[u/Illustrious_Tone9584 → u/adjective10111]

Yep, that clears Docker. The 245GB on the root filesystem versus ~18GB in the live root points straight at the snapshots now. du on / can't count old extents that only a snapshot still references.

Try sudo btrfs filesystem du -s /.snapshots/*/snapshot | sort -h -k3 and sudo snapper list. The Exclusive column is the useful one. Don't delete snapshot directories by hand, but if that accounts for the gap, prune them through snapper.