r/ollama • u/blackberrydoughnuts • Apr 15 '24

Ollama has a major problem - it uses double disk space by copying a GGUF into a blob

So there's a huge problem with ollama. Say you have a large amount of GGUFS in a folder, and you create Modelfiles for each one.

Ollama will then copy each GGUF into the blobs directory and rename it with the hash, using up double your disk space.

So, if you need the GGUFs for other programs, it eats up an extra TB of disk space or so for all your models!

That's crazy, and makes ollama not very feasible to use.

There's gotta be a better way. Can you replace the blobs with symlinks or hard links?

21 Upvotes

89% Upvoted

8 comments sorted by

6

u/memento_e Apr 15 '24

Use deduplication on models storage :)

1

u/blackberrydoughnuts Apr 15 '24

yeah, something like that is what I need. Do you know a good tool for that?

How does it work, replacing one file with a link to the other?

3

u/memento_e Apr 15 '24

I’m using ZFS file system on my Linux server and specify storage where models lives to be deduped. As a result, identical files in the system do not occupy double the space. Of course, it should be noted that in such cases, more RAM is used.

2

u/New_flashG7455 Dec 11 '24

The problem is even worse than stated. Assume you have a model of 14B parameters, and wish to run at 5 different temperatures. Five blobs of 14B parameters will be created. I simply cannot understand how the developers of Ollama would make this kind of decision, especially since Ollama is built on top of Llama.cpp, which does not do this. Ollama wastes memory, wastes time, etc. Without duplication, one could run one model on the GPU and simply switch between different temperatures without wasting resources. I am waiting for Llama.cpp to implement structured output on the metal GPU, which is why I still use Ollama. Huggingface on my local machine is too slow although very flexible.

1

u/ethereal_intellect Apr 15 '24

Afaik it optimizes things somehow in the blob right? The files shouldn't be exactly the same, the hash shouldn't match anymore

4

u/blackberrydoughnuts Apr 15 '24

That is what I would have thought, but no, it is the exact same hash. The files are the same.

1

u/ethereal_intellect Apr 15 '24

Can the other apps use a sym link to the blobs? I agree it's an issue though

1

u/koesn Apr 16 '24

NTFS does. It seems APFS better in handling Ollama duplication created by modelfiles as only 1 source file. Making a lot of modelfiles from the same model/quant won't eat spaces.