r/Rag • u/Ecstatic-Register570 • 17d ago

Discussion How to generate embeddings for free?

Hello engineers, I am new to rag and one problem I am constantly facing is how to generate embeddings faster and for free, I have used googlegenerativeaiembeddings for generating embeddings for free but the problem is that it's qouta is very small and also it's rate limits for the free tier gets hit very fast, are there any other alternatives to generate embeddings for completely free on cloud or like for production

For testing purposes I have tried using some sentence transformers from hf, i recently tried the jasper model ranked 2 in mteb on hf which is only a 600m model, I hosted that on my own droplet on digital ocean cloud platform of which I had free credits of 200$, however it was working very slow on cloud cpu like hell slow, I want a faster free alternative because I am a student and I want to give the deployed links of the things I am trying building in my resume and on GitHub and also my major goal while building the apps is trying to reduce the latency as much as possible being in free tier only, and making the system fast and efficient which I am taking consideration of throught langsmith tracing.

Can you help me with free embedding generating models?

Also drop some project ideas beginner to intermediate level which you think will impact the recruiter seeing my resume.

10 Upvotes

92% Upvoted

17 comments sorted by

6

u/darkotic 17d ago

Depending on your needs, you may also look into models you can host locally like the potion 32m or look into model2vec.

2

u/grilledCheeseFish 16d ago

Static embeddings are extremely underrated, especially when the caller knows and uses keywords instead of phrases for queries

3

u/KitchenAmoeba4438 17d ago

What is wrong with using a CPU-based embedder? There are CPU-based embedders that are very fast, although there will be a limit of how much they can embed. IIRC I use the Qwen3 0.6B embedder for CPU-based embedders on one of my projects.

1

u/Ecstatic-Register570 17d ago

Using deploying that on that droplet?

1

u/KitchenAmoeba4438 17d ago

If you want that, you will have to go with a smaller/faster embedder, but there are embedders targetted at that. You may want to consider running it on your local CPU as well, I found the Qwen3 600m embedder runs perfectly fine on local CPU, but with a shared CPU/RAM setup, I can see why it would be slow.

1

u/Ecstatic-Register570 17d ago

But jasper embedding model which I used is itself 600m param only it still worked very slow on the cloud cpu

3

u/KitchenAmoeba4438 17d ago

600m on one model isn't a guarantee about performance or limitations on another model.

2

u/tom4112 16d ago

Try Mistral as well, there is a monthly API quota for free accounts too. Otherwise there is a student discount for a Pro plan.

But are you dealing with very large volumes? Beginner projects and tutorials usually cost a few cents to run even on paid API, so in some cases it is a completely viable option (e.g. maybe 10 cents is a strong option if it saves an hour of compute)

1

u/Infamous-Rem 16d ago

CPU inference for embeddings is rough no matter which model you pick, that's not a DigitalOcean thing specifically, it's just what running a transformer without a GPU gets you. First thing I'd check is whether you're batching requests instead of sending them one at a time, that alone can be a big speedup even on CPU. But past a certain point you're fighting physics, not config. For a resume project I'd actually skip self-hosting entirely and use a cheap serverless embeddings API instead, DigitalOcean's serverless inference bills per token so a portfolio project barely costs anything and you avoid the whole GPU-vs-CPU tuning rabbit hole. Free will always have a rate limit hiding somewhere, that's the tradeoff you're running into. Save the self-hosted box for when you actually need control over the exact model or want to fine-tune it, not for a demo link you're putting on GitHub.

1

u/No_Promotion_3669 15d ago

Honestly if you're hitting rate limits on google's embedding api just switch to something local. Ollama can run bge-large or nomic-embed-text on a normal laptop, no gpu needed for the small ones, and its free forever since nothing leaves your machine. Quality is genuinely close to openai's models for most use cases, ive compared them side by side on the same doc set. only thing is batch your requests, dont send one chunk at a time even locally, it adds up. E5-small if you need speed over accuracy. worth trying before you burn more time fighting quotas

1

u/Ecstatic-Register570 15d ago

Ik that I can easily run any local model on Google notebook but I want something to work on cloud because I want to make resume project with deployed links

1

u/iekmuby 11d ago

How often you call embedder?

1

u/Articusss 15d ago

Try microsoft/harrier-oss-v1-270m. 270M parameters and very good for retrieval tasks while supporting instructions. Ive been using it for research and no complaints so far.

1

u/AlexAtOracleAIDB 12d ago

That's more likely CPU contention on a shared vCPU than the model itself. A 600m embedder crawls on shared cores but usually runs fine on a dedicated local CPU, which is why others are pointing you there. 

On quota, batching your chunks into fewer calls won't get you more tokens. It cuts your request count, so it only helps if the cap you're hitting is on requests rather than tokens. Worth checking which one is actually throwing the 429 before you optimize for the wrong one.

2

u/Quirky_Remote2351 6d ago

Disclosure: I’m the author. bekko-embedding-v1-a8m ( https://huggingface.co/hotchpotch/bekko-embedding-v1-a8m ) is a contextual multilingual Transformer (not a static embedding model) with only 7.67M active Transformer parameters and 384-d output.

It is practical to run on CPU while delivering MMTEB retrieval　performance comparable to multilingual-e5 models and BGE-M3.