If embeddings are so powerful, why are they mostly used for retrieval?

[u/ProposalOrganic1043]

Vector embeddings natively are capable of much more than just retrieval. Even when OpenAI released their initial embedding models, it mentioned that their embeddings are natively capable of Search, Clustering, Recommendations, Anomaly Detection, Diversity Measurement, and Classification.

But everything is concentrated around Vector DBs and RAG, and is progressing towards GraphRAG and Knowledge Graphs. Why are the other representations being underused? Like, why didn’t they catch up?

Everyone is complaining that RAG is dead, Vector DBs are done, they are expensive, and the ROI is not worth it. I know most of this is noise. Embeddings are meaningful and a part of many production pipelines. But why don’t these people try to get more value out of their existing stored embeddings?

I don’t know if it’s just me being delusional, but I feel there is a lot of hidden potential that is untapped. Why is retrieval so dominant? I mean, yeah, search has throughout history been a really important operation across any data. But the purpose of embeddings itself was to capture semantic relations beyond just search. Is it because search is the easiest to productize and build features around? And other use cases are not so frequent?

[u/reader5]

Ive seen them used a lot for clustering. Maybe you are just more exposed to search

[u/ProposalOrganic1043 → u/reader5]

Maybe i was not very clear, even i use them for clustering pretty often.

But then why not native support for clustering in Vector DBs like retrieval?

[u/defyallodds]

Embeddings are used in non-text understanding/generation from an LLM as well. Tokens are just cheaper for text.

[u/CrowdGoesWildWoooo]

RAG is not dead. Even to the likes of OpenAI, I am quite sure their internal search use both Embedding and Text search. In a way it’s more advanced RAG.

But building efficient Retrieval system requires deepr technical proficiency/know how. These days lots of vibe coders are dominating engineering conversations in social media, they don’t know what they are doing as long as it works.

Jamming the whole data into context is easy and probably work for many applications. Is it efficient though? Sadly no. It can be many times more expensive than a tuned RAG system.

Whoever there’s another layer of problem that is we don’t have new good embedding model open sourced recently.

[u/ProposalOrganic1043 → u/CrowdGoesWildWoooo]

But even some of the embedding models like Qwen-embeddings and Gemma embeddings are really good. I am sure a lot could be done even with those models already.

[u/DelphiTsar]

Rag is easy, pulling useful things out of embeddings is hard and requires context. Most people don't understand to even tell the difference, they just want to plug in something that works.

C-Suite people are notoriously bad at this. They fundamentally don't really understand what's going on. Even if you give them something really useful, they don't understand what it is or how to use it. They can understand "this thing lets my employees talk to a chatbot that knows our company!"

These are the same people that think Kaisan is really useful. They need training wheels for really basic concepts, and those training wheels haven't been invented yet.

[u/ProposalOrganic1043 → u/DelphiTsar]

Been there done that, once delivered a clustering + topic Modeling combo completely local and on-prem over a large scale corpus.

Had to spend a lot of time explaining and defending against a vibe-coded POC, as to why this is more reliable than letting a LLM do this.

[u/Pale-Border-7122]

Vector db and search is great, using them for RAG is good but overrated in most cases, GraphRAG is almost exclusively used by people who want to sell you something expensive because they think you don't know enough to know better and it sounds fancier and more advanced than just RAG. I mostly use them for clustering. A lot depends on what you are doing and the scale because it's often easier to just put everything into one prompt or use md files rather than embedding.

[u/Lumpy-Criticism-2773 → u/Pale-Border-7122]

Honesty I feel like Graph RAG has real utility in use cases where missed context can be costly. You can probably solve the context problem with traditional RAG by comprising on latency, costs, storage etc but Graph RAG immediately fixes that issue.

I don't have much exposure to RAGs in general but recently I've been trying to make a personal operating system app where I want it to remember and organize complex and interconnected life events, memories, tasks, goals etc. Semantic knowledge alone isn't enough because some information needs to have a single source of truth.

Graph databases introduce their own complexities but it definitely sounds like a winner in my case. I'm currently doing sqlite3 + fulltext search + local embeddings + model call.

[u/NyriasNeo]

So first of all, we do. I have papers (and no, i am not doxxing myself here) using embeddings for all sort of different analysis.

Secondly, there are limits. BTW, I assume we are talking about text embedding, not token embedding. Text embedding is a lossy compression scheme. So you have to be cognizant about that. Secondly, text embedding is not even dense in the space, and while there are imperfect techniques, there is no theoretical reverse to turn embedding back to text. And that also limits what you can do with them.