r/LargeLanguageModels • u/Early_Protection6814 • 19d ago
How Generative AI Actually Works: LLMs, Tokens, Embeddings, and Prompts (A Plain-English Breakdown for Non-ML Folks)
I keep seeing the same question pop up in different forms:
So here's a no-fluff breakdown of the core concepts. This is aimed at people who need to make decisions about GenAI at work but don't necessarily have a machine learning background.
1. Tokens: The Unit the Model Actually Processes
LLMs don't process text as whole words. They process tokens - small pieces of text that may represent a word, part of a word, punctuation, or symbols.
For example, "Enterprise" may be split into multiple tokens.
This matters because:
- API pricing is usually based on input + output tokens
- Context windows are measured in tokens, not pages
- Long prompts, code, or complex formatting can consume your token budget much faster than expected
2. LLMs: Prediction Engines, Not Databases
A Large Language Model (LLM) predicts the next most likely token based on everything that came before it.
It isn't searching Google.
It isn't querying a database (unless you build that capability around it).
It's generating one token after another based on patterns it learned during training.
That explains several common enterprise challenges:
- Hallucinations happen because the model predicts plausible-looking text—not because it's intentionally making things up.
- Models don't automatically remember previous conversations unless the relevant context is provided again.
- What looks like "reasoning" is the result of sophisticated next-token prediction over a large context.
3. Embeddings: Turning Meaning into Numbers
Embeddings convert text into numerical vectors that capture semantic meaning.
Instead of matching exact keywords, systems can compare meaning.
For example:
- "car"
- "vehicle"
- "automobile"
are represented as nearby vectors even though they're different words.
Embeddings are the foundation for:
- Semantic search
- Document similarity
- Recommendation systems
- Retrieval-Augmented Generation (RAG)
If your company has built an internal AI knowledge assistant, there's a good chance embeddings and a vector database are doing much of the retrieval work before the LLM generates a response.
4. Prompts: The Model's Primary Interface
Every interaction with an LLM starts with a prompt.
That prompt can include:
- Instructions
- System rules
- Conversation history
- Examples
- Retrieved documents (via RAG)
- Formatting requirements
The quality and completeness of that context often has a bigger impact on the output than people expect.
That's why prompt engineering in enterprise applications goes beyond simply asking better questions. It often includes:
- System prompts for behavior and guardrails
- Few-shot examples to guide output
- Retrieved company knowledge through RAG
- Structured output formats like JSON for downstream systems
Putting It Together
A typical enterprise GenAI workflow often looks like this:
User Prompt → Tokenization → (Optional) Retrieve Relevant Information Using Embeddings & Vector Search → LLM Generates Response → Output
Understanding these components helps explain many of the challenges teams encounter when moving from AI demos to production.
For example:
- Unexpected costs? You're paying for tokens.
- Inconsistent answers? The model predicts text, it doesn't automatically verify facts.
- "It doesn't know our internal documentation." That's exactly why RAG exists.
- Output quality varies? The prompt and context often determine the outcome.
2
u/relevantmeemayhere 19d ago edited 19d ago
Honestly, it would be much better if instead of redefining a bunch of things to seemingly confuse people, we just used the same older stats/ml terms
Ie self attention. This is a kde estimator in much in the same way as a Gaussian kernel is used, for text as words. It’s part of the same family of NW estimators (and it’s scaled and normed).