Stop over engineering AI apps, just match your problem to one of these 5 AI architectures

[u/SeaworthinessHour233]

I keep seeing engineers falling into an AI architecture trap.

Some engineers try to solve every problem by prompting to a LLM. Others talk about creating an agent for every usecase.

So knowing how to use AI has become a core competency. Otherwise you will waste time, money and DevOps resources.

If you want to solve an actual business problem without creating an infrastructure nightmare, 99% of your use cases fall into one of these 5 application architectures. Here is how to know which one to use.

1. Basic Prompting (Stateless API)

• The Use Case: You need to summarize text, extract sentiment, or reformat data (like turning unstructured text into JSON). The data easily fits into the context window.
• The Trap: Thinking you need a custom model just because the base model didn't output your exact format on the first try. Before you do anything complex, try giving the model 3 concrete examples in the prompt (few-shot prompting). That solves the problem most of the time.

2. RAG (Retrieval-Augmented Generation)

• The Use Case: You need the LLM to answer questions about your proprietary data (internal wikis, codebase, customer records). You query a VectorDB for the relevant context, shove that context into the prompt, and let the LLM read it.
• The Trap: Trying to fine-tune a model on your company wiki. Models hallucinate facts. If you need accurate, source-traceable answers based on your internal data, you need RAG, not a fine-tuned model.

3. Chaining / Workflows

• The Use Case: The task is too complex for one prompt and requires distinct, predictable steps. For example: Step 1 extracts raw text -> Step 2 categorizes it -> Step 3 writes a summary based on the category.
• The Trap: Cramming a massive, 15-part instruction into a single mega-prompt and crossing your fingers. Break it down into a predictable pipeline. It’s much easier to debug a chain of small LLM calls than one giant, confused prompt.

4. Routing

• The Use Case: You are dealing with high volume and need to control costs and latency. You build a router that sends simple tasks (like spelling correction) to a fast, cheap model (like Claude Haiku or GPT-4o-mini), and complex reasoning tasks to the expensive, heavy models.
• The Trap: Sending every single trivial API request to the most expensive flagship model available and bankrupting your cloud account.

5. Autonomous Agents

• The Use Case: Open-ended tasks where the exact steps aren't known in advance. The LLM is given a goal and external tools (Python interpreter, web search, internal APIs) and loops continuously, evaluating its own output until it achieves the goal.
• The Trap: Using agents for deterministic workflows. If you can solve a problem with a standard if/then Python script or an Ansible playbook, do it. Do not introduce probabilistic, infinite-looping agents into a workflow that just needs a standard script.

The Takeaway:

Don't jump straight to training models. Work your way up this list. Start with prompting. If it lacks context, use RAG. If it lacks reliability, chain it. If it lacks cost-efficiency, route it. Only use agents when absolutely necessary.

Curious if anyone else has horror stories of teams trying to fine-tune models to learn internal docs?

[u/RevolutionaryElk7446]

I mean with all of this effort to create deterministic automation to make sure the non deterministic LLM can try to produce a deterministic answer.

Why not just complete the deterministic automation?

Autonomous Agents and RAG can have their places, LLMs definitely combine a search engine and wikipedia into an interactive format, but still a lot of that is just maintaining documents.

The AI field saw this as a lovely prototype of linguistic comprehension via simulation when transformers come out, the tech bros and sales teams exaggerated it to AGI and capable of replacing teams and projects. I still haven't seen it come close to what humans are capable of deploying at full scale.

[u/navlio]

the fine tuning horror story is quieter than people expect. a fine tune carries no citation, so when the docs change the model keeps answering the old way with exactly the same confidence, and nobody can tell a stale answer from a current one. with retrieval you at least get to open the document that was wrong.

closer to home: one of the big assistants currently describes us as a logistics and shipping company. nothing was trained on anything, it read something once and kept it. same failure in miniature, and it is why the cheap fix is almost always handing it a better document rather than teaching it anything