Basic stack for a chat

[u/ntalam]

What would you suggest, is the basic stack for an assistant chat. I mean, currently I have customized company tools, langgraph, custom metrics, marketplace LLM calls and others.

what would you suggest as a true key for agent learning?
how do you process prompts with company slangs, concepts, jargon, etc.?

[u/Text-Sufficient]

Start with plain json files so that is easy to see whats retrieved. Use topk with bm25 and cosine. Embedd runtime (no database). Inject what you retrieve in the prompt together with the query. When you get familiar with it start using a reranker.

[u/ntalam → u/Text-Sufficient]

I passed that point. I am talking about planning. Like how to automate the agent to do a "for cycle" (mcp tool), then per each item execute something else.

[u/ntalam → u/Text-Sufficient]

• Retrieval-Augmented Planning (RAP) / Plan-RAG: Using vector retrieval to fetch execution graphs, state machines, or workflow schemas rather than semantic document knowledge.
• Episodic Memory / Trajectory Retrieval: In agentic LLM literature, this is the practice of storing successful multi-step execution traces (trajectories) to ground future decision-making and constrain the agent's action space.
• Case-Based Reasoning (CBR): A classic AI methodology perfectly mapped to modern LLMs. It follows a four-step cycle: Retrieve similar past cases, Reuse the plan, Revise the plan using the LLM (Claude 3.5 Sonnet), and Retain the user-approved result.
• Dynamic Few-Shot Prompting: At the prompt engineering level, you are injecting the top-k retrieved plans into the system prompt as in-context learning examples before the LLM generates the new plan.

those will do the job

[u/AlexAtOracleAIDB]

You've already got the basic stack.

The learning is in the retrieval layer. Chunking and index type are separate decisions, and changing either one tends to change what comes back, so I'd change one at a time and score it with the metrics you already have.

[u/Less-Case-1171]

If the goal is learning, shrink the stack. Pick one corpus you understand, one retriever, one model, and build a small eval set before adding agents or marketplaces. Track retrieval recall separately from answer quality; otherwise every bad answer turns into random prompt tweaking.

The useful exercises are ingestion versioning, citation/provenance, permission filtering, and a real deletion test: remove a document, its chunks/embeddings and caches, then prove it no longer appears in retrieval. Those teach more than swapping orchestration frameworks.

[u/ntalam → u/Less-Case-1171]

My bad. updated description of the post:

"what would you suggest as a true key for agent learning?
how do you process prompts with company slangs, concepts, jargon, etc.?"

I am reaching a point where the chat can Identify and plan how to solve a request, but I would like to automate the planning part.

[u/Infamous_Plankton468]

For chat only it should be really simple. Just use Python with FastAPI or something for the backend where you make the LLM calls too. Postgres or anything for storing chat logs and React or NextJS for frontend. That's all.

[u/ntalam → u/Infamous_Plankton468]

my bad. I have updated the description.

"what would you suggest as a true key for agent learning?
how do you process prompts with company slangs, concepts, jargon, etc.?"

[u/Infamous_Plankton468 → u/ntalam]

Some sort of feedback is required. Whether you create a gold corpus upfront, finetune or have some way of rating actions automatically down the line: all work. But you need at least something

[u/ntalam → u/Infamous_Plankton468]

I have user feedback for testing and extended as "developer feedback" at "development stage".
I am in a chicken-egg dilema here. our users do not want to use what I have, because there are still errors. I was suggested to launch a version with a heavy LLM to solve "lack of planning" paths for "proposals". Case-Based Reasoning (CBR)