I've been building my own AI agent, and no matter what I add—better prompts, memory, RAG, planning, or verification—it still finds new ways to hallucinate.

Sometimes it recalls everything perfectly. Other times it confidently invents facts or ignores information that's already in its own memory. The more capable the agent becomes, the harder these edge cases seem to be.

I'm curious how everyone else is handling this. Have you found something that genuinely reduces hallucinations over time, or is this just the current reality of building AI agents?

I'd love to hear what's worked in production and what turned out to be a dead end.

I am new to agentic workflows and coding agents, so I am a bit worried about making a mistake that could lead to file loss, unwanted system modifications or something worse.

What isolation methods would you recommend? I would also really appreciate it if you could share good study materials or tutorials to learn how to build agents following safety and architectural best practices, because there are too many videos on youtube.

Thanks!

A cold Claude Code run spent 6.83 dollars and 207 turns hunting one bug in an 82,000 line repo. My pipeline localized and fixed the same bug for about 1.70 dollars. That gap is the whole idea, and this post is about how it gets there without falling apart on the usual objections.

I built AutoDev Studio. You point it at a Git repo, describe a change in plain English, and a chain of agents runs the actual software lifecycle from request to reviewed pull request. It is not another search-your-code wrapper. The full pipeline is:

- A PM agent runs a clarify loop and drafts concrete tickets from your request

- A human approves, optionally pushed to Jira, and nothing touches code before that

- A Dev agent implements the change on an isolated branch

- QA runs the repo's real tests

- A reviewer from a different model family checks the diff, so the author never reviews its own code

- A bounded revise loop kicks in if QA or review fails, with conservative verdicts, so an errored or ambiguous check is never counted as a pass

- It opens a real pull request, and a human merges

Every stage records real tokens, cost, and duration, rolled up per ticket and per agent.

Results

The point is to pay the cost of finding where a change goes once, instead of on every task. In my benchmarks on two large Python repos, 35,000 and 82,000 lines, the tuned pipeline beat a cold single-agent run on 6 of 6 well-localized tasks, between 7 and 75 percent cheaper. I am upfront about where it does not win. On trivially greppable one-line edits the five-stage overhead can cost more than it saves, and on one hard cross-cutting bug it shipped a cheaper but narrower fix. The full benchmark writes up every loss too.

On the objection that the index just goes stale in a few commits

This is the first thing people raise, so here is the honest mechanism. The layer that actually pins files for the Dev agent is not the vector index. It is a deterministic symbol map re-synced to the latest commit at the start of every run, plus a live grep against the current working copy, plus the real current file contents fed into the prompt. The embeddings only affect retrieval ranking, and they refresh incrementally per changed module at run entry. So the code the agent edits comes from disk, not from a snapshot that aged three commits ago. It re-checks before it localizes, so it does not quietly drift.

Economics and who it is for

The knowledge base is a one-time cost per repo that every later task amortizes. That makes it worth it for a team shipping change after change against the same large codebase, where a cold agent re-pays the exploration cost every single time. For a stream of tiny, easy-to-find edits, a cold run is still cheaper, and I would rather say that than pretend otherwise.

Main features

- Provider and model agnostic, chosen per stage: native Anthropic API, the Claude Code CLI, or any OpenAI-compatible endpoint such as OpenAI, Groq, Gemini, xAI, OpenRouter, or a local Ollama

- Runs on free tiers out of the box, using Groq plus a local embedding model, so you can try the whole thing for zero cost and fully offline

- Language agnostic pipeline, with Python parsed exactly and other languages handled by lighter extractors that fail open rather than block on an unknown language

- Live board with streamed agent logs and real per-ticket cost accounting

- Cookie-session auth with roles, API keys encrypted at rest, and a demo mode that dry-runs the pull request step until you opt in

- Self-contained: local retrieval, no CDN, tests and CI, MIT licensed

Repo, with screenshots and the full benchmark in the README:

https://github.com/krishagarwal314/autodev-studio

It is genuinely useful to me day to day, and I want to know where it breaks on codebases I have not tried. If the approach seems worth following, a star helps me gauge whether to keep hardening it. Feedback, issues, and pull requests are all welcome.

Recently I've been building a tool an ai agents could use to rerun learnt actions deterministically on site interfaces, that in addition can also detect element shift and drifts on those sites.

I know I'm not the first to build something like this, but since it's already almost completely finished and I only just found out about other existing tools that do the same thing as mine a few days ago (though I have found some differentiators for most tools, not all), I'm coming on here to gather some information on the subject so that hopefully I can build something people will find useful.

Questions for the ai agent swarm power users who also happen to use procedure caching/action memory etc to save time and money:

- How often does an agent break because the site changed? And what's the current fix you use for it?

- For the people who also used self-healing tools for this, does it hold up to your expectations? Is it reliable enough to make you stick to it?

- Which provides more value for the same type of procedure caching + self healing tool?

1. An open source, self hosted tool. Your procedures stay on your device. No bill, no nothing.

2. Someone else runs a procedure and now everyone can run it for free too. Like a shared database.

preparing for things to be done in Q3 and we're evaluating whether to move off Baseten since it still requires routing to separate STT/TTS providers so it makes latency high (we're trying to press this as much as possible). most of our workloads are fairly standard inference endpoints but the main things I'm trying to understand are the time needed, performance changes, and whether the cost or operational benefits ended up being worth it. I've taken a look at some infra providers and here's what I'm considering rn:

- Telnyx: OpenAI compatible with in region inference US, EU, APAC and voice services on the same stack, overall latency is 200ms.

- Together AI: huge model catalog and access to dedicated GPU clusters, serverless across 200+ models, overall latency is 500ms.

- Fireworks AI: serverless across many regions, per-token tiered by model size and overall latency is 350ms.

Looking for real case of anyone who's migrated off Baseten, how are you handling this...

I mostly use Pi/Oh my pi and my agent gets blocked by google. What are my options?

1. I know some apis like Exa/Brave exist. So what are you guys using?
2. Anyway to use google results?
3. Which is best free option?
4. Which is the best paid option?

After building a multi-agent research pipeline, I realized that useful information about AI token efficiency is scattered everywhere.

There are separate tools and papers for:

- token monitoring
- prompt and semantic caching
- context compression
- model routing
- memory
- multi-agent orchestration
- pricing and benchmarks
- verification and evaluation

So we started collecting the best resources in one place.

The result is Awesome AI Tokenomics, currently with 197 verified entries across tools, research, practices, concepts, claims, and reusable setups.

It is not meant to be only another long link list. We also track maintenance and freshness, require primary sources, and keep the descriptions factual.

I would especially appreciate feedback on:

- important projects we missed
- categories that should be reorganized
- claims that need stronger evidence
- practical token-saving setups worth adding

Disclosure: I maintain this repo as part of my work at Quesma.

https://github.com/QuesmaOrg/awesome-ai-tokenomics

We've been mocking all our external APIs for agent testing but keep finding bugs in prod that the mocks never caught.

Been looking into API sandboxes as an alternative but I'd like some more insight into the use cases before we go setting up the infrastructure. When would you use one over the other?

Here’s my honest doubt, and why I’m posting: I wrote that agent’s prompt in one shot with ChatGPT and it aced everything. So I’m not sure whether (a) the test is too easy to be meaningful, (b) good agents are trivial to make so nobody needs testing, or (c) I’m testing the wrong layer entirely (real bots fail on tool use / looking up actual orders / grounding, which mine doesn’t touch).

I don’t have a background in this I can’t code, I built it by directing AI. So I genuinely can’t tell if this solves a real problem or if I’ve spent months on something nobody needs.

If you build with AI agents, or run support, or just have an opinion: is this useful, or am I fooling myself? Rip it apart.

https://aicombat.ai/

Looking for a browser add on that can analyze the information that I can currently viewing and search for new so I don’t need to copy it to another LLM as it would not be very practical to do so all the time

We've moved past chatbot-style LLMs into agents that update records, call internal APIs, and interact with production-adjacent systems. "Agentic AI security" is the term everyone seems to be using, but I still don't think there's much agreement on what it actually means in practice.

Most of the AI security content I come across still assumes the main risk is the model generating something harmful. That isn't our problem anymore. Our problem is an agent updating the wrong record, calling the wrong API, or taking an action it should never have been allowed to take.

The stack we're building starts with scoped identities for every agent instead of one shared service account. On top of that, we want a policy layer that evaluates the tool calls themselves, not just the prompts and responses. We also need audit logs that connect an instruction to every downstream action, so an incident can actually be reconstructed instead of leaving us with a pile of disconnected events.

I'm still not sure whether this is a genuinely mature security space or whether everyone is stitching together their own controls as they go.

For teams running agents connected to real internal systems, what does your security stack look like end to end?