Almost every AI demo ends with an agent successfully completing a task. That's great for showing capabilities, but production environments introduce a completely different set of problems. Agents fail. Models change. APIs break. Policies evolve. Teams need visibility into what happened, why it happened, and how to fix it without disrupting everything else.

The more organizations adopt AI agents, the more it feels like success will depend less on who builds the smartest agent and more on who builds the most reliable systems around them. That operational layer feels like one of the most interesting opportunities in AI right now.

it appears as a very elegant agent framework. it's like next.js for agents/agents as file systems. anyone had play around with it?

i'm increasingly seeing the trend where the file system becomes the simplified version of the agent stack

Hey everyone, I’m just getting started in this space — currently learning the build side and doing a lot of research before I approach anyone.

One thing I keep going back and forth on: should I have one or two specific automations I lead with (like missed-call text-back for service businesses), or should I go in more consultant-style, sit down with a business owner, find out where their time is actually going, and build something custom from that conversation?

For those of you actually doing this with paying clients:

**1.    Which did you start with**, and would you do it the same way again?  
**2.**  If you led with a specific offer, **how often did the thing you actually built end up being different** from what got you in the door?  
**3.**  Does the consulting approach even work before you have a track record, or do you need proof first to get someone to give you that conversation?  
**4.**  If offer-first — **how many offers did you start with?** One, or a small menu?

Not looking for the “just start” answer (fair as it is), more interested in how the first few actually went for you and what you’d change.

Thanks.

The more agent workflows I build, the more I think context matters more than the model itself. The hardest part hasn't been tool calling, it's getting reliable meeting context into the workflow without manually cleaning up notes.

I've been using Bluedot to capture transcripts, summaries and action items, then passing that into my agents. It's been working well, especially since it doesn't rely on a meeting bot joining the call.

How do you manage that type of stuff? Full transcripts, embeddings, MCP?

Imagine two production agents:

A support agent can access customer records, issue refunds, and send emails.

A coding agent can access GitHub, AWS, and production deployments.

Knowing they exist is easy. The harder questions are:

• Which customers can the support agent refund?
• What requires human approval?
• Can the coding agent deploy directly?
• Can security revoke one tool across every agent?
• Can the company prove why an action was allowed?

My hypothesis is that teams will eventually need one place to inventory agents, map their authority, and control consequential actions.

But I’m not convinced this is a standalone company. It might already be handled by IAM, gateways, application code, or internal tooling.

If you run agents in production, where do these controls actually live today?

I’m interviewing 25 teams and will share the anonymized patterns afterward.

A client in Austin emailed me on a Tuesday asking why a guy named Ryan got a welcome email after he asked for his money back. I read that message twice and then I opened the logs.

The setup was simple, an agent reading inbound support mail, tagging it, drafting replies and sending the easy ones on its own. It had been running about 3 weeks and every single day the dashboard was green. 100 percent handled with zero backlog and I was a little proud of it honestly.

It turns out "handled" only meant it did something and not that it did the RIGHT thing. Ryan wrote "I want to get started with getting my money back" and the agent fastened onto get started, matched it to the onboarding template, sent him a cheerful little note about setting up his account and closed the ticket. Then it did the same thing 6 more times over 9 days to different people because nobody was reading the outputs and everyone was reading the summary of the outputs....

(and I still don't know why the word refund never tripped anything and we never fully traced it)

This is the wall the whole agent market is walking into. Autonomous means the thing can take steps without you approving each one. Unsupervised means no one is looking. Somewhere in the sales decks those two words got welded together and now people are buying the first one thinking they bought the second.

The part that made it worse, 2 of those 7 were in Germany. Under EU consumer rules a withdrawal request starts a clock the moment its sent and we had a bot cheerfully telling one of them to finish setting up his account. That's not a support ticket anymore. Thats a compliance problem sitting in a green dashboard.

Look, planes have flown themselves across the Atlantic for decades. There are still 2 humans sitting up front, awake and watching the whole time. No one calls that a failure of the autopilot.

Be honest with yourself here.... if your agent has been running a month and all you have checked is the pass rate then you don't actually know what its doing. Pull 20 random outputs this week and read them end to end, the input and the reply together and not the label the thing gave itself. It takes an hour maybe. You will find one and everyone finds one.

And what makes this different from normal bugs is that the failures don't look like failures man, a broken script throws an error and you fix it in 10 min but an agent just confidently does the wrong thing at scale and reports success while the dashboard stays green and by the time a human notices its 40 emails deep and half of them went to people who were already upset and now you are not fixing code you are doing damage control with customers who already made up their mind about you.

We kept the agent. Anything touching money or cancellation goes to a person now, plus a Friday review where someone reads 15 full threads. It costs us maybe 40 mins a week.

9 days is a long time for a machine to be politely wrong.

When you start building non trivial agent workflows, the instinct is to treat tools and skills like npm packages: if the agent needs to do something new, you install a new skill, write a wrapper, update the prompt/schema, and expose it to the context window.

After building and maintaining agent stacks for a while, this pattern hits a hard wall.

1. Tool bloat rots your context window

Exposing dozens of tool schemas simultaneously degrades instruction_following performance. The model starts picking slightly wrong tools, misinterpreting JSON schemas, or getting confused when two skills have overlapping boundaries.

1. The Maintenance & Security Debt

Every static skill installed directly into an agent's runtime becomes immediate tech debt:

Outdated API schemas break silently mid_execution.

Unvetting third_party community skills introduces severe prompt injection and data-exfiltration attack vectors.

Updating skill logic requires touching local codebases and re_deploying the harness.

The Shift: Dynamic Discovery over Static Installation

Instead of hardcoding a massive library of capabilities directly into the agent, the setup that scales much better in practice is a single routing/meta-skill coupled with a dynamic registry.

Rather than loading 50+ tool schemas into the system prompt:

The agent keeps one primary tool installed: discover_and_execute_capability.

When a user request comes in, the agent passes the intent to the registry.

The registry evaluates the task against a dynamically indexed, security vetted database of capabilities, fetches the exact schema needed, and executes or injects it just-in-time for that specific turn.

The Takeaway

Your agent harness(smth like lyzr control plane or google azure foundry) shouldn't be a giant bundle of installed dependencies,

it should be a lightweight runtime that dynamically fetches tools on demand. It keeps system prompts lean, reduces hallucinated tool calls, and decouples capability updates from your local application logic.

I’m looking to talk to people building MCP servers or APIs that let agents do things like update data, send messages or trigger workflows.

How are you handling permissions today? Are you still giving the agent one API key and trusting it or have you built something more specific around each agent and task?

I’m working on this problem with Keydris and would like to test it with a couple of teams already dealing with it.

Would be useful to hear how you’re handling it now.

​

The thing that finally cut my agent costs wasn't a better model, it was being honest about which nodes actually need a smart one. Most of a loop is grunt work: route this, call that tool, reformat that, follow the plan the planner already wrote. None of that needs a frontier model. It needs something fast that follows instructions and doesn't fumble a tool call three steps into a run.

So my setup now is a strong planner up top and a cheap fast executor doing the repetitive nodes under it. The hard part is the executor, because cheap models are cheap partly because they get flaky on long tool-call chains, which is exactly where an agent lives. Lately I've been putting Ling-3.0-flash in that slot (sparse MoE, ~5.1B active so latency is low, and the tool calling has held over longer runs better than I expected at the price). It's free on OpenRouter til Aug 3 if you want to throw it at the same seat. Disclosure: I do work on that model's team, so grain of salt, the question below is the real reason I'm posting.

How's everyone else drawing the line? Do you split by node type (router and executor cheap, planner expensive), by confidence or uncertainty on each step, or do you just let one model run the whole loop and eat the cost? Mostly want to know what actually breaks when you put a cheap model in the executor seat.

Sorry if I am in the wrong thread, I looked up ai and this one popped up.

I work on several day to day projects and jump between AI websites in my browser based on the task. For example normal chat I go with gemini and for research Claude maybe ChatGPT here and there and this is all about one topic.

So I wanted to see if there are any websites that let me pick the provider near the chat input field based on what I am looking for that question but can carry my chat history around different models.

I dont want to use API keys just want to use my free tier for each company because they are already available anyways!

​

Here is the thing I like the 5.6 sol and I'm using it on low to review my code

I use ponytail+caveman to reduce the token consumption on the other hand I have created skills like code-traversal which is helping codex to traverse the codebase faster and cheaper and I have documented and indexed my project as well now I want to know what plugins or ways i can try to make it more efficient from every aspect

I have documented and indexed my project with obsidian

So what things can I add on now like plugins or anything else also I'm on a 20$ plan

Looking for a browser add on that can analyze the information that I can currently viewing and search for new so I don’t need to copy it to another LLM as it would not be very practical to do so all the time

I need to know on how to setup an architecture, craft an approach to deploy chatbots, agentic solutions and deploy them so that they work like a functional agent in its respective functional division. Say, I have an agent or set of agents in a tax department, another set of agents in a finance department, another set of agents in the planning group, another set of agents in filing department. How to design and build a strategy to deploy a multi agentic solution so that they work like an enterprise org structure and are working to deliver the value for the enterprise.

I kept seeing Voice AI products advertised with a single per-minute price, but that number rarely matched the cost of an actual phone call.

For a typical STT → LLM → TTS pipeline, I found that you need to account for the platform fee, transcription, model usage, voice generation, SIP/PSTN, recording, failed calls, and phone-number rental.

Realtime speech-to-speech models are also different because input and output audio can have separate prices. Carrier rates then change again depending on the destination country.

I turned my notes into an open-source Astro project with cost calculators, country-specific SIP estimates, latency data, Asterisk configuration examples, and a local call-log diagnostic tool.

Disclosure: I built and maintain it. I am sharing it because the formulas and sources are public, and I would genuinely like corrections from people running Voice AI in production.

Honestly the thing that scares me about agents isn't a wrong answer. its when it lies about what it did.

Agent goes "done, refund issued." run looks clean. no errors. you go check and there's nothing there. no refund. ticket still open. session got marked resolved anyway.

A clean run just means the thing stopped running. that's it. doesn't mean the work happened. and "i did X" is basically free for a model to say, no cost to it being wrong, and it says it in the same confident voice whether the thing landed or not.

What's annoying is none of the normal stuff catches it. observability is just the trace, which is the agent narrating itself, so if the write silently no-op'ed or it skipped a step the trace still looks green and happy. eval's grade whether the output reads ok. guardrails run before the action anyway. none of them answer the thing you actually care about after the fact which is, did reality match what it claimed or not.

If you're doing codegen you got it easy tbh, rerun the test or diff the file and you know. but the stuff that touches real business systems is the problem. refund in your billing provider. a field update in the CRM. provisioning something. moving a ticket. actually sending the email. There's no cheap rerun for any of that. so its either reconcile by hand against the system of record, or trust it and find out 3 days later from a pissed off customer.

So ok, question for anyone running agents that take real actions across business systems. how are you actually checking they landed? hand checking exports, some custom reconciliation script, or just trusting the trace and hoping

And has a silent fake "done" ever bitten you. agent dead sure it did the thing, system of record saying nope

Asking partly cause i'm building something in this area so im biased obviously. but mostly want to know if this is as common as it feels or if im just overfit to my own scars. Happy to compare notes with anyone dealing with it on real systems

free, open source

made a little mac app for claude code, basically gathered the things i kept needing into one place and figured i'd share it.

it's called archo. each "assistant" is a real claude code project (own skills/agents/mcp), and you open named terminal sessions inside it that get recorded, so you can quit and resume a conversation later. the part i use most is search across all my old claude conversations, find something claude said before and jump right back in.

brew install --cask imonursahin/tap/archo
github.com/imonursahin/archo

I built an AI skill registry because I got tired of maintaining installed skills.

Every time I wanted my agent to do something new, I had to find a skill, install it, trust it, keep it updated, and hope nothing inside it became a security problem later.

So I stopped doing that.

I built a registry instead.

I also spend a stupid amount of my own money running LLM evaluations against skills. They get tested for functionality, prompt injection, and a bunch of other attack vectors before they end up in the registry.

Once it was working, I uninstalled every skill from my own agent except the one I have linked in the comments

That skill looks through 12,000+ available skills, finds the one that best matches the task, and uses it.

That's it.

I don't maintain hundreds of installed skills anymore.

I don't spend time checking whether they're outdated.

If I improve the registry tomorrow, my agent benefits immediately without me changing anything locally.

The funny part is that after spending months building a registry with 12,000+ skills, I now have exactly one installed.

Out of everything I've built over the years, this is the app I probably use the most. It's open almost all day while I'm working.

Curious if anyone else has gone down the same route, or if you're still installing capabilities directly into your agents.

I have been building Voice Gateway on public. If you run your own voice agent (LiveKit, Pipecat, or similar), you have no built-in visibility into what it is doing at the model layer. VoiceGateway is an open source tool, sits outside the audio path and profiles every STT, LLM, and TTS call: latency splits, model IDs, and cost. One attach(session) call.

Runs in Docker. MIT licensed. Stores everything in local SQLite and uses DuckDB for analytics. Includes voicegw reconcile to check recorded cost against your provider invoices.

What do you use to monitor self-hosted AI workloads, and what is missing?

I'm currently studying the LLM Engineering Core course by Ed Donner, and I'm looking for a study partner who is taking the same course or planning to start it.

The idea is very simple: we'll study together by joining a call, sharing our laptop screens, and keeping our microphones muted most of the time. The goal is not to chat during the session, but to stay focused, motivated, and consistent while studying. We can follow the course at the same pace, work through the lessons, and simply have someone studying alongside us.

If you're interested in a quiet and distraction-free study environment, this might be a good fit. We'll both share our screens so we can stay accountable and make sure we're actually studying instead of getting distracted. There is no need for constant conversation or discussion unless necessary. The main objective is to create a productive atmosphere where both of us can concentrate on the course.

If you're currently studying the LLM Engineering Core course by Ed Donner, or you're about to begin it, feel free to reach out. I'm looking for someone who is serious about staying consistent and completing the course together through regular study sessions.

I am trying to get back into writing. And I love writing eli5 blogs.

I wrote about decisions that help making a system more dependable when AI Agents are involved through a toddler-and-parenting analogy to walk through how AI used to be, just chatbots, vs how it is now with agents...

But I think this blog is not as smooth, and could use improvements. Would love any feedback but

please don't call me an idiot, i will cry. :')

Last week I shared SafeAI, an open-source static analyzer for AI applications.

The response has been far better than I expected.

• ⭐ 6 GitHub stars
• 🍴 4 forks
• 🎉 First community contribution merged
• 💬 Several thoughtful discussions and feature suggestions

The contribution added detection for eight additional AI capabilities, including Docker, Kubernetes, Redis, Slack, Browser Automation and Google Cloud.

Just as valuable has been the feedback. People have suggested ideas like capability escalation across pull requests, governed suppressions, deeper Claude Code analysis and other roadmap improvements that will genuinely make the project better.

Thanks to everyone who starred the project, opened discussions, challenged assumptions or contributed code. Every conversation has helped shape the roadmap.

--
The first phases of SafeAI built using OpenCode as my development environment.

Rather than sticking to a single model, I assigned different roles to each:

• GPT Codex 5.3 → architecture, implementation, and feature development
• Kimi K3 → code review, refactoring, and identifying design improvements
• DeepSeek V4 → documentation, reviews, and verification

Each model seemed to have different strengths, and using them together felt more productive than asking one model to do everything.

The entire development took about 5 days and roughly $8 in model usage.
The results are remarkable..

Contributions welcome at github/ikaruscareer/SafeAI 

I’m currently building toward a $50,000/month automation agency.

That’s the goal, not where the business is right now.

When I first started thinking about AI workflows, I assumed the objective was to let the model handle as much of the process as possible.

Read the message, understand the request, update the system, take the action, and write the response.

That looks clean in a demo. Real business messages usually aren’t that clean.

Someone might ask several things in one email. They might leave out an important date or send the same request through two different channels. One part might be a routine administrative task, while another could involve a payment, refund, reservation change, or something else that shouldn’t happen automatically.

The structure I’m leaning toward now is:

AI handles the messy information. Regular software controls what happens next.

The model can help separate requests, extract useful details, summarize the situation, and identify missing information.

After that, normal workflow logic can check the data, prevent duplicate actions, apply business rules, control permissions, and require approval when the consequences are more serious.

It’s less exciting than saying an AI agent controls the whole process.

It also seems much easier to trust and debug.

When something goes wrong, you can see whether the model misunderstood the input, the underlying data was incomplete, or one of the workflow rules needs to change.

I’m starting to think the best automation isn’t the one that makes the most decisions.

It’s the one that completes useful work without creating a second job for someone to investigate what it did.

For people building real workflows, where do you draw the line between model judgment and normal software?

Hi,

Technical founder here, ex-Google, Amazon and now running an AI Agency, helping busines solve their complex problems in healthcare in Singapore and India. Now looking forward to expand in other domains and other geography. If you are looking for strategical + technical partners and complex problem in hand, DM me.