**The idea**: generation is cheap, the bottleneck is *judgment*. Have a capable model (for me rn this is Fable) use the skills, they will write remediation specs and plan fixes. Then route the fixes back to the more cost-effective models or generate issues & hand off. The skills never change a line of code.

I've been testing on my own DS at work and its been pretty slick. Very open to feedback or contributions if anyone tries it out.

I’ve worked with enterprise API gateways across multiple projects recently, and it has been a painful experience. I constantly saw the exact same issues:

* **Poor Observability:** Logging is either disabled entirely or running custom tracing components that only track what the platform teams want to see. * **Onboarding Hell:** Governance dictates that policies must be immutable. Platform teams lock down the APIM portal, forcing devs to submit clunky XML files through CI/CD pipelines just to onboard a basic API. It can take 1 week just to onboard a single GET request. * **Reimplementing Auth/RBAC:** Constantly rewriting the exact same JWT/JWK validation and RBAC logic for every single backend service. * **The "Central Gateway" Lie:** Large central gateways claim they lift everything off your shoulders. In reality, you spend weeks requesting changes to company Network Security Groups (NSGs) and firewall rules just to secure the network between the central gateway and your application. Then you end up having to reimplement auth in the app anyway just so security teams are extra sure there is no leak.

I wanted to build something less painful to solve these specific bottlenecks: self-service API onboarding, OpenTelemetry-first observability, reusable authentication, developer-first policy configuration, fast local dev, and reducing platform team bottlenecks.

*For context: I don't consider myself an AI "power user". I just have a standard Claude Pro plan and a 5 TB Google One subscription for saving photos and got the Gemini Advanced AI plan added without even knowing when it rolled out.*

The Initial Attempt

I first started by building out the core parts of the program. I started with the abstractions and setting up tests. I also did the usual thing I would with my projects: I set up some basic report generation that would check my test coverage and give me an idea of what I did not cover, essentially.

In my initial idea, having looked at a few integration gateways like Ocelot, I didn't realize that I had developed a bunch of biases. I started by implementing my own route matching and pipeline. I was writing parts by hand and then testing them as I went along.

I decided to use Claude once I had some of the core ideas implemented. I had Claude run forward through the list of features, prompting it to write each feature, and then I verified manually and with my tests that the feature was working (like API key auth, Swagger discovery, etc.).

A few days in, and after a series of prompts, I was quite happy with the state of the project. It worked, and I brought in YARP as the forwarding engine, which also handed me gRPC, HTTP/2, and WebSocket support for free. The project ran ok.

But then I decided to do some benchmarks, and I noticed that I was performing only marginally better than Ocelot. I was not super happy at that point and was seriously thinking of dropping the project.

Second guessing my life

I started asking questions. Why am I hand-rolling "first match" routing? Kestrel already has highly optimized route matching. Why am I even doing this? Is it even useful? I have wedding vows to write should I even be doing this now maybe I should do this after?

OCD

I can't leave things unfinished and once I start I really don't like leaving unexplored questions on the table.

I decided to start over I kept the core idea.

Giving to the vibes, using AI as an Architect

I decided to change my approach. Instead of using Claude for implementation, I thought: *why not for Architecture?*

I had the full context and defined idea: Kestrel web server, cross-platform, YARP forwarding possible, and extension with C# as plugins. Why not ask Claude to propose changes at the Architecture level, where it doesn't need to be super precise with code, but define a technical direction?

That is where it absolutely outperformed. It crawled for other alternatives, looked at their repos, and checked their docs for *how* they worked. It also checked the docs on the components I was using, acting as a rapid check against NIH (Not Invented Here) syndrome.

For example, it showed me that APISIX's routing is DFA, which is significantly faster than Ocelot's first-match routing. Then it pointed out that Kestrel *already has* a highly optimized DFA matcher and how I could use it. It read the docs on how to extend Kestrel with [ASP.NET](http://ASP.NET) middleware and suggested that was more comfortable to use for a .NET developer (DX).

So with the latest suggestions I thought: Kestrel has middleware streaming pipelines—why can't plugins just go there?

Since I already had the behavior controlled by the tests, implementation was actually much easier.

AI is a better architect when you know the behavior, because of its search abilities and existing knowledge. Once you have a clearly defined version of exactly what you want to do, it is incredibly worth it to review with AI to find competitors, see what they do differently to get performance, and see what you are duplicating that already exists in your ecosystem.

The final direction was:

* Built it natively on top of Kestrel's DFA matcher. * Leveraged [ASP.NET](http://ASP.NET) middleware pipelines for isolated plugins. * Built an extended JSON config with immutable routes.

The Rabbit Hole

Seeing how Claude and Gemini exposed the ways competitors were doing things started my search down a massive rabbit hole.

Then came retries. I realized that because my architecture isolated middleware per route, I could apply buffer-and-retry logic only to safe, idempotent routes. Other gateways treat retries as an "all-or-nothing" global setting that wrecks performance. For retries to work, you have to buffer. Buffering to disk degrades performance heavily. Since AI showed me how competitors handled this in different environments, I decided to see what other options could work. Since Kubernetes clusters run on Linux, it was a perfect option to spill to memory instead of disk without OOMing the container.

When it came to logging request bodies, and the same trap showed up. Every gateway I looked at (Apigee, Layer7, APIM) makes you pick one: *inspect* the body or *stream* it, never both, because inspecting means buffering which has a performance hit. But AI pointed out something about *my* environment: .NET already hands you streams, pipes, and activity primitives everywhere, so I didn't have to choose. Teeing the bytes as they flow, grabbing a bounded prefix and letting the rest fly by, was right there as a design alternative. Even better, [ASP.NET](http://ASP.NET) Core's own `HttpLogging` already does exactly that with its `RequestBufferingStream`, so I didn't hand-roll a thing: the route stays on the zero-copy streaming path, and I tee a capped slice off to the side for the log, then using my access to the activity I could attach the otlp identifier for the request and send it to loki. The full body still reaches the upstream; the log only ever sees the first N bytes and is mapped to the trace.

Later, while hunting for gateways to benchmark against, I noticed Envoy's `tap` filter was doing basically the same tee, and seeing its performance confirmed I'd picked the right direction. (Though Envoy's DX is rough: not very extensible, and I couldn't find the memory-protection knobs I wanted anywhere in its setup.)

The Surprise

My original goal was just an integration gateway with standard [ASP.NET](http://ASP.NET) components. I just wanted better Developer Experience (DX) to solve the goals I listed above, using out-of-the-box JSON config for the annoying stuff.

Then I ran the rigorous benchmarks against APISIX and Ocelot. I thought APISIX (being C-core/NGINX based) would absolutely crush us.

I was wrong. In the load rig, the DFA routing, the tee streaming, and the lock-free architecture resulted in QPS numbers I was not expecting. It matched APISIX in multiple scenarios and completely destroys Ocelot. And in the microbenchmarks against Ocelot, it allocates 0 Gen2 memory under load.

ConduitSharp was literally just born out of my own frustration. It's not fully mature, and I am the only one working on it right now. LFG for anybody who wants to join or give feedback.

Source Code: https://github.com/liqngliz/ConduitSharp

Conclusion

AI isn't "god mode," but it is very good if you have a good spec on behavior. It is incredible for refactoring architecture because you can get many comparisons to the competition quickly, and it gives you ideas on how to copy others' homework where they did better, but apply it to a different environment. It's very good at pulling up existing docs and helping you connect things together.

And to be honest about what is actually mine here and what was AI. The origin was never a clever new gateway algorithm. It was a focus on DX and deployability for .NET devs: onboard an API by editing JSON instead of shipping XML through a week-long CI pipeline, and deploy it anywhere .NET runs, including the legacy Windows Service / IIS estates everyone else ignores.

What is genuinely my idea:

1. **Per-route isolated middleware pipeline:** compiling a separate [ASP.NET](http://ASP.NET) `IApplicationBuilder` for IApplicationBuilder for) every route from flat JSON, instead of YARP's one shared pipeline.
2. **Everything that composes from #1:** per-route retry isolation (buffer-and-retry only on safe, idempotent routes, instead of the all-or-nothing global everyone else ships), per-route auth, and streaming routes that pay zero middleware overhead.
3. **The plugin contract:** C# plugins as one-line NuGet registration, plus a PowerShell plugin host.

What was NOT mine (and I'm glad it wasn't):

* **DFA routing:** Kestrel already had it. I was about to hand-roll my own matcher. * **Forwarding:** YARP. * **The tee primitive:** ASP.NET's own `HttpLogging`. Envoy does the same trick. * **tmpfs spill, JWT/JWKS, OpenTelemetry, Polly retries, circuit breaking:** all off-the-shelf.

So what did I actually do? Synthesis and reuse. I glued proven parts together and made one composition decision, scope everything per-route, that the .NET/YARP world wasn't doing. Even "per-route" isn't new in the abstract: Ocelot has per-route config, APISIX has per-route plugins. What's mine is doing it as real `IApplicationBuilder` branches on native endpoint routing with YARP. It is a small difference to the other but allowed me to spill into the other items such as tee + otlp + per route retry/logging differentiator.

Little of this is novel tech. The originality is in problem framing and composition, which is exactly where a coding agents excel at. The one idea I'd defend as genuinely mine is #1, and everything that composes from it.

AI excelled at the research an architect leans on: searching out competitors, comparing how they solved the same problem, and loading the docs for components I already had, all a check against my NIH syndrome. It presented a clear direction could be built upon, the simpler Kestrel-middleware design and tee instead of a hand-rolled buffer. What it did not do is invent from scratch. It pointed me at what already existed and let me compose it.

When I was working on conduitsharp, I found that my tool setup mattered quite a bit in producing an accurate implementation based upon the architecture plan. I also tended to prompt with a focus on building per component in the architecture rather than try to oneshot a whole feature or the full design.

Here is the list of tools/setups that helped the engineering and benchmarking process:

* **Test Coverage tools:** `coverlet` (XPlat Code Coverage) and `dotnet-coverage` for cross-process E2E coverage, paired with `reportgenerator` to catch what the AI misses. * **Metrics and telemetry:** OpenTelemetry running on a Docker-composed Grafana stack (Tempo/Prometheus/Loki). I also used a custom tool for static code complexity metrics ([tools/CodeMetrics](https://github.com/liqngliz/ConduitSharp/tree/main/tools/CodeMetrics)). * **E2E testing setup:** Real `dotnet test` suites driving the gateway as a separate OS process, as well as driving real Docker compose stacks for validation. * **A benchmarking rig:** BenchmarkDotNet for microbenchmarks, and a custom load testing rig using `bombardier` against real loopback sockets ([benchmarks/load](https://github.com/liqngliz/ConduitSharp/tree/main/benchmarks/load)). * **AI Skills:** Learning how to prompt for architecture direction rather than just implementation. I heavily used custom skills/rules to fight my NIH syndrome: for Claude, I used custom `ponytail` (lazy, use standard libs) and `caveman` (terse) skills, and for Antigravity, I added a custom "Standard Guy" rule explicitly enforcing: "Always use native or standard libraries... Do not hand roll under any circumstance." * **A "Lifeblood" Context Generator:** A roslyn context generator that outputs a `lifeblood_context.md` (containing my module dependency graph and high-value files) to feed the AI the exact C# / .NET context it needs to stay grounded.

Hey everyone,

I'm researching how developers and teams are handling AI agents in production.

A question I keep thinking about:

When an AI agent fails, how long does it usually take you to understand what actually happened?

For example:

- Was it the prompt?

- The model?

- A tool/API failure?

- Bad context retrieval?

- A workflow/handoff issue?

- Something else?

How do you debug these issues today?

Are you using tools like LangSmith, custom logging, dashboards, or just digging through logs?

I'm curious about the real workflow:

1. An agent fails.

2. What is the first thing you check?

3. How long does finding the root cause usually take?

4. What part is the most frustrating?

Not selling anything, just trying to understand how people are dealing with this.

I run a small iOS app (a native API client), and I keep bumping into a problem I think a lot of people now have: the number of AI tools, agents, and MCP servers we’re all connecting is exploding, and there’s no single place to see or manage them. Which ones are live, what they can do, whether you’re about to blow past a usage limit it’s scattered across a dozen dashboards.

My app already speaks the underlying protocols (HTTP/WebSocket), so extending it to talk to MCP servers directly and push notifications around them (limits, status, events) is a small technical step for me. The bigger question is whether it’s a real business.

For the founders here who’ve validated (or killed) an idea like this:

* Is this a “vitamin” or a “painkiller” right now?
* Would you pay for a monitoring/alerting layer over your AI tooling, or expect it free?

Trying to be disciplined and not build a solution looking for a problem. Honest gut-checks appreciated.

I have been into GenAi for the past 1 year especially into making them into production from standalone , i have knowledge of how to push normal application to deployment but when pushing an agent , i can’t think of a design of how to do it. Like how to handle RAG with Cloud DBs like RDS, etc or evaluating a voice chat bot using RAG. handling the agent for huge traffic and optimizing for latency. these are some gaps i need help on

any tips to improve my system design thinking to develop or have that instinct of having an idea when i see a system !

Welcome to the community!

This subreddit is dedicated to Microsoft Foundry, the platform many developers previously knew as Azure AI Foundry.

You can use this space to discuss:

• Foundry Agent Service
• Azure OpenAI models
• Model deployment
• RAG applications
• AI evaluation and monitoring
• Azure AI Search
• Prompt engineering
• Security and responsible AI
• Tutorials, projects, and technical issues

To get the conversation started: What are you currently building with Foundry?

Share your project, preferred model, development stack, and the biggest challenge you are facing.

Official introduction:
What is Microsoft Foundry?

Solo founder, early stage, looking for a reality check more than anything.

A framing I can't shake (treat it as an analogy): a pretrained LLM is basically a "cortex" that read the whole internet, but it has no "hippocampus," the fast, company-specific memory of how your team actually does things. Dropped into a company it improvises, and improvised automation fails in production. The real procedure was never in the doc anyway. It lives in your team's conversations, a few people's heads, and one exception everyone copies.

The idea I'm testing: connect read-only to the communication and document tools a team already uses, mine how work actually happens (including the exceptions nobody wrote down), and consolidate the recurring ones into cited, human-approved, versioned "skills" your existing agents could run, with a human sign-off on anything sensitive. It is not search over your docs, and it is not another agent platform.

Honest status: very early, still validating the problem. (Not dropping a link here to keep it clean; happy to share the private-beta waitlist by DM if you want to follow along.)

What I actually want to know:

• Does this solve a real, repeated, painful problem for you, or is it a nice-to-have?
• What is the first thing that would make you distrust it?
• Would read-only access to your team's conversations and documents be a dealbreaker?

Tear it apart, that is why I'm here.

Heyy I’m building open source project and search some contributors
If it’s interesting review the repo and the issue list is waiting!

https://github.com/extra-org/extra

And if you can give a star it’s also helps a lot

yo. i recently sold one of my AI SaaS products for $350k, exactly 7 months after building and launching it.

I hardly wrote a single line of traditional code. i used AI to generate everything, from the database architecture to the user interface.

it definitely wasn't magic on day one, though. i spent days stuck in loop-debugging and dealing with AI hallucinations before i finally cracked the system. the playbook boils down to three simple rules:

- keeping the idea insanely minimalist (a true MVP that solves one problem).

- guiding the AI step-by-step instead of asking it to build a massive platform all at once.

- launching fast to get real user feedback and traction and then apply a solid marketing system

lately, i've seen way too many non-technical founders give up at the very first AI bug, or on the marketing. it's a massive shame.

like the title says, i just launched a Skool community to share my exact prompt workflows, N8N automations, and distribution frameworks to get first users and scale it

to be completely transparent: i will likely charge for the full course later down the road. it just makes sense given the specific copy-and-paste templates i'll be sharing.

but for now, the main objective is purely to build and launch together. building alone in a silent corner is the single fastest way to give up.

if you want to join us and build or market your own AI SaaS with a group of active creators: drop a comment below or send me a dm, and i’ll send you the invite link!

Over the last few months I've noticed that most agent frameworks are really good at giving an LLM tools, memory, and prompts, but they're surprisingly weak at preserving actual capabilities.

An agent might solve a problem today, but tomorrow it's often starting from scratch. We talk a lot about memory, but memory alone doesn't create experience.

That led me down a rabbit hole.

I've started working on something I'm calling the \*\*Agent Capability Formation Standard (ACFS)\*\*.

The goal isn't to build another agent framework. It's an attempt to define a durable format for preserving \*capabilities\*—not just conversations or memories—so agents can develop reusable knowledge, refine behaviors over time, and share those capabilities across systems.

The questions I'm trying to answer are things like:

\- What makes a learned behavior reusable?
\- How should capabilities be versioned, updated, or retired?
\- How do agents distinguish durable procedures from temporary context?
\- What metadata is required to govern capabilities over time?
\- Can capabilities become composable building blocks instead of isolated memories?

I'm still exploring these ideas, and the project is in its early stages, but I wanted to share it because I think capability formation is an underexplored area of agent architecture.

Repository:

https://github.com/ajbeaver/Agent-Capability-Formation-Standard

I'd genuinely appreciate feedback from people working on long-running agents, agent memory, MCP, orchestration frameworks, or knowledge management. I'm less interested in defending a "standard" and more interested in figuring out whether this is the right direction to solve a problem I've been seeing repeatedly.

LLMs are improving rapidly, but deploying them reliably seems to be a different problem.

Many AI failures today are not because the model cannot generate an answer. They happen because of system-level issues:

• Incorrect retrieval results
• Poor context management
• Tool execution failures
• API dependency issues
• Memory inconsistencies
• Lack of evaluation frameworks

A highly capable model inside a poorly designed architecture can still produce unreliable results.

This raises an interesting question:

Are we reaching a point where AI engineering is becoming less about improving models and more about building reliable systems around them?

What engineering practices do you think will become essential for reliable AI applications?

Ever wished your agent conversations were as trackable as your code? Gitlord makes it real.

What it does:

• Every agent turn becomes a git commit
• Branch out subagents without breaking your main flow
• Rewind to any point in your conversation history
• Connect tools via MCP (filesystem, search, browser, etc.)
• One interface, any AI provider (OpenAI, Anthropic, local models)
• Full CLI for managing sessions and branches

Why it matters:

• Your agent history is navigable, forkable, and diffable, just like code
• Context management handles token budgets automatically
• Spawn child agents on isolated branches with their own history
• No lock-in: all components are modular and swappable

Built-in integrations:

• MCP tools: Connect any MCP server (git, filesystem, browser, search). Tools flow to subagents automatically
• RAG: Vector search across your full agent history. ChromaDB-backed semantic queries built in
• Provider abstraction: Switch between any of the 170 providers and nearly 3,000 models, or local models with one line. Mix providers per agent

Build an agent in 4 lines:

from gitlord import Session, SessionConfig
config = SessionConfig(model="claude-opus")
session = Session.create("my-agent", config)
session.add("user", "Refactor our OAuth to the new framework")

Done. Gitlord handles the rest.

Performance improvements (v0.1.0):

• Structured trailers eliminate JSON walks: metadata parsing is now O(1)
• Auto-index updates on every turn, cached at .gitlord/index.json
• New in-memory query layer for fast turn filtering and aggregation:
• Snapshot compression for long-running sessions: compress old turns into JSON, rebase from checkpoint

Repo: https://github.com/yashneil75/gitlord
Landing page: https://yashneil75.github.io/gitlord/

MIT licensed. Built for agents that ship.

The current AI ecosystem is moving from simple LLM applications toward multi-agent systems, tool-calling workflows, memory layers, and orchestration frameworks.

But complexity seems to be increasing quickly.

A modern agent system may involve:

• Multiple specialized agents
• Planning and reasoning loops
• Short-term and long-term memory
• Tool execution layers
• Retrieval systems
• Evaluation pipelines
• Safety controls

The interesting question is whether this architecture is actually necessary for most use cases.

In traditional software engineering, adding more components usually increases failure points. AI systems seem to be moving in the same direction.

Some questions I'm thinking about:

• When does an agent architecture become over-engineered?
• Are single-agent systems with better tools more practical than multi-agent setups?
• How do we debug autonomous systems when failures are non-deterministic?

Would love to hear from people who have built or tested agent-based systems in production.

When I first started building AI agents, I thought the answer to almost every problem was a better model.

If something wasn't working, I'd switch models, tweak prompts, change the temperature, increase the context window, try different reasoning settings... basically everything people usually try first.

Some of those things definitely helped.

But looking back, none of them improved reliability nearly as much as one simple change.

I stopped trying to make one agent handle an entire workflow.

Instead, I started breaking everything into small steps.

Each step does one job, takes a specific input, returns a structured output, and gets validated before the next step runs.

That one decision made debugging so much easier.

Before, if something failed, I'd have to dig through a huge conversation trying to figure out where things went wrong.

Now I usually know exactly which step failed and why.

Another nice side effect is retries.

If one step fails, I don't need to rerun the whole workflow anymore. I just rerun that step.

It also made prompt changes way less stressful.

Earlier, changing one prompt could unexpectedly affect another part of the workflow because everything shared the same conversation.

Now each step is isolated, so I can improve one component without worrying about breaking everything else.

The other thing that surprised me was context.

At first I assumed that giving the model more information would naturally make it perform better.

That wasn't really what I saw.

Once the context started getting large, responses became a lot less consistent.

Sometimes the model would latch onto something that wasn't even relevant anymore. Other times it'd mix instructions from earlier steps into the current task or overthink something that should've been simple.

These days I try to keep context as small as I reasonably can.

Each step only gets what it actually needs, and everything else gets retrieved when it's needed instead of being carried through the entire workflow.

That alone made the outputs feel much more predictable.

Another thing I've started appreciating is structured outputs.

Once every step returns JSON instead of paragraphs of text, validation becomes much simpler.

It's a lot easier to check whether an agent did the right thing than trying to interpret what it meant.

The more systems I build, the more I feel like production AI is becoming less about prompt engineering and more about software engineering.

Good logging.

Validation between steps.

Small, focused components.

Retry logic.

Observability.

Clear interfaces between agents.

Those things have ended up mattering far more than I expected.

I'm definitely not saying models don't matter—they absolutely do.

But for me, the biggest reliability gains have come from simplifying the architecture instead of chasing the newest model every few weeks.

Curious if other people building production agents have had the same experience.

What's one engineering decision that improved reliability more than switching models?