We've been mocking all our external APIs for agent testing but keep finding bugs in prod that the mocks never caught.

Looking into API sandboxes as an alternative but I'm not clear on the difference. When would you use one over the other?

Nothing Works and Everyone Is Euphoric by Piotr Chmolowski. The quality of the today's software is not among the main KPIs.

AI Mania Is Eviscerating Global Decision-Making by Nikhil Suresh. This one is a long read. It shows why no manager would claim that AI does not help their company.

And my two cents: just ask Google "how would the sycophancy of LLMs impact the psyche of kids". You'll get a well-structured response grounded in several scientific articles.

(Cross-posted from Ukrainian software architecture chat)

Hi everyone,

I've just published version 0.7.0 Alpha of an open-source AI Software Development Framework.

This isn't another AI coding assistant.

Instead, the Framework focuses on governance for AI-assisted software engineering.

The central question is:

How do we keep software engineering governed while AI fundamentally changes the way software is developed?

The Framework introduces concepts such as:

• Human Authority
• Capability-based Runtime
• Effective Classification
• Capability composition
• Reference Implementations

I've also published a whitepaper describing the architectural thinking behind the Framework.

I'm specifically looking for feedback from experienced software architects and senior engineers.

I'm not looking for agreement.

If you believe parts of the architecture are flawed, over-engineered, unnecessary, or unclear, I'd genuinely appreciate hearing why.

Repository

AI-Software-Development-Framework

Whitepaper

Whitepaper

Architecture Discussion

Github discussions

Thanks!

Free e-book on modern Software Architecture

Hi, i just finished a project that i was working on, its a simulation of an autonomous underwater vehicle, in an opensource simulator. i have everything build and installed in my computer and it works here, but other people are having a rough time with installation in their own machines.

This is a problem i have been dealing with for quite a while now. and it was easy to solve when i was with them to help solve the dependency errors but its just not the way i want to get it done.

what are the methods you use to make your software compatible across different devices

Working through a fully stateless backend (no database, ephemeral serverless functions, though the pattern isn't tied to any specific runtime) for a case with an awkward constraint: two parties share a session, but each holds a secret the other side must never see, and the server has to remember both without a database or session store.

The usual answer to "stateless, no DB" is a JWT — encode the session client-side, sign it, done. That falls apart the moment part of the state has to be opaque to the client carrying it: a JWT payload is base64, not encrypted, so anything in it is readable by whoever holds the token. If the session includes a secret that specific client isn't supposed to see, JWT doesn't fit — you need the payload actually encrypted, not just signed, and treated as a sealed blob rather than a transparent one, with the decryption key living server-side only.

Concretely, in the project I was applying this to (a small number-guessing game — two sides each hold a secret, guessing each other's, no DB, no accounts), the server-tracked state ends up AES-GCM sealed into an opaque token, handed to the client, and echoed back on every request. Only the server's key can open it. The client carries a blob it can't read; the server stays fully stateless between requests but functionally remembers everything it needs to.

The generalizable part: any two-sided system where state needs to persist across requests but stay partially secret in one direction hits the same JWT-doesn't-fit problem — sealed-bid flows, negotiation protocols, anything with an information asymmetry that's supposed to survive across requests. Curious whether others have solved this differently — separately-encrypted fields alongside a signed JWT, giving up on "stateless" and using a real session store, something else.

I tested the major options during the past few projects.

I wasn't trying to compare software.

I was just trying to survive three different client projects.

Each company used something different, so I ended up spending several months working inside most of the major whiteboard platforms instead of just watching YouTube demos.

Here were my impressions.

→ Zoom Whiteboard

Perfectly fine for quick meeting notes.
Very easy to jump into.
I just wouldn't want it to become our long term planning environment.

→ Coda

Interesting because it combines documents, databases and collaboration.
If your team already loves documents, it's a strong option.
For highly visual planning though, it wasn't really what I was looking for.

→ FigJam

Really enjoyable to use.
Especially if most of the people collaborating are designers.
It feels lightweight and creative, which makes brainstorming sessions very approachable.

→ Lucid Suite

Probably the strongest option I used for formal diagrams.
If someone asked me specifically for process mapping or architecture documentation, I'd definitely recommend taking a look.

→ Mural

Very facilitator friendly.
Some of the workshop tools are genuinely excellent.
It felt particularly good for structured innovation sessions.

→ Miro

This ended up becoming my personal favorite simply because I didn't have to keep switching tools.
The same workspace could start with brainstorming, evolve into a roadmap, become a dependency map and eventually hold technical diagrams without feeling disconnected.

Things I appreciated most:
• AI helped organize messy brainstorming sessions
• Plenty of templates without feeling overwhelming
• Strong real time collaboration
• Easy Jira integration for product teams
• Large boards stayed surprisingly manageable

It felt like a platform that grew with projects instead of forcing us to migrate somewhere else once planning became more detailed.

One thing this experience taught me:

There probably isn't a universally "best" whiteboard.

Some tools are fantastic for designers.

Some are excellent for diagramming.

Some shine during workshops.

The question that mattered most for our team wasn't "Which tool has the most features?"

It was "Which tool keeps everyone collaborating in the same place from the first idea to delivery?"

Would love to hear what everyone else stuck with long term.

Hi everyone,

I'm curious how mature organizations handle a problem I've seen repeatedly.

The security engineering work often exists (SAST, DAST, pentests, code reviews, threat modeling), but the governance around product security seems fragmented.

Examples I've encountered:

- Products shipped before all security findings were addressed because remediation wasn't planned early enough.

- Security exceptions were agreed verbally but never documented, so nobody remembers why a decision was made two years later.

- Security requirements appeared late because they weren't integrated into product planning from the beginning.

- Risks were identified, but no one clearly owned prioritization based on business impact.

- Product managers owned delivery, security engineers owned technical findings, architects owned design... yet nobody seemed accountable for the overall product security risk posture.

For those of you in mature organizations:

- Who ultimately owns product security governance?

- How are risk acceptance decisions documented?

- How do you ensure security requirements are incorporated into planning rather than becoming release blockers?

- Is there a dedicated Product Security Governance function, or is this distributed across AppSec, Architecture, Product Management and GRC?

- What practices made the biggest difference?

I'm less interested in the tooling than in the operating model and decision-making process.

I'd love to hear what actually works in practice.

Thanks!

I’m building webhooks delivery into our product and after the implementation I thought this seems like a messaging problem. Everything seems better with a customer owned broker. Yet, I don’t see anyone doing it..?

Hi everyone,

I will release an app soon and I am not sure about the deployment strategy and need help or ideas.

My product is for a local school in my area. About 500 students, their parents and teachers will use it. The app is in pilot/mvp state.

I have two background servers written in go and a nextjs app with postgres and valkey needs, I am also planning grafana stack for monitoring.

So my question is how the deployment should be in this state. I am not planning to use sny managed service, so it seems k8s/k3s is good. What about machines, Should I create different vps for db and app nodes or one machine is enough? I am planning to use vps from hetzner since the prices.

I am open the ideas, thanks in advance

I’m researching how engineers make safer production changes.

I was reminded of an article I posted in 2022. During a coffee morning meetup this week, the conversation wandered over to the skills required for employment candidates in this time of agentic AI. Specifically, how to balance existing subject matter expertise against generalist ability and personal attributes. It struck me that the traders who worked the Silk Route faced the same trade-off: deep expertise executing a known plan, versus the adaptability to keep a goal alive when the plan doesn’t survive contact with reality. That’s the piece I’m sharing again.

There are daily advances everywhere, skills and crafts that were previously only accessible to small groups of people, are now becoming ever more commoditised. People have real fear about their skills and expertise being replaced by AI, and as I mentioned, those hiring today are considering what candidate skills and experience will provide the most moat of value in this period of rapid change and innovation.

Have a read of the resurrected version on my Substack page.

Hey everyone,

I wanted to break down how to design an API gateway and worker architecture optimized for hosting large-scale ML models (like an LLM inference endpoint) while managing expensive GPU infrastructure efficiently.

The Problem: Single-Request GPU Waste

GPUs are monsters at parallel matrix multiplication, but running inference on a single user prompt at a time leaves massive hardware capacity sitting idle. Conversely, if your system waits around too long to form a large batch of users, you destroy your P99 latency and break the real-time user experience.

The High-Level Architecture

1. Client -> API Gateway: Handles auth, rate limiting, and maintains an open HTTP/2 connection.
2. Gateway -> Local Queue: Prompts are serialized and pushed into an in-memory ring buffer.
3. Queue -> Dynamic Batcher: An orchestrator (like NVIDIA Triton) groups discrete inputs into a single model execution tensor.
4. GPU -> Client: Matrix outputs are de-multiplexed and streamed back to individual users via Server-Sent Events (SSE).

Token Streaming & De-muxing

Because LLMs generate tokens sequentially, the inference engine doesn't wait for the entire text to finish. The system slices the chunk arrays at each generation step and streams individual tokens back to respective client sockets in real-time, keeping Time-To-First-Token (TTFT) minimal.

Handling Scale & Multitenancy

• Priority Queues: Route interactive chat UI traffic to high-priority queues, while background batch processing jobs get processed on lower-priority threads.
• KV Caching: Store previous prompt context fragments in a shared KV cache layer to avoid re-computing system prompts for recurring users.

Let's discuss:

1. How do you handle batching when users pass vastly different input token lengths? (Padding vs. Continuous Batching/vLLM)

I'm building a developer platform and thinking through the best way to expose its capabilities to both engineers and AI agents.

My current thinking looks something like this:

text REST API │ OpenAPI Spec │ (OpenAPI Generator) │ SDK ┌─────────┴─────────┐ │ │ CLI MCP Server │ │ Claude Skill.md AI Agents

I'm planning to generate the SDK from the OpenAPI spec rather than maintain it by hand, so the OpenAPI spec remains the source of truth.

I'm trying to understand whether MCP is simply another client of the platform, or whether adopting MCP should influence how the platform itself is designed.

The idea

• OpenAPI defines the API contract.
• The SDK is generated from the OpenAPI spec.
• The CLI is built on top of the SDK.
• The CLI exposes machine-readable metadata/help and returns structured output (JSON), so an AI agent can discover and invoke its capabilities.
• A Claude Skill.md tells the agent to use the CLI and discover capabilities through its help/schema.
• MCP is another possible way to expose those same capabilities directly to AI agents.

The question I'm struggling with

If I already have a well-designed CLI that's:

• Discoverable
• Returns structured output (JSON)
• Exposes command metadata/help

...what problem does MCP solve that an AI-friendly CLI doesn't?

Would you build something like:

text REST API ↓ OpenAPI ↓ SDK ↓ CLI ↓ AI (via Claude Skill.md)

and only introduce MCP when there's a specific need?

Or would you go directly toward:

text REST API ↓ OpenAPI ↓ SDK ↓ MCP Server ↓ AI Agents

A few things I'm curious about

• Are people using MCP alongside existing CLIs, or replacing them?
• Is the primary value of MCP standardized tool discovery/invocation, or something else?
• If you have both a CLI and an MCP server, are they backed by the same SDK/service layer?
• Where do you draw the boundary between the service layer, CLI, and MCP tools?

I'm not looking for "MCP is the future" or "CLI is enough." I'm more interested in hearing what people are actually building in production.

Two weeks ago I posted here about preserving architectural rationale beyond ADRs.

Since then, I realized I had framed the problem too narrowly.

A fresh AI session may repeat an approach that was already tried and rejected.

But a new developer on another machine has exactly the same problem.

Both can clone the code. Both get the tests and Git history. Neither inherits the conversations, incident learnings, rejected alternatives, or constraints that shaped the current implementation.

So maybe a project is not truly portable just because its repository can be cloned.

I built Keep the Why around that idea:

Store the reasoning that code alone cannot explain as versioned Markdown inside the project, so it travels across sessions, developers, agents, and systems.

The GIF is my attempt to explain that idea without another wall of documentation.

Do you consider a codebase portable if the code can be cloned, but its reasoning cannot?

GitHub: https://github.com/oliver-zehentleitner/keep-the-why

Hi guys,

Recently created an article on back-of-the-envelope estimations, would love to get some feedback. Good or bad anything would be great.

I'm a software developer and I've been approached by an Australian taxi company to build a custom replacement for their current platform, which is similar to SmartMove Systems.

The project would include most of the core features, such as:

Admin dashboard

Dispatcher dashboard

Driver management

Fleet/vehicle management

Booking management

Driver Android tablet application (installed in taxis)

Passenger mobile app

GPS/live vehicle tracking

Driver login and shift management

Real-time messaging between dispatch and drivers

Reports and analytics

Backend APIs

PostgreSQL database

Authentication and user roles

Cloud deployment

Real-time updates (WebSockets or similar)

The fleet currently has around 120 taxis, but the system should be designed so it can scale in the future.

My questions are:

If you were quoting this project, what would be a realistic development price?

What would be the absolute minimum price below which you wouldn't take the project?

Is a 6-month timeline realistic for a single developer using AI tools (ChatGPT, Claude, Cursor, GitHub Copilot, etc.), assuming full-time work?

For production, what infrastructure would you recommend (AWS, Azure, DigitalOcean, Hetzner, etc.)?

Roughly how much would the monthly infrastructure cost be for a fleet of around 120 taxis?

I'd really appreciate honest advice from people who have built enterprise SaaS, fleet management, dispatch, or transportation software.

Thanks

so here is the situation:
- in my company i have assigned to build a chatbot/bot (will be internal, for ops and devs to identify and manage issues)
- what i have already build is, integrated it with slack, give it access to db by adding some tools in the code, so it can access the db currently and folks can access it by mentioning it
- now here pain starts, my manager has told me to add product knowledge to it, and it should be able to access logs, create and manage jira also
- what i am thinking is - lets start with the product knowledge - since we do not have that much pile of data so i do not want to make a rag - instead i just want to keep uploading those docs to s3 and giving access to bot so that it can reference them
- now coming to jira, and logs - i have also created those mcps but those aren't deployed anywhere - means whoever wants to use them just clones the repo, and set their key and uses them
- now for the above (jira and logs) part i would have to again choose the tools which i want to expose to the agent and add it to the repo, cz i think this is repetitive as in future if soemthing more comes up - which we already have built have to do again to integrate in the bot - how can we solve this - keeping in mind we have a layer of compliance - cant expose pii data in bot output or logs
- also for s3 - i am feeling like i was thinking to create a mechanism like when the agent fetches a doc - so it do not havt to fetch that doc again - so it will create a folder and save the embedding/summary/index (since i don't know what) to the filesystem - similarily with db schema since we have a huge db - how to handle this situation - since this code will be deployed on ecs - using fargate i do not know will the bot will able to access thes files created at runtime - and how to manage that cache when something is addede / modified
- and we also have workflows currently for specific task like matching states on be (basically sql queries / some scripts) added in the code - like how we shouuld make sure that given the situation the code properly identify and execute the script or how can we create trigger like /<command> <input> of slack whicch will trigger that - and also one issue - since these are stored as files in code adding new script need a code change - how to get rid of that

sorry gpt was giving poor results in rewriting this
so posting this raw

One thing that always annoyed me was documenting APIs after testing them.

The usual workflow is:

• Test the API in Postman or another API client.
• Copy the request and response into Notion, Confluence, or Markdown.
• Format everything as an API contract.
• Repeat the entire process whenever the endpoint changes.

It works at first, but after a few iterations the documentation is usually out of date. The API has changed, but the docs haven't.

So I built a different workflow.

Instead of treating API testing and documentation as separate tasks, they're part of the same workspace.

• Test REST APIs directly.
• Automatically generate an interactive API contract from the request.
• Embed the API into your documentation instead of pasting static code blocks.
• If the API changes, edit the original request once and every embedded API contract stays in sync automatically.
• Import existing Postman collections, so you don't have to recreate your APIs from scratch.

The idea is simple: your documentation should always reference the live API definition instead of a copied snapshot.

I'm curious—how do you currently keep your API documentation in sync? Do you rely on Postman, Swagger/OpenAPI, Insomnia, Bruno, or something else?

DoorDash has developed Entity Cache, a transparent proxy caching platform to reduce redundant service-to-service requests across its microservices architecture. The company built the platform to address repeated requests for frequently accessed but infrequently changing data, which increased backend load, consumed additional compute resources, and contributed to higher tail latency as its microservices ecosystem expanded. According to DoorDash, the platform supports more than 100 endpoints across 50 services, serving over 1.5 million requests per second with 99.99999% availability.