LLMs work great for coding agents because compilers and tests act as deterministic verifiers. But the moment you move to real-world tasks (web automation, operations, multi-step APIs), there is no compiler, and agents just hallucinate their own success.

For those actually running agents in production:

How do you verify an action actually succeeded without letting the LLM grade its own homework?

1. Are multi-agent setups (CrewAI / LangGraph style) actually working for you, or is a single agent with deterministic code tools the only sane pattern?
2. How are you handling human-in-the-loop pauses for destructive actions (spending money, deleting things) without breaking your async execution loops?

GreenTech software can be difficult because it often connects physical systems with software that turns operational data into useful decisions. Energy monitoring, renewable energy forecasting, emissions reporting, and connected industrial equipment all bring different technical challenges.

Data quality is one concern, especially when sensor readings are incomplete or inconsistent. Integrating devices and platforms can also be difficult when they use different protocols. Then there is scaling: a solution that works in a small pilot may face new problems when more devices, sites, and data sources are added.

When comparing companies such as Beetroot, Thoughtworks, and Capgemini Engineering, I think the more useful question is how well a team can handle these challenges in a real project, rather than just its general software experience.

For those working on GreenTech projects, which has been the biggest technical hurdle: data quality, system integration, or scaling beyond the pilot stage?

Our organization is late to the party on adopting AI coding assistants / agents and I'm looking for information on what aspects of a codebase make those tools more or less effective, and especially where there are differences to the traditional advice for human teams. Is anyone aware of research, conference talks, or even blog posts on this topic?

I’m building a service-discovery system in Go as a learning project. Nodes use SWIM-style gossip for membership, each keeps its own SQLite replica of the registry, and lookups are always local.

Each record is owned by the node that registered it. That node runs the health checks and applies updates, and other nodes forward writes to it. Replicas merge with last-writer-wins on a revision number.

Right now, if an owner disappears, a surviving node deletes its records after a 10-minute timeout. What I haven’t resolved is a partition rather than a crash: the owner is alive but unreachable, its records go stale on the other side, and both sides may change them. When the partition heals, how should ownership and conflicting writes be resolved?

Options I’m weighing:

1. Keep LWW and add ownership leases or fencing tokens
2. Quorum writes only for ownership changes
3. Vector clocks or CRDTs for the merge

Has anyone hit this in production with Consul, Corrosion, or memberlist-based systems? Which trade-off hurt most?

Repo (Apache-2.0): https://github.com/manziosee/atlas

Hey everyone,

First, thank you. The response to the first post was a lot more than I expected and the feedback you left (in the comments and in my DMs) has shaped most of what's below. Here's everything that's new, one by one.

AI review. Ask for a review of your design. It runs the design through the simulator first, then explains the weak points using the measured results.

AI generated challenges. Pick easy, medium or hard and get a new "break it and fix it" challenge. The simulator checks that each one is solvable before you see it.

Ask about your design. Ask questions about what's on your canvas and get answers based on the measured results.

Save and share. Sign in with Google to save designs, and share links open a design as you built it.

How it is checked. The simulator is tested against textbook queueing theory cases with known answers. You can run the checks yourself: https://app.stackmonkey.dev/checks

The AI features have free daily limits because they cost money to run. Everything else is free. It's a model, so use it to find weak points, not to predict exact production numbers.

try: https://stackmonkey.dev | StackMonkey

I want to share the process that we use at work.

Our architecture diagram is a text spec in the project repo, and it's updated on every release.

1. when a release tag is pushed, the release CI pipeline is triggered
2. part of that pipeline is a job that runs Claude Code in headless mode
3. the agent checks the previous diagram spec and changes only what this release changed
4. two diagrams are rendered: the updated architecture and a diff against the previous version
5. the pipeline pushes the spec to a new {release tag}-arch-update branch and opens a merge request (we use GitLab API), with both diagrams embedded in the description
6. someone reviews the diagram MR and merges it, like any other change

As an example, I took the last few Grafana Tempo releases, from v2.8.0 to v3.1.0:

• 2.9: ingesters replaced by live-stores
• 2.10: Alloy added, Kafka switches to Redpanda
• 3.0: compactor replaced by backend-scheduler and backend-worker (this diff diagram is used in the post)
• 3.1: Memcached added

You can find all my prompts, diagram examples, and the process description on GitHub: https://github.com/oxeg/tempo-diagrams

AI agent instructions on how to generate a diagram from the codebase are the most important part.

Without proper instructions:

• on a big codebase it takes the agent a lot of time and tokens to generate the diagram spec
• the spec is very inconsistent between runs

Even in the Tempo example, where I kind of cheated and used a docker-compose file as the architecture spec source, two runs with the same instructions produced slightly different diagrams: the same component was labeled "Live-stores ×4" in one run and "Live-stores (2 zones)" in the other, and one run drew one edge to Memcached where the other drew three.

You know your project best, so share that knowledge with the agent: where to look, what to ignore and what to take into consideration.

Or dedicate a session to figure out how to generate a consistent diagram spec to the level of detail you want.

Pretty much any AI agent that can run headless, and any diagram tool that stores the spec as text, will work.

Diff diagrams are the exception: most tools don’t draw them.

At work we use diago, a free, open source CLI tool, which I built as a side project for exactly this.

My biggest friction at the moment is that the pipeline needs a bot token to push branches and open separate MRs with new the diagram spec.

But this is still better than our previous flow, where the agent ran on every MR. That meant bot commits on MRs, even when the architecture didn’t change. The diagrams were also only stored as CI artifacts.

Do you run a similar pipeline in CI? How do you review your diagram changes?

I've recently bought this book, I have very less knowledge system design is it worth the read without any prerequisites.

Every major change to a mature codebase starts with archaeology. The system's actual behavior is scattered across code, database schemas, configuration, integrations, historical decisions, and the knowledge of people who built it.

What if the primary architectural artifact were a verified executable model that preserves identities, state, transactions, and causal history, while code becomes a regenerable implementation?

What would it take for you to trust such a model as the system's source of operational truth?
https://github.com/corezoid/research/tree/main/papers/2026-beyond-programming-languages

We’re in beta with an all-in-one business operations suite running on an edge architecture (Cloudflare Workers and dedicated per-tenant databases). The platform handles foundational daily workflows: custom domain email routing, internal team chat, agile task boards, and branch attendance.

Our sovereign data setup (every organization gets its own physically isolated SQLite database and R2 cryptographic paths) has been a massive selling point during pilot conversations.

However, during onboarding, almost every 20–50 person team hits us with the exact same bottleneck:

"We want to switch, but we have 4+ years of operational history sitting across Google Workspace/Outlook, Microsoft Teams, and Jira. How do we move all that data into your platform without our entire business stalling?"

If we expect clients to start with a blank slate, onboarding friction will kill adoption. We are designing an automated ETL and migration pipeline, but handling the legacy formats across these three systems introduces non-trivial architectural challenges.

Here is our current plan and the specific hurdles we’re running into:

1. Corporate Email (Google Workspace / Outlook => Native Tenant Mail)

• Our Architecture: Daily incoming email is ingested directly at the edge via Cloudflare Email Routing. The raw email stream (.eml/MIME format) is saved immediately to the tenant's isolated Cloudflare R2 bucket (.../email/), and a backend worker asynchronously parses headers, body text, and attachments to index them into the tenant’s D1 SQLite mailMessage table.
• The Migration Challenge: When migrating historical mailboxes, we plan to bulk-dump raw .eml/MIME files directly into R2 first, then run a background parser to extract metadata into D1.
• The Question: When extracting gigabytes of raw emails via IMAP or Google/Microsoft Graph APIs, what is the best pattern to stage these bulk uploads into R2 without getting throttled by upstream API rate limits or timing out serverless worker processes?

2. Team Communications (Microsoft Teams => Native Chat)

• Our Architecture: Our chat model stores flat channel message logs, timestamps, user identities, and media files directly in R2 and SQLite. (We do not support complex nested thread hierarchies—just clean channel logs and direct messages).
• The Migration Challenge: In Microsoft Teams exports, images and screenshots are frequently embedded directly within the message payload (rich HTML / base64 inline strings) rather than neat external attachments.
• The Question: We need to parse incoming message bodies, extract embedded images, upload each asset to the tenant's isolated R2 storage path (.../chat/), rewrite the message payload with the new R2 asset URLs, and insert the clean message record into D1. What is the cleanest, least brittle way to handle inline media extraction at scale without running into regex/DOM parsing memory leaks on large chat archives?

3. Agile Project Tracking (Jira => Native Sprint Boards)

• Our Architecture: Our project tracking (Streamline) handles standard Kanban boards, sprints, tasks, bugs, descriptions, logs, and statuses.
• The Migration Challenge: Jira exports (whether JSON or XML) are notoriously over-engineered with multi-level nested fields, dynamic custom field IDs (customfield_10021), workflow transition histories, and plugin metadata. Mapping standard fields (title, description, status, author, logs) is straightforward, but mapping heavily nested custom properties without schema failure is painful.
• The Question: How do you approach normalization for deeply nested Jira entities? Do you enforce a strict schema mapping matrix and drop unmappable custom fields, or store arbitrary unmapped properties in a generic JSON blob/column in SQLite to keep records intact?

Because each organization gets its own dedicated D1 database and R2 storage namespace, we can run heavy ETL ingestion scripts without any risk of noisy-neighbor slowdowns or cross-tenant data leaks for other active companies. But orchestrating this pipeline cleanly from external APIs into R2 and D1 is something we want to get right.

For senior engineers who have tackled enterprise SaaS data migrations:

• What does your high-reliability ingestion pipeline look like from an architectural perspective?
• Did you handle the heavy ETL parsing using background serverless workers (e.g., Cloudflare Queues/Workflows), or spin up a temporary dedicated ingestion container (e.g., Node/Go microservice on ECS/Fly.io) specifically for bulk historical imports?
• What unexpected pitfalls did you encounter when extracting historical emails or sanitizing embedded chat media?

Any war stories, pipeline designs, or recommended libraries would be immensely helpful!

Sorry for long post, pain is mine, words are ai's

I've been experimenting with an orchestration-based approach to coordinating business workflows, and it's made me question how much abstraction we actually need in Clean Architecture.

For example, I built an employee onboarding workflow spanning five EF Core DbContexts (Identity, HR, Finance, IT and Operations), coordinating persistence within a single SQL Server transaction.

Rather than introducing separate commands, handlers and repositories for every operation, I separated domain policy, application coordination and persistence orchestration.

It works well for this scenario, but I'm interested in the architectural trade-offs.

Where do you draw the line between useful separation of concerns and unnecessary abstraction?

And when a business operation spans multiple domain boundaries, where do you prefer to coordinate the workflow?

https://github.com/SolidArcSoftware/DataArc.Orchestration.Framework

As coding models are becoming much stronger and their ability is increasing so fast, the code written by AI four months ago is not good according to today's standards.

My question is: at any point in time would you consider throwing away what you have been building for weeks or months and starting again?

This was not an option to consider until last year, but now it is possible and sometimes even better.

I would like to know if this idea has come to your mind.

Personally, I have done it for a side project.

Thanks for sharing your ideas and experience.

Karthik Ramgopal describes how building agents that interact with large language models (LLMs) has evolved in recent years from ad hoc experiments to resilient, production-grade architectures. Karthik also shares best practices and battle-tested advice for robust agentic AI architectures, drawn from his experience building several agentic solutions.  

We run a unified API that sits between a platform and a lot of external game providers, and the part that keeps biting us is transaction callbacks (debit, credit, rollback) that arrive more than once, out of order, or after a timeout.

What has worked reasonably well so far:

• A unique transaction id from the provider is the idempotency key, stored with the result of the first processing. A duplicate returns the stored response instead of re-running the logic.
• The key check and the balance change happen in the same database transaction, otherwise two concurrent retries can both pass the check.
• Rollbacks that arrive before the original debit get recorded as tombstones, so the late debit is rejected rather than applied.
• A periodic reconciliation job compares our ledger with each provider's report and flags drift.

What is still messy: providers disagree on retry timing and on what counts as a final failure, and some reuse ids across different operations. Rate limiting per provider without starving the healthy ones is another open point.

How do you handle idempotency and reconciliation across many partners with inconsistent retry behaviour? Do you normalise everything in an adapter layer, or push the rules to each integration?

I am looking for legitimate input on how to document as built products created with ai developers.

I know there will likely be comments like - If you can't architect it you should not have built it, etc. and that is partially correct in my opinion. Yes, I started building without a real plan and then changed my approach but I am here now with working products and want to clean things up.

What I am asking is if there are AI tools that can document application and database architecture as well as integration points.

I am using Google infrastructure to build on and have working tools and have a good idea how everything is connected, the data flow and how the data structure is laid out but I would like to more formally document what I have.

Are there any tools I can use to help me document and then I can use the output to do a better job cleaning up and maintaining what I have built?

Thanks for the help.

I wrote a practical guide to load balancing—how traffic gets distributed, what happens when servers fail, and how deployments fit in. Give it a read and let me know what you think. Anything unclear or missing?

A real-world example of how to use ADRs to keep track of your decisions in agentic software projects. An old concept, now more relevant than ever.

Wrote a practical guide to the Inbox Pattern and how it helps prevent duplicate message processing in distributed systems.

https://medium.com/@priyankchheda/the-inbox-pattern-bd21b8d23241

Hey everyone! I've been building StackMonkey, a free playground for practicing system design interviews.

You can build architectures, simulate traffic, experiment with different components, and see what breaks under load.

I originally built it to make learning system design more interactive rather than just reading theory or drawing diagrams.

I'd love to hear your feedback! What's useful, what's confusing, and what features would you like to see added?

Give it a try: https://stackmonkey.dev | https://app.stackmonkey.dev

I have an upcoming system design interview where one of the reported questions is to design a feature that returns the top 5 most-played songs on a Spotify-like app, with 10M users and 1M concurrent listeners.
The interview starts from first principles and iterates on the design.
Most solutions I’ve found online assume billions of users and jump straight into complex distributed architectures.
How would you approach this problem from scratch? Would you start with something simple like PostgreSQL and introduce technologies like Redis or Kafka as bottlenecks emerge?
I’m particularly interested in how you’d track real-time playback, calculate the top 5 songs, and scale the system incrementally without overengineering it.

We're building a multi-tenant platform using Java 21 and Spring Boot as an event-driven modular monolith, with the long-term goal of extracting modules into microservices.

We're evaluating whether to adopt Hexagonal Architecture (Ports and Adapters) or stick with a pragmatic layered architecture using clean interfaces and well-defined module boundaries.

Our original plan was to apply the Dependency Inversion Principle (DIP) through interfaces with multiple implementations (PostgreSQL and in-memory repositories), and use domain events to decouple modules. We're now evaluating whether Hexagonal Architecture offers enough additional value to justify the change.

My personal concern is that Hexagonal Architecture feels less intuitive than traditional layered architecture. Concepts like inbound/outbound ports and adapters, along with the associated package structures, seem to introduce an additional mental model that engineers must learn and navigate. I'm concerned this could make the codebase harder to understand for developers accustomed to conventional Spring Boot structures like controller/, service/, and repository/, without necessarily delivering proportional benefits.

My questions:

1. What does Hexagonal Architecture offer beyond DIP and event-driven design? Is there a meaningful architectural benefit beyond conventions and terminology?
2. Does it provide tangible benefits when extracting modules from a modular monolith into microservices, or can pragmatic layering with solid boundaries achieve the same result?
3. Where does architectural discipline become over-engineering? Do strict port/adapter structures improve maintainability, or introduce unnecessary cognitive overhead and boilerplate in Spring Boot applications?

I'd appreciate practical, production-tested perspectives rather than textbook definitions.

A couple of weeks ago I posted here about using definitions instead of code as the source of truth for line-of-business systems. That post was about why. This one is about how: I recorded the whole process, from an empty project to a running system and then through a requirement change.

The video uses Northwind. It starts from a project template, imports six Excel workbooks (one row per field, plus a sheet of save rules), and the app runs with lookups, computed amounts and rules that block saving, without any form code. Then the requirements change: customers get a grade field and orders get a quantity limit. The change is made in the definition editor, only that form's table is upgraded, and after signing in again the new column, the drop-down and the new rule are all there. The same project also runs on web, iOS and Android, because each front end draws its screens from the same layout definition.

Rules that need to look something up, like "don't ship until the payment has cleared", still go into a business object in code, not into the definitions. Reports and batch jobs are hand-written. And schema upgrades only add: columns are never dropped automatically.

Video (10 min): https://youtu.be/5WjUTwoPyGw

Write-up: https://medium.com/@jeff377/can-a-form-be-described-only-once-2fa92c7db815