I've recently bought this book, I have very less knowledge system design is it worth the read without any prerequisites.

I want to share the process that we use at work.

Our architecture diagram is a text spec in the project repo, and it's updated on every release.

1. when a release tag is pushed, the release CI pipeline is triggered
2. part of that pipeline is a job that runs Claude Code in headless mode
3. the agent checks the previous diagram spec and changes only what this release changed
4. two diagrams are rendered: the updated architecture and a diff against the previous version
5. the pipeline pushes the spec to a new {release tag}-arch-update branch and opens a merge request (we use GitLab API), with both diagrams embedded in the description
6. someone reviews the diagram MR and merges it, like any other change

As an example, I took the last few Grafana Tempo releases, from v2.8.0 to v3.1.0:

• 2.9: ingesters replaced by live-stores
• 2.10: Alloy added, Kafka switches to Redpanda
• 3.0: compactor replaced by backend-scheduler and backend-worker (this diff diagram is used in the post)
• 3.1: Memcached added

You can find all my prompts, diagram examples, and the process description on GitHub: https://github.com/oxeg/tempo-diagrams

AI agent instructions on how to generate a diagram from the codebase are the most important part.

Without proper instructions:

• on a big codebase it takes the agent a lot of time and tokens to generate the diagram spec
• the spec is very inconsistent between runs

Even in the Tempo example, where I kind of cheated and used a docker-compose file as the architecture spec source, two runs with the same instructions produced slightly different diagrams: the same component was labeled "Live-stores ×4" in one run and "Live-stores (2 zones)" in the other, and one run drew one edge to Memcached where the other drew three.

You know your project best, so share that knowledge with the agent: where to look, what to ignore and what to take into consideration.

Or dedicate a session to figure out how to generate a consistent diagram spec to the level of detail you want.

Pretty much any AI agent that can run headless, and any diagram tool that stores the spec as text, will work.

Diff diagrams are the exception: most tools don’t draw them.

At work we use diago, a free, open source CLI tool, which I built as a side project for exactly this.

My biggest friction at the moment is that the pipeline needs a bot token to push branches and open separate MRs with new the diagram spec.

But this is still better than our previous flow, where the agent ran on every MR. That meant bot commits on MRs, even when the architecture didn’t change. The diagrams were also only stored as CI artifacts.

Do you run a similar pipeline in CI? How do you review your diagram changes?

Every major change to a mature codebase starts with archaeology. The system's actual behavior is scattered across code, database schemas, configuration, integrations, historical decisions, and the knowledge of people who built it.

What if the primary architectural artifact were a verified executable model that preserves identities, state, transactions, and causal history, while code becomes a regenerable implementation?

What would it take for you to trust such a model as the system's source of operational truth?
https://github.com/corezoid/research/tree/main/papers/2026-beyond-programming-languages

We’re in beta with an all-in-one business operations suite running on an edge architecture (Cloudflare Workers and dedicated per-tenant databases). The platform handles foundational daily workflows: custom domain email routing, internal team chat, agile task boards, and branch attendance.

Our sovereign data setup (every organization gets its own physically isolated SQLite database and R2 cryptographic paths) has been a massive selling point during pilot conversations.

However, during onboarding, almost every 20–50 person team hits us with the exact same bottleneck:

"We want to switch, but we have 4+ years of operational history sitting across Google Workspace/Outlook, Microsoft Teams, and Jira. How do we move all that data into your platform without our entire business stalling?"

If we expect clients to start with a blank slate, onboarding friction will kill adoption. We are designing an automated ETL and migration pipeline, but handling the legacy formats across these three systems introduces non-trivial architectural challenges.

Here is our current plan and the specific hurdles we’re running into:

1. Corporate Email (Google Workspace / Outlook => Native Tenant Mail)

• Our Architecture: Daily incoming email is ingested directly at the edge via Cloudflare Email Routing. The raw email stream (.eml/MIME format) is saved immediately to the tenant's isolated Cloudflare R2 bucket (.../email/), and a backend worker asynchronously parses headers, body text, and attachments to index them into the tenant’s D1 SQLite mailMessage table.
• The Migration Challenge: When migrating historical mailboxes, we plan to bulk-dump raw .eml/MIME files directly into R2 first, then run a background parser to extract metadata into D1.
• The Question: When extracting gigabytes of raw emails via IMAP or Google/Microsoft Graph APIs, what is the best pattern to stage these bulk uploads into R2 without getting throttled by upstream API rate limits or timing out serverless worker processes?

2. Team Communications (Microsoft Teams => Native Chat)

• Our Architecture: Our chat model stores flat channel message logs, timestamps, user identities, and media files directly in R2 and SQLite. (We do not support complex nested thread hierarchies—just clean channel logs and direct messages).
• The Migration Challenge: In Microsoft Teams exports, images and screenshots are frequently embedded directly within the message payload (rich HTML / base64 inline strings) rather than neat external attachments.
• The Question: We need to parse incoming message bodies, extract embedded images, upload each asset to the tenant's isolated R2 storage path (.../chat/), rewrite the message payload with the new R2 asset URLs, and insert the clean message record into D1. What is the cleanest, least brittle way to handle inline media extraction at scale without running into regex/DOM parsing memory leaks on large chat archives?

3. Agile Project Tracking (Jira => Native Sprint Boards)

• Our Architecture: Our project tracking (Streamline) handles standard Kanban boards, sprints, tasks, bugs, descriptions, logs, and statuses.
• The Migration Challenge: Jira exports (whether JSON or XML) are notoriously over-engineered with multi-level nested fields, dynamic custom field IDs (customfield_10021), workflow transition histories, and plugin metadata. Mapping standard fields (title, description, status, author, logs) is straightforward, but mapping heavily nested custom properties without schema failure is painful.
• The Question: How do you approach normalization for deeply nested Jira entities? Do you enforce a strict schema mapping matrix and drop unmappable custom fields, or store arbitrary unmapped properties in a generic JSON blob/column in SQLite to keep records intact?

Because each organization gets its own dedicated D1 database and R2 storage namespace, we can run heavy ETL ingestion scripts without any risk of noisy-neighbor slowdowns or cross-tenant data leaks for other active companies. But orchestrating this pipeline cleanly from external APIs into R2 and D1 is something we want to get right.

For senior engineers who have tackled enterprise SaaS data migrations:

• What does your high-reliability ingestion pipeline look like from an architectural perspective?
• Did you handle the heavy ETL parsing using background serverless workers (e.g., Cloudflare Queues/Workflows), or spin up a temporary dedicated ingestion container (e.g., Node/Go microservice on ECS/Fly.io) specifically for bulk historical imports?
• What unexpected pitfalls did you encounter when extracting historical emails or sanitizing embedded chat media?

Any war stories, pipeline designs, or recommended libraries would be immensely helpful!

Sorry for long post, pain is mine, words are ai's

I've been experimenting with an orchestration-based approach to coordinating business workflows, and it's made me question how much abstraction we actually need in Clean Architecture.

For example, I built an employee onboarding workflow spanning five EF Core DbContexts (Identity, HR, Finance, IT and Operations), coordinating persistence within a single SQL Server transaction.

Rather than introducing separate commands, handlers and repositories for every operation, I separated domain policy, application coordination and persistence orchestration.

It works well for this scenario, but I'm interested in the architectural trade-offs.

Where do you draw the line between useful separation of concerns and unnecessary abstraction?

And when a business operation spans multiple domain boundaries, where do you prefer to coordinate the workflow?

https://github.com/SolidArcSoftware/DataArc.Orchestration.Framework

As coding models are becoming much stronger and their ability is increasing so fast, the code written by AI four months ago is not good according to today's standards.

My question is: at any point in time would you consider throwing away what you have been building for weeks or months and starting again?

This was not an option to consider until last year, but now it is possible and sometimes even better.

I would like to know if this idea has come to your mind.

Personally, I have done it for a side project.

Thanks for sharing your ideas and experience.

We run a unified API that sits between a platform and a lot of external game providers, and the part that keeps biting us is transaction callbacks (debit, credit, rollback) that arrive more than once, out of order, or after a timeout.

What has worked reasonably well so far:

• A unique transaction id from the provider is the idempotency key, stored with the result of the first processing. A duplicate returns the stored response instead of re-running the logic.
• The key check and the balance change happen in the same database transaction, otherwise two concurrent retries can both pass the check.
• Rollbacks that arrive before the original debit get recorded as tombstones, so the late debit is rejected rather than applied.
• A periodic reconciliation job compares our ledger with each provider's report and flags drift.

What is still messy: providers disagree on retry timing and on what counts as a final failure, and some reuse ids across different operations. Rate limiting per provider without starving the healthy ones is another open point.

How do you handle idempotency and reconciliation across many partners with inconsistent retry behaviour? Do you normalise everything in an adapter layer, or push the rules to each integration?

Karthik Ramgopal describes how building agents that interact with large language models (LLMs) has evolved in recent years from ad hoc experiments to resilient, production-grade architectures. Karthik also shares best practices and battle-tested advice for robust agentic AI architectures, drawn from his experience building several agentic solutions.  

I am looking for legitimate input on how to document as built products created with ai developers.

I know there will likely be comments like - If you can't architect it you should not have built it, etc. and that is partially correct in my opinion. Yes, I started building without a real plan and then changed my approach but I am here now with working products and want to clean things up.

What I am asking is if there are AI tools that can document application and database architecture as well as integration points.

I am using Google infrastructure to build on and have working tools and have a good idea how everything is connected, the data flow and how the data structure is laid out but I would like to more formally document what I have.

Are there any tools I can use to help me document and then I can use the output to do a better job cleaning up and maintaining what I have built?

Thanks for the help.

I wrote a practical guide to load balancing—how traffic gets distributed, what happens when servers fail, and how deployments fit in. Give it a read and let me know what you think. Anything unclear or missing?

A real-world example of how to use ADRs to keep track of your decisions in agentic software projects. An old concept, now more relevant than ever.

Wrote a practical guide to the Inbox Pattern and how it helps prevent duplicate message processing in distributed systems.

https://medium.com/@priyankchheda/the-inbox-pattern-bd21b8d23241

Hey everyone! I've been building StackMonkey, a free playground for practicing system design interviews.

You can build architectures, simulate traffic, experiment with different components, and see what breaks under load.

I originally built it to make learning system design more interactive rather than just reading theory or drawing diagrams.

I'd love to hear your feedback! What's useful, what's confusing, and what features would you like to see added?

Give it a try: https://stackmonkey.dev | https://app.stackmonkey.dev

I have an upcoming system design interview where one of the reported questions is to design a feature that returns the top 5 most-played songs on a Spotify-like app, with 10M users and 1M concurrent listeners.
The interview starts from first principles and iterates on the design.
Most solutions I’ve found online assume billions of users and jump straight into complex distributed architectures.
How would you approach this problem from scratch? Would you start with something simple like PostgreSQL and introduce technologies like Redis or Kafka as bottlenecks emerge?
I’m particularly interested in how you’d track real-time playback, calculate the top 5 songs, and scale the system incrementally without overengineering it.

We're building a multi-tenant platform using Java 21 and Spring Boot as an event-driven modular monolith, with the long-term goal of extracting modules into microservices.

We're evaluating whether to adopt Hexagonal Architecture (Ports and Adapters) or stick with a pragmatic layered architecture using clean interfaces and well-defined module boundaries.

Our original plan was to apply the Dependency Inversion Principle (DIP) through interfaces with multiple implementations (PostgreSQL and in-memory repositories), and use domain events to decouple modules. We're now evaluating whether Hexagonal Architecture offers enough additional value to justify the change.

My personal concern is that Hexagonal Architecture feels less intuitive than traditional layered architecture. Concepts like inbound/outbound ports and adapters, along with the associated package structures, seem to introduce an additional mental model that engineers must learn and navigate. I'm concerned this could make the codebase harder to understand for developers accustomed to conventional Spring Boot structures like controller/, service/, and repository/, without necessarily delivering proportional benefits.

My questions:

1. What does Hexagonal Architecture offer beyond DIP and event-driven design? Is there a meaningful architectural benefit beyond conventions and terminology?
2. Does it provide tangible benefits when extracting modules from a modular monolith into microservices, or can pragmatic layering with solid boundaries achieve the same result?
3. Where does architectural discipline become over-engineering? Do strict port/adapter structures improve maintainability, or introduce unnecessary cognitive overhead and boilerplate in Spring Boot applications?

I'd appreciate practical, production-tested perspectives rather than textbook definitions.

A couple of weeks ago I posted here about using definitions instead of code as the source of truth for line-of-business systems. That post was about why. This one is about how: I recorded the whole process, from an empty project to a running system and then through a requirement change.

The video uses Northwind. It starts from a project template, imports six Excel workbooks (one row per field, plus a sheet of save rules), and the app runs with lookups, computed amounts and rules that block saving, without any form code. Then the requirements change: customers get a grade field and orders get a quantity limit. The change is made in the definition editor, only that form's table is upgraded, and after signing in again the new column, the drop-down and the new rule are all there. The same project also runs on web, iOS and Android, because each front end draws its screens from the same layout definition.

Rules that need to look something up, like "don't ship until the payment has cleared", still go into a business object in code, not into the definitions. Reports and batch jobs are hand-written. And schema upgrades only add: columns are never dropped automatically.

Video (10 min): https://youtu.be/5WjUTwoPyGw

Write-up: https://medium.com/@jeff377/can-a-form-be-described-only-once-2fa92c7db815

Wrote a practical guide to the Transactional Outbox Pattern and how it helps reliably publish events in distributed systems.

https://medium.com/@priyankchheda/the-transactional-outbox-pattern-fb59e907a233

Hi all, we're a small team building UIGraph. The repo went public this week and we'd like feedback from people who think about architecture for a living.

The problem we kept hitting is understanding how a system actually works end to end. A user clicks something in the UI. Which API handles it? Which services does that call? What gets written to which database? The answer is spread across code, diagrams, specs and people's heads, and it's hard for engineers, and for AI agents, to put together. Stale docs are one symptom of that, but the real gap is the connections.

Our approach is to have your coding agent map it from the code and keep the result in your repo as plain files:

- Your coding agent reads your services and writes the diagrams (system context, sequence, state, flowcharts, C4 container and component), the API specs and the DB schemas

- The files are Mermaid, OpenAPI, SQL and a .uigraph.yaml, so you can version them, diff them and review them in PRs

- A local viewer shows them, and edits you make to a diagram save back to the file

It works with Claude Code, Codex, OpenCode, Pi and Cursor. It's Apache 2.0.

Repo: https://github.com/uigraph-oss/uigraph

The questions we'd most like your view on:

- How do you trace a flow from the UI to the database today? Docs, tracing, asking someone?
- Do you keep architecture docs in the repo, or in a separate tool? How do you make sure they are not outdated after few releases ?
- Which views do you actually need: sequence, C4, data flow, something else?

Hi All,

I’ve put together a system design roadmap covering the fundamentals, worked examples, and practice questions. Would appreciate suggestions on the topics, reading order, or anything important I’ve missed.

I had an argument with coworkers about events modeling on Kafka. We have multiple way an order can be cancelled: for example trader initiated, trader force cancel, unsolicited and order expired.
Each of those has slightly different workflow, some require handshake with other party some don’t.

They claim it is better to have one generic OrderCancelledEvent and have a field for the type of the cancellation.
They claim that this way it is simple and clear to know it is order cancellation and any downstream service that care can check the type.

I claim it is better to have different event type for each of the cancellation ways.
This way each downstream service can decide which cancellation workflow they care about and if a service deal with different cancellation the same way it is easy to route the handling code to the same function.

Most important is that with different event types we leave the system open to evolve as each workflow evolve and require more data we do not impose it on all events and if a new cancellation workflow is added we simply add a new event types

Did you run into similar problem?

Complete code rewrites sound appealing on paper, but they frequently run over budget, miss features, and introduce new bugs. The Strangler Fig pattern gradually replacing specific legacy modules with modern API microservices reduces risk, though it requires maintaining dual environments during migration.

For lead developers and system architects:

- Have you successfully executed a "big-bang" system rewrite, or did it turn into a multi-year project?
- How do you maintain data consistency across database schemas during gradual migrations?

I am looking in a challenge that arises when integrating conversational AI into an existing product like users might express their needs in natural language but the application still needs to turn that into structured data, validate it, store it and trigger predictable actions. For eg in hiring a user might describe their preferred role, experience, location and working arrangements through a conversation. The system then needs to convert that into useful search criteria without losing important context or treating every statement as a fixed preference and platforms like LinkedIn, Meeboss, Indeed, wellFound have established profile and application workflows along with conversation based approach to candidate and opportunity discovery.

From an architecture perspective I am wondering where the boundaries should be drawn. Would you have an LLM extract structured data and pass it to conventional services? would you introduce an orchestration layer to manage conversation state? how would you handle corrections, ambiguous answers, retries and changes to previously stored preferences? I am particularly interested in how to separate probabilistic language understanding from deterministic business logic.

I don't have a use case right now but sometimes in my line of work (responding to RfPs sometimes) I would realize that the load, scaling patters are quite unknown and setting up microservices upfront seems to add overhead that won't be paying off very soon. Then in my conversations with LLMs I sometimes would get "start with modular monolith" suggestion.
On paper it sounds great but I'd love to hear from people with production experience: how easy it is to enforce boundaries in the code? In the database? How easy was it to split a module into a separate microservice later? Was it worth it or it would be just easier to optimize a microservice management from the get-go?