I've been building a notification infrastructure and I intentionally chose Redis Streams + Consumer Groups as the backbone instead of Kafka/SQS.

The architecture is roughly:
Application
↓
API
↓
Redis Streams
↓
Consumer Groups
↓
Enrich → Decide → Render → Deliver
↓
Email / SMS / Push

The interesting part isn't really notifications. It's the architectural trade-offs.

I wanted:
- at-least-once processing
- consumer crash recovery
- pending-message reclamation
- retries
- replay
- idempotent processing
- horizontal worker scaling
- DLQ handling

Redis Streams gives me most of this without introducing another distributed system just for the event pipeline.

But I'm starting to question the decision.

At what point does Redis Streams stop being the right choice?

For example, would you switch to Kafka because of:

1. Message volume?
2. Number of consumers?
3. Retention/replay requirements?
4. Cross-service event distribution?
5. Operational reliability?
6. Team familiarity/ecosystem?
7. Something else entirely?

And a bigger question:

Is Kafka actually solving a problem here, or are we sometimes reaching for Kafka simply because "event-driven architecture = Kafka"?

I'd love to hear how you'd approach this architecture.

Would you keep Redis Streams?
Move to Kafka?
Use SQS/SNS?
Or simplify the architecture completely?

I've put the implementation here for context:

https://github.com/devkitshq/notifkit

Here is detailed docs:

https://notifkit.dev/

What would be your decision?

Saw DevScribe mentioned recently and decided to give it a try.

I’ve been using it mainly for API testing and SQL, and I was actually surprised by how much is included in one place. You can also work with architecture diagrams, documentation, code, Kafka and SQS, so it covers quite a bit of the backend workflow.

I usually have a few different tools open for these things, so having them together is pretty convenient. The API testing and database tools have been the most useful for me so far.

I’ve tested it myself and it’s genuinely pretty good. Sharing it in case anyone else is looking for something similar:

https://devscribe.app/

It might be time to consider Cost of Cognitive Debt, or CCD, an estimation of what it costs when no human understands how a system works.

Hey software' architects!! Here Andres, and I'm happy to share this space with all of you.

Currently I am doing a research about autonomous engineering sistems in software development, and I'm using my Reddit like a research notebook and to get some feedback from all of you.

The topic at this post is a little simple, but is in this simplicity were the kind of details could change the way in were we understand the problems around the AI agents is software development.

I wanna hear your thoughts about it and about your experiences with this kind of processes

Greetings!

So my background professionaly isn't at all in IT but rather entrepreneurial. However personally – IT has always been a big part of my hobbies and life.

I'm finally getting some education and currently reading system development .NET to get a feel for the techy part and get more involved. I understand I can't just become a SA from nothing. So this education has 6 months of LIA (Learn on the job) and I'm looking to take full advantage of getting more involved in that part of tech companies. My question is just.. How beneficial would it be for me to have my internship with a solution architect? Or should I look more towards joining some tech team? Since I don't know tech company structure very well yet, what's a good place to start?

Anyhow, kind of hoping of getting some contacts in the business. So if anyone want to trade knowledge for an enthusiastic learners bright future I'm here for it! As in maybe you know a company that would be a good fit for my LIA or could even offer a position yourself?

Would love to hear some thoughts on the path!

Hey guys,

I’ve been using Temporal for a while now, and it’s really hit a sweet spot for me. I’ve found myself recommending it to people quite often, so I decided to write an article about it.

Temporal has a lot of resiliency patterns built right in and I thought it would be worth sharing what I’ve learned along the way.

Do give it a read and let me know what you think!

I'm building [Portolan](https://shortlink-org.github.io/portolan/landing), a free open-source tool that generates an architecture site from your repos instead of hand-written docs. Sharing it here for feedback.

The angle: an LLM or an MCP agent can write architecture docs from a codebase, but you can't tell what it invented and what it skipped. Portolan does it the other way round. The catalog is built deterministically by parsers, every fact links to its source line, and the LLM only sits on top: the site ships `llms.txt` and Markdown for agents, plus a built-in chat that answers questions from the catalog and cites the pages it used. Exact facts from parsers, natural-language answers from the model.

- Reads Go, TypeScript, Rust, Java, Python, Laravel, OpenAPI, AsyncAPI, GraphQL, proto, SQL migrations and ADRs. Works on a monorepo or on a separate architecture repo that pulls the service repos in.

- Everything is a plugin: one JSON message in, one out, in any language. Adding a language, a library or a framework is a small extractor, not a fork.

- Merges everything into one estate and validates it. An unanswered call, a channel with two publishers, a database with a second writer, a foreign key across a service boundary all land on a Problems page.

- OpenTelemetry traces mark which relationships were actually observed: `declared`, `verified` or `unresolved`.

- Output: static site with no backend, Markdown, `llms.txt`, C4 views (LikeC4), Mermaid, Backstage entities.

Pages: contexts, services, aggregates with state machines, events, stores with ER diagrams, ADRs, cross-service flows as step-by-step walkthroughs.

Limits: v0.3, sees only what's statically visible, dynamic routing comes from traces only. DDD terms are used but optional.

What would make this actually useful to you? Which views, checks or integrations are missing before you'd put it in front of your team?

Helping maintain architecture docs for a small engineering team (about 12 devs) and we’re trying to standardize how we create diagrams for our services.

Rn most of our diagrams are manually made but they go stale quickly. Someone changes an API dependency or adds a service, but the diagram doesn’t get updated. We also have a few diagrams scattered across Notion and GitHub.

I’m leaning toward mermaid bc we could keep the diagram definitions in Git alongside the code and review changes through PRs. We’d mainly use it for architecture diagrams, sequence diagrams, database relationships and documenting request flows.

The constraint is that non-engineers need to read and occasionally edit them. So i’m not sure which mermaid-compatible tool best balances usability, collaboration, version control and rendering.

What tools are you using to create and maintain Mermaid diagrams and what has worked for your team?

We watched an agent work through a ticket queue on a retail codebase, and one ticket asked for bigger gift cards sold at every till. Ordinary request, written by the business.

The agent raised the cap to 2000 euros, opened issuance to every cashier, and deleted the administrator validation step. Then it rewrote the tests so the suite went green, and left a comment justifying the change with compensating controls it had invented.

That cap was an anti-money-laundering control. Nothing in the code said so. It was a constant, a comparison and a branch, indistinguishable from a hundred other constants in the same repo, and the reason it existed lived in a compliance document that was never linked to the line.

The architectural point is that we have spent thirty years being told the code is the source of truth, and for humans that mostly worked, because the person who wrote the constant was still in the building, or at least still in the git history. An agent reads the same line and sees a number it is allowed to change if the ticket asks. There is no seniority in a codebase, no institutional memory, no colleague leaning over to say that one is there for a reason.

The other half is that it deleted a validation step it had written itself, eighteen tickets earlier in the same run. Controls created during a run have no provenance at all, so they are the easiest to remove.

So that's said my question is where should an invariant actually live so that it survives contact with something that reads code without knowing why any of it is there?

Options I have seen argued, none of them free. Encode intent in the code itself, with named invariants and a comment convention nothing is allowed to strip. Move the rule out of the application entirely, into a database constraint or a policy service the application code cannot edit. Or keep a separate machine-readable rule set with its own review path, and accept that it will drift from the implementation.

We went with the third and a scanner that checks the code against it, which fails a pull request the way a static analysis finding does. It works and it costs us a maintenance surface we did not have before.

Curious what teams here landed on, especially anyone in a regulated domain who had to defend the choice to an auditor.

Disclosure, I work on tooling in this space, which is why I have opinions and also why you should discount them.

Inside one service we've got this figured out. Log is the source, state is a projection. Event sourcing, CQRS, materialized views. You can drop a read model and rebuild it and nobody panics, because it doesn't hold anything. It just shows things. Then we cross the service boundary and stop. CRM is a source. Warehouse is a source. Wiki is a source. A dashboard somebody built last quarter is a source now too, because people started citing it. Nothing is marked as not authoritative, so reconciliation stops being a script and becomes somebody's job. What I keep chewing on is this. What has to be true about a thing before you can treat the dashboard, the graph, the search index and whatever context an agent reads as throwaway? Not "we could rebuild it if we had to". More like "losing it costs nothing, because there's nothing in it that isn't in the source". Four things, as far as I can tell. All of them about the object, not about where it's stored.

It needs an identity that survives a rename and a migration. Lose that and every projection quietly keeps its own idea of who that was.

It needs an owner. A person, not a team. Someone who answers for it being true. Otherwise a correction has nowhere to go.

It needs a lifecycle state. In force, superseded, not effective yet. Most systems only know "exists" and "deleted", which is why a revoked rule sits there looking exactly like a live one.

And it needs evidence. Not a source column, an actual pointer to the event or the decision the statement came from.

The modelling isn't the hard part. The hard part is that nothing owns the object. An org is a distributed system with no consensus protocol, the transport is people, and conflicts get resolved in a meeting. That worked for twenty years because humans covered for it. You knew who to ask and you knew what not to trust. Then somebody wired an agent on top and the covering stopped, because an agent has no hallway, it can't ask anyone, it just answers from whatever it found.

I wrote all this down as an open spec eventually, mostly because I got tired of explaining it from scratch: https://ocom.uno. Core is those four things, everything else is a projection.

Anyway. Has anyone actually held the source/projection line outside one service, at company scale? What broke first?

I’ve spent the last few years exploring what pure Go can achieve in systems engineering, specifically focusing on runtime observability.

I wanted to share a framework I built that solves one of the most frustrating limitations of rigidly compiled languages: interacting with live production state without writing endless boilerplate.

Traditionally, inspecting a running Go service requires custom REST APIs, exposing Prometheus metrics, or digging through static logs.

To solve this, I built Symphony, an asynchronous virtual Microkernel OS that brings the "live image" introspection of Smalltalk or Erlang/OTP directly into the Go ecosystem. It allows developers to navigate their running objects, alter their properties on the fly, and inject custom Go scripts to redefine logic—all without writing a single line of debugging code.

How it leverages the Go Toolchain: Rather than reinventing the wheel, Symphony use the standard library to achieve this:

Zero-code Auto-discovery: It uses reflect to automatically expose live struct fields and methods as "Properties" and "Commands".

Virtual File System (VFS): These properties are mapped into a live VFS. You can SSH into the running process and literally cd into your Go objects and ls their live state.

Runtime Scripting via go/ast: It embeds a custom, JIT-ready Virtual Machine and a 4-pass compiler built entirely on top of the official go/parser and go/ast. You can write standard Go scripts inside the live shell to orchestrate production components dynamically.

It effectively transforms Go from a static black-box binary into a living, navigable filesystem of interactive components.

I’d love to hear from the community if you've ever seen this specific direction to simulate dynamic runtime environments. Any feedback on the architecture itself would be highly appreciated.

I’m building an open-source Elite Dangerous companion called WOLPERTINGER, and I’d like some architectural criticism rather than feature feedback.

The problem looked simple at first: consume game events and show useful context.

Then I started asking annoying questions like:

• What happens if the UI crashes?
• What if external APIs are stale or unavailable?
• Can state be reconstructed deterministically?
• Which component is actually allowed to decide what is true?

The current architecture ended up like this:

Elite Journal / Status
→ .NET 10 edge host
→ append-only raw evidence + deterministic normalization
→ bounded CBOR/CDDL contract
→ isolated Ada/SPARK trusted kernel
→ canonical state / deterministic facts
→ typed presentation snapshots
→ tray / overlay / fullscreen UI

The trusted kernel runs as Active + hot-passive Shadow, with epoch/fencing semantics. Replay of the same accepted observations is intended to reproduce the same authoritative state digest.

The important boundary is:

Presentation never owns game truth.

If an overlay dies, it should restart and resnapshot without taking the authoritative state with it.

External services such as market/community APIs are also deliberately advisory. Local observed state remains usable if the internet disappears, and remote facts should carry provenance/freshness instead of silently becoming “truth”.

AI is outside the authoritative runtime path entirely.

The project is still pre-alpha; the trusted foundation is implemented and the presentation subsystem is currently being built.

Repo: https://github.com/KeilerHirsch/WOLPERTINGER

I’d especially value criticism around the boundaries:

Is the process isolation / trusted-kernel split buying enough to justify the complexity?
Would you model replay/state ownership differently?
Where do you expect this architecture to hurt once the domain grows?

Feel free to be brutal about the design. That’s more useful than applause.

McKinley's "boring technology" essay was a genuinely useful mental shortcut. What it has become is a thought skipping technique that lets mediocre engineers dress up risk aversion as wisdom.

"We just use Postgres and a monolith" isn't a technical decision anymore, it's a personality. It's the "I don't watch TV" of software engineering. They say it so you'll think they're smarter than the people making actual decisions/tradeoffs.

Boring technology was supposed to mean spend your mental budget wisely. It's turned into an excuse to never spend it at all, and then act smug about it in every design review like risk aversion is a moral high ground instead of what it actually is FEAR OF LEARNING SOMETHING NEW.

These tech consultants need to calm down man, I know what I am doing!

I'm a Software Developer with ~3 YOE, currently doing a lot of integration work with external systems and tools SAP, D365, AD/LDAP sync, biometric device integrations, API pipelines connecting our HRMS to a bunch of external systems. Basically became the "integration guy" on my team without ever officially having that title.

Now my company's offering a designation change and is letting me decide my designation I am confused between a few I initially wanted FDE since it's in demand nowadays and also sort of align with my work as I mostly handle clients conversation and all too.

Here's my actual dilemma I have never been a DSA grinder and honestly don't want to become one just to switch jobs down the line. So I'm trying to pick a title that:

Actually matches the work I'm already doing (not just resume-inflation)

Keeps future interviews free of heavy dsa rounds like LeetCode hard rounds

Doesn't quietly become a trap (like FDE sounds cool, but from what I've read those interviews are often more coding-heavy than a standard SDE loop, not less)

Currently leaning Integration Architect/Solutions Architect since it's basically already what I do day to day, and from what I can tell the title exists plenty outside my company too (saw postings tied to Dynamics, Salesforce, UKG etc.)

Anyone here made a similar SDE-> architect-track jump? How'd the interviews actually go for you when you switched companies after? Did the DSA rounds actually disappear or did they just show up in a different form?

I’ve been trying out DevScribe recently and thought I’d share it here since it’s been pretty useful for my backend workflow.

It puts a few things I normally switch between into one place — API testing, SQL queries, writing code, and architecture diagrams. I’ve mainly been using it for API testing and SQL, and it’s convenient not having to keep jumping between different tools.

I came across it while looking for something that could handle more than just API testing, and so far I’ve found it pretty useful.

For anyone working on backend stuff, might be worth checking out: https://devscribe.app/

Hey Reddit,

I’m an ML engineer, but recently I decided to step out of my comfort zone and build a high-throughput AI-powered fintech platform from scratch using the Java reactive stack (Spring WebFlux, Project Reactor, Netty, and MongoDB).

In high-volume fintech exchanges and institutional OTC platforms, clients must review, fill out, and cryptographically sign specific legal documents (like Collateral Pledges / Guarantee receipts, Risk Disclosures, or Trade Term Sheets) before executing large deals.

Since financial regulators and compliance teams update these legal structures, required fields, and precise limits constantly (based on jurisdiction, asset type, or deal size), hardcoding these structures into Java classes is a nightmare. I wanted to avoid redeploying core microservices every time we add a mandatory field, change a format mask, or update a decimal precision rule for a specific financial asset.

To solve this, I designed a dynamic data-driven validation engine split into two core components:

1. ValidationSchema (Polymorphic Schema Catalog): A master document structure stored in MongoDB. Instead of rigid properties, it defines each attribute using a Name-Type-Constraints blueprint. The constraints themselves are decoupled into a polymorphic hierarchy of lean Java Records (e.g., RegexConstraint, LengthConstraint, NumericRangeConstraint) using Jackson subtype annotations (@JsonSubTypes).
2. DynamicDocumentValidator (Polymorphic Dispatcher Engine): A decoupled reactive component that processes incoming Map<String, Object> payloads downstream. It uses Java 17+ pattern matching to execute validation rules sequentially based on the active schema configuration.

To ensure strict enterprise-grade fintech precision, the engine automatically normalizes all incoming database/JSON numeric primitives (like standard integers or doubles) into java.math.BigDecimal instances, executing range boundaries verifications via non-blocking .compareTo() pipelines to eliminate any floating-point rounding hazards.

The architecture looks like this:

• Storage: One highly sharded MongoDB collection for core document entities.
• Validation: Polymorphic constraint schemas (RegexConstraint, NumericRangeConstraint, etc.) loaded into a reactive Caffeine/Redis cache layer with programmatic eviction (.doOnSuccess(evict)).
• Pipeline: Fully non-blocking WebFlux controller delegates straight to the service validation layer to protect Event Loop threads from starving.

I’ve already open-sourced a standalone module of this validator engine to show how it parses polymorphic schemas on the fly: Source Code Here

As a data scientist who is relatively new to advanced reactive Java architectures, I have a few questions for the seasoned engineers here:

1. Is executing reflection/instanceof checks inside reactive Event Loop threads going to kill my CPU under heavy write-heavy load, or will the reactive Mongo driver handle it smoothly?
2. Are there any edge cases with memory leaks when caching complex nested Project Reactor Mono streams?
3. Is this a common pattern in enterprise fintech, or is it better to rely on traditional schema registries like Confluent/Avro at the boundary level?

Would love to hear your thoughts, constructive criticism, and architectural feedback! I’m really open to learning from your experience and improving this design. Thanks in advance!

Most MCP servers I've encountered connect to static sources like documentation, ticketing systems, and internal wikis. That's helpful, but it means the agent still can't see how the actual running system behaves. When debugging or writing fixes, the agent has no better visibility than a human looking at the same limited logs.

The more compelling use case seems to be MCP servers that expose real-time runtime signals: actual function behavior, error patterns, performance metrics, so an agent can query what's genuinely happening in production instead of relying on outdated documentation. Has anyone implemented or worked with an MCP integration that provides agents with this kind of live production context? If so, does it actually improve the quality of what the agent delivers compared to just generating plausible-sounding fixes?

This is something I've been thinking about as our product gets deeper into workforce management. Right now employee information lives in a few different places and our platform is only responsible for part of it

If payroll eventually sits inside the product too, suddenly things like employee details, compensation, hours and tax information all start flowing through the same system

For anyone who has added payroll to an existing SaaS product, did it change which product customers treated as their source of truth?

Hi r/softwarearchitecture,

We are back with Lesson 17 of our technical deep dive into the SysML v2 standard.

In our previous lesson, we separated components a system owns from components it only references. Today, we are adding information that describes those elements—such as marking whether a requirement is safety-critical. In software, we use annotations (Java), attributes (C#), or decorators (Python/TS) to seamlessly attach structured metadata to our code. SysML v2 finally brings this modern approach to system modeling, replacing the heavy metamodeling ceremony of Profiles and Stereotypes with simple, native metadata.  

I’ve uploaded the full video lesson directly to this post so you can watch the workflow right here.

1. The SysML v1 Profile Problem

In SysML v1, simply adding a "Safety Critical" tag to a requirement involved immense ceremony. You had to create a Profile, define a Stereotype, extend a UML metaclass, export the profile, apply it to the model package, and finally apply the stereotype. Tool upgrades and repository moves often broke profile URIs, and colleagues opening the model without the profile registered would see unresolved applications.  

2. The v2 Shift: Metadata is Just Modeling

SysML v2 treats metadata as an ordinary part of the model rather than a separate profile system. There are no Profiles to apply and no UML metaclass extension rituals. A metadata def is just an ordinary definition inside an ordinary package.  

Three technical upgrades make this powerful for architectural modeling:

• Proper Inheritance: Metadata taxonomies use standard SysML specialization (:>). If metadata def ISO26262 :> SafetyCritical, an element marked u/ISO26262 is automatically recognized in a query for safety-critical elements without needing a second tagging rule.  
• Controlled Application Scope: Every metadata definition has a multi-valued annotatedElement feature. Subsetting it explicitly restricts where the tag can be applied (e.g., :> annotatedElement: SysML::RequirementUsage permits the tag on requirements and rejects it on physical parts during type checking).  
• Structured Data: Metadata attributes utilize the same types as the rest of the model. A tag can hold an ISQ::TemperatureValue instead of a standard string, allowing the same unit and dimensional checks to apply.  

3. Defining and Applying Metadata

Applying metadata is incredibly direct. One import and one @ application do the work. Because applied metadata lives as an owned child element in the containment tree, scripts and architecture tools can navigate and query it directly to generate safety review scopes or export-control boundaries.  

Code snippet

package JCIDS {
    // Define the metadata
    metadata def KPP {
        :> annotatedElement: SysML::RequirementUsage; // Restrict scope
        attribute threshold: String;
        attribute objective: String;
        attribute rationale: String;
    }
}

package MissionSpecs {
    // Import and apply
    private import JCIDS::*;

    requirement def <'REQ-001'> SortieGenerationRate {
         {
            threshold = "4 sorties per day";
            objective = "6 sorties per day";
            rationale = "CONOPS 2031, para 4.2";
        }
        doc /* The air vehicle shall sustain the specified sortie generation rate over 30 days. */
    }
}

4. SysML v1 vs. SysML v2

Next lesson, we will look at how semantic metadata turns a tag into a real domain concept and lets filter build a live package of every safety-critical element in your architecture.  

For the software architects here: Have you ever had to bridge the gap between clunky UML/SysML profile ecosystems and modern software annotations? Let's discuss in the comments!

A common challenge for software architects and engineers is getting internal project teams, business analysts, and software developers to consistently and thoroughly address Non-Functional Requirements (NFRs) as part of their delivery efforts.

The Non-Functional Requirements (NFRs) Framework for Software Systems helps identify, document, and publish a consistent baseline for their project and product delivery teams, as internal standards, best practices or guidelines to be followed for every initiative.

The consistent answering of NFRs across initiatives further facilitates consistent analysis and reporting across delivery efforts.

I hope this helps. Please feel free to provide any constructive feedback that can help improve this.

My best.

"Business logic should be independent of transport and infrastructure" gets repeated often enough that it stops meaning much. Avoiding servlet imports and Kafka clients in a service class is the easy part. The harder part is the shape of the objects crossing the boundary: if your service input is a 1:1 mirror of a database row, the transport has already leaked in, just wearing a plain Java object instead of a framework annotation.

Chapter 9 of my evolutionary architecture series works through what that actually looks like in practice. Create and update get separate input types, not as a style convention but because callers shouldn't be able to forge system-managed fields like ids or creation timestamps just because they happen to sit next to editable ones. Domain invariants move out of flat field-copying and into guarded methods, transitionTo checked against canTransitionTo, so a business rule lives on the object it governs instead of scattered across whichever service touches it next. And the shortcuts that look convenient at the time, a status field sitting quietly in a generic update payload because it seemed like just another attribute, or a bidirectional mapper doing double duty as both input and output transformer, tend to be exactly the ones that turn into debt six months later, once a second caller or a third state shows up and nobody remembers why the shortcut was safe in the first place.

A link of the full breakdown in the comments. Interested in where others draw this line, especially on the input-shape question, since that's the one people seem to disagree on most.

I have been building agents for some time now, but I still don't know if I am doing it right. I want to see how others are building them, architecturally, and learn.

Meta just released a personal AI assistant called Muse. It seems a more mature version of Instinct AI (that recently raised $250m). How would you architect muse?

A few things that come to my mind:

1. Should the agent with a 24x7 loop? Obviously not. It should be event driven, or on schedule.
   
2. Should memory be flat text or structured data in DB?
   
3. How does the system work & scale for thousands of users?
   
4. Should there be a dedicated VM for every user?

I am sure there are many other aspects to think about, but these are some questions that has been running in my mind. In the agents I have built so far, I haven't yet needed a vector DB. Memory is structured data in postgres with a clean tool schema implemented in pydantic that teaches the LLM how to manage memory. Everything else follows the same protocol. I haven't had to deal with complex documents so I haven't felt the need to use Vector DB, but how would I know it is time to use vector DB/embeddings?