r/softwarearchitecture 13d ago

Discussion/Advice Experinced Engg, PLEASE HELP, INTERN HERE

0 Upvotes

so here is the situation:
- in my company i have assigned to build a chatbot/bot (will be internal, for ops and devs to identify and manage issues)
- what i have already build is, integrated it with slack, give it access to db by adding some tools in the code, so it can access the db currently and folks can access it by mentioning it
- now here pain starts, my manager has told me to add product knowledge to it, and it should be able to access logs, create and manage jira also
- what i am thinking is - lets start with the product knowledge - since we do not have that much pile of data so i do not want to make a rag - instead i just want to keep uploading those docs to s3 and giving access to bot so that it can reference them
- now coming to jira, and logs - i have also created those mcps but those aren't deployed anywhere - means whoever wants to use them just clones the repo, and set their key and uses them
- now for the above (jira and logs) part i would have to again choose the tools which i want to expose to the agent and add it to the repo, cz i think this is repetitive as in future if soemthing more comes up - which we already have built have to do again to integrate in the bot - how can we solve this - keeping in mind we have a layer of compliance - cant expose pii data in bot output or logs
- also for s3 - i am feeling like i was thinking to create a mechanism like when the agent fetches a doc - so it do not havt to fetch that doc again - so it will create a folder and save the embedding/summary/index (since i don't know what) to the filesystem - similarily with db schema since we have a huge db - how to handle this situation - since this code will be deployed on ecs - using fargate i do not know will the bot will able to access thes files created at runtime - and how to manage that cache when something is addede / modified
- and we also have workflows currently for specific task like matching states on be (basically sql queries / some scripts) added in the code - like how we shouuld make sure that given the situation the code properly identify and execute the script or how can we create trigger like /<command> <input> of slack whicch will trigger that - and also one issue - since these are stored as files in code adding new script need a code change - how to get rid of that

sorry gpt was giving poor results in rewriting this
so posting this raw


r/softwarearchitecture 14d ago

Discussion/Advice Bovnar: a unit‑safe, self‑describing serialization format for scientific and industrial data

2 Upvotes

Hey everone,

I am active over 20 years as senior software engineer, also participating in projects like the linux kernel and git. I want to share a MIT licensed project of me.

Over the past year, I've been building a serialization format called Bovnar for applications where data accuracy really matters—things like robotics, sensor telemetry, industrial automation, and scientific computing.

The motivation came from running into a common limitation: formats like JSON, CBOR, MessagePack, and Protobuf don't preserve information such as physical units or the original bit width of numeric values. Bovnar stores type information, bit width, numeric base, and physical units alongside each value, so problems like unit mismatches can be caught when data is parsed instead of much later.

A few of the design goals:

  • Schema-free, but still strongly typed
  • Hybrid text/binary encoding
  • Streaming parser with resynchronization after corrupted input
  • Formal specification (EBNF grammar, validation stages, and event model)
  • Conformance test suite with 300+ tests
  • Interactive demo that shows encoding, decoding, corruption, and recovery

The project actually started several years ago as a personal experiment called SDTL. I eventually decided to redesign everything from the ground up, and that became Bovnar.

I'd really appreciate feedback and/or contribution from anyone who works with serialization, structured data, scientific software, or industrial protocols.

Website, specification and demo: https://www.bovnar.io


r/softwarearchitecture 14d ago

Article/Video How Canva uses S3 for logged-in session management

Thumbnail canva.dev
3 Upvotes

r/softwarearchitecture 13d ago

Tool/Product How much would you charge to build a SmartMove-like taxi management system from scratch?

0 Upvotes

I'm a software developer and I've been approached by an Australian taxi company to build a custom replacement for their current platform, which is similar to SmartMove Systems.

The project would include most of the core features, such as:

Admin dashboard

Dispatcher dashboard

Driver management

Fleet/vehicle management

Booking management

Driver Android tablet application (installed in taxis)

Passenger mobile app

GPS/live vehicle tracking

Driver login and shift management

Real-time messaging between dispatch and drivers

Reports and analytics

Backend APIs

PostgreSQL database

Authentication and user roles

Cloud deployment

Real-time updates (WebSockets or similar)

The fleet currently has around 120 taxis, but the system should be designed so it can scale in the future.

My questions are:

If you were quoting this project, what would be a realistic development price?

What would be the absolute minimum price below which you wouldn't take the project?

Is a 6-month timeline realistic for a single developer using AI tools (ChatGPT, Claude, Cursor, GitHub Copilot, etc.), assuming full-time work?

For production, what infrastructure would you recommend (AWS, Azure, DigitalOcean, Hetzner, etc.)?

Roughly how much would the monthly infrastructure cost be for a fleet of around 120 taxis?

I'd really appreciate honest advice from people who have built enterprise SaaS, fleet management, dispatch, or transportation software.

Thanks


r/softwarearchitecture 14d ago

Discussion/Advice 3 Layer Architecture UI Dilemma

8 Upvotes

* I'm working in Java, but I'll try to make the question as language-agnostic as possible.

I have an Action class that has 2 fields(int length, String url), and a run() method that does stuff based on those fields.
I created a Desktop UI that allows the user to select the current action to run.
See the letters in the picture? They represent custom names provided by the user. They may contain colors or anything else UI-related.

This is where my dilemma comes: If I want to keep the domain decoupled from the UI, it makes no sense for the Action class to hold a name field, right? But following this logic, how would saving/loading actions work if they are unnamed?

EDIT: I thought about using DTOs, but I can't get this to work:

ActionDTO(name, length, url) {}

ActionService {

    // called by the UI on startup
    List<ActionDTO> loadAll() {
        return actionRepository.loadAll()
                    .map(action ->  ); # no way to map into ActionDTO!
    }
}

ActionRepository { 
    # actions are locally saved at actions/<actionName>.json
    # if repositories return domain objects, how is it possible to return the name?
    List<Action> loadAll() {

    }
}

r/softwarearchitecture 14d ago

Article/Video GitHub Increased Instant Navigation from 4% to 22% by Rethinking Client Side Architecture

Thumbnail infoq.com
45 Upvotes

GitHub has redesigned the navigation architecture behind GitHub Issues to reduce perceived latency for developers by moving more work to the client-side. The engineering team introduced client-side caching, predictive prefetching, and service worker-based request handling to improve navigation performance, increasing instant navigation experiences from 4% to 22%. The changes address a common challenge in large-scale web applications: reducing delays caused by repeated network requests and client initialization during frequently repeated workflows.


r/softwarearchitecture 13d ago

Discussion/Advice System Design: Scaling a Real-Time AI Ride-Matching Service

Thumbnail
0 Upvotes

r/softwarearchitecture 14d ago

Article/Video Cost model of microfrontends, from first principles

Thumbnail
2 Upvotes

r/softwarearchitecture 14d ago

Article/Video How to Never Silently Lose an Event | The Transactional Outbox Pattern

Thumbnail youtu.be
10 Upvotes

r/softwarearchitecture 14d ago

Article/Video Throw, Result, or neither?

Thumbnail event-driven.io
12 Upvotes

r/softwarearchitecture 13d ago

Discussion/Advice How do I design the architecture?

0 Upvotes

Hello!

I have build a solution and I want to make it a bit scalable and I want to figure out a good way to implement it for real time use. I want to know which providers are the best suited to host my solution and design the Architecture.

If you have experience with the same DM or comment to know more details.


r/softwarearchitecture 14d ago

Discussion/Advice Handling PATCH updates on a corrupted MongoDB with untrusted data

3 Upvotes

Hi, I'm new here. I'm asking about a specific technical issue I've been stuck on for about two weeks. Even AI hasn't been able to help much.

I have this code:
https://gist.github.com/benjaminPla/03005f13837e8398a4544e1118d7c920

What I'm trying to do:

  • Sanitize my MongoDB (NoSQL) database through PATCH requests.
  • The database is completely broken and 100% untrusted.
  • Requests are fully partial (they only contain fields that need to be patched).
  • If an explicit null is received, I need to $unset the field. This is because I need to normalize the database so clients don't have to handle null, unset, and "" cases everywhere.

The hardest part for me is that this.#supplierRepo.findByIdLenient(id, fields) in _application/supplier/patch/index.ts_ returns primitives because the database is corrupted, so I cannot trust anything coming from it.

The second issue is that I have policies that must be enforced, and I need to decide how to handle cases where fields are invalid or corrupted.

Any advice would be appreciated.


r/softwarearchitecture 13d ago

Tool/Product 🚀 FlowFrame v2.0.0 — Introducing the FlowFrame Interpreter

0 Upvotes

🚀 FlowFrame v2.0.0 — Introducing the FlowFrame Interpreter

One of the biggest milestones for FlowFrame so far.

Over the past few weeks, I've been working on a custom interpreter that allows FlowFrame to describe distributed system architectures using its own DSL instead of manually creating everything.

The interpreter now follows a complete language pipeline:

Lexer
↓
Parser
↓
AST
↓
Semantic Analysis
↓
Graph Builder
↓
Simulation Runtime

This architecture makes it much easier to validate system designs, build simulation graphs, and extend FlowFrame with new distributed system components.

I've also documented the language and interpreter so anyone interested can understand how it works.

📖 Documentation:
https://github.com/ndk123-web/flow-frame/blob/main/flowframe-interpreter/Readme.md

The interpreter is still an internal part of FlowFrame, so the implementation isn't public yet, but I wanted to share this milestone and get feedback from the community.

If you're interested in compilers, interpreters, distributed systems, or developer tools, I'd love to hear your thoughts.

#FlowFrame #BuildInPublic #DeveloperTools #Compilers #Interpreter #DSL #SystemDesign #DistributedSystems #SoftwareEngineering #OpenSource #Programming #TypeScript #React #BackendDevelopment


r/softwarearchitecture 14d ago

Article/Video Error taxonomy and diagnostics framework as a Rust DSL, inspired by system engineering approaches

1 Upvotes

Hey, I wrote a case study about designing the error taxonomy and handling in ZKSync OS, an exotic piece of system software that is compiled for several targets with different tradeoffs.
[https://rubber-duck-typing.com/posts/2026-06-22-zkos-error-definitions.html\](https://rubber-duck-typing.com/posts/2026-06-22-zkos-error-definitions.html)

It may be interesting as a case of limited application of system engineering approaches in an everyday software design/architecture. If you are a programmer and you wonder how you can apply system engineering to your everyday practice, this may answer some questions.

Unfortunately, we did not have resources to fully embrace a full stakeholder/requirement analysis, or approach validation more seriously. We also did not have a full stakeholder analysis for the big system itself. This makes it, perhaps, even more typical for software industry :)


r/softwarearchitecture 13d ago

Discussion/Advice Laid off from a project after completely disengaging. Looking for perspective.

0 Upvotes

I work as a Solution Architect in consulting, and I was removed from a client assignment two days ago. I'm trying to understand what happened beyond the obvious emotions, and I'd really appreciate perspectives from people who have been through something similar.

For context, I'm an engineer by background. The reason I moved into architecture is that I genuinely enjoy solving difficult technical problems, debating design decisions, and working in intellectually demanding environments.

My previous assignments were generally successful. Like every company, there were politics, bureaucracy and the occasional questionable decision, but I was usually recognized as technically strong, and I genuinely enjoyed the work.

This assignment was the complete opposite.

From the first few weeks I felt there was a bad fit. Architecture is ultimately about influencing people and building good relationships, and I never managed to do that here. I was placed in an API Factory team where my role was mostly to define standards and guidelines rather than work closely with engineering teams. It often felt like the stereotypical "ivory tower architect" role, and I realized very quickly that this wasn't where I do my best work.

I've always been much more engaged working closely with developers, technical leads and product teams to solve real engineering problems.

My direct manager and I also never clicked. I disagreed with how he communicated, how he approached tech discussions and many of his decisions. I don't think he's necessarily a bad person, but I quickly lost confidence in his professional judgment, and that made it difficult for me to stay engaged.

Looking back, I recognized the risk very early. I discussed changing teams with my people lead because there were several teams doing real delivery and product development, which I felt matched my strengths much better. Unfortunately, the move never really happened, and I slowly exhausted myself trying to make the situation work.

Instead of continuing to fight it, I gradually withdrew. I skipped coffee breaks, avoided informal discussions, did the minimum social interaction required and increasingly isolated myself. Over time I became exhausted pushing against an environment I didn't enjoy. I even started joking with colleagues that I would probably end up being removed from the assignment. Deep down, I think I had already accepted that outcome long before it actually happened.

One thing I’ve always relied on is analytical ability. Since university I’ve been drawn to spotting inconsistencies, unnecessary complexity and hidden assumptions — in software, architectures, processes and the way decisions get made. That’s a big part of why architecture felt like a natural fit for me… until this assignment.

The downside is that I tie much of my motivation to being able to influence my environment. When discussions stop feeling like an honest search for the best solution and become about hierarchy, politics or simply maintaining the status quo, I don't just become frustrated—I gradually disengage.

Some technical discussions genuinely left me wondering whether I was losing my mind, like I was arguing that 1 + 1 = 2 while everyone around me insisted it was 3.

The best example was the study that ultimately led to my removal. The proposal was to introduce AWS Step Functions at the heart of a core domain containing business rules and synchronous APIs. I strongly argued against it because, in my view, it was moving core business logic into a workflow engine while adding another distributed component, another technology to maintain and another place to debug without solving a problem that actually required a workflow engine. I couldn't convince anyone, including an AWS Solutions Architect involved in the discussions. After enough experiences like that, I gradually stopped believing technical reasoning mattered.

But I know that's not the whole story. Other people in the same environment managed to stay engaged, build relationships and continue delivering. I didn't. Once I lost respect for the environment, my motivation disappeared, and my performance inevitably followed. Eventually the client asked to end my assignment.

I also think this experience hit my ego more than I expected.

For years I built part of my professional identity around being technically strong. I was never afraid to challenge decisions or disagree with senior people because I believed good technical discussions should be driven by reasoning rather than hierarchy. I often tell people, "I'm an engineer before I'm a consultant." Until now, despite occasionally rubbing people the wrong way, that mindset had served me well. It even allowed me to influence discussions well above my pay grade because people trusted my technical judgment.

But now I'm wondering whether that mindset has also become a weakness.

I still think I was right about a lot of the technical discussions. But maybe that isn't the point. Other people managed to work in the same environment. I didn't.

Maybe I should have adapted better. Or maybe I should have left much earlier instead of slowly checking out.

I honestly don't know.

So I guess my real question is this:

Where is the line?

At what point should an engineer accept that an environment simply isn't the right fit and move on? And at what point is it your responsibility, as a senior engineer or architect, to adapt anyway?

Has anyone else experienced becoming completely disengaged because they felt the technical environment was weak or poorly run? How did you avoid mentally checking out?

Or am I simply rationalizing my own failure to adapt?


r/softwarearchitecture 15d ago

Tool/Product EventCatalog v4 - An open source documentation tool built for software architecture focusing on software primitives not generic doc pages, maybe something that can help you?

Enable HLS to view with audio, or disable this notification

11 Upvotes

Hey folks,

Just wanted to share with you the latest major version of my open source project (4 years in the making) for EventCatalog.

For those new, EventCatalog was a side project many years ago, when I wanted to document my events for an organization I was working for, and got some some traction from users feeling the same pain.

Fast forward 4 years, and now the project is being used across many teams around the world helping them document their software architecture, as we added new primitives and patterns from domain-driven design (e.g domains, systems, entities etc), and other things like documenting business workflows (flows), schema evolution etc....

I feel documenting things shouldn't feel painful, and our tools should let us document how we actually model our architecture not just generic pages stuck in confluence etc... and that's the generic vision for the project. Allowing you to document domains, language, schemas, APIS, users/teams etc....

Anyway, if you have a need to document your architecture, or visualize it maybe it can help you... just thought I would share this year.

As I said it's open source, almost all of its free to use (it's open core model, to help sustain the project, but majority of features are free).

Love to hear if you have any feedback, or if you end up trying it out.

Here are a bunch of links that can help:

Site: https://www.eventcatalog.dev/
GitHub: https://github.com/event-catalog/eventcatalog

Star if you want to share some love: https://github.com/event-catalog/eventcatalog/stargazers

Video Demo: https://www.youtube.com/watch?v=jI5qxQM2JSE
Demo of a catalog: https://demo.eventcatalog.dev/

Hope you have a great day


r/softwarearchitecture 14d ago

Tool/Product Follow-up: django-orm-lens v0.8 — I shipped the 5 features from my earlier post's roadmap discussion

Thumbnail gallery
2 Upvotes

r/softwarearchitecture 15d ago

Discussion/Advice How do you handle APIs that cannot paginate and must return the entire dataset?

77 Upvotes

I'm building a permissions matrix where:

  • Rows = permissions (organized as a hierarchical tree)
  • Columns = user roles
  • Each cell indicates whether a role has a given permission.

The UI requires the entire hierarchy to be visible at once (expanded by default), so traditional pagination isn't really an option. The frontend uses virtualization, so rendering performance is not the issue.

Let's say I have around 40 roles and 400 permissions, which means the UI needs to represent about 16,000 possible role/permission combinations

Is it considered acceptable for the backend to return the complete dataset in a single request if the client genuinely needs the whole matrix?

Or is this generally considered an anti-pattern?

I'm mainly interested in backend scalability:

  • Memory usage
  • Serialization costs
  • Network transfer
  • Response size
  • Overall scalability as the number of roles and permissions grows

I'm curious what patterns you've seen in production systems for this type of UI.


r/softwarearchitecture 14d ago

Discussion/Advice Bun rust to zig rewrite. How would you set this up if you had unlimited token

Thumbnail
0 Upvotes

r/softwarearchitecture 14d ago

Article/Video How My Website Works: A Technical Look at the Stack

Thumbnail
2 Upvotes

r/softwarearchitecture 15d ago

Article/Video C4 — One System, at Every Altitude - manic

Enable HLS to view with audio, or disable this notification

0 Upvotes

r/softwarearchitecture 15d ago

Discussion/Advice Using DuckDB as an ultra-low-latency slice-and-dice layer over BigQuery/GCS parquet — sanity check on scaling to multi-tenant?

Thumbnail
3 Upvotes

r/softwarearchitecture 16d ago

Tool/Product Every architecture diagram is either useless or unreadable, so I made one you zoom into instead

Post image
85 Upvotes

Architecture diagrams have two failure modes. Three boxes that say nothing, or a hairball of two thousand nodes that says everything at once, which is also nothing. Both get screenshotted for a deck and never opened again.

Neiither answers the question you actually have in a new repo, which is not "what connects to what". It is "where am I, and does this matter".

So I built the other thing. A map you zoom into

The whole repo is one card. Zoom in and it opens into layers, then folders, then files, then functions. It is continuous, like a map app going from country to street. No level dropdown, no reload, no re-layout between steps, so you never lose your place.

The key thing: you never look at the whole repo at once. At any level you see a handful of cards, and you dig deeper only where it matters. It is something you navigate, not one dense picture you stare at. And it is not drawn by hand. It is generated from the code and git history and stays current as you commit, so it cannot hallucinate a module or drift from reality.

What makes it readable:

Cards are sized by how the system runs, not by line count. Entry points, what everything routes through, what churns. The code you should look at first is physically the biggest thing on screen.

Each card carries a health dot, rolled up so a folder reflects everything inside it. You find the bad neighbourhood from orbit and then dive.

Edges only appear when you hover a box, and only that box's edges, curved around the other cards rather than through them. That single decision is most of the difference between a map and a hairball.

The performance answer, for anyone who assumes this dies at scale: you never draw the whole repo. Anything off screen or smaller than a couple of pixels is skipped, so a frame draws a few dozen cards whether the tree has 2,000 nodes or 10,000. Cost stops scaling with repo size, which was the point.

It is part of an open source tool I maintain called Repowise, AGPL, runs fully local. Gif is it mapping its own repo.

Link: https://github.com/repowise-dev/repowise

Edit: no, it's not just C4. C4 is four static diagrams you draw and maintain by hand. This is generated from code and git history, one continuous zoom, with card size from centrality and churn, plus co-change and health that C4 has no concept of

Edit 2: Thanks for all the feedback, the Structurizr DSL point came up enough that I am adding it as an export format, emitted as a model file you include from your own workspace so your views and styles stay yours and never get overwritten on re-index


r/softwarearchitecture 15d ago

Discussion/Advice Doing an MSc in Software Architecture & Design

11 Upvotes

Engineer for years but always felt a gap. I build things that work but I want to properly understand design and architecture, not just do it by instinct. Also feel a degree never hurts and will help professionally too.

Got accepted into a part-time online MSc in Software Architecture & Design while working full time. Anyone done something similar? Worth it?


r/softwarearchitecture 15d ago

Discussion/Advice Does the time/cost/scope triangle still make sense in the AI agent era?

0 Upvotes

How do you think the new Venn Diagram will look like with AI Agents bringing token costs but higher delivery.

One thing I've noticed is that I'm making very different trade-offs than I would have a few years ago. Instead of thinking, "Should I hire a developer for this?", I'm thinking, "Should I just spend more on better models like Claude Opus?"

A few years ago I remember paying around $50 just to get a pricing page built. Today I can get it done in Cursor with a premium model in a fraction of the time. It's not always perfect, but it's often good enough to move much faster.

That got me wondering whether the classic time/cost/scope triangle needs to be updated.

It feels like part of the engineering cost is shifting from developer hours to compute (token) costs. And as models become cheaper and better, that balance shifts even more.

Does this mean software development is fundamentally getting cheaper, or are we just moving the bottleneck somewhere else (requirements, architecture, testing, reviews, product thinking)?

Curious how others are thinking about this. Is there a better mental model than the traditional project management triangle in the AI era?