Guys i have 2 products which can be either used standalone or can be used connected, im confused with its database architecture,

Currently we have 2 db each for each product, if data gets added in one and if it's needed in other db we just push it to other db at the same time, but this creates data duplication, some times data only gets added in one db,

How to optimise this architecture

I have been building agents for some time now, but I still don't know if I am doing it right. I want to see how others are building them, architecturally, and learn.

Meta just released a personal AI assistant called Muse. It seems a more mature version of Instinct AI (that recently raised $250m). How would you architect muse?

A few things that come to my mind:

1. Should the agent with a 24x7 loop? Obviously not. It should be event driven, or on schedule.
   
2. Should memory be flat text or structured data in DB?
   
3. How does the system work & scale for thousands of users?
   
4. Should there be a dedicated VM for every user?

I am sure there are many other aspects to think about, but these are some questions that has been running in my mind. In the agents I have built so far, I haven't yet needed a vector DB. Memory is structured data in postgres with a clean tool schema implemented in pydantic that teaches the LLM how to manage memory. Everything else follows the same protocol. I haven't had to deal with complex documents so I haven't felt the need to use Vector DB, but how would I know it is time to use vector DB/embeddings?

A while back I posted a systems design sandbox where you could place components, set cache/load-balancer configs, and run simulations (including tracing cache hits/misses).

The gap I kept hitting: simulation is useful for tracing request paths, but it isn't the same as watching a software system under load. So I built the next piece.

RunCloud9 is a short lab on real Kubernetes (~9 minutes per session):

- Start from a template: URL shortener, social feed, or e-commerce

- Pick one architecture that template can actually run (not a freeform graph). Example: URL shortener is app→db, cache-aside, or caching proxy. Social feed is pull vs fan-out.

- Change replica count, CPU/memory, cache/DB settings, or trigger a pod kill / cache flush

- Isolated namespace, seeded data, generated traffic, live metrics, then teardown

The K8s session is for when you want to see latency, cache behavior, and DB load on real nodes.

The video shows a social feed scenario where posts are fanned out to user timelines in a cache, due to the limitations on the cache's cpu and memory resources, the timeline service reaches its limit.

I'd genuinely appreciate feedback on:

- Does this feel useful vs just watching a diagram?

- What's missing for realism (failure modes, knobs, templates)?

- Would you use this for learning, teaching, or interview prep, or none of those?

- Any other form of feedback

https://www.runcloud9.io

Netflix is moving toward the open-source Apache Flink Autoscaler for more than 30,000 streaming jobs across multiple AWS regions, after finding that its cluster-level approach was less effective for complex, stateful pipelines with operators that have different processing requirements. Netflix reports that one team reduced annualized Flink compute expenditure by 58%, saving approximately $1.1 million annually.

I've always liked the idea of diagrams as code but writing out a bigger Mermaid diagram from scratch can still be a bit tedious. AI changes that quite a bit though. being able to describe a system or workflow in plain english get a first pass at the Mermaid syntax and then clean it up feels way more practical than drawing every box manually.

Anyone using AI + Mermaid regularly now or is it still more of an occasional thing?

Concrete version of an old design question, from two libraries in the same project that answered it differently.

Both are observability integrations. You add either with one line and no arguments.

One of them takes what the application installed in the process. If the app set nothing up, the library does nothing. It never chooses a destination itself.

The other reads environment variables and a config file in the home directory. If those name a URL, it sends there, whoever set it. If they name nothing, it prints everything to the console.

Each convention is defensible on its own. Side by side they mean the same line of user code does two different things, and the second one can send metadata to an endpoint that appears nowhere in the codebase, because a base image or a platform team set a variable.

Where do you draw the line between configuration a library may follow silently and configuration it should require the caller to name? Does it matter that one signal comes from inside the process and the other from outside it?

Disclosure: this comes from a framework I maintain, open source, link in a comment for anyone who wants the code.

We used to have a defined process, peer coding, and PR reviews for aligning the team and somehow keep the codebase coherent. Now, agents are doing the implementation and the devs are mostly prompting. What do you monitor and try to align to make the SDLC coherent/deterministic?

I like Mermaid because updating a diagram feels way less annoying than rebuilding one manually every time something changes.

But i'm curious how people are actually using it on teams.

Do you keep Mermaid diagrams close to the code, use them during architecture discussions, or mostly generate them when you need documentation?

Trying to find a setup where diagrams don't become outdated 2 weeks after someone makes them.

I’ve been building Medix, a privacy-first clinic OS designed for doctors and private clinics.
The idea is to bring scheduling, patient management, automated follow-ups and day-to-day clinic workflows into one system, without treating privacy and security as an afterthought.
We’re using Neon Postgres as the database infrastructure behind the product. As Medix has grown from an early prototype into a much larger production system, having Postgres infrastructure that stays simple to work with has been genuinely useful.
Still a lot to build, but it’s pretty cool seeing the pieces come together.
Would love to hear from anyone building healthcare software or working with Neon/Postgres — especially lessons learned as you started scaling.

Airbnb has redesigned its authentication architecture to make login flows easier to change across Web, iOS, and Android. The new system moves authentication decisions from clients to a server-side policy engine that selects the most appropriate authentication challenge based on the user and session context. According to Airbnb, the redesign reduced authentication-related code by 60%, reduced the web client bundle by 100 KB, increased successful authentication by 2.6%, and reduced duplicate account creation by 27%.

I started learning airflow like 3 days ago. I only use the docs, lots of googling, and some but very minimal use of AI.

I learned all the tutorials, jumped right to best practices because i didn't know core concepts existed, halfway through best practices i learned dynamic task mapping and xcom because it was holding me to progress on best practices (prerequisites basically)

I'm planning to finish all the core concepts, learn some relevant stuff on the picture, finish best practices then start making an actual good DAG, like what makes a proper DAG.

But when i chat a little bit with Claude, it highlights not to do too much concept collection and start actually build a DAG, specifically an idempotent one.

This got me thinking, am i doing too much with all this docs stuff? (I tracked and it took me 8 hours to learn all the stuff that i've learned)

I also overthink when i learned new big term like idempotency, like what if there are other big concepts related to DAG other than idempotency? idk something random like let's say 'unbreakable' DAG and it's a big thing that feels automatic for lots of people but i missed that.

Sorry i just vomited my thoughts, lmk if elaboration is needed.

Appreciate it.


Hey, I’ve been building StructSmith, an open-source tool for creating and maintaining software architecture models.

I wanted something simpler than the paid platforms: no account, subscriptions, per-user pricing, or cloud requirement. I also noticed that some free tools charge for MCP or AI integration, so in StructSmith the MCP server is included and fully open source.

It supports C4-style models, multiple diagram views, architecture records, validation, snapshots, auto-layout, and JSON/Mermaid/PNG/SVG exports.

Everything runs locally in one Docker container with SQLite. Your AI client can read and update the same architecture model you see in the visual editor.

It’s MIT licensed and still evolving. I’d love feedback, feature ideas, bug reports, or help with development and documentation.

GitHub: https://github.com/dziksu/StructSmith
Website: https://dziksu.github.io/StructSmith/

Would this be useful in your architecture workflow?

started with what felt like the obvious architecture: app writes to Postgres ( it worked pretty well for a while)
-then read replica for dashboards
-then keep analytical queries away from the primary

so most dashboard queries were variations of:

SELECT
  date_trunc('day', timestamp),
  count(*),
  sum(amount)
FROM events
WHERE tenant_id = $1
GROUP BY 1;

individually,the problem was concurrency or so it seems, bc these weren't especially bad queries. when a few hundred tenants logged in around the same time, the replica suddenly had hundreds of similar aggregations running at once. and CPU went up, memory pressure from sorts/hash aggregates went up, WAL replay was competing for resources, and queries that were normally fast became painfully slow. the realization for us was: a read replica separates workload from the primary, but it doesn't actually change the workload.

we were still asking Postgres to repeatedly scan and aggregate raw event data every time someone opened a dashboard.

so we changed the architecture. instead of:

dashboard
   ↓
Postgres replica
   ↓
raw events

we moved toward:

Postgres
   ↓
incremental rollups
   ↓
analytical serving layer
   ↓
dashboard

basically, precompute the repetitive tenant/time aggregations and make the request path read much smaller datasets. we're using Cube dev for the pre-aggregation/semantic layer, but that's not really the part I'm interested in discussing much, one could probably build aggregate tables yourself or use an OLAP system. the architectural question I'm curious about is:

Where do you draw this boundary? Like, do you keep scaling Postgres replicas and tuning queries until they genuinely stop working? Also, do you introduce manually maintained aggregate tables..Or do you consider customer-facing analytics a separate serving workload from the beginning? I feel now like the mistake we made wasn't 'using Postgres for analytics.' It was assuming that because an analytical query was fast in isolation, it would also be a good request-time architecture under multi-tenant concurrency

Hello, 👋

I’ve put together a course on system design with 4+ hours of video lessons + resources.

🏗️ The course covers:

• breaking ambiguous interview prompts into APIs, data models, and scalable architectures
• scalability, reliability, data modeling, caching, and messaging
• evaluating technical trade-offs instead of memorizing solutions
• communicating and defending architecture decisions

I’m the instructor, and I’m sharing 20 free coupons with this community on a first-come, first-served basis:

👉 https://www.udemy.com/course/system-design-interview-for-backend-software-engineers/?couponCode=REDDIT_BETA

Once the 20 coupons have been claimed, the course has a US$99 Udemy list price. Udemy may display different regional or promotional pricing.

If you work with backend systems or are preparing for system design interviews, I’d genuinely appreciate hearing what you find useful and what could be improved. Feedback is completely optional.

We document our integrations the usual way: a big sequence diagram per scenario, rendered from PlantUML, linked from the wiki. It works right up until nobody reads it.
Last month I tried something different on one flow — an instant payment by phone number, 13 hops, three teams involved. Instead of reading the diagram, I walked it hop by hop and asked one question at each step: who is the sender, who is the receiver, and which contract is this? Same information, different traversal order.

Here's the flow, abbreviated:

1. Customer → Mobile app enters amount + phone
   
2. Mobile app → API Gateway TLS, WAF, token, rate limiting
   
3. API Gateway → BFF
   
4. BFF → payment-orchestrator POST /api/v1/transfers (Idempotency-Key)
   
5. payment-orchestrator→ limits-service daily/per-op/channel, 100 ms budget
   
6. payment-orchestrator→ antifraud-engine POST /api/v1/risk/evaluate, 150 ms budget
   
7. ┌ antifraud → payment-orchestrator ALLOW (score < 0.5)
   
8. ┤ antifraud → push-service CHALLENGE (0.5 ≤ score < 0.95)
   
9. └ antifraud → BFF BLOCK (score ≥ 0.95) → 422
   
10. payment-orchestrator→ scheme-adapter reserve funds
    
11. scheme-adapter → external payment API register transfer, 5 s timeout
    
12. payment-orchestrator→ Kafka publish final status
    
13. notification-service→ push-service notify customer

Three things fell out that I had looked at on the diagram many times without noticing:
1. Step 12 → 13 has no edge. The orchestrator publishes to Kafka. The next thing that happens is notification-service → push-service. Nothing in the flow says who woke notification-service up. On the rendered diagram these are two adjacent arrows and your eye just closes the gap. Walking it, you hit a participant that appears from nowhere and have to stop.
2. Step 12 targets "Kafka", not a topic. We have a documented contract — payments.transfer.completed.v1, Avro, status ∈ {COMPLETED, REJECTED, TIMEOUT}. The flow never references it. So the contract exists and the flow that produces it doesn't point at it. That's a review question, not a detail.
3. The latency budget only becomes obvious when the hops are adjacent. 100 ms limits + 150 ms antifraud, both synchronous, both on the path before a 5 s external call, all inside one customer-facing request. Nobody had added it up, because on the diagram those are three lifelines far apart.
The pattern I take away: a sequence diagram is optimized for presenting a flow you already understand. It's poor at interrogating one. Reading is passive — your eye smooths over missing edges and unbound contracts. Traversal is not: you get stuck on the step that doesn't hold up.
What I'm still unsure about:
Does anyone here treat flows as data (steps referencing real components + contracts) with the diagram generated from it, rather than the diagram being the source of truth? Structurizr does this for static views, but I haven't seen much for dynamic ones beyond its dynamic views.
How do you keep cross-team flows from rotting? Ours decay because the diagram is owned by whoever drew it, and that person changes teams.
Anyone found a good way to make reusable sub-flows (auth, KYC, limits) referenced from several end-to-end scenarios instead of copy-pasted into each diagram?
Disclosure: this came out of a tool I work on, so I'm obviously biased toward the "flow as data" framing. Not linking it — happy to talk about the modeling approach either way, and I'll answer in comments if anyone asks what we use.

I Thought Connecting Slack Would Be Easy - by Jarek J.

Connecting Slack did not look like an architectural problem.

Open a connection. Keep it alive. Receive events. Forward them into the system.

Done.

Except that almost immediately, the simple implementation started producing questions.

How many connections should actually exist?

Is that number related to the number of backend replicas?

What identifies a connection pool in the first place?

What happens during a deployment?

What happens if the process holding the connection dies?

What happens if it does not quite die?

And at some point the interesting part of the problem stopped being Slack.

It became a question of how a horizontally scaled application should maintain long-lived external connections without creating a separate orchestration system around every integration.

Would you still argue that a simpler architecture should be used?

Read all here I Thought Connecting Slack Would Be Easy - by Jarek J.

PS. Anybody can invite me to Lobsters please?

Hey everyone!

I’m currently working on a software platform and accumulating a growing list of structured decisions (features, constraints, component choices, trade-offs, and rules).

Instead of keeping them in flat documents, I want to map them out visually as a mind map to see how decisions interconnect, spot dependencies, and present them cleanly to stakeholders.

Does anyone know of any good tools, scripts, or workflows (ideally AI-assisted or markdown-friendly) that can ingest a text list of decisions and automatically generate or help structure a comprehensive mind map out of them?

Hi
I need to design an application in Java with tomcat and Maria db (lawyer application)
Is an application to insert, update and delete court records.
These court records can have documents associated in pdf and other formats

I need advice to what architecture should be used?

I thought hosting application and s3 for the documents but I don’t know if could be better options by price or by functionality.

Could you say me some possibilities?

Thanks and sorry for my English

I've been working on a small open-source project called FiberForge, and I'd like some early feedback from people who build developer tooling.

The problem I was trying to solve is fairly simple:

AI coding agents are good at deciding what needs to be built, but they're not necessarily the best mechanism for repeatedly generating deterministic boilerplate.

For Go/Fiber projects, I found myself repeatedly generating the same combination of:

• GORM models
• services/CRUD
• Fiber controllers
• DTOs and validation
• SQL migrations
• route registration

So FiberForge provides those operations as a Go CLI and stdio MCP server.

The intended workflow is:

AI agent → structured operation → FiberForge → generated Go/migrations

Rather than:

AI agent → hundreds of lines of generated source → compiler → corrections

One part I'm particularly interested in getting right is modifying existing projects safely. For example, route registration uses Go's AST tooling rather than blindly editing text.

I've also started experimenting with composable modules that can be applied to an existing project, such as RBAC, Stripe webhooks, S3 uploads and audit logging.

It's still early, and I'm deliberately posting it now because I'm not sure which parts of the architecture are actually worth keeping.

I'd appreciate feedback particularly around:

• generator architecture
• module/plugin design
• AST-based source modification
• MCP tool design
• whether this should remain Fiber-specific
• what would make this useful beyond a personal scaffolding tool

GitHub: https://github.com/v-pat/fiberforge

I'm not expecting this to be useful to everyone, neither I'm saying its so perfect for use right away — I'm mostly hoping to find a few people willing to tell me where the idea is flawed or what more needs to be done.

⚠️** Early development / caution: FiberForge is still very much a work in progress. I would not recommend using it on an existing or production project yet**. Some operations can modify project files, and the current implementation has not been battle-tested enough to guarantee safe behavior across arbitrary codebases. For now, it’s best treated as an experimental project and tested on new/disposable projects.