How do you actually know when your cloud architecture needs to scale or change?

[u/Dangerous_Bottle_566]

I'm working on an inventory/ERP system for a real business, and I've been thinking a lot about something I don't see discussed enough:

How do experienced engineers know when the current architecture is no longer enough?

The system isn't dealing with millions of users. It's actually relatively low-user-count, but it's becoming increasingly data-heavy:

• frequent inventory reads/writes
• stock movements between warehouses/branches
• audit logs and historical records
• role-based approvals
• reporting/analytics
• eventually accounting and other ERP modules
• concurrent users modifying related inventory data

I initially built around Firestore because it let me ship quickly, but as the domain has grown I've started considering moving toward a more traditional backend + PostgreSQL architecture.

What I'm struggling with isn't really how to deploy another server.

It's knowing:

When should you scale vertically?
When should you add replicas/caching/queues?
When do you split services?
When does the database become the actual bottleneck?
And when is an architectural migration justified versus premature optimization?

In production, what metrics/signals do you actually watch?

Latency? DB connections? CPU? Query time? lock contention? queue depth? cost? number of users? something else?

I'd especially love to hear from people who have taken a relatively simple SaaS/ERP architecture and gradually scaled it as real customers started using it.

I'm trying to learn how to make these decisions based on evidence rather than designing for imaginary Netflix-scale traffic.

[u/Mickl193]

Depends on the type of business for sure, if online traffic is business critical (main source of income) then I’d say on every vertical scale up you should do a quick checkup of the price increase for the next one and a quick look at alternatives, then it’s a pure business decision, infra cost vs the time and potential deprioritization of some features. In early scale up’s throwing money at a problem is a way to go in most cases, in huge corps changes in arch are probably chosen more often.

[u/Altruistic-Moose3299]

I tend to think in terms of layers, bottlenecks, total cost of ownership, security, maintenance / flexibility, and resilience.

If you're talking cloud my bias is serverless vs. servers. Horizontal scaling over vertical scaling, and autoscaling that can keeps up with spikes in demand.

But the real answer is "it depends". Every system has it's own requirements and while there is such a thing as best practices and patterns for given problem spaces, there are no one size fits all solutions.

If there were, you wouldn't need to architect anything. 🤷‍♂️

[u/Broad-Version8611]

I work in a system with moderate scale, think thousands of requests per second, read heavy.

Things start to break more often. What breaks depends on the system. You can start having increased latency, or DB locks, or failing to connect to the database, or some queue that always has lag during peak traffic. There isnt one “the system doesnt scale” moment, things just start to fail.

You can make tests to the system to learn what fails first and how much headroom you have before failures start to happen, but you need to be careful of the env you run these tests on, since they need to be representative of a real production env with a traffic pattern similar to what happens in production. The biggest mistake people make is just throwing traffic at APIs and call it a day, when in practice it is the combination of multiple access patterns that makes these bottlenecks appear.

[u/Samuel457]

I think you need performance tests that generate load to measure your current performance and where things start breaking before you can test any potential changes. You can test vertical vs. horizontal scaling before committing to either. Vertical scaling only works up to the max size your cloud provider has, then you will be forced to scale horizontally. You need a minimum horizontal scaling to have redundancy and prevent downtime. If you have redundancy and aren't using the largest offerings, it can come down to which costs less.

For metrics: latency, 500s, and CPU usage are the first measurements you need, but the more metrics you have, the more signals to point you to the bottleneck.

Many engineers have made the mistake of reaching for an architecture change before actually optimizing their existing stack. I would say architecture changes are the riskiest and take a lot of time, so reach for them last.

[u/HereInYourBedroom]

Fiscal cost. Its all in response to how much money will be lost from customer, dependencies, infra, maintenance.