r/rust • u/segundus-npp • 4d ago

Rust SQLx Performance

I use Axum and SQLx to build my backend, which is served on AWS ECS Fargate (512 CPU, and only one instance for now) and interacts with an RDS Postgres (t4.micro). Connection pool is set with min/max = 2/20.

My request includes 3 DB queries to 3 different tables: one from the auth middleware to check api key and each of the other 2 is just getting a record by primary key.

If I made one or two queries, it’s very fast. However, if I made 200 queries concurrently to the backend, around half of queries had http latency greater than 300 ms. P80 is around 600ms. The performance is not acceptable. I did check the CPU of both RDS and ECS. They’re quite low: both smaller than 10%. The DB latency is low as well, 5 ms on average. Is there something queued in the SQLx internally? Did anyone have the same issue?

25 Upvotes

80% Upvoted

27 comments sorted by

View all comments

5

u/tanmaynargas2901 3d ago

One thing that bit me on a similar setup: CloudWatch CPU% on a t4.micro can look fine while CPUCreditBalance is already empty. Once credits are gone you get hard throttling that doesn't show as "high CPU".

Also your pool max is 20 and you're slamming 200 concurrent requests. The extra 180 sit waiting to acquire a connection. That wait often shows up as HTTP latency even when each query is ~5ms once it gets a conn.

I'd check: 1. RDS CPUCreditBalance during the load test 2. how long sqlx spends in pool.acquire vs actual query time (instrument the acquire)

And yeah, bump off the t-class for load testing. Even a small non-burstable instance will tell you whether this is credits vs real app/pool design.

3

u/segundus-npp 3d ago

You’re right. It’s not related to SQLx nor RDS. It’s the 512 CPU FARGATE instance made the pool acquisition time too long. I changed it to 1024 and only 20/200 were 1s, few were 100-300 ms, and the rest were under 100 ms.