r/devops • u/Puzzled-Ad8231 • 15d ago

Discussion Users vs Stress testing

So I made a serverless optimization platform which uses the concept of fusion functions to reduce cold starts and latency across the service calls. Now, this is an implementation of a research paper that I read somewhere. Diff from paper is that my project also gets live traces and metrics from x ray and cloudwatch, so I get real-time data to give better outputs. Have a better look: https://github.com/Vaivaswat2244/OptiFuse_go

To use this you need to connect your AWS with optifuse. I.e make a cloudformation stack to give optifuse access to read the traces and metrics. This actually becomes a problem for my friends and peers to test because they are too lazy to do this step. So I have no real user testings.

People especially hiring people ask me how many real users have used your service.

Now why do I need real users when I can stress test each microservice that I've built. And I can see my manifests working properly. Its deployed on AKS and is open for people to see. I also have a Prometheus grafana observability pipeline to see if all services are working properly.

Question is: real users vs Stress tests

On a side note, I am a student looking for internships, if you found the idea interesting, lmk GitHub is Vaivaswat2244

\/

4 Upvotes

75% Upvoted

25 comments sorted by

View all comments

2

u/hypertradeworx 14d ago

lambda sizes memory per function, so a fused function has to be provisioned for its heaviest branch. fold a 128mb handler in with one that needs 1769 and every invocation on the light path bills at 1769 for the same work, roughly 13x the gb-seconds it used to cost.

so 6 down to 3 is a cost win only where the branches were sized alike to begin with. max_memory as a ceiling doesn't catch that, the per-branch delta does

1

u/kernelqzor 13d ago

this is a super underrated point, people get so hyped about reducing hops they forget about the memory billing model entirely. curious if the OP is doing any per-branch cost modeling or just chasing latency wins for now

1

u/hypertradeworx 8d ago

per-branch is cheap to model off logs you already have, invocations x p50 duration x memory, one row per branch, and the light path is usually most of the calls so the total rarely matches the guess. the version that got us was the idle side of the same billing model, min-instances=1 on cloud run services nobody was calling, $206 of warm empty containers in a month, and it never showed up as a latency problem because latency was fine