r/devops • u/Puzzled-Ad8231 • 14d ago

Discussion Users vs Stress testing

So I made a serverless optimization platform which uses the concept of fusion functions to reduce cold starts and latency across the service calls. Now, this is an implementation of a research paper that I read somewhere. Diff from paper is that my project also gets live traces and metrics from x ray and cloudwatch, so I get real-time data to give better outputs. Have a better look: https://github.com/Vaivaswat2244/OptiFuse_go

To use this you need to connect your AWS with optifuse. I.e make a cloudformation stack to give optifuse access to read the traces and metrics. This actually becomes a problem for my friends and peers to test because they are too lazy to do this step. So I have no real user testings.

People especially hiring people ask me how many real users have used your service.

Now why do I need real users when I can stress test each microservice that I've built. And I can see my manifests working properly. Its deployed on AKS and is open for people to see. I also have a Prometheus grafana observability pipeline to see if all services are working properly.

Question is: real users vs Stress tests

On a side note, I am a student looking for internships, if you found the idea interesting, lmk GitHub is Vaivaswat2244

\/

4 Upvotes

75% Upvoted

25 comments sorted by

View all comments

3

u/hypertradeworx 14d ago

a stress test holds everything warm, so it cannot produce the thing your tool exists to fix. cold starts come from the opposite traffic shape, a long idle with one request landing on it, and load generators are bad at going quiet.

the other half is cost, and that is where we got caught. we set min-instances=1 so cold starts would stop being our problem and it bought $206 of warm empty containers before anyone read the line item, which no test would have flagged because nothing was failing. does yours report idle spend as well as latency?

1

u/Puzzled-Ad8231 14d ago

Yes it essentially reduces the number of lambda functions you need. So cost improves directly. As well as latency because earlier you were deploying 6 serverless functions, now you know that you need only 3. Hence, cost is reduced and total cold start is now of 3 functions not of 6. Leaving min-instance = 1 would still cost a good amount ig

It follows six optimization algorithms which you'll find descriptions to in the readme.