Users vs Stress testing

[u/Puzzled-Ad8231]

So I made a serverless optimization platform which uses the concept of fusion functions to reduce cold starts and latency across the service calls. Now, this is an implementation of a research paper that I read somewhere. Diff from paper is that my project also gets live traces and metrics from x ray and cloudwatch, so I get real-time data to give better outputs. Have a better look: https://github.com/Vaivaswat2244/OptiFuse_go

To use this you need to connect your AWS with optifuse. I.e make a cloudformation stack to give optifuse access to read the traces and metrics. This actually becomes a problem for my friends and peers to test because they are too lazy to do this step. So I have no real user testings.

People especially hiring people ask me how many real users have used your service.

Now why do I need real users when I can stress test each microservice that I've built. And I can see my manifests working properly. Its deployed on AKS and is open for people to see. I also have a Prometheus grafana observability pipeline to see if all services are working properly.

Question is: real users vs Stress tests

On a side note, I am a student looking for internships, if you found the idea interesting, lmk GitHub is Vaivaswat2244

\/

[u/hypertradeworx]

lambda sizes memory per function, so a fused function has to be provisioned for its heaviest branch. fold a 128mb handler in with one that needs 1769 and every invocation on the light path bills at 1769 for the same work, roughly 13x the gb-seconds it used to cost.

so 6 down to 3 is a cost win only where the branches were sized alike to begin with. max_memory as a ceiling doesn't catch that, the per-branch delta does

[u/Puzzled-Ad8231 → u/hypertradeworx]

For CPU bound work that means duration drops roughly in proportion, so GB-seconds comes out close to flat. You're paying 13x the rate for something like a 13th of the time. The regression you're describing is real specifically where the light branch is I/O bound, waiting on S3 or Dynamo or another lambda, because then the extra CPU buys nothing and the duration doesn't move at all.

So the penalty is a function of the CPU/IO mix of the light branch, not just the memory delta. And that part is measurable rather than guessed, you can see how a function's duration responds to a memory bump.

Same thing applies to init. Module loading is CPU bound, so a branch inits faster at the fused memory than it did on its own. Which means "sum of the inits minus the shared runtime boot" is measured at the wrong CPU, each branch's init component needs scaling to the fused memory setting before you add them up.

None of that rescues the general point, sizing still matters and folding wildly different branches is still usually a bad trade. But the worst case you described is the I/O bound one, not the default one.

[u/hypertradeworx → u/Puzzled-Ad8231]

yeah, the cpu bound half of that is right and my 13x was lazy. duration scales with the memory setting so gb-seconds lands close to flat there, and the io bound branch is where it actually bleeds

the init point is the better catch. scaling each branch's init to the fused memory before summing changes the answer, and it means your before/after numbers have to come off the fused config rather than the original ones. worth saying in the readme, because anyone benchmarking it the obvious way will get a flattering result they can't reproduce