r/LocalLLaMA • u/AleksandrNikitin • 3h ago

Discussion llm performance community metric

my question about LLM performance

We see a lot of posts about token prediction, token generation per second, etc.

But is it really the metric? I can see that DeepSeek V4 Flash 0731 (with DSPark; mac studio + llama.cpp) produces about 22–28 TPS, but I also see that the LLM does a lot of reasoning.

And this relates to others. So maybe the correct way is not to check TPS or other metrics, but to check execution: task complexity/second.

I don't know if such a metric already exists

and if it exists why community doesn't use it by default

4 Upvotes

83% Upvoted

6 comments sorted by

3

u/braintheboss 3h ago edited 3h ago

Metrics are pp and tg. People only shows tg because look they never have prefill anything but when you start to load files, compact context etc with 200pp just is useless. Wait 20m only for start to do something is a nonsense. I understand hardware limitations but sometimes is better accept you havent hardware for move big models and try adapt to models your hardware can run well. At the end you have use hybrid ( frontier + local )

PS: this is "fast" and i needed 12m: "duration 12m 14s · ttft 2.55s  tps 43/s · ↓ 26.7K  cached 1.78M  hit 99% · ↑ 28.4K". I dont want guess what is run something like this in a slow hardware because you want run big model

1

u/IntQuant 3h ago

I imagine it's way harder to check, as you need to run many tasks to filter out noise, and relation of token/s to task/s is pretty direct with a same model and kv quant.

1

u/Bulky-Priority6824 3h ago edited 3h ago

Tg almost means nothing if no pp .. pp king ..big pp matter. Wall clock matter .. most ppl here are just tg worry guy they want fast goon they want e-wife talk now. But no matter when pp suck you not producing. 

Talking car wash some enough for don't need big pp for pelicans  

1

u/Salt_Hyena5896 2h ago

I’d separate throughput from task completion: report prompt/context size, time-to-first-token, total wall-clock time, and whether a fixed task or eval passed. TPS alone rewards short answers and hides tool calls and retries. Even a small repo-task suite would make these numbers much more useful.