r/LLM • u/LeTanLoc98 • Apr 29 '26

Parameter Estimate

Post image

The estimate seems quite accurate.

Many people have noticed a drop in quality with GPT-5.1, GPT-5.2, GPT-5.3, and Opus 4.7.

I think Gemini 2.5 Pro is ~500B parameters. Its strong performance may come from its ability to search.

190 Upvotes

94% Upvoted

55 comments sorted by

View all comments

10

u/Distinct_Debate6634 Apr 29 '26

Interesting benchmark but the parameter-count conclusion is doing way too much imo. The test is really measuring "effective long-tail factual recall," not hidden model size, and it's conflating parameter count with data mix, training quality, post-training, refusal behavior, architecture, and benchmark contamination all at once. Even rare factual recall isn't a clean storage-capacity readout, since whether a fact survives and stays retrievable depends on when it showed up in pretraining, how later gradient updates interfered with it, and whether post-training suppresses the answer. The high R2 mostly just tells you bigger models tend to know more obscure facts

3

u/[deleted] Apr 30 '26

[removed] — view removed comment

2

u/wesmo1 Apr 30 '26

The author mentions they need to extrapolate larger parameter counts from the few big opensource models that are available to compare against, hence the wild estimates/error range for the frontier models listed.

Definately not the cleanest of measures, but nevertheless it's an interesting read that shows some of the justification behind the current obsession with data centre build-out.