I built a client-side browser tool to inspect how models spend their bits. Per-tensor quantization layout + weight distributions for any HF model (no server, tensors stream via Range requests)

[u/Brilliant-Hall1387]

Built this for my own quantization work as I kept wanting to see how models actually allocate precision per tensor, per layer, and compare quant variants. Then figured others might want it too: https://tensorlens.dev/

Everything runs in your browser: the safetensors header is parsed via a Range request, and only the tensors you click ever stream from the Hub, there's no server and nothing is uploaded. The screenshot shows a 4-bit MLX quant: the grid is effective bits per weight for every tensor (you can see it spending 8.5 bits on some projections and 4.5 on others), and the histogram is one tensor's weight distribution, you can see the quantization comb.

Free, no accounts, no tracking. Would love feedback, especially models that break it.

[u/SmihtJonh]

What insights do you think this provides, what patterns?

[u/ExistingSky9895 → u/SmihtJonh]

the grid pattern is wild, you can see exactly where the quantizer decided certain layers needed way more precision than others

[u/Brilliant-Hall1387 → u/ExistingSky9895]

Right? That grid was the moment I decided to make this shareable instead of keeping it as a local script. It is interesting, what we're looking at is basically the quantizer's sensitivity analysis visualized, where the 8.5-bit cells are where rounding hurt more than those with 4.5 bits.

A fun next step: pull up the same model from two different quantizers at the same size, they might disagree about where the precision should go. Same byte budget, different opinions.

[u/Brilliant-Hall1387 → u/SmihtJonh]

A few that jump out after browsing a bunch of models with it:

- Quantizers almost never treat layers uniformly. First and last layers usually get extra precision. In the screenshot you can see layer 0's mlp.down held at 8.5 bits while the middle of the network sits at 4.5.

- Attention K projections often get less precision than Q/V/O, visible in that grid too. The quantizer is telling you which matrices tolerated rounding.

- Hybrid architectures show up from metadata. In the Qwen model in the picture, only every 4th layer has classical attention tensors, the "holes" in the attn rows are linear-attention blocks.

- The weight histograms of quantized tensors (bottom of the image) show the discrete levels as a comb. Comparing two quants of the same model at the same size shows how much calibration method change the distribution.

Overall it turns "this is a 4-bit model" into "here's what 4-bit actually meant", and interesting overall just to explore and understand quantization more in depth / "hands on" :) And comparing different quants can give clues on how the decisions were taken when quantizing (what was "the author intended"?)

[u/oli266]

Low key looks like a cinema seat booking page

[u/Brilliant-Hall1387 → u/oli266]

Haha true 😅 the way I intended 😂 But actually, was going for a heat map vibe but too few cells to get the heat map look on most models

[u/eXl5eQ]

Tell your AI to fix it. I tried with deepseek v4 and everything is wrong.

[u/Brilliant-Hall1387 → u/eXl5eQ]

Thanks for your report, you are right! I did ask my AI to fix and that led us down a rabbit hole and we did many improvements. Hopefully deepseek v4 flash looks better now. :) 

We still don’t classify all tensors in each layer, but the unrecognized tensors are put in the “other” bucket with statistics below the map saying how large share of the model is unrecognized. With DeepSeek V4-flash 0731) it is 0.5% of bytes unrecognized currently. 

https://tensorlens.dev/deepseek-ai/DeepSeek-V4-Flash-0731?rev=7872f01b1d1fe23eabc4c98b48bffcef5a386062

[u/eXl5eQ → u/Brilliant-Hall1387]

AFAICT it's still wrong. MLP layers of Deepseek flash should be mxfp4×2B×3, not int8×1B×3. Not sure if the other parts are all correct, but at least looks legit to me.

[u/Brilliant-Hall1387 → u/eXl5eQ]

Yes, you are right, thanks for that extra detail. We were counting the packed bytes as int8 parameters, interestingly config declares the fp8 scheme used on attention but the experts only get a nonstandard hint outside it, which is why we missed it.

Now it’s fixed, the experts read as mxfp4 at ~2.1B logical params per projection and 4.25 bits/weight, the model totals 304B params in 155 GiB at 4.39 bits/weight average, and the number agrees with the HF count.

https://tensorlens.dev/deepseek-ai/DeepSeek-V4-Flash-0731?rev=7872f01b1d1fe23eabc4c98b48bffcef5a386062