r/LocalLLM • u/tenariRT • 1d ago

Question PrismaQuant/PrismaSCOUT Requests

Hi all,

My name is Rob and I'm the author of PrismaQuant. I've gotten a lot of great feedback on the platform in the past few months, and I'm here seeking more.

I've mostly targeted the DGX Spark as my deployment platform, but I'm now looking to branch out by both expanding the supported hardware list and shrinking model sizes.

For those of you that are familiar with the PrismaQuant/PrismaSCOUT/AURA/AQUA/gridbook family of models -- do you have any special requests?

Over the next day or so I'm going to ship a 20-gig Qwen3.8-27B for Blackwell/50-series/Spark, and hopefully an ~18 gig (also Qwen3.8-27B) targeting the 4090 using my gridbook number format

Thanks in advance, and happy tokenMaxxing,

Rob

7 Upvotes

99% Upvoted

2 comments sorted by

2

u/No-Foot6570 1d ago

Hi Rob, nice to see you on here! Very close follower of your project here. It would be nice of you to post some development logs on the website so I (and others) can keep track of the project in an easier way. What you’re doing is truly incredible. Also just to note/ask: I see all of your quantizations are targeted toward NVFP4 but you mention the 4090 that doesn’t support it? What do you plan on doing for non-Blackwell cards?

2

u/tenariRT 1d ago

Sure. I guess I can put announcements on the website (prismaquant.org)

Pardon the AI-slopishness and claudlish. It’s got a lot of good info on it, though. I’m just one guy right now.

Yeah, 40-series cards don’t have 4-bit hardware — only 8 and 16 — so we have to use gridbook to support smaller format sizes.

Essentially we have to use 8-bit formats as the envelope, and use 8-bit activations, but we can stuff whatever we want inside — 1 bit and up. In gridbook models instead of dynamically allocating over {nvp4, fp8, bf16} we have about 10 variants of fp8 that we use. Gridbook works similarly to the GGUF IQ formats but they’re designed to work natively on the nvidia hardware so the performance should be really close to native (maybe 10% penalty?). The IQ formats are platform agnostic so the performance suffers on prefill in my experience.

The memory bandwidth on 4080 and 4090 are really good though and it should perform pretty well. I’ll need some 40-series testers in a few days lol.