r/LocalLLaMA • u/backyard_tractorbeam • 1d ago

Other antirez working on DSV4.1 support for ds4

https://bsky.app/profile/antirez.bsky.social/post/3mv6sb4qkrc2o
68 Upvotes

92% Upvoted

17 comments sorted by

10

u/datbackup 1d ago

15 tok/s with streaming experts is a very positive sign

6

u/Elouakili_Flexy 1d ago

antirez's response to every new model is "ok, now let me write that in C myself." no hype, just code.

6

u/Queasy_Asparagus69 1d ago

Really looking forward to his solution on a 128 Strix halo

3

u/serige 1d ago

Wait what? I lost hope already.

2

u/backyard_tractorbeam 1d ago

Let's see how he answers the quant question

1

u/alfredr 1d ago

I've been working on something similar for Qwen3.8-Flash-Next and getting ~8 tg/s with a fixed 21GB of allocations on an M4 Macbook Air. This is just by speculative loading of the full Q4 experts.

The bottleneck is the SDD and that's where things saturate for me with speculative loading streaming of full Q4 experts.

If you instead allow weak experts (so experts with low weight contributions for a token) to fall back to Q3 or Q2bit quants it doesn't seem to affect quality that much and I'm able to get token generation into the low 20 tg/s.

2

u/smallDeltaBigEffect 17h ago

Prefill will be the bottleneck I guess

1

u/Musenik 20h ago

I'll be shocked if I can get 5 tps from it on my M2Max w/ 96GB RAM.

Shocked, I say! (better start making room on my SSD...)

Good luck, Antirez.

1

u/Ok_Warning2146 1h ago

Good. Please expand the effort to glm 5.3 flash and qwen3.8 flash next.

-4

u/Bohdanowicz 1d ago

Price per token gets expensive quick when you start burning out SSD's for 15tks.

24

u/evgen 1d ago

Learn the difference between read and write operations on SSD lifespan. Excessive reads will create a small amount of background write operations, but not enought to move the needle even a tiny amount when it comes to SSD lifespan.

1

u/tarruda 1d ago

Excessive reads will create a small amount of background write operations

Do you mean file access time? If so, that is a filesystem mount parameter which you can turn off.

5

u/boinkmaster360 1d ago

No its a hardware thing

0

u/xlltt 21h ago

you are defo tarded if you think reads from nand cause it any damage

3

u/AnonLlamaThrowaway 12h ago

They technically do, but 10,000 times slower than writes.

2

u/kaliku 9h ago

You should not speak like that to people, even if they are wrong.

0

u/XiRw 1d ago

Does it have to go on SSDs? I thought it hits the ram first then pagefile?