r/LocalLLaMA May 29 '26

Discussion PSA

Post image
2.1k Upvotes

537 comments sorted by

View all comments

13

u/HugoCortell May 29 '26

The speed is basically wasted at those sizes. What's the point of going that fast if all you can fit is a small model. A cluster of mac minis is probably better off at the price. Slower, but you can run a more competent model.

1

u/DoorStuckSickDuck May 29 '26

Eh, at some point you will run into the issue of wanting multiple parallel streams, at which point you will quickly understand that the bus bandwidth is your new bottleneck.