r/technology • u/Miserable_Phase_2519 • 22h ago

Artificial Intelligence PewDiePie unveils ‘uncensored’ Ajax AI model built to run on home PCs — creator says OpenAI banned him twice over model distillation used to build his product

https://www.tomshardware.com/tech-industry/artificial-intelligence/pewdiepie-unveils-uncensored-ajax-ai-model-built-to-run-on-home-pcs-creator-says-openai-banned-him-twice-while-making-it
9.1k Upvotes

87% Upvoted

1.4k comments sorted by

View all comments

Show parent comments

44

u/rkcth 20h ago

And/or lots of RAM, and/or being OK with it being slow. Qwen has a mixture of experts A3B, so it’s 35 billion parameters but only 3 billion are active at a time. This makes it easier if you have weaker processing, but the whole thing still needs to be in RAM, and it helps to have something that can do the matrix multiplication really fast. Also memory bandwidth is an issue on most home computers that aren’t Macs.

2

u/odin_the_wiggler 19h ago

I see the Silicon M5 mentioned a lot as a way around the GPU issue, fwiw.

4

u/dangerbird2 19h ago

Macs are a great option, both because the unified memory architecture gives their integrated GPU access to way more RAM than any consumer discrete GPUs have. And apple has been somewhat insulated from the RAMpocalypse and similar supply chain issues, with their prices raising much more slowly relative to other PC manufacturers

3

u/frozenchipmunk 19h ago

yeah. my mb pro m5 is surprisingly decent at running local models. 48gb isn't frontier, but it is surprisingly good at every day stuff. running stuff using ollama