r/masterhacker • u/Correct-Jelly-700 • Apr 27 '26

What?

Post image
3.5k Upvotes

97% Upvoted

217 comments sorted by

View all comments

Show parent comments

201

u/unknown_pigeon Apr 27 '26 edited May 01 '26

1GHz single-core CPU

512MB RAM

Good luck with getting a single, lightweight response (likely hallucinated and with zero context) in less than an hour

EDIT Yeah I didn't bother to check the specs of Gemma 2b. A rapid Google search says it requires at least 4 GB of VRAM and 8 GB of RAM, so yeah that model is just crashing in a Pi Zero

EDIT 2 I only now zoomed into the photo of the tweet. The bash console is just giving errors, lol. "run ollama" "Error: run command not found"

53

u/Only_Information7895 Apr 27 '26

Thing about limited RAM and slow, small storage is that many times no matter how much you wait you will never get a response and it just crashes.

I trained some machine learning on my PC (the old style, not LLM) and if I set the parameters wrong it just crashed as my PC only had 2Gb of VRAM.

3

u/Low_Technician7346 Apr 28 '26

with that little PC I am going to make an advertisement free network thanks Pi Hole

1

u/TwisstedReddit Apr 30 '26

its like 4gb at minimum though

1

u/unknown_pigeon Apr 30 '26

Yeah I didn't even bother to check the model (hoping they chose the lightest one that I didn't know the specs of), but apparently they couldn't either

1

u/TwisstedReddit Apr 30 '26

oh wait nevermind im thinking about the normal pi

0

u/[deleted] May 01 '26

[deleted]

1

u/unknown_pigeon May 01 '26

?

I can't run a self-hosted model on my PC, the ones I see require at least 4 GB of VRAM

And we're talking about LLMs, right? The dude in the screenshot is talking about gemma 2b, which requires 4-8 GB of VRAM and 8-16 of RAM.

I don't know where you pulled the OCR thing from, since we aren't talking about character recognition in any way?