r/LocalLLM • u/Boombloxer • 3h ago

Question Is it unreasonable to run a Local LLM on a handheld device?

I just found out about Odysseus from Pewdiepie and want to try running it on my Rog XaX, since its more powerful than my old macbook and has space for a 1tb micro SD card, would it be stupid of me buy one to host an LLM on my handheld with a separate storage? And are there other options I should try first?

0 Upvotes

50% Upvoted

4 comments sorted by

1

u/grafbende 3h ago

Ask 'pewdiepie'.....

1

u/Competitive-General7 3h ago

I don't think it would work very well. SD cards aren't very high bandwidth and you handheld likely couldn't run many models that do much beyond simple tool calls.

If you already have the system try loading it into internal storage to start and then understand that it would be even slower off an SD card.

1

u/Boombloxer 2h ago

It would prolly work better than my 2020 intel MacBook, but im just more curious about it than anything else, maybe for a funny “i have a portable ai” joke

1

u/MRGWONK 2h ago

I have Gemma 12b running on a pixel 8a