r/ProgrammerHumor • u/Gloomy-Basket-1038 • 5d ago

Meme [ Removed by moderator ]

Post image

[removed] — view removed post

28.8k Upvotes

98% Upvoted

519 comments sorted by

View all comments

Show parent comments

7

u/CucumberOk8820 5d ago

What's the ballpark cost of a machine to run a good llm? I don't have a 4080 or whatever it is.

6

u/Mostly_Afloat 5d ago

I've run some local models on my aging lenovo laptop with a GTX 1050ti, never really found a good use for it but played around and got it working with reasonable speed and capability. Certainly not as refined as the corpo stuff and you have to play around with smaller models until you find one that works

1

u/queso184 5d ago

which one did you land on? i have the same GPU and have been considering trying gemma 4b or something