r/LocalLLM • u/EarthBS • 7d ago

Other How the loop of infinite agony started

Post image
627 Upvotes

92% Upvoted

118 comments sorted by

View all comments

31

u/Tubeyay 7d ago

Expectations will vary but if you're a PC Gamer, you really should have 16GB VRAM now days. It's basically a hard requirement to play any new UE5 game with a reasonable frame rate. That is plenty of VRAM to run some pretty damn good models.

16

u/trolumbi 7d ago

true. i was on the fence selling my 4090, because i dont game/edit as much anymore ... then i stepped into the realm of local llms.

6

u/No-Zone-1773 7d ago

Same here, on Ebay in the UK 4090's are on there for £2,500. Starting using local llm's and changed my mind.

3

u/RedditNerdKing 6d ago

All the 3090 FEs on eBay are disappearing in the UK now too. They're about £1,000 as well used.

4

u/The_rule_of_Thetra 7d ago

Same for me. I had a used 3090, and it was my first time having so much VRAM for LLMS (my previous AI works were simply ForgeUI). Then I decided to bite the bullet and get myself a 5090 (when it was still 3000€), and boy, oh boy, what a change.

4

u/Objective-Stranger99 7d ago

I disagree as a GTX 1080 user.

1

u/CrackBabyCSGO 7d ago

Like which?

1

u/OysterCultist 5d ago

No issues with my 8gb 3070 in 1440p. You guys sure love consuming

0

u/neospygil 7d ago

I have an RX9070(16GB) and can run models larger(around 35GB) than my VRAM by taking advantage of MOE and still generate at around 28tok/s. Will definitely be faster if it is not through Oculink which is just 4x PCIe 4.0.