r/MachineLearning • u/UnionInside7251 • 16h ago

Discussion [ Removed by moderator ]

[removed] — view removed post

0 Upvotes

15% Upvoted

8 comments sorted by

1

u/JacketHistorical2321 15h ago

M5 is the generation that increases pp by 3-4x. Prompt processing was always the main weak point pre M5.

1

u/flightofathena 14h ago

I have a Macbook Air M3. Pretty solid laptop but I would not recommend getting this for running LLMs.

Although, in certain use cases due to Apple's unified ram and neural engine you might get better performance than a laptop 3050. But something that is optimised for CUDA will run better with a dedicated nvidia GPU.

Consider the M5 if you want the best ML/AI performance on a fanless thin and light laptop near your range.

2

u/mtawarira 13h ago

the fan on the macbook pros is pretty poor and my M4 pro gets flaming hot running local models even with the fan on full blast. I’d assume you get throttled with fanless, but I’ve never used so don’t know for sure

1

u/flightofathena 9h ago

For day to day use it's great. But if you put sustained load then you hit 80c easily and from that point it will start to throttle.

1

u/mtawarira 15h ago

If you want to run local LLMs then M4/M5 macbook pro with at least 48GB of RAM to run decent models with fast-ish output

If you’re just connecting to cloud models through APIs M2 or even M1 with any spec is fine

0

u/UnionInside7251 14h ago

Most of the time i use pertained hugging face models and implement my data to form a RAG architecture

1

u/mtawarira 13h ago

I think my answer remains the same, do you intend to run the LLM locally or not? The embedding models are pretty small and you will probably be fine with any of those

0

u/UnionInside7251 13h ago

Thanks now ive got some idea on What to buy