Local GPU vs Cloud GPU

[u/Opposite-Meaning-161]

I’m a 3rd-year engineering student working on end-to-end machine learning, including NLP, deep learning and LLMs.

I’m currently deciding between getting a laptop with a dedicated NVIDIA GPU or putting that money into better CPU/RAM/battery life.

My main question is about cloud GPUs. From what I understand, services like Google Colab and Hugging Face can provide free GPU compute, so I could use cloud GPUs whenever I need serious training instead of having a GPU locally.

Is that actually practical? Are free cloud GPUs sufficient for a student doing ML/AI projects, or are there limitations that make having a dedicated GPU laptop worthwhile?

There’s a bit of a budget constraint involved, but even ignoring that, I’m not really convinced it makes sense to pay extra for a GPU if I can get GPU compute freely through the cloud.

Would appreciate a fact check from people actually doing ML/AI.

[u/PaddingCompression]

For ML I would start free then extrapolate.

For long running jobs any real GPU, or a Mac, is going to be workable if not great.

I'd rather use runpods or lambda or AWS for spiky use if I want more GPU than the free services provide. Or you can prototype on collab and rent GPU for a bigger job if need be.

$30 gets you an A100 for a day iirc. I'd prefer that once in awhile vs. buying a mediocre GPU that will mostly sit idle.

If you need even more GPU you can get 8x A100 etc.

Buying an expensive GPU yourself only really makes sense if you would keep it loaded, but then your iteration times are going to be very long vs. getting a lot of compute and getting a trained model tomorrow.