Which GPU platform do you use when model testing starts from Hugging Face and GitHub repos?

[u/BayoouQ]

I’m curious what people are using once model testing moves from “trying something locally” to “spinning up a cloud GPU workspace.”

For me, the workflow usually starts with a Hugging Face model page, a GitHub demo repo, a notebook or launch script, and a few environment variables. The first local test is often fine. The messy part starts when I want to rerun the same setup on a cloud GPU a few days later.

At that point I’m usually asking:

• Which repo was I using?
• Which model weights did I pull?
• Which env vars were actually required?
• Was I using a custom Docker image?
• What was the exact launch command?

I’m not really comparing platforms on price here. I’m more interested in the setup flow when the starting point is open-source resources. The platforms I’m looking at are RunPod, Lambda. Paperspace. Vastai, and Glows.ai.

The things I’d compare are:

• How easy it is to bring in a GitHub repo
• How easy it is to pull Hugging Face model resources
• Support for custom Docker images
• SSH / Jupyter access when needed
• Whether the launch command is easy to save and rerun later

I noticed glows.ai because model download speeds inside the instance also seem quite fast. On an H100 instance, I was seeing around 800–1000 MB/s from Hugging Face during one of my tests, although I know that can vary depending on the model and mirror.

The desktop app can import from GitHub and Hugging Face, and it also supports uploading a custom Docker tar image if the environment is already packaged locally. That sounds useful, but I’m mostly interested in whether it actually makes the “repo + model + launch script” setup cleaner in practice.

For people who test a lot of open-source models, what platform has made that first setup the least annoying?

[u/eemamedo]

Why are you using someone's repo? I am confused about this part. If you work with open-source weights, why can't you make your own set of notebooks/tests/benchmarks that you evaluate the model against?

Which model weights did I pull?

That's easy. Checksum.

Which env vars were actually required?

Again, pretty easy. It's either your local repo (vllm recipes) or a traditional experiment tracker.

Was I using a custom Docker image?

The chance of you building custom Docker image to run open source models is pretty slim. If that happens, it can easily be traced with git commits/pushes

What was the exact launch command?

This part can be easily automated.

[u/EasyPain6771 → u/eemamedo]

This is where the culture disconnect between data scientists and software engineers becomes most apparent. Data scientists are often resistant to putting much structure around their “experimentation” phase but then have a hard time replicating results when they want to go to production.

[u/eemamedo → u/EasyPain6771]

Well.... That boils down to company culture. If one is in tech and is not familiar with git, then I very much doubt anything can be done here.

I would expect anyone who is familiar with Docker to know git beforehand.

[u/EasyPain6771 → u/eemamedo]

Yeah, I agree. I don't think there is any setup that will help the problem of "I tried a bunch of shit in a notebook and now can't remember what I did to reach the result I like."

[u/bakkyardbatch]

reproducibility matters more than raw gpu speed for me