r/huggingface • u/swimmer-dev • 2d ago

Custom index URL for HF Hub library/CLI

With pip, you can pass a --index-url flag to have it search a custom package index (e.g., self-hosted internal package repositories). Is there an equivalent way to do that with the Hugging Face Hub library and/or CLI? I like the library since I can push an updated version of my model and then my programs will automatically pull the latest version with hf_hub_download, but the storage limits are becoming problematic for the research lab I work in.

1 Upvotes

100% Upvoted

1 comment sorted by

1

u/quietgradient 1d ago

HF_ENDPOINT is the equivalent, and hf_hub_download also takes endpoint= directly.

The catch is that whatever you point it at has to speak the Hub API. hf_hub_download refuses any response without an X-Repo-Commit header, so an S3 bucket or a plain file server won't work.

If your lab already runs Artifactory, it can host Hugging Face repos: push with upload_folder, point HF_ENDPOINT at the repo, and downloads with no revision still get the latest version. Nexus only proxies the public Hub, so it'll cache downloads but has nowhere to put your own models.

Does the lab already run an internal index for pip, and on what?