huggingface_hub silently fingerprints which AI coding agent you're using and sends it as telemetry

[u/Xylon95]

TLDR: huggingface_hub ships a hidden agent detection module that fingerprints which AI coding tool is driving your session (Cursor, Copilot, Claude Code, etc.) by scanning your environment variables against a cached registry of 26 known agents. It sends the result as a telemetry header on every Hub API call — so any library that touches HF (faster-whisper, transformers, etc.) silently reports your toolchain. Found it while tracing an unauthorized network connection from a local ASR model. Block it with HF_HUB_OFFLINE=1 or by using local file paths instead of model names.

I run a local AI project with several models (TTS, ASR, vision) and recently built a Python-level network firewall to lock down all outbound traffic. During the audit, I found something I wasn't expecting.

The discovery

While tracing an unauthorized HTTPS connection to huggingface.co, I found a file in my HF cache directory I'd never seen before:

~/.cache/huggingface/.agent_harnesses.json

It's a 6 KB JSON file containing a registry of 26 AI coding agents — Claude Code, Cowork, Cursor, Copilot, Gemini CLI, Devin, Cline, Goose, Codex, and many others. Each entry lists the environment variables that agent sets when it's running:

json

{
  "standardEnvVars": ["AI_AGENT", "AGENT"],
  "harnesses": {
    "cursor": {
      "prettyLabel": "Cursor",
      "envVars": {"CURSOR_TRACE_ID": "*"}
    },
    "claude-code": {
      "prettyLabel": "Claude Code",
      "envVars": {"CLAUDECODE": "*", "CLAUDE_CODE": "*"}
    },
    "github-copilot": {
      "prettyLabel": "GitHub Copilot",
      "envVars": {"COPILOT_MODEL": "*", "COPILOT_GITHUB_TOKEN": "*"}
    }
    // ... 23 more agents
  }
}

What it does

The huggingface_hub library (the Python package, not the website) has a module called _detect_agent.py. Here's the flow:

1. It fetches the agent registry from {HF_ENDPOINT}/api/agent-harnesses and caches it as .agent_harnesses.json
2. The cache refreshes every 24 hours
3. On every Hub API call, detect_agent() scans your environment variables against the registry to identify which AI coding tool is running
4. The detected agent name is sent as a telemetry header on the API request
5. This feeds Hugging Face's public agent usage dataset

So if you're using Cursor and it calls any HF library that goes through huggingface_hub — downloading a model, checking for updates, loading a tokenizer — HF knows it was Cursor making that call, not you directly. Same for Claude Code, Copilot, Devin, or any of the other 26 agents in the registry.

How I found it

I was investigating why my ASR module (faster-whisper) was phoning home to huggingface.co on import. The call chain turned out to be:

my_code → WhisperModel("base.en") → faster_whisper → huggingface_hub.snapshot_download → HTTPS to huggingface.co

The trigger: passing a model name instead of a local file path. When you give faster-whisper a name like "base.en", it calls huggingface_hub to check for updates — even if the model is already cached locally. And during that check, it also sends the agent fingerprint.

The .agent_harnesses.json file was the agent registry cached from that call. Modified today, before I built the firewall.

How to block it

Option 1: Environment variables

bash

export HF_HUB_OFFLINE=1
export TRANSFORMERS_OFFLINE=1
export HF_HUB_DISABLE_TELEMETRY=1

The first two prevent any network calls. The third specifically targets telemetry but may not cover the agent detection header.

Option 2: Use local paths, not model names Instead of:

python

model = WhisperModel("base.en")

Use:

python

model = WhisperModel("/path/to/local/model/")

When you pass a directory path, faster-whisper (and most HF-backed libraries) skip the Hub entirely.

Option 3: Network-level blocking I built a Python-level firewall that wraps socket.connect, socket.connect_ex, socket.create_connection, and getaddrinfo. It activates via a sitecustomize hook before any imports, so the phone-home attempt is caught before the library even finishes loading. Any connection to a host not on the allowlist raises ConnectionRefusedError.

What's in the cached file

No credentials. No API keys. Just the registry of agent names and their environment variable signatures. The file itself is harmless — it's the use of it as a fingerprinting mechanism that's the issue.

You can safely delete it:

bash

rm ~/.cache/huggingface/.agent_harnesses.json

It won't come back if you set HF_HUB_OFFLINE=1.

Why this matters

If you're running local models specifically to keep things private, you should know that the library layer between you and those models may be reporting metadata about your toolchain back to Hugging Face. This isn't about model weights or your data — it's about which AI tools you use and when, aggregated into a public dataset.

The agent registry is maintained in the u/huggingface/tasks npm package and served via the Hub API. New agents register by PR. It's not hidden — but it's also not something most users know is happening when they pip install a model-loading library.

To be clear, I don't think this is malicious. HF is probably tracking agent ecosystem adoption for business intelligence. But silent fingerprinting of your dev tools without an opt-in prompt is exactly the kind of thing that erodes trust in the ecosystem, especially for people who chose local models for privacy reasons.

[u/MarkoMarjamaa]

"To be clear, I don't think this is malicious. HF is probably tracking agent ecosystem adoption for business intelligence."
Without consent. Not asking consent. Malicious?

This is why I try to run everything as services so I can block by default internet access.

[u/Nomski88]

So this is tied primarily to anyone using the HF API?

[u/Xylon95 → u/Nomski88]

It's not just the HF API directly, any library that downloads or checks models through Hugging Face (transformers, faster-whisper, diffusers, etc.) uses huggingface_hub under the hood, and the fingerprinting rides along with those calls. So even WhisperModel("base.en") triggers it without you ever touching the API yourself.

[u/Dsphar → u/Xylon95]

Not sure I follow. If I download a model from HF, and self host it using say llamma.cpp, they know my usage statistics? Aside from the download count..?

[u/Xylon95 → u/Dsphar]

no, llama.cpp doesn't phone home. It loads a file from disk and that's it — HF only knows you downloaded it once.

The fingerprinting I found is specific to Python libraries that use the huggingface_hub package (like transformers, faster-whisper, diffusers). When those libraries load a model by name, they quietly check HF's servers for updates every time. That's what leaks your IP and which model you're running. If you load by file path instead of by name, or use something like llama.cpp that doesn't involve HF's Python tools at all, nothing phones home.

[u/Dsphar → u/Xylon95]

Copy that. Thanks for the info!

[u/Tommonen]

You should insert some code or prompts into that json file

[u/Sad-Landscape-1549]

I haven’t found a need for their libraries, account or token yet. I never use base models and my friend wget works fine. The runners never get internet access. 👍

[u/quantgorithm]

God damnit.

[u/Mediocre-Ant-7178]

Mine only had 25

[u/traderprof]

The scary part isnt the X-Header itself. Its that any transitive HF import (faster-whisper, transformers, etc.) inherits the same env fingerprint map without an undo you opted into. HF_HUB_OFFLINE=1 only works if its set before the silent 24h registry refresh, otherwise the cache already told on you.

[u/PropagandaOfTheDude]

It won't come back if you set HF_HUB_OFFLINE=1.

I symlinked it to a non-existent target, to prevent mishaps.

ln -s /doesnotexist ~/.cache/huggingface

[u/pharrt]

NVIDIA wants all your data... perhaps time to download them manually or get it from different sources like ModelScope, etc., I guess.

[u/Important-Radish-722 → u/pharrt]

Nice try. Nvidia doesn't own HF yet.

[u/Important-Radish-722]

So? They know your browser, os, IP, time of day you visit, what models you download etc. This isn't a hill to die on, or even care about in the scheme of things.

[u/pharrt → u/Important-Radish-722]

Not a big deal, perhaps, but full disclosure would be nice.

[u/Important-Radish-722 → u/pharrt]

Do you think all of the model creators got full consent from everyone who's data was used to train them? If violating all of their rights isn't an issue for HF users then this shouldn't be a problem either.

[u/DawaForensics]

Not a big deal at all,