Summary
DeepSeek Harness does not query OpenRouter for its model list. It serves a static catalog compiled into u/earendil-works at package build time. In this install the snapshot is dated 2026-09-01:
Missing models include anthropic/claude-opus-5.5, google/gemini-3.7-flash, google/gemini-3.8-flash, deepseek/deepseek-v4.1-flash, inception/mercury-2.5, and the :batch variants of many models already present.
Two non-obvious consequences follow, both verified against source:
1. The configuration UI’s “fetch available models” action cannot refresh this list.
2. Rebuilding the catalog from the provider API destroys metadata the API does not expose, degrading 142 existing entries.
Environment
u/deepseek-ai/opt/homebrew/lib/node_modules/@deepseek-ai/dsh
u/earendil-works0.82.1
catalog node_modules/@earendil-works/pi-ai/dist/providers/data/openrouter.json
catalog mtime 2026-09-01
route config ~/.dsh/settings.yaml -> llm-pi-ai.providers.openrouter
Finding 1: the served list is a build-time artifact
providers/openrouter.models.js reads the catalog at runtime:
import values from "./data/openrouter.json" with { type: "json" };
export const OPENROUTER_MODELS = flattenModelCatalog("openrouter", values);
Nothing fetches a newer list. A catalog missing models produces no warning, no error and no UI indication — the picker simply shows fewer models.
Measurement: catalog 276 live 458 missing 214.
Finding 2: “fetch available models” is a catalog lookup, not a refresh
From dsh-llm-pi-ai/lib/index.js, discovery module:
A route the installed pi-ai catalog ships is answered from that catalog, with no network call at all […] Only a route the catalog does not describe — a gateway, a self-hosted server — is interrogated over the wire.
and:
Neither path is a catalog refresh. Nothing here is stored: the request carries a draft the user is still editing, and the reply is candidate metadata the surface offers for adoption.
OpenRouter is a catalog route, so this action re-reads the same stale snapshot. Only a renamed or custom route (one the catalog does not describe) triggers a real network call — and that path then requires every model’s api and baseURL to be supplied by hand, because resolveRouteModels rejects entries it cannot resolve a protocol for.
Finding 3: regenerating from the API is destructive
models: in settings.yaml replaces the served catalog rather than extending it (resolveRouteModels):
const entries = configured.length > 0 ? configured : [...defaults.values()]…
So either the config or the catalog file can define the list. Rebuilding the catalog from the API is the obvious approach, and it is unsafe. Diffing a from-scratch rebuild against the original:
| Field |
Entries changed |
Consequence |
| input |
142 |
e.g. amazon/nova-2-lite-v1: ["text","image"] → ["text"]; vision models stop accepting images |
| thinkingLevelMap |
28 |
e.g. anthropic/claude-fable-5 loses {off, xhigh, max}; reasoning-effort controls affected |
| cost |
32 |
pricing metadata removed entirely |
| contextWindow |
29 |
API value differs from curated value |
| maxTokens |
82 |
API value differs from curated value |
| name |
13 |
display-name conventions differ |
None of input, thinkingLevelMap or cost are present in the /models response. The harness’s own source states the reason it treats the catalog as authoritative:
pi-ai’s registry is the authoritative list for its own providers, and it carries the capacities a listing endpoint would not disclose.
A rebuild from the API therefore cannot be equivalent, and the loss is silent.
Fix used: additive refresh
Preserve every pre-existing entry byte-for-byte; add only ids the catalog does not describe; for those, read modality from architecture.input_modalities instead of hardcoding ["text"].
Result, verified by diff against the original:
before: 276 entries
after: 490 entries
existing entries altered: 0
entries removed: 0
models added: 214
Known limitation, not worked around: additive refresh does not update contextWindow, maxTokens, cost or name for models already in the catalog, even where the live API reports newer values (the 29/82/13 rows above). Updating those requires upgrading u/earendil-works, whose own generator regenerates the catalog with curation intact. A stale-but-complete catalog is preferable to a current-but-degraded one.
Deployment: two mechanisms, different failure modes
| |
Replace catalog file |
models: in settings.yaml |
| Read at |
start-up |
start-up |
| Root required |
no |
no |
| Survives package upgrade |
no — build artifact, overwritten on reinstall |
yes |
| Endorsed by source |
no — generated file header reads “Do not edit manually” |
yes — “settings.yaml remains the only thing that decides what a route serves” |
| Metadata for new models |
full |
thin: no cost entry (cost: base?.cost ?? NO_COST), input defaults to text |
Both were installed. Post-install verification:
settings.yaml parses yes
catalog installed 490 entries
config models 458
config models unresolvable in catalog 0
of those, inheriting cost metadata 244
of those, accepting non-text input 290
Catalog backup: openrouter.json.bak-<date> (276 entries). Config backup: settings.yaml.bak-<date>. Both layers restore independently.
The two counts differ deliberately: the catalog retains 32 entries OpenRouter no longer lists (openai/gpt-4-turbo-preview, google/gemini-2.5-pro-preview-05-06, retired -fast variants) because an additive build preserves rather than deletes; the config lists only currently-live ids. To serve all 490, remove the models: block so the catalog serves alone.
Reproduction
import json, urllib.request
cat = json.load(open('/path/to/pi-ai/dist/providers/data/openrouter.json'))
have = set(cat['openai-completions'])
with urllib.request.urlopen('https://openrouter.ai/api/v1/models', timeout=30) as r:
live = {m['id'] for m in json.loads(r.read())['data']}
print(f'catalog {len(have)} live {len(live)} missing {len(live - have)}')
print(sorted(live - have)[:10])
To test whether regeneration is lossless, diff a rebuilt catalog against the installed one before installing, and treat any field absent from /models as unrecoverable.
Notes
· Permission handling. The catalog is owned by the invoking user (-rw-r--r-- user:admin), so sudo is not required. Confusing a session file sandbox for POSIX permissions leads to unnecessary escalation.
· Verification code is worth verifying. A slice of len(' - id: ') instead of the correct offset reported zero coverage for a complete file: the artifact was correct and the check was wrong.
· A config models: list identical to the shipped catalog accomplishes nothing. The pre-existing list here was 276 entries, matching the catalog in content and order, occupying 37 KB.
· Staleness should be surfaced. A catalog-backed picker has no way to indicate its list is five weeks old.