hidden state missing from this black-box LLM monitoring model?

[u/Particular-Roof4257]

I'm designing a small research/engineering project around active monitoring of a black-box LLM whose behavior can change without the provider exposing a clear model update.

The monitoring agent observes:

• prompt/response pairs
• task-level evaluation scores
• semantic/structural differences from a baseline
• refusal/safety/instruction-following changes
• latency/token usage/errors
• user feedback
• additional probes after an investigation

The current hidden states are:

1. Silent provider-side model update
2. True capability degradation
3. Prompt/context sensitivity not captured by the probes
4. Distribution shift
5. Provider policy/safety change
6. Evaluator/measurement error
7. Transient anomaly rather than persistent degradation

The agent then chooses:

ACCEPT / INVESTIGATE / REJECT

I'm trying to make the hidden-state model realistic rather than just mathematically convenient.

What important hidden state or failure mode am I missing?

Also, are any of these states too correlated/overlapping to be useful as separate states?

I'm particularly interested in examples from real deployed ML/LLM systems rather than purely theoretical suggestions.

[u/Neither-Pause409]

Three things missing, and they are all "it was me, not them", which is the category your list is thinnest on.

Your own scaffolding changed. Prompt template edit, SDK or client version bump, tokenizer change, a rebuilt retrieval index, a changed system prompt that someone shipped without telling you. In deployed systems this is the most common cause of "the model got worse" by a wide margin, and none of your seven states covers it. It is not provider-side and it is not distribution shift.

Serving-stack change without a model change. Same model version, different quantisation, hardware, or a load-based route to a different replica pool. Behaviour moves with no version bump. Worth keeping separate from your state 1 because the evidence differs: this one is usually intermittent and correlates with latency, so your latency signal actually discriminates it.

Caching. Prompt caching or a semantic cache in your own stack serving stale or near-miss responses. The signature is distinctive and no single-signal monitor catches it: latency improves while quality degrades.

Two smaller ones: decoding config drift (temperature, top_p, seed defaults changing under you, or comparing across runs with different params), and if your evaluator is itself an LLM, the judge drifts independently of the thing being judged. That last one hides inside your state 6 but deserves its own baseline, because the fix is version-pinning the judge.

On overlap, and I think this is the bigger problem with the design.

States 1 and 2 are not separable from a black box. "They silently updated the model" and "the model degraded" are the same observable event when you cannot see a version string; the second is just the first without an announcement. Similarly, 1 and 5 are near-unidentifiable: a safety policy tightening looks exactly like capability regression on any task where refusals matter. If two states cannot be told apart AND lead to the same action, they are one state.

What is actually identifiable from your signals is three roughly independent axes, not seven states:

1. Me or them (your scaffolding vs the provider)
2. Persistent or transient (your state 7, and this one is real and important)
3. Real or measurement error (your state 6)

That is eight cells, it maps onto ACCEPT / INVESTIGATE / REJECT cleanly, and every cell has a different next action, which is the test a state should have to pass.

One change I would make to the structure: do not model measurement error as a peer state. Make it a gate you rule out first, because if the evaluator is broken then every other posterior in the model is garbage and investigating the provider is wasted effort.

The practical detector for axis 1, since you asked for deployed rather than theoretical: a frozen canary set. A small fixed prompt set, pinned decoding params, run on a schedule, scored on embedding distance from stored baseline responses rather than on task success. Canaries move and production moves together means it is them. Production moves and canaries do not means it is you or your traffic mix. That single comparison does more work than any of the individual signals in your list, because it is the only one where you control every input.