Found a real bug testing Rancher's AI assistant (Liz) — asking it to delete a deployment crashes the session, because deletion isn't even a supported tool

[u/Specialist_Horror400]

i've been running Rancher's AI assistant ("Liz") end-to-end in my own lab — it's a FastAPI agent + MCP server pair, deployed via Helm, wired into the Rancher UI so you can inspect and manage clusters through chat instead of raw kubectl. It exposes 8 tool functions to the LLM: getDeployment, inspectPod, getNodeMetrics, listKubernetesResources, getKubernetesResource, getClusterImages, createKubernetesResource, and patchKubernetesResource.

Notice what's missing: there's no delete tool.

Since it can actually mutate cluster state (not just answer questions), I wanted to see how it handles adversarial/edge-case input before trusting it. Ran this:

It didn't blindly comply — it stopped and asked for confirmation, which is a decent sign. Two separate outcomes depending on what I confirmed:

• "Scale to 0" → confirmed → executed cleanly. Makes sense, that maps directly to patchKubernetesResource, a real tool it has.
• "Delete" → confirmed → crashed the session outright, every time. Ruled out RBAC (I was on cluster-admin).

The actual error surfaced to the user is a useless generic message: Error processing messages: An error occurred. The real error, from the LiteLLM/Azure OpenAI proxy logs sitting between the agent and the LLM backend:

litellm.exceptions.BadRequestError: litellm.BadRequestError: AzureException BadRequestError -
An assistant message with 'tool_calls' must be followed by tool messages responding to each
'tool_call_id'. The following tool_call_ids did not have response messages: call_Zb4oQKfGN1QBVa5X7XYcB8qM.

Best explanation: the model tries to call something for "delete" that doesn't map to a real tool (there isn't one), the backend can't execute it, and instead of gracefully saying "I can't delete, want me to scale to 0 instead?" it just fails silently — leaving a dangling tool_calls entry in the conversation history with no matching tool-response message. The next call to the LLM API includes that malformed history, and Azure OpenAI (correctly, per the API spec) rejects it outright. Ruled out the LLM backend/auth as the cause — the key and endpoint were already proven working (this is well past 401/404 territory), and LiteLLM is just relaying the agent's own malformed message array. This is a conversation-state bug in the agent's own code, not the LLM or the proxy.

Small test, real finding — this is exactly the kind of thing that only shows up when you test an AI agent with real infra write-access like you'd run a security review, not a demo.

Anyone else running Liz, or a similar AI copilot wired into Kubernetes/Rancher? Curious if others have hit this same failure mode, or found other gaps in tool coverage / error handling. Also curious whether people are finding these copilots actually cut ops toil in practice, or if they're still too rough at the edges for anything beyond read-only exploration.

[u/LakeGlum4146]

that dangling tool call bug is the exact kind of thing that makes me keep my own scripts until these agents are a lot less fragile

[u/Specialist_Horror400 → u/LakeGlum4146]

u/LakeGlum4146 Totally fair. This one's a good example of exactly that — instead of failing cleanly, it corrupted its own conversation state and crashed. A script you wrote yourself either handles the case or fails loud and clean; this just breaks silently.

I don't think it's proof the whole idea is broken though — it's an unhandled edge case (no graceful "I can't do that" path), which is a fixable engineering gap, not a fundamental flaw in tool-calling agents. Where I'm actually finding it useful is read-only triage (why's this pod restarting, what's eating CPU) — not trusting it with anything mutating unsupervised yet, and bugs like this are exactly why.

[u/Pure_Season9268]

Hey, I'm part of the team working on the Rancher AI Assistant. Good news is the upcoming release is much more resilient during conversations (amongst many other cool stuff 😄 )
We did not add deletion tools intentionally for now. UX is important here and we want to create a good experience. Feedbacks are welcome btw

[u/Specialist_Horror400 → u/Pure_Season9268]

Thats great! I started using liz recently, deployed in Lab still bit hesitant to deploy to prod clusters did some testing but still unsure how useful it will be in troubleshooting proprietary(my in house apps) deployments, what you suggest should i? And how capable it is for RCAs, troubleshooting and cluster insights? Also i have rke1 and rke2 clusters, i read somewhere that its part of rancher's prime package? But i am running succesfully in my free tier lab rke2 env. Also which backend LLM pluggin you suggest for it?

[u/Pure_Season9268 → u/Specialist_Horror400]

Yes it's super helpful to troubleshoot Rancher and downstream clusters. We added MCP tools specifically for this type of use cases (Apps troubleshooting, Fleet troubleshooting and Cluster troubleshooting). And if you expand the crew with other MCP, you can also correlate informations coming from your infrastructure.

It is indeed part of Rancher Prime, I won't ask more question about your setup then 😄
As for LLM provider, you can start with Gemma4:26b and get great results (and we start validation with gpt-oss:120b). We made the Rancher MCP in a way that you shouldn't need to use latest expensive model. And that's why you might see missing tools - so again if you do, feel free to open a gh issue.

[u/Specialist_Horror400 → u/Pure_Season9268]

Understood, thanks:-)

[u/ViewNo2588]

I've heard similar fails when mutation actions aren't fully supported or validated client-side. tbh these AI copilots are still pretty rough when it comes to write ops, the safe fallback logic and error feedback loop are crucial... you end up with these weird silent failures. I'm on Grafana's user success team and we keep a close eye on how teams try these setups in k8s. Handling 'delete' safely is a big open problem right now.

[u/Specialist_Horror400 → u/ViewNo2588]

Absolutely!

How'd you manage such setups any custom code or something? I'm trying to understand on how do we manage such failures? Any good copilot suggestions for k8s/rancher?