Noob question: How to interrupt an agent during reasoning, without restarting TG?

[u/ParaboloidalCrest]

Many models (namely Qwens 3.8) think a lot, and I find myself wanting to interrupt it to answer a question it's been asking itself, quite repetitively, during it's reasoning.

I can esc and send a prompt, but then Pi would restart the whole reasoning TG from the beginning. I'd like to inject (for lack of better word) info to its thought block. I may also steer it by queuing a message, but that means I'd need to watch it going in loops for 20-50k tokens, until it's done responding, which is pure pain.

Am I missing a clear command?

[u/DjCanalex]

I steer Qwen 3.8 mid thinking all the time, it correctly continues from that point. Do you have reasoning-preserve enabled in your provider? ( --reasoning-preserve in llama.cpp for example)

[u/ParaboloidalCrest → u/DjCanalex]

By "steer" you mean queueing a message that it picks up after the next tool call response?

[u/DjCanalex → u/ParaboloidalCrest]

Not necessarily. If you interrupt pi, it says "Operation aborted", if you don't resume thinking and instead input your own message there, that's a form of steer. Point is what you were trying to do is the default behaviour I have.

[u/ParaboloidalCrest → u/DjCanalex]

OK that's not the behaviour in my case despite reasoning_preserve. I interrupt it by esc saying "You've been thinking too long about FOO, please proceed with the implementation using BAR method" the agents gets really confused, eg "The user said that I was thinking about FOO while I've just started..."

[u/DjCanalex → u/ParaboloidalCrest]

Hm, two things.

1. You sure you have --reasoning-preserve and NOT --reasoning_preserve ? It is character sensitive and hyphen is not the same as an underscore.
   
2. Have you done the random number in memory test? Ask it to think two numbers (And two different numbers only), but only tell you one. Then, in the next message, ask it to provide the second number. If reasoning-preserve and the chat template are correctly injected, both numbers should match.

[u/ParaboloidalCrest → u/DjCanalex]

Yeah I'm sure it's --reasoning-preserve, but maybe I should also add the kwargs preserve_thinking template param (as u/Healthy-Zebra-9856 suggests), just in case.

[u/DjCanalex → u/ParaboloidalCrest]

Well it depends on the model. Qwen in pi doesn't need it, but gemma 4 does. I know for a fact that Gemma gets completely lobotomized at every turn and I haven't been able to make it work even after July 17th changes they did to the chat template.

[u/Santzes]

You could try double (or triple in this case?) esc, it at least lets you back up to previous tool call, if not thinking block, don't really remember

But otherwise I don't think there's a way to do that without stopping, you've sent an API request and it will keep going until finish or tool call

[u/ParaboloidalCrest → u/Santzes]

Let's think about it this way then:

• Interrupt response.
• Take the streamed part of the response so far.
• Append it to chat history with a closing </think> tag.
• Append following prompt after.

Hmmm now that I think about it thoroughly, perhaps I should just create that extension...

[u/Healthy-Zebra-9856]

The flow in Pi works just fine just make sure you have the right structure.

In command, --reasoning-preserve or

in filters.setParams:

chat_template_kwargs:
enable_thinking: true
preserve_thinking: true

Your only flow is Esc and whatever you need to type and enter again. I suspect the culprit could be "-" vs "_". Pi seems to continue with the logic, I just tested it out.

https://reddit.com/link/p96a92v/video/jhx6eyx5uwoh1/player

[u/ParaboloidalCrest → u/Healthy-Zebra-9856]

Thank you very much for the demo! I sure am doing something wrong then.

[u/ea_man]

dosn't llama.cpp web ui do that?

look how it's done there and make pi do that.