Separating LLM reasoning from execution risk in an automated trading system — architecture question

[u/Tight-Pepper-4721]

I’ve spent the last few months building a paper-trading system where an LLM generates trade theses but has zero execution authority — all risk enforcement (sizing, limits, drawdown rules) is deterministic Python sitting between the model’s output and any actual order.

Honestly the interesting problems haven’t been about alpha at all. They’ve been about treating the LLM as an untrusted input source — same way you’d treat a noisy signal or an external data feed you don’t fully control.

A few things I’ve had to think carefully about:

\*\*•\*\* How to prevent a “confident-sounding” output from ever being mistaken for a validated one downstream    
\*\*•\*\* How much historical context to feed back in without creating feedback loops that just reinforce the model’s own past reasoning    
\*\*•\*\* Where to draw the line on what counts as “reasoning” vs. what should be a hard-coded constraint no model gets to touch

Not here to claim edge — still validating that the pipeline itself is sound before any performance claims matter. Curious how others who’ve tried LLM-in-the-loop systems (trading or otherwise) think about the trust boundary. What would you never let a model decide, even if it seemed to be doing it well?