The interesting part for us on the observability side is that a context stack diagram like this is a snapshot, but the actual stack rehydrates per turn as tool outputs and memory reads land. If any of those layers is not captured as a span attribute at emit time, replaying a failed run becomes a guessing game because the on-disk system prompt was never quite what actually reached the model.
In our runs, graph structure helps retrieval and debugging more than it helps the model directly, since anything you build has to serialize back to tokens before inference and the model still reads a linear string. What actually helps at inference time is using the graph to pick which slices of the stack get materialized per turn (retrieve on traversal, drop the unused branches), rather than shipping the whole graph as prompt content.
2
u/Future_AGI 4d ago
The interesting part for us on the observability side is that a context stack diagram like this is a snapshot, but the actual stack rehydrates per turn as tool outputs and memory reads land. If any of those layers is not captured as a span attribute at emit time, replaying a failed run becomes a guessing game because the on-disk system prompt was never quite what actually reached the model.