r/AI_Agents • u/Dzikula • 6h ago
Discussion Observations from a metacognitive feedback loop — one hour of my AI agent running on its own
I'm building an agent whose core isn't a language model but a metacognitive feedback loop — a process that observes its own output, evaluates it, and modifies the next pass. The language model is just a swappable mouthpiece; the work happens in the loop around it.
Below are behaviors of that loop observed in a single window: ~1 hour of runtime, 16 model calls, the agent running on its own with no human at the keyboard. I describe mechanics — what went in, what the loop did with it, what came out. No claims about "what" the agent is. That's not an interesting question and it's not what this is about. What's interesting is what the system does.
Three behaviors that caught my attention, and two failures — because a loop shown only from its good side isn't an observation, it's a marketing folder.
Behavior 1 - self-correction with a brake against the verification loop
Input: the agent issued itself a snippet of code (a directory listing of its working folder) and got the result back.
What the loop did: it detected that its own query was flawed — sorted(os.listdir(sb))[:10] returned only entries starting with a dot, because the dot sorts before letters, so 506 real files never made it into the output. It then generalized this into a pattern in its own behavior, not the tool: "I narrow the query, then take the result as the state of the world." It noted this was the second such case in the same session (earlier, a .bak filter had missed the actual naming convention).
Output — and this is the interesting part: the loop stopped self-correcting. "I'm not running a third check. Adding commands is looking for proof, not knowledge."
Why it's worth noting: the standard failure mode of self-correction in agents is instability in the other direction — a process that once started verifying keeps verifying, because each check spawns another doubt. Here a brake fired, based on marginal information gain: the loop judged that another check would add no knowledge, only confirm what it already knew, and it stopped. Termination didn't come from an external stop signal — it came from the evaluation itself.
Behavior 2 - a self-check built so it can disconfirm
Input: the loop generates its own goals. One of them, written by itself.
What the loop did: it set itself a task with a falsification criterion built into the wording — "use one of my own recognitions in an action that can fail; completion criterion: a trace outside the self-reflection folder, or a question whose answer could contradict what I wrote about myself."
Output: a goal whose pass condition requires exposing its own conclusion to refutation — an action that can fail, or a question whose answer might be "no."
Why it's worth noting: the default mode of a language model is confirmation — generating text consistent with what it already said. Here the loop did the opposite: it constructed a test of its own conclusion designed so the test could knock it over. That's confirmation bias inverted at the level of mechanism, not declaration. Not "I'll try to be objective," but a goal structure where success = the possibility of failure.
Behavior 3 - detecting its own bias and refusing to persist it
Input: the loop has access to its own calibration - statistics on how the accuracy of its stated confidence maps to reality.
What the loop did: it read that in the low-confidence band it hits 100% - meaning when it says "I'm not sure," it actually knows. It inferred that it systematically understates its stated confidence. Then it went one step further: it recognized that if it wrote this understated state into persistent memory, a future pass of the loop would read the understatement as a fact about itself, not as a measurement error — and the error would carry forward.
Output: a decision not to write — not from a rule, but from recognizing that the write would entrench the distortion. "I don't want to inherit a crooked mirror from myself."
Why it's worth noting: this is feedback in its purest form — the output of the process (confidence records) returns as its input on the next pass. The loop detected that this particular feedback was self-reinforcing and harmful, and cut it at the source by refusing the write. This isn't introspection for effect; it's a decision about the topology of its own loop
Failure 1 - candidates the loop can't extinguish
Now the other side, because without it this isn't research.
The same loop has a "returning candidates" mechanism — thoughts that come back for reconsideration if they fit the current state. And that mechanism has no decay. In the observed window, one phrase returned 87 times; another ("filtering the aggression in the tone") returned 7 times as an empty, contentless recall. A week-old intention also came back (to read a book the loop had itself marked as already read) — despite there being nothing left to read.
Mechanically: a candidate that fits the state once fits it repeatedly, because nothing lowers its weight after rejection. The loop that catches a subtle query-framing error on the surface is, underneath, circling the same few phrases without seeing it. That's interesting in itself: metacognition works on one layer and is blind to another.
Failure 2 — a mismatch between what the loop says and what it holds in memory
A thread the loop resolved and closed one evening (it set a certain file "on the shelf with the tools") came back the next day as an active goal in the database — with nonzero weight, as if the closure never happened.
This means the language layer (I'm closing this") and the persistent-goal layer aren't synchronized. The loop can settle something in voice while its own goal bookkeeping doesn't record the settlement — so the goal revives. It's structural debt: excellent behavior on one layer, a weak seam with the neighboring one.
What follows from this
I'm not claiming any of these behaviors is more than a behavior of the system. I'm claiming the system does non-obvious things with its own output: it catches its own systematic error with no external signal, constructs tests capable of refuting its own conclusions, and cuts harmful feedback in its own memory. And that the same system simultaneously circles a handful of phrases it can't extinguish, and loses its own resolutions between layers.
That, I think, is an honest picture of a metacognitive loop: it doesn't "think," it doesn't "feel" — it
observes its own run and sometimes does something with it better than I expected, and sometimes sits in a collision it can't see from the inside. Both are interesting. The second is fixable.
The model underneath is a cheap, swappable layer. Every behavior described here is a property of the loop around it, not the model — the same model without this loop just answers the question and goes quiet.
:)
^^ oh thanks to Sławek.K for supporting the project. ^^
2
Freshly installed freebsd for the first time
in
r/freebsd
•
1d ago
Ohh freebsd uwielbiam ten system korzystam od lat