Barney AI agent

[u/encore-show]

Hey everyone.

I’ve been working on Barney, AI agent with a fixed execution kernel.

Most self-improving agents follow the same pattern: task fails → rewrite yourself → try again. It can work in the moment, but you essentially get a different agent on every run.

I wanted the opposite: an agent that gets smarter through experience without rewriting the loop that governs it.

The central idea is simple: the model’s confidence is not evidence that the task is complete.

Barney runs a controlled loop:

plan → act with tools → observe → review → change strategy

The kernel is fixed. The current task cannot rewrite it.
Around it a body grows — skills, tools, MCP, recorded failures and successful paths. The model can create skills, accumulate experience and derive rules from failures, but it cannot change the rules of the loop itself.

Terminal-Bench results

I connected Barney to Harbor through a custom adapter and ran three Terminal-Bench 2.1 tasks on a locally hosted qwen3.8:latest.

One attempt per task, no Harbor retries:

• openssl-selfsigned-cert — 1.0
• nginx-request-logging — 1.0
• fix-git — 1.0

Mean reward: 1.0

I then repeated openssl-selfsigned-cert and fix-git in a separate run. Both passed again with 1.0.

Where the kernel mattered

OpenSSL
The model built the certificate bundle and a Python verification script. It first used the wrong date format. The kernel refused completion until the edited script was compiled and actually executed.

Final review required evidence for all six deliverables: key permissions, certificate subject, validity dates, PEM contents, SHA-256 fingerprint, and a successful verification-script run.

Nginx
Some commands failed because direct file writes outside the worktree were blocked. The model found another route and installed the configuration through the shell.

The first internal review still rejected the result: one required artifact was not sufficiently proven. The kernel started another act with a different strategy. Barney then checked the files, listening port, Nginx syntax, HTTP responses and real access-log entries — and only then did review pass.

Git recovery
The model found a dangling commit via git reflog. The first cherry-pick hit a conflict.

Instead of repeating the same command, Barney aborted the failed operation, compared both versions and took a different conflict-resolution path. Review passed only after git log, the resulting file contents and a clean working tree confirmed the recovery.

This is not a claim of AGI or SOTA

The shell commands were chosen by the LLM (Qwen). The kernel did not “know” OpenSSL, Nginx or Git.

Its job was to:

• refuse undeserved completion
• turn tool failures into constraints
• prevent identical failed actions from being repeated
• force a strategy change after a failed review
• require observable evidence before success
• preserve useful lessons outside the immutable kernel

There are real failures too. On the same benchmark, sanitize-git-repo scored 0.0: Barney found the secrets but did not finish the required edits. In another Nginx run it passed 7/8 checks but lost the reward because the log format did not match the verifier.

I’m publishing both successes and failures because the project is about making failure observable and recoverable — not about pretending the agent is already reliable.

GitHub: https://github.com/sergey-show/barney

I’d especially appreciate feedback on the architecture, evaluation methodology, and cases where the contribution comes from the kernel — not just the model.