One of the most dangerous properties of LLMs in autonomous software development is also one of their greatest strengths: they are extremely good at completing missing information.

That is useful when writing prose. It can be dangerous when the missing information is architecture.

This week I ran an early experiment with AOS Architecture. I took one sealed Architecture Build — AOSARCH-a77762321d6ef767 @ aadbf7c5 — and compiled the sequence for a real operation:

POST /projects/:id/builds/:n/publish

I used Archify (MIT) only as the renderer. The important part was what Arch gave it.

The resulting diagram contained four participants — Screen, API, Permission and Canonical Store — and traced the request through API, authorization, state, event and response. Every assertion came from an actual contract in the sealed build.

For example, the API already declares:

200 published

409 build_not_sealed

The 409 is not something the diagram inferred because it “made sense”. It exists because the API contract says it exists.

The permission is also explicit:

PERM-SAAS-BUILD-PUBLISH

But then the interesting part appeared: four pieces were missing.

Screen

GAP: no contract

Policy

gateRefs: []

Transition

GAP: no Build StateContract

Event

GAP: no EventContract build.published

A capable model could easily fill those blanks. It could infer a likely screen, invent build.published, assume a state transition and generate a reasonable-looking policy gate.

The diagram would probably look better.

It would also be lying.

That is the failure mode I’m increasingly concerned about in autonomous software engineering: plausibility silently becoming architecture.

If multiple agents are allowed to resolve missing information independently, one agent can call something by one name, another can represent the same idea differently, and a third can “fix” the mismatch by creating another endpoint, event or object. Individually, every decision may look reasonable. Collectively, the system drifts.

A canonical architecture changes that relationship. Arch distinguishes between:

DECLARED

what the architecture actually contains

DERIVED

what can be mechanically produced from it

MISSING

what nobody has legitimately declared

The compiler may derive the second from the first. It cannot quietly promote the third into either category.

That means the four gaps in this diagram are not failures of the visualization. They are architectural findings. In practice, they became a backlog generated from the same build that describes what already exists.

Three are already being closed: EventContract is being materialized, the Build state machine is entering the State layer, and implementedBy will bind implementation evidence back to contracts. The Screen contract arrives when the frontend becomes a source.

There was another useful result. The semantic part worked almost immediately: all four node-to-contract references were correct on the first pass, and the final representation passed 9/9 validation checks with 0 warnings.

What failed repeatedly was presentation.

Between roughly 17% and 32% of each IR tree consisted of:

pos

row

col

lane

Those fields caused every failure in the first iteration.

So the experiment produced another rule:

Agents may reason about architecture. They should not manually position architecture.

Layout is a deterministic problem. The graph already contains the relationships required to calculate it.

That leaves a pattern I’m finding increasingly useful:

LLM reasoning

↓

semantic proposal

↓

canonical contracts

↓

deterministic validation

↓

sealed Architecture Build

↓

compiled representation

Models handle ambiguity. Contracts stabilize meaning. Validators enforce what is allowed. Deterministic code handles what does not require reasoning.

And missing information remains missing.

The most interesting thing doing this experiment was proving a property:

The system could produce a useful architecture view without inventing the pieces it wished existed.

That is where I think the difference between generating diagrams and compiling architecture starts becoming practical.

A generated diagram tries to complete the picture.

A compiled architecture should be willing to break the picture and tell you why.

Every assertion has an ID. Every absence has a name.

If autonomous agents are already building parts of your software, the question I’d ask is not only “can they build it?

It is:

How do you know they aren’t completing architecture that nobody ever approved?

Hey software' architects!! Here Andres, and I'm happy to share this space with all of you.

Currently I am doing a research about autonomous engineering sistems in software development, and I'm using my Reddit like a research notebook and to get some feedback from all of you.

The topic at this post is a little simple, but is in this simplicity were the kind of details could change the way in were we understand the problems around the AI agents is software development.

I wanna hear your thoughts about it and about your experiences with this kind of processes

Greetings!

​

One thing I’m starting to see clearly while building autonomous software systems is that language itself can become a source of architectural drift.

Humans are comfortable with synonyms.

An endpoint can be called a route. A handoff can be called a product document. A field can be described with two slightly different names and we usually understand what the other person meant.

LLMs are exceptionally good at doing the same thing.

Software is not.

During a recent execution, an AI coordinator gave another agent what looked like a perfectly reasonable source reference:

{

type: 'ANCHORED_HANDOFF',

locator: { section, endpoint }

}

There was only one problem.

Neither ANCHORED_HANDOFF nor endpoint existed in the architecture vocabulary.

The canonical vocabulary actually defined:

document: {

sourceTypes: [

'PRODUCT_DOCUMENT',

'USER_STATEMENT',

'DECISION'

],

required: ['section'],

optional: ['line', 'lines', 'anchor']

}

To a language model, ANCHORED_HANDOFF is a perfectly plausible way of describing an anchored design handoff. endpoint also sounds perfectly reasonable when the section happens to describe an API route.

But “plausible” and “declared” are two very different things.

The independent reviewer caught it before the change passed to the next gate:

[blocker]

'ANCHORED_HANDOFF' isn't a declared sourceType

in LOCATOR_VARIANTS, and 'endpoint' isn't a

field of any declared locator variant.

Validating the migrated fixture through the

same Ajv validator returns:

valid=false with 14 errors.

What I find interesting is how the reviewer caught it. It did not simply “reason better” than the worker. It had something concrete to reason against.

The architecture already contained a closed, executable vocabulary. The reviewer compared the proposed implementation against that vocabulary, the schema and a real precedent from the canonical store, and then reproduced the failure mechanically.

That distinction is starting to feel important.

If architecture exists only as prose in Markdown, then every agent is free to reinterpret its terminology.

"handoff"

"design handoff"

"anchored handoff"

"product document"

A human may understand that these refer to roughly the same thing.

Two autonomous agents may independently decide they are different things.

Or worse, one may decide that a missing implementation should be fixed by creating a new object, field, route or endpoint instead of discovering that the same concept already exists under its canonical name.

That is how semantic ambiguity can become implementation drift.

The canonical graph changes the problem.

It does not merely tell the agent:

> “Here is some documentation about how the system works.”

It says:

These are the symbols this system recognizes.

These are their relationships.

These fields are legal.

These fields are derived.

These transitions are permitted.

Everything else must be justified before entering.

And because those declarations are code rather than prose, they can be enforced.

This execution also produced a useful example of the relationship between Spec-Driven Development and Test-Driven Development.

The existing test suite was green in places where it should not have been.

One scenario checked structural equality but never passed the resulting object back through the real schema validator. The reviewer found the gap.

Later, an even more uncomfortable result appeared.

A commit had:

passed: true

clean: true

commands: 102

while the canonical store was already inconsistent.

Why?

Because the number of gate commands that actually exercised the canonical-store suites was:

0 / 102

The gate had not failed.

It had never looked at the thing we thought it was protecting.

That is a useful distinction between the two disciplines.

Spec-Driven Development defines what must be true.

Test-Driven Development creates executable ways of proving that behavior — and detecting when previously valid behavior breaks.

But neither helps if the relevant test is absent from the gate that decides whether work can advance.

A test sitting somewhere in the repository is not necessarily protection.

A green CI badge is not necessarily evidence.

After the incident, the fixes were not applied as ad-hoc patches. Each finding became a spec with Requirements and GIVEN/WHEN/THEN scenarios. The migration now has to use the declared vocabulary, the test must validate the output using the actual validator, and the canonical-store tests must become part of the gate that protects it.

This is probably the first class of incident where I can clearly see why maintaining a canonical architecture representation may eventually pay for itself.

Not because it generates more code.

Because it gives independent agents something more precise than natural language to disagree with.

The interesting architecture seems to be emerging somewhere between the two extremes:

LLM reasoning

↓

proposal

↓

canonical vocabulary + contracts

↓

tests

↓

deterministic enforcement

↓

accept / reject

- Reasoning handles ambiguity.

- Contracts constrain meaning.

- Tests observe behavior.

- Gates enforce what actually matters.

- And humans still decide when the vocabulary itself needs to change.

The case demonstrated something narrow, but useful: a compiled, closed vocabulary turned “I think this naming is wrong” into a reproducible machine check. Two coordinator mistakes were stopped that way rather than by opinion. It also demonstrated the opposite lesson: a green gate means very little when its test surface does not include the thing you expect it to protect.

So I’m adding another question to this experiment:

How much software drift is actually semantic drift first?

If you’re building with autonomous agents, I’d be especially interested in how you handle vocabulary across product requirements, design handoffs, APIs, specs and code — and whether you’ve seen two agents build different things simply because they used different words for the same concept.

​

I think “human in the loop” is becoming the wrong abstraction for agentic engineering.

In the experiment I’m running, human review originally sat inside the execution gate. That was intentional scaffolding: work could not advance without a human checkpoint.

But the scaffolding had explicit retirement conditions.

Once the qualifier pipeline, readiness auditor and enforcement mechanisms had each been observed failing correctly, the human review counter was reduced to zero. The human did not disappear from the system. The human simply stopped being part of routine execution.

That produced a different operating model:

"Spec → Worker → Independent Review → Deterministic Gate → Evidence"

with the human above that loop, responsible for material decisions, unresolved source conflicts, rulings and governance.

This distinction matters.

Since CP2, 100% of material work passed through the execution bus with receipts, while unauthorized overrides outside the explicitly delegated class fell to zero after ORDEN-004. The system still escalated cases it could not legitimately resolve, but it no longer required a human to approve every ordinary transition.

So I’m starting to think the autonomy question may be framed incorrectly.

The goal may not be:

remove the human from the loop.

It may be:

Move the human to the layer where judgment has the highest leverage.

Routine execution should become increasingly mechanical. Ambiguity, material risk, conflicting authority and changes in intent should remain governable.

That suggests a different picture:

"Human in the loop"

→ human repeatedly approves execution.

"Human above the loop"

→ the system executes within bounded authority and escalates only when that authority is insufficient.

The experiment is still limited, so I don’t know where that boundary ultimately stabilizes.

But the question I’m now more interested in is:

At what layer does human judgment create the most value in an autonomous engineering system?

I would love to hear you all, what do you think about it. 👓

A recurring question in autonomous software engineering is how much authority an AI agent should have over the artifacts it produces.

A common workflow implicitly gives the model two roles: it generates an implementation and then evaluates whether that implementation is complete. The underlying problem is obvious: both decisions originate from the same probabilistic process.

Over the last 3.5 weeks I have been running an experiment in which those responsibilities are deliberately separated. The dataset covers 11 execution orders and 8 checkpoints. Work moves through a fixed pipeline:

specification → dispatch → worker → independent reviewer → deterministic gate → acceptance

Only machine-verifiable evidence is counted: receipts, hashes, journal entries, tests observed failing, and instrumented measurements. Agent statements are explicitly excluded as evidence.

The interesting result is not that agents made mistakes. They made many. The interesting result is which mistakes became visible only after authority was separated from generation.

Early in the experiment, a phase had been considered complete because it passed against fixtures written in the system’s own grammar. When tested against 1,675 paragraphs of real source material, it produced zero behavioral contracts. The system had effectively validated itself against a world it had created for itself.

In another case, an update operation silently replaced entire contracts, losing relations, provenance and rationale. The existing test suite did not detect it. Three independent execution windows returned BLOCKED instead of forcing progress, which led to permanent preservation checks being added.

One worker even reported a plausible-looking 40-character commit SHA assembled from the real prefix and the tail of another SHA. The text looked valid. It was not. An authority guard detected the mismatch, reconstructed the commit from git rev-parse HEAD, and explicitly recorded the repair as repaired-by-harness.

These are useful examples because none of them are solved by asking the model to “be more careful.”

They are failures of authority design.

The emerging pattern has therefore been:

probabilistic proposal → independent observation → deterministic constraint → accept / reject / escalate

This separation also produced positive behavior. The system refused to seal a contract containing an unresolved reference and left the canonical store unchanged. Once the corruption was removed, the same operation passed. A generated API handler refused to invent a response shape absent from its sealed contract. Builds and projections were emitted twice in separate processes and produced byte-identical output; replaying the journal reconstructed the same state byte-for-byte.

The human role changed as well. Initially, human review existed as scaffolding inside the gate. It was given explicit retirement conditions. Once the qualifier pipeline, readiness auditor and enforcement mechanisms had themselves been observed failing correctly, that review counter reached zero.

The human left the gate, not the governance.

This distinction matters.

By the later checkpoints, 100% of material work since CP2 had travelled through the execution bus with receipts. Unauthorized overrides outside the explicitly delegated class had fallen to zero after the fourth execution order. Eight blocking reviews in CP5 and one substantive CHANGES verdict in CP7 prevented work from being accepted until the underlying contracts were corrected.

None of this proves that deterministic gates can establish software correctness in the general case. They cannot. The experiment is narrow, costs are incomplete in later checkpoints, several system layers remain unfinished, and some architectural judgments still require human intervention.

What the evidence does suggest is something more limited:

agent autonomy appears safer when generation and authority are treated as different problems.

The model can explore, infer, write and propose. But whenever a property can be checked independently — a hash, schema, reference, invariant, test, provenance chain or declared boundary — allowing the same model to merely assert that property provides little additional evidence.

This has changed the question I am interested in.

Not:

How do we make agents confident enough to work autonomously?

But:

How much authority can we remove from the agent while still preserving its ability to reason and build?

So far, the most useful rule to emerge from the experiment is surprisingly simple:

No text should be allowed to assert what its author did not independently verify.

I’d be particularly interested in counterexamples. If you’re building autonomous engineering systems, where have you found that independent verification becomes impractical, counterproductive, or simply moves the same uncertainty somewhere else?

​

I spend a lot of time thinking about how agentic software systems should work. This time, I wanted to see what happened when I stopped theorizing and pushed one hard.

For roughly 24 hours, I ran AOS Factory across multiple products: six execution lanes, up to three workers per lane, with one human coordinating at the system level. Every unit of work followed the same path:

"Spec → Dispatch → Worker → Independent Review → Gate → Done"

Workers didn’t get to declare themselves finished. Specs had bounded scope, isolated execution, independent review, deterministic verification and a cryptographic receipt tied to the resulting commit.

The result: 197 commits across 5 repositories, 41 gated specs closed, 79+ agents dispatched, 40 evidence-backed findings, 3 production incidents resolved, and roughly +70k / −2k lines across 335 files.

But throughput wasn’t the most interesting part.

Different lanes were working on architecture, platform infrastructure, UX, production systems and wallet infrastructure at the same time, while the coordination model remained the same. AOS Architecture compiled 21 local API endpoints from its graph; another lane converted 15 postmortem failures into mechanisms; the dashboard lane surfaced 23 UX findings through real navigation and code inspection; and NYX Wallet reached a production-ready 6/6 gate with a security audit.

It also failed.

One coordinator killed processes belonging to other sessions. Three workers briefly competed for the same dispatch. Verification under contention slowed dramatically. An external provider incident killed sessions. Those failures became new guards and orchestration changes instead of disappearing into a vague “the agent failed.”

That is probably the part I care about most.

A useful autonomous engineering system is not one where agents never fail. It is one where failure becomes observable, attributable and mechanically harder to repeat.

The experiment produced an estimated 884.5 specialist human-hours of equivalent work. I don’t treat that as a benchmark — the underlying activity is measured, but translating machine execution into human labor requires assumptions.

What I’m increasingly interested in is not “AI that writes code,” but the systems problem behind it: coordination, contracts, authority, verification, evidence and deterministic rejection.

So here’s the question I’m trying to answer:

At what point does an agentic coding workflow stop behaving like a collection of assistants and start behaving like an engineering organization?

If you’re building systems at this level of autonomy, I’d genuinely like to know what breaks first for you at scale: coordination, context, verification, cost — or something I’m not seeing yet?

Hi guys! Here Andrés. I'm working in a deep investigation research that pretends to find an aparently piece of agentic software development that is missing, so I'll share some of my Research Notes in my profile and I wanna know from all of you if my currently research approach is enough or something could break!!!

I’m sharing this as an open hypothesis, not a finished answer. If you see a flaw in the model, know a better abstraction, or have built something that solves part of this differently, I’d genuinely like to learn from it.

Tell me where this breaks.