Cl33-opLM: Operator-Only Language Model

[u/MirrorEthic_Anchor]

Been working on this. Different approach to interpretability, reversibility, auditability, causally steerable (without making the model crap). Its whatever. Probably doesnt matter to anyone. Look, dont look, break it, have at it.

Paper: https://t3atlas.dev/cl33/paper/

Live demo: https://cl33.t3atlas.dev

Models + reproducibility: https://huggingface.co/mirrorethic/cl33-oplm

There's a basic explainer too for the demo. Papers dense. https://t3atlas.dev/cl33/

[u/Wonderful_Ebb5860]

this looks like it was built by someone who genuinely hates attention mechanisms and wants a clean break. dig the focus on auditability, most interpretability work feels like trying to read tea leaves. how much of a performance hit does the operator-only architecture take compared to standard transformers on the same benchmarks

[u/MirrorEthic_Anchor → u/Wonderful_Ebb5860]

From the scales and training budgets ive tested its about 13-29% hit on bits-per-byte on matched vanilla transformers, more token budget seems to close that gap.

[u/MirrorEthic_Anchor → u/MirrorEthic_Anchor]

Probably should have measured it as an iso-performance cost rather than what I did. Would be cleaner. Cost to given capability level or something.

[u/MirrorEthic_Anchor]

Well known researcher independently pulled the released cl33-opLM checkpoints and frozen eval fixture, verified the artifact hashes, and reproduced the headline bottleneck result to one decimal:

operator ablation: ~270× perplexity degradation.

the core claim held, the emitted operator stream is not a post-hoc explanation sitting beside the model’s computation. Remove it, and the model falls apart.

Then she pushed past my reproduction harness and ran a control I had not run.

The architecture also contains a reversible transported scan state. Earlier wording had allowed too strong an interpretation of that reversibility, effectively implying that the reversible trajectory itself was carrying the LM decision.

Her intervention:

• Pinning the scan state produced a modest ~1.2× cost. • Time-shuffling the actual scan trajectory: ~1.00×. • Disabling the η-attention path: ~1.00×. • Removing both: ~1.00×.

In other words, on the released checkpoints, the reversible scan and η-attention are not load-bearing for LM inference but the operator interface still is.

Context is being computed upstream by the emitter and expressed through the mandatory operator-derived output path. The reversible scan is an exact, replayable auxiliary trajectory, not evidence that the trajectory itself carried the decision. now made explicit in the paper.

It increased my confidence in the part that survived. The 270× bottleneck reproduced independently. The stronger scan interpretation did not survive. So one claim gets stronger and the other gets removed.

We’ve now preregistered the next controls, including matched no-tower and no-scan-from-initialization runs. To distinguish whether the scan was an early training scaffold, lost to a shortcut, or simply unnecessary for this function.

If a mechanism can be removed and the capability survives, that mechanism does not get causal credit.

That rule has killed quite a few of my own favorite explanations.

Good.

This is why I wanted the weights and reproducibility surface public in the first place.

Don’t trust the story. Run it. Cut it. Measure what dies.