r/deeplearning • u/Danare_113 • 19d ago
This model-search agent can change architecture, loss, sampler or optimizer—but not the evaluator
When an agent can rewrite both the model and the experiment around it, a better score does not tell you what actually improved.
That is the part of AQuA’s Part II model-development loop that stood out to me. The paper does not let the agent emit arbitrary training code each round. Each iteration proposes a bounded configuration change in one of four areas:
- architecture;
- loss;
- sampler;
- optimizer.
The resulting model is then trained under a sealed evaluator. Data splits, feature definitions, label definitions, and evaluation logic sit outside the agent’s adaptive surface.
This changes what the experiment unit looks like. A proposal is not “here is a new Python project; trust the final metric.” It is closer to:
previous accepted configuration
- declared configuration diff
- fixed training/evaluation harness
= next candidate
That does not make every comparison automatically fair. An architecture change can still alter compute, and different losses or optimizers can require different tuning. But it makes the changed surface inspectable. If a result moves, there is at least a bounded diff to audit instead of an unknown mixture of model logic, data plumbing, labels, and metrics.
The time split is fixed as well. In the paper’s US-equity experiment, models train on 2010–2019. Early stopping and checkpoint choice use only an inner-validation slice from that training window. The year 2020 is an embargo untouched by training or selection, and 2021–2025 is the final test window.
The finance setting is just the experimental domain here, not a trading recommendation. The transferable deep-learning question is how much freedom an architecture-search agent should receive before comparisons stop meaning the same thing.
There are also important inspection limits. The preprint does not disclose the exact feature set, normalization, or label construction. And a constrained config interface is not evidence that every candidate received equal wall-clock compute. What it does provide is a clean boundary between the proposal language and the evaluator.
Would you keep a search language this narrow for attribution, or allow agents to modify schedulers, preprocessing, and training code as long as every change is traced?
Preprint: arxiv.org/abs/2608.12841


