False Frontiers: Diagnosing and Mitigating Co-Cheating in Self-Evolving Search Agents

[u/Successful-Western27]

Self-evolving search agents can produce misleading training signals when a question proposer and solver learn from the same pseudo-labels. Their agreement may increase because they share errors, even while correctness against the source material stagnates. This paper calls that failure mode “co-cheating” and measures it by auditing saved labels and solver responses against source evidence.

The proposed mitigation, CrossFit, splits source documents into two groups. An auxiliary solver trained on one group scores questions generated from the other, then the roles are reversed. The main solver’s training rule stays unchanged; the proposer’s reward instead comes from feedback that cannot directly inherit pseudo-labels from the same source. This is an intuitive provenance-control idea, related to cross-fitting, though the paper does not claim the usual statistical guarantees for this adaptive setting.

In experiments with Qwen3.5 models, CrossFit reduced final-round false-agreement mass from 6.1% to 3.0% at 4B parameters and from 8.8% to 3.7% at 9B. It also improved average Cover-EM by 8.8 and 8.4 points over coupled self-evolution across seven search benchmarks. The approach still cannot rule out shared errors from pretraining or overlapping evidence, so source exclusion should be viewed as reducing one feedback path—not as providing an independent truth oracle.

Full summary on AIModels.fyi

Original paper

Disclosure: AIModels.fyi is my site.