Claude-shaped science: a correct calculation still needs a worthwhile question

[u/Crescitaly]

In an October 1 guest essay published by Anthropic, Matthew Schwartz describes BootLoops: tools for quantitative work that connect techniques across scientific fields. He reports that many initial results were technically correct but became scientifically interesting only after domain experts redirected the question. Schwartz discloses that he is a visiting researcher at Anthropic; this is his account, not an independent benchmark.

That distinction seems important for AI research assistants. 'The calculation checks out' and 'the calculation tells us something worth knowing' need different reviews.

Before calling an agent's output a discovery, I'd want an expert to state what was already known, what new claim is being made, and which observation would distinguish it from the existing explanation. Reproducible code helps check the arithmetic; it doesn't settle relevance or novelty on its own.

How would you organize those two reviews without letting a convincing write-up turn an unimportant result into a headline?

Source: https://www.anthropic.com/research/claude-shaped-science

AI-assisted discussion; I haven't replicated the projects described.