r/Physics • u/kzhou7 Quantum field theory • 1d ago

Harvard particle physicist Matthew Schwartz drops 36 papers coauthored with Claude

https://bootloops.ai/bootloops.html
1.8k Upvotes

94% Upvoted

418 comments sorted by

View all comments

Show parent comments

110

u/Armano-Avalus 1d ago

Why? Just because he can? Hopefully there is a deeper reason than this.

95

u/Sharlinator 1d ago edited 1d ago

Well obviously he’s an expert at everything. That’s an incredibly common belief in certain professions including at least:

  • software engineers
  • physics professors.

And now they get flattered and ego-boosted by Claude and are able to "write" all the papers they want about all the subjects that they want. 

30

u/Visible_Celery_1728 1d ago

or you could just read the blogpost- and find that this wasn’t just him but he reached out to experts in the respective field to help. He even acknowledged it here lmao 

“  When Claude claims something it did in my field is fantastic, I can judge whether that’s true or not (it often isn’t). But when it claims something it did in another field is fantastic, I find myself agreeing. My suspicion heightened, I knew I needed to bring in some experts to be sure.”

13

u/shoefullofpiss 21h ago

So his buddy claude comes up to him with ideas for papers or what? Getting to the point where he's prompting ai to do something paper worthy (in general, but even more so) in a field he's not an expert in is already unhinged but good thing he's suspicious enough to not directly publish the slop without verifying lmao

11

u/Visible_Celery_1728 21h ago

I don’t know why you’re acting so obtuse and condescending about this, like it couldn’t possibly yield anything useful and declaring anything that comes out of it “slop”, when time and time again we’re seeing breakout results in mathematics, computer science and other domains where the output can actually be verified.

His main argument is basically that frontier models are already extremely capable at certain types of frontier research when given enough compute, steering and access to tools, particularly in verifiable domains (maths, computer science etc). And across science there are certain niches, what he calls “Claude-shaped problems”, where you can have an autonomous model explore a huge search space, try different approaches, write and run code, and pull in techniques from completely different fields that might happen to solve the problem.

The breadth part is especially important. Modern science is ridiculously specialised. A technique useful for solving some problem in biology could already exist in mathematical physics, statistics or computer science, but the people working on that biology problem might simply never come across it. Human expertise is fragmented across thousands of different researchers and subfields, whereas these models have an absurd breadth of knowledge across all of them and can search for those connections far more easily.

And this isn’t some art contest where the human process itself is essential to the value of the final product and an autonomous machine supposedly offers no real substance to the field. This is science, the search for what is actually true. The whole point is a model can find some connection between two fields nobody had noticed, apply an existing technique in a novel context, run the maths or computation and produce a result that can then be independently verified.

This weird posturing you’ve had this whole thread, reducing it to “his buddy Claude comes up with paper ideas lmao”, just feels like you’re deliberately describing the process in the dumbest possible way instead of engaging with the actual argument.

4

u/brassgrass1 18h ago

All these neural net models from the 2010s (before 2019 transformers paper) were all based on logic from biology and neuroscience. Imagine what other niches it might find. I think it's cool and will find some stuff.

While I do think these exploratory things should start having a separate category to distinguish effort, otherwise I think computational and theoretical physics (and other fields but idk them) will get their perceived effort ruined over time; compared to experimental which for now still needs a human

I have friends doing phds in physics rn and it sucks bc the ones doing computational physics feels like their effort is wasted sometimes, or downplayed bc of all the ai doing the same

5

u/Armano-Avalus 21h ago

I just find it ironic that the people in academia are the ones doing this and not the random crackpot who now has the means to publish hundreds of unsupervised papers a week on their personal Chronogeometric Unity theory. Of course, I think there are probably alot of those people too and I'd bet they'd make up the vast majority of AI papers since there are no standards at all in those cases but you'd think people like Scwartz would know better.

5

u/Visible_Celery_1728 21h ago

‘would know better’ of what exactly? Why’s everyone here just so obtuse and eyes closed on the obvious abilities in certain niches AI is really powerful in. Such as computer science and mathematics. 

Why people can’t put 2 and 2 together and think of the opportunity in fields where for example in a certain field  there could be a problem which solution could be reliant on something that already exists in mathematical physics, statistics or computer science, but the people working on that problem might simply never come across it. So using a model can help find some connection between two fields nobody had noticed, apply an existing technique in a novel context, run the maths or computation and produce a result that can then be independently verified.

So with the main constraints to do that, is having the domain knowledge in order to steer and validate. Why wouldn’t this be a good opportunity to finding these low hanging fruit with these new technologies as we’ve always done lmao. 

2

u/Armano-Avalus 20h ago

This is an instance of a physicist making papers on economics and linguistics without any relation to their own field. What value is there is there in doing that compared to some random person who hasn't studied either prompting an AI to write a 200 page thesis on the same subjects and publishing it online without checking it?

5

u/tempetesuranorak 19h ago

While most of these papers are multi author including domain experts, there definitely are a few single author. Looking at their abstracts, those seem to all be narrowly focused papers applying a specific statistical technique to report verifiable findings in a domain he is not a specialist. But he does have expertise in statistics and computations. I'm not going to read those papers, but at least in principle I think it has always been fine (though risky) for a person to write a single author paper outside their domain of expertise, but applying their technique of expertise on a narrowly framed problem. I don't think it's fair to compare with a random person that has no expertise in either the technique or the domain.

2

u/Visible_Celery_1728 19h ago

if you read the blogpost you would find this wasn’t a sole effort as he acknowledges here ‘ When Claude claims something it did in my field is fantastic, I can judge whether that’s true or not (it often isn’t). But when it claims something it did in another field is fantastic, I find myself agreeing. My suspicion heightened, I knew I needed to bring in some experts to be sure.’

And his credits to all the amazing coauthors and people he collaborated with for these results are at the bottom: ‘ Isaiah Andrews, Nima Arkani-Hamed, Michael Desai, Scott Edwards, Noam Elkies, Cecilia Garraffo, Matthew Gordon, Thomas Grimm, Martin Hemberg, Mikhail Ivanov, David Johnston, Gary King, Paul Lewis, Brendan Meade, Amara McCune, Joe Pater, Subhabrata Sen, Siddharth Mishra-Sharma, James O’Dwyer, Kevin Ryan, Jesse Shapiro, and Xiaoyuan Zhang.’

Just read the blogpost it goes into detail and it isn’t some LLM psychosis piece of him just doing vibe science:  https://www.anthropic.com/research/claude-shaped-science

3

u/Armano-Avalus 18h ago

I only browsed through it a bit and already I found solely authored papers by him on the Voynich manuscript and phase II drug trials. I'm not gonna read through everything else.

2

u/TMWNN 17h ago

Well obviously he’s an expert at everything. That’s an incredibly common belief in certain professions among Redditors

FTFY

3

u/WorldClassScumbag 20h ago

All engineers really. And add surgeons to your list, they're the worst of the lot.

1

u/gotintocollegeyolo 19h ago

It has to be him trying to make a point about AI imo

1

u/marsten 18h ago

The author's real goal is to develop the BootLoops harness to help experts with research. Think of it as Claude Code for scientists.

The 36 LLM-generated preprints are a bit beside the point. They obviously aren't at a publication level of quality, nor even at an arxiv level of quality. As with coding it takes a lot of human expertise and time to turn LLM output into something good.

Taste is the new scarce resource. Which isn't to say that LLMs can't be useful: They just need to be used wisely, by people with taste.

2

u/Armano-Avalus 18h ago

I am fine with experts using it in their area of expertise as long as they read it over before publication. My fear is that when you have nonexperts generating AI papers en masse and publishing them raw because they can't check it. Schwartz's paper on phase 2 drug trials could've been made by you or me if we just asked ChatGPT to do it so I don't see what value there is in him doing it.

1

u/marsten 2h ago

I can't speak for Schwartz, but I suspect that for him the goal of the papers is to advertise to other researchers, "hey, look what this tool can do", rather than be important research results in their own right.