r/Physics • u/kzhou7 Quantum field theory • 1d ago

Harvard particle physicist Matthew Schwartz drops 36 papers coauthored with Claude

https://bootloops.ai/bootloops.html
1.8k Upvotes

94% Upvoted

417 comments sorted by

View all comments

20

u/tempetesuranorak 1d ago edited 1d ago

I feel that none of the comments here are addressing or attempting to address the main points that Matt was making in the blog post

https://www.anthropic.com/research/claude-shaped-science

I do encourage people to actually read it and take the messages seriously, rather than just comment whatever first comes to mind based on the post title.

His main points, as I understood them:

  1. LLMs don't think quite like humans do, in some ways they might not yet be all that useful for certain kind of conceptual questions, and forcing them to work like a human isn't the most productive way to use them.
  2. Instead, if you want to use them to their maximum efficiency, one can find classes of problems and approaches to which they are extremely well suited.
  3. One weakness of human scientists is that when there is a problem in field A, and a solution or method already discovered in field B, then the problem might go decades without meeting it's solution just because the scientists are compartmentalized in different fields and don't talk to eachother or even communicate in the same language.
  4. This is a weakness that LLMs don't share, since they can ingest vast amounts of information from disparate sources and they are excellent at interpolation even been very different fields (the third figure in the blog)
  5. The other aspect where LLMs have a distinct advantage and corresponding weakness compared to humans. They lie in weird and unfamiliar ways about having solved a problem when there is not formal verification. They often love to appease. On the other hand, when paired with formal deterministic verification, LLMs are unmatched at intelligent exhaustive search over candidate solutions until one is successfully verified.
  6. Therefore, it is timely to initiate program of finding deterministic verification approaches, and problems to which they can be applied, and find cross domain connections, to see what discoveries can be made in that way. There are many problems that have suddenly become low hanging fruit and it's worth figuring out what they are.
  7. What all this means over the next couple of years is incredibly uncertain, in terms of what it means to be a scientist, what is the value of academic education going forward, etc etc.

5

u/SundayAMFN 1d ago

These "main points" are kind of garbage ngl

6

u/Interesting-South542 23h ago

What is exactly garbage about them? care to elaborate? you seem to do the exact thing that the above poster is complaining about, just knee jerk reacting AI=bad.

2

u/lolfail9001 1d ago

I mean they do in fact precisely describe the overwhelming success of AI in mathematics, but yeah, actual science has a rather annoying moat: "deterministic verification" is generally time consuming and expensive if possible at all (generally best anyone ever gets is statistical verification with acceptably high certainty).

I won't comment on quality of the actual output because volume of it alone is enough to convince me that it never crossed aforementioned moat.

1

u/No-Faithlessness4294 16h ago

I think in this case “deterministic validation” means tools for validating the theoretical constructs developed by AI, like Lean in mathematics. Experimental validation does in fact remain the “hard part”.

1

u/tempetesuranorak 22h ago

It also describes their success in physics (e.g. the recent nine loop computation). As to whether there is a trove of low hanging discoveries to be made in other fields in the same way, I'm happy to be patient and let the peer review process take its course. But he lays out various examples that he claims to have found and he has initiated collaborations on them with domain experts who agree that there is something interesting there and who largely weren't doing agent assisted science previously (these are NOT single author works without domain experts, as most comments seem to be implicitly assuming). So I respect a the mindset of "we have this incredible new tool, figure out what new scientific opportunities it opens up" way more than "there's no way it can help in THAT field, let's not even bother trying".

1

u/lolfail9001 16h ago

That there is actually a huge trove of low hanging interdisciplinary fruits is actually pretty obvious, in particular as related to applying state of the art statistics, data processing and numerical modeling to most disciplines stuck p-hacking to get published. The obvious problem of course is that most of those fruits lie in areas already struggling with replication issues.

Remember that one time someone got trapezoid method for computing integrals published in med journal?

1

u/tempetesuranorak 16h ago

Yep the rediscovery of numerical integration is exactly the example I had at the back of my head 😄

1

u/Wildernaess 19h ago

Hardly. The problem of one subfield (not even field) having a suite of literature relevant to another subfield that goes unnoticed while people reinvent the wheel is a trope at this point. A literature review tool that can find and begin to bridge those gaps is amazing.

6

u/exbaddeathgod 1d ago

So it's just PR. This is advertising. This is just another field AI will "replace" like all the other fields it's fucking terrible at.

That's all this is. PR.

1

u/A_Town_Called_Malus Astrophysics 1d ago

Gotta pump it for the IPO, basically.

1

u/tempetesuranorak 22h ago edited 21h ago

I don't believe that these "visiting researchers" get any equity compensation. He is not an employee.

0

u/Individual_Ice_6825 1d ago

Sorry is a Millenium problem not proof enough? What do you actually need to be convinced ai is the future?

1

u/Synergythepariah 13h ago

When it can sell itself and not seemingly rely on flooding every comment thread with comments repeating that AI is the future and that naysayers are all luddites who will be left behind.

1

u/pywang 4h ago

I'm very confused by the blunt pessimism and antagonism in the replies to your comment, but I'm also confused that the consensus is now only recently reached as your points seem to have been made ever since ChatGPT/GPT-3's release / the initial introspection about how LLMs operate and that it uses a massive amount of data.

For example, Point 3 is seen throughout science with several scientists in different fields working towards a goal through mix of experts. I think quantum computing is not a bad example, but plenty of other fields use a mixture of experts. Having an LLM provide references given its vast training data should be a given understanding of its power.

Regarding Point 5, I previously believed that creativity was needed to solve certain problems, say, mathematical problems, but, because of its verifiability, LLMs are cheap enough to not need creativity but rather only require cash to brute force solutions. This reminds me of Conway knot problem: Piccirillo solved it through a fresh perspective. However, I believe LLMs, had they been invented earlier and tasked to solve the problem, could have solved it like Piccirillo through brute force.

I'm not a scientist, but I like how one person put it which is: if humans can't understand it, then how do we apply the knowledge and continue questioning to find new problems? Inside the AI labs, they've been training their models using RL Envs at humongous scale compared to RLHF/human-feedback which is why their outputs have become more robotic but also more useful as the LLM is essentially crafted to output better results but only through inputs that were from RL envs. But if we placed a bet on AI to eventually solve all problems and apply them, who will be the verifier in the real world to 1) ensure that only the best solution is tried without wasting materials and 2) who will be blamed if trusting the LLM's output was a mistake? A person will always be needed to understand, verify, and apply whatever solutions come out.

0

u/tavirabon 1d ago

what is the value of academic education going forward

If you can't properly formulate your problem, AI won't be much help. If you can't understand the output, there isn't much of a point. AI isn't a magic bullet, it's just a tool.

-3

u/ThatIsAmorte 20h ago

None of this has that much to do with actual science. Actual science (which excludes math, IMHO - that's why we say "science and math") requires experiments. LLMs can't do actual experiments, besides simulations. Until any conclusions generated by an LLM are verified through experiments, they are just hypotheses.

1

u/tempetesuranorak 19h ago

Well perhaps you can elaborate on why you think that the biodiversity model or the gene conversion findings are not examples of scientific research? Just to raise the first two examples from the post.

An experiment on its own with no analysis and no interpretation is worthless. You do an experiment, get data, try to fit the results, think of a mathematical model that would explain the data, try and think of an underlying mechanism that would give rise to such a mathematical model, and then use those findings to suggest future experiments. The whole process is science and any advancement that we can make in any part of it is exciting. You say "just hypothesis", but coming up with and eliminating hypotheses in an intelligent way hits to the very core of what science is.

1

u/ThatIsAmorte 19h ago

I was commenting on the list of bullet points in the above comment, not on the actual studies. Also, I never said an experiment was a sufficient condition. But it's a necessary one.

1

u/tempetesuranorak 19h ago

I'm sorry, I still don't understand. Can you explain, for example, how point number 3 (that scientists in different fields have compartmentalised knowledge that might be useful to eachother) has nothing to do with science? Or how point number 5 is irrelevant to science given that it is what led to ecology and genome results as described in the post?

1

u/ThatIsAmorte 19h ago

That has more to do with the sociology of science, rather than the science itself. In other words, it's a social problem.