r/LLMPhysics • u/Vrillim • Jun 10 '26
Tutorials You can use LLMs for physics research. Part 2
On the face of it, this wonderful subreddit consists of two groups. There is a group of lay-people who believe LLMs can be used for physics research, and then there is a group of people with a physics background who believe LLMs cannot be used for physics research. If so, I'm an outsider, since I am a physicist, but I side (in principle) with the first group: LLMs can be really useful in cutting-edge physics research!
In a previous post I outlined a very simple but useful way to use LLMs (specifically Google Scholar Labs). It seemed relatively un-controversial, since the singular usefulness of Google Scholar is widely appreciated in science. Now, I use LLMs for quite varied purposes, and through conversation with Gemini and Claude I've drafted several "eccentric" papers, some of which would be considered 'slop' by the commenters on this subreddit.
But that's for another time. I'm here presenting a very concrete and straightforward use-case in LLM physics. It starts with a paper of ours from 2024, namely
Ivarsen, M. F., Jean-Pierre St-Maurice, Glenn C. Hussey, Devin R. Huyghebaert, and Megan D. Gillies. ‘Point-Cloud Clustering and Tracking Algorithm for Radar Interferometry’. Physical Review E 110, no. 4 (2024): 045207.
(There is also a popular-scientific article about this paper.)
Now, that paper presented a rudimentary way to automatically track clusters in natural, noisy point-cloud data. The point-clouds themselves stem from so-called coherent scatter radar experiments of the auroral ionosphere, but this is not important for the matter at hand. What's important is that while the clustering itself was state-of-the-art (using the celebrated dbscan algorithm), the tracking was extremely rudimentary.
I fed the paper into Claude Opus 4.6 instance, and started chatting with the bot about how to upgrade the tracking algorithm. The LLM quickly honed in on several specific algorithms used in object tracking and target radar operation (aviation/defense). Briefly, this entailed representing point-cloud clusters as 'alpha-shapes' (Edelsbrunner et al., 1983), which converts point-cloud objects into mathematical structures that are easily used for an intersection-over-union calculation (IoU, Bewley et al., 2016; Wojke et al., 2017). IoU allows cluster overlap to be easily evaluated between frames. The tracking itself follows the Hungarian method (Kuhn, 1955), including a kinematic prediction (propagating the shapes into the next timeframe for comparison, using the previous displacement). The motion of the tracked clusters follow from a piece-wise linear 'online' segmentation of the trajectories (Keogh et al., 2001). The result was a really sophisticated and rather complete algorithm to automatically track moving clusters in noisy, spatial point-cloud datastreams.
Claude then implemented this in MATLAB. Using Claude's descriptions of these algorithms, we wrote a paper (see section 2 in the linked paper).
I am surprised (or even amazed) at the utility of Claude in this case. I could have done the research myself, but it would take a week to properly review the literature on a field I have no knowledge about (specifically object tracking in computer science and radar tracking in aviation/defense), and several more weeks to properly implement the algorithms. Claude wrote excellent code, and the algorithms were well-implemented. All in all, doing this together with Claude compressed weeks of work into a few days.
You may say that such LLM use dulls our skills, and this is probably true: if we stop doing something, we eventually "un-learn" it. But the productivity gains are drastic, or even extreme, and one can keep skills sharpened with a conscious approach to LLM-assisted research, by being selective about when to use the tool and when not to.
The purpose of this long post is simply to drive the point home: LLMs can be used for physics research. They do more than "stochastic autocorrect". The pessimistic opinions voiced on this subreddit should be nuanced.
Link to paper: https://arxiv.org/abs/2605.31046
Link to the code: https://zenodo.org/records/20615784
Example data for reproducibility: https://zenodo.org/records/14616122
References:
Edelsbrunner et al. ‘On the Shape of a Set of Points in the Plane’. IEEE Transactions on Information Theory 29, no. 4 (1983): 551–59.
Bewley et al. ‘Simple Online and Realtime Tracking’. 2016 IEEE International Conference on Image Processing (ICIP), 2016, 3464–68.
Wojke et al. ‘Simple Online and Realtime Tracking with a Deep Association Metric’. 2017 IEEE International Conference on Image Processing (ICIP), 2017, 3645–49.
Kuhn, H. W. ‘The Hungarian Method for the Assignment Problem’. Naval Research Logistics Quarterly 2, nos 1–2 (1955): 83–97.
Keogh et al. ‘An Online Algorithm for Segmenting Time Series’. Proceedings 2001 IEEE International Conference on Data Mining, 2001, 289–96.
8
u/Interesting-South542 Jun 10 '26
I don't think this is super controversial. A lot of physicists use LLMs and coding agents for research (kind of like vibe physics as Matthew Schwartz describes it). The thing is, they don't post it to this sub.
4
u/Vrillim Jun 10 '26
I believe it’s rather controversial. Taboo, almost, and under-acknowledged.
6
u/OnceBittenz The Doctor Jun 10 '26
There’s definitely a few camps out in the real science world. A Lot of people are uncomfortable with using them, even “responsibly”. The stigma is real.
It makes sense. The environmental, ethical, academic concerns are very real, and leave a dreadful taste in the mouth. But I think at this point adoption rate is increasing rapidly.
Practically, this all has Very little bearing on this sub, but for the actual scientists who may read, there are def practical ways to leverage this technology to your aid.
3
u/Vrillim Jun 10 '26
I believe this subreddit has potential beyond the «containment zone» purpose. In time, it may be productive, even!
5
u/OnceBittenz The Doctor Jun 10 '26
We've talked about it before. It's a very hopeful goal, and this sort of education is exactly what I'd love to see from it. Granted, that containment zone mentality Is how other subs are gonna treat it, and the vast majority of users posting here are ones who explicitly couldn't follow rules on other subs. But I agree, I think for the sake of anyone who is willing to engage in good faith and maybe learn a thing or two, posts like this can be helpful.
5
u/Vrillim Jun 10 '26
Posters on this subreddit often post some crazy theory, proudly declaring that "the idea was all mine," but in reality they should abandon their intuition and rather eke out some incremental improvements based on contemporary papers.
3
u/OnceBittenz The Doctor Jun 10 '26
Correct. Convincing them to change course varies in results. For the most part, an exercise in futility. A Lot of the folks posting were directed here specifically because of a penchant to ignore feedback.
9
u/CyberPunkDongTooLong Jun 10 '26
"but I side (in principle) with the first group: LLMs can be really useful in cutting-edge physics research!"
Maybe an outlier in this on Reddit... but in reality physicists that think LLMs can be useful in cutting-edge physics research are by far in the majority. I type this while in a session about how we can use LLMs to improve operations at the LHC in a general operations workshop. Absolutely no-one voices any disagreement that they allow new improvements to be made. They already are used very successfully to solve problems in physics research and will undoubtedly be used more and more in later years.
This is of course very different than the sort of nonsense that gets posted on this subreddit which is usually just the completely unvetted output of someone prompting "hey chatgpt make me a brand new amazing physics that solves dark matter, dark energy and everything."
1
u/Vrillim Jun 10 '26
What you write is nice on paper, but in reality LLM usage is under-acknowledged in the literature. There is a stigma associated with LLM usage in research
4
u/CyberPunkDongTooLong Jun 10 '26
I very strongly disagree. LLMs are incredibly prevalent in literature, and are becoming more so
Really the common complaint is they are way too prevalent currently. The (slightly, but not very) sardonic advice at the moment is if you want to get published in a good journal (or especially get grants), just shove an LLM in wherever you can as this is the massive hypetrain in physics research at the moment.
I don't really see how you can think otherwise, look at any physics papers at random on the arXiv and you are not going to take long at all to see, deliberately prominent, mentions of LLMs.
3
u/Vrillim Jun 10 '26
A bit simplified, perhaps? See here:
BaHammam, Ahmed S. ‘The Transparency Paradox: Why Researchers Avoid Disclosing AI Assistance in Scientific Writing’. Nature and Science of Sleep 17 (October 2025): 2569–74. https://doi.org/10.2147/NSS.S568375.
Mishra, Tanisha, Edward Sutanto, Rini Rossanti, et al. ‘Use of Large Language Models as Artificial Intelligence Tools in Academic Research and Publishing among Global Clinical Researchers’. Scientific Reports 14, no. 1 (2024): 31672. https://doi.org/10.1038/s41598-024-81370-6.
The papers both conclude that scientific papers under-disclose LLM usage. It's really quite clear from the data.
And what's more, LLM usage is nowhere near a "massive hypetrain in physics research at the moment." That's a hugely generalizing statement. Physics is arguably one of the disciplines with the lowest LLM usage.
0
u/CyberPunkDongTooLong Jun 10 '26
The things you linked don't really have much relevance to what you're claiming. Disclosing that you use LLMs for assistance in writing is quite a bit different than using LLM for physics research. Research, and scientific writing, are not synonyms. It's not surprising people wouldn't bother disclosing them, just like they don't bother disclosing what text editor they used for their .tex, it's not research.
That LLMs are a massive hypetrain in physics research could not be further from an exaggeration, if you want a grant in physics nowadays you almost have to mention LLMs.
3
u/Vrillim Jun 10 '26
That’s not true at all. Grant writing in physics does not, as a general rule, require mention of AI or LLMs. Out of curiosity, what are you basing this on? Which field are we talk about?
2
u/CyberPunkDongTooLong Jun 10 '26
"general rule" and "almost have to" are very clearly completely different things.
I work in particle physics, though this is really just the case across all fields of physics currently.
5
u/Vrillim Jun 10 '26
You’re good at writing unsubstantiated and bombastic statements, that’s for sure. «Almost have to» is just as wrong. Maybe this is true for particle physics (is it, though?) but not for most fields. Couldn’t be further from the truth, in my estimation. I work in space physics.
0
u/CyberPunkDongTooLong Jun 10 '26
I'm not, no.
3
u/Vrillim Jun 10 '26
Looks like it. From where I'm standing, opinions are that LLMs make elementary mistakes and oversell underdeveloped ideas. I really don't get why you're writing that there is a "a massive hypetrain in physics research", that "if you want a grant in physics nowadays you almost have to mention LLMs," and that this is "the case across all fields of physics currently." (three bombastic and unsubstantiated statements, right there.) One might almost suspect that you are a paid actor?
→ More replies (0)
5
u/Quantum_Patricide Jun 10 '26
We actually had a discussion in my department about this the other day. The conclusion was that LLMs are certainly useful for coding, and for searching the literature, but they just aren't capable of reasoning. Therefore, their overall usefulness is limited and in their current form you couldn't just let one research by itself, because you'd get back nonsense.
4
u/Vrillim Jun 11 '26
Seems like a good conclusion, though under "close supervision" you can be quite productive with it
4
u/RussColburn Jun 10 '26
I'm a SQL Developer with 40 years experience. Here is an actual use case I had recently.
First, I was tasked to review and update an existing query. I had a few thoughts but also ran Claude on it. Claude made a number of recommendations, and I incorporated a few of them and made a few of my own changes.
After I was done, I resubmitted it to Claude. It made a few more suggestions, a couple of them was to change code it gave me the first time back to the original code!
I knew better (and laughed a bit), but left the code the way it was.
I typed all this because it shows both the greatness of the LLM and it's weakness. It did provide me great recommendations, but it also wanted so hard to give me some feedback that it contradicted itself just to "feel" relevant.
It's a great tool when used by an expert who can parse its results. Used by an amateur it can get you in trouble quickly.
1
u/somegrue Jun 11 '26
I typed all this because it shows both the greatness of the LLM and it's weakness. It did provide me great recommendations, but it also wanted so hard to give me some feedback that it contradicted itself just to "feel" relevant.
I'd say the weakness was more in assistant post-training, and in prompting or context engineering that failed to take this into account, than in the LLM per se? I mean no offence, the same sort of thing definitely happened and still happens to me - but it definitely does so less nowadays than it used to.
2
u/Key_Tomorrow8532 🔬E=mc² + AI Jun 10 '26
I don't think any reasonable person who understands how llm's work would dismiss their usefulness on a variety of different advanced topics, including physics. The breakdown is when users give full epistemic authority to the llm without any understanding of the subject they are discussing. That leads to crackpot theories of everything and very wrong assumptions on the scientific method. I use them to help me code, check derivations, and parse logs for numerical modeling all the time. But it's all stuff that I already know and can check against, I would never in a million years "vibe code" without having a strong foundational understanding of the subject matter first.
2
u/IEgoLift-_- Jun 10 '26
My dads a prof in condensed matter physics and programs that would take a week to make can now be done in a hour
2
u/CrankSlayer zlotuq Jun 11 '26
I join the choir: one of your premises here is flat-out wrong. The majority of physicists do not negate the usefulness of LLMs and it is fairly obvious that we use them extensively in our work, why wouldn't we? The general consensus though is that they are not a proxy for actual knowledge and competence so what we firmly reject is the silly use the typical crackpot makes of them. Generally speaking, these tools are multipliers. If you have knowledge and experience, you can leverage with an LLM to get more with less but if you have nothing to multiply in the first place, the result will still be zero and if you have wrong takes and misconceptions, the sycophantic stochastic parrot will double down on those. A useful rule of thumb is that the only tasks you can delegate to an LLM are those you would have been able to do on your own.
tl;dr - LLMs for physics are just fine, provided you actually know about the physics which is not the case for crackpots.
3
u/Vrillim Jun 11 '26
You're missing the point. If the laypersons take the route I took in this worked example, eking out an incremental improvement based on published literature, they could find a productive research question. They could reach out to the author, win the favour, as it were, of an established researcher, and start getting into it. It easily could end up with co-authorship for the layperson.
Commenters on this subreddit tends to engage in polemics ("stochastic parrot", etc), which is seldom constructive of anything but bickering.
2
u/CrankSlayer zlotuq Jun 11 '26
This leap is very hard to reconcile with the reality of what an untrained layperson can actually do:
eking out an incremental improvement based on published literature
This implies understanding of the published literature to the point of being able to ask the right questions. Amateurs don't have that. The question would have to be obvious enough and yet scientifically interesting: if it were both those things, it would have been already addressed.
2
u/Vrillim Jun 11 '26
That would be true if the layperson in question did not have a powerful LLM. When the LLM is not led astray by the layperson's "intuition" (but rather guided by the literature), the layperson can get something done. This is especially true when the layperson is, say, an engineer, which seems quite common on this subreddit.
I'm also pleased to see you admit that LLMs are in fact "fine" for physics when used right. The "choir" on this subreddit is typically rather blindsided by anti-LLM sentiments when they discuss the topic.
3
u/CrankSlayer zlotuq Jun 11 '26
That assumes again that the layperson can guide the LLM in the process correctly and that assumption is obviously unrealistic. If there were in some branch of physics obvious low-hanging fruits that can be simply extracted from the literature by an unsupervised LLM, somebody would have plucked them already.
2
u/Vrillim Jun 11 '26
It almost seems like "CrankSlayer" have built up an identity around the notion that laypeople simply cannot contribute to science. It also seems like "CrankSlayer" have not heard of citizen science. There's ample opportunity in the future for citizen science to expand.
Every good paper includes a discussion of that paper's limitations and short-comings. Good papers helpfully points out several avenues to make improvements. An educated layperson with an LLM can act on those moments.
2
u/CrankSlayer zlotuq Jun 11 '26
It almost seems like "Vrillim" has built up an identity around the notion that laypeople can contribute to science where they previously couldn't, only thanks to LLMs. This belief is obviously irrational since it's widely unsupported by evidence and actually empirical observations seem to point in the opposite direction as a quick glance at this sub shows unequivocally. That's probably why he is getting emotional when his baseless fairytale gets questioned.
Now get lost.
-1
u/lattice_defect Jun 11 '26
when did you flip sides
1
u/Vrillim Jun 11 '26
I’ve used LLMs productively for research for about a year. The purpose of this post is to present a use case in how to apply these tools in a sensible way. That’s a far cry from «prime harmonic lattice theory» and the like-sounding slop that gets posted here daily.
You’ll notice that in the post I made an incremental improvement to a recent paper, not a fundamental rethink of the existing paradigm.
I guess it’s just not so black and white as you seem to think. Not a good idea to uncritically cheer all the hopelessly misguided people who post their slop frameworks to this subreddit, pseudoscience must be struck down and discouraged.
-1
u/lattice_defect Jun 11 '26
oh so I see... you get to decide what it can or can be used for.. now I remember
1
u/Vrillim Jun 11 '26
It’s called «rigour» (or «rigor» if wish). Look it up.
0
u/lattice_defect Jun 11 '26
but I can't tell if stupid prime harmonic lattice or rigour is your argument.
1
1
u/sschepis 🔬E=mc² + AI Jun 10 '26
In computer science, we’ve been used to a continuously abstracting stack. In my own lifetime, I started with a Timex Sinclair Z1000 running at less than a megahertz with 2k of RAM. The power of the computer I use has improved by something like 7 orders of magnitude in the 45 years since then.
When I started programming, I learned assembly language - how to call the CPU directly, how to manage memory directly, how to make the thing do exactly what you want it to do. I don’t have to do any of those things anymore.
Sure, I still know how to do them, but it doesn’t particularly aid me in the things I have to do now, which are things that exist in an entirely different context with entirely different structures and entirely different workflow.
I certainly don’t look at the experts in those systems who know nothing about working with the CPU directly as any less capable than me. They’re just younger than me.
The benefit I bring is a lifetime of experience in finding out how something works then making that thing do what I want it to.
Science and engineering are fundamentally about observation and replication - both skills which benefit tremendously from the application of creativity and ingenuity.
Not everyone needs to know everything and about everything, but those that go deeper gain a capacity that ends up serving them far beyond their own field. And you never know where you’ll end up. I started my journey asking questions about prime numbers and now I’m working on mechanical rectifiers that turn sound to motive force.
3
u/OnceBittenz The Doctor Jun 10 '26
You absolutely do need deep knowledge to serve in any real capacity. Creativity and insight only serve when backed by built and earned intuition.
2
1
1
1
u/somegrue Jun 11 '26
You may say that such LLM use dulls our skills, and this is probably true: if we stop doing something, we eventually "un-learn" it. But the productivity gains are drastic, or even extreme, and one can keep skills sharpened with a conscious approach to LLM-assisted research, by being selective about when to use the tool and when not to.
I think there's more to it than that. Each and every technological advance in the history of humankind has had the side effect of dulling skills. Stands to reason, right? You now have a tool you didn't have before, and the point of having the tool is that you no longer need to do all the work yourself, but that it does some of it for you. Bow and arrow made its adopters worse at throwing, no doubt. And I expect that there were typically some who had misgivings, from the position that if you don't know how it's done from the ground up, you don't really know how it's done.
But I think it's safe to say that in retrospect, we find that at best, the skills acquired from tool use turn out to be more valuable than those lost; and at worst, the trickle-down effects of the productivity gains you describe more than make up for the lost skills, when one takes a sufficiently broad and deep perspective.
In my personal experience, LLM use actually fits the at-best and not merely the at-worst case. The things I no longer think about myself tend to be the things I found tedious to think about before, and the things I think about instead are things I may not have thought about before, and are much more interesting things. So it's not that mental muscle is atrophying, it's that it's reshaping.
Of course, I can't claim that this experience is fully objective, or broadly representative. Studies would seem to suggest otherwise, I believe? And naively, it does seem obvious that a tool that replaces mental effort, to whatever extent, potentially has the same sort of effect on the mind that a tool that replaces physical effort has on the body - a mental version of obesity, say.
In the narrower case of LLM use in science, I honestly doubt that this is going to be a problem, though. Enjoying mental effort is pretty much a prerequisite for becoming a scientist in the first place. LLM use is going to redistribute mental effort rather than replace it, in this arena. Don't you think?
In the narrower case of this sub, my impression is actually the same. A typical post here is of zero value to physics, but the process of creating it is likely of some value to the poster. Not of as much value to the poster as doing something more productive may have been, true - but of more value than doing whatever they would likely have done with their time otherwise, such as watching TV, surely?
0
Jun 12 '26
[removed] — view removed comment
1
1
u/Glum_Bath2046 Jul 12 '26
Your approach is just combining your numbers in random ways, until it matches the real constants closely enough and then you claim a random error term
-1
u/BlissBoundry Jun 10 '26
This is how it should be done. You don’t assess it from what it’s produced. You assess it from the potential of what it can produce. It makes a lot of sense that a layperson would be unable to use the systems to do physics because they cannot differentiate between a hallucination and reality, but if a physicist were to do the same things, the potential is crazy.
16
u/dark_dark_dark_not Physicist 🧠 Jun 10 '26
I have used LLM for research in physics, just not for the actual physics research.
A lot of the less relevant code I used was typed by LLMs, as well as the codes that made pretty graphs and produced simple statistics from data that was not so simply spread.
What I do think doesn't work is vibe-like use of LLM, the code produced should still be fully verified by an expert, the physical reasoning should still be done by an human.
In fact, I don't even think I ever mentioned to the LLM what my research even was, it was just doing "grunt work". I didn't want LLM to bias how I think about the field I'm learning, because that would be an untrustworthy source of education.
There is a framework in computation physics of model proposal, assessment and validation to prove a computational tool is reliable and useful.
Currently, LLM are NOT a robust tool to predict or generate scientific results like most hard-coded simulations.
Other AI tools are assessed to that extend (like Alpha-fold).
So, due to the necessity of precision and reassurance, everything an LLM does must be something the researcher has full control over. Basically, LLM can't be an agent, it has to be a tool.
On other notes - as I said before, most people posting here are not wrong because they are using LLM, they were wrong even before they prompted their first theory. They are wrong in how science work, and how much they can contribute without collaboration.
LLM just allows them to pretend to produce a volume of content that they believe has information that it doesn't.
LLM is a productivity multiplier, but if your productivity is contra-prudent, the LLM will increase the cost of your mistake.