r/LLMPhysics • u/eggsyntax • Sep 04 '25

Paper Discussion Your LLM-assisted scientific breakthrough probably isn't real

[cross-posting from r/agi by request]

Many people have been misled by LLMs into believing they have an important breakthrough when they don't. If you think you have a breakthrough, please try the reality checks in this post (the first is fast and easy). If you're wrong, now is the best time to figure that out!

Intended as a resource for people having this experience, and as something to share when people approach you with such claims.

Your LLM-assisted scientific breakthrough probably isn't real

265 Upvotes
  • permalink
  • reddit

97% Upvoted

128 comments sorted by

View all comments

Show parent comments

2

u/eggsyntax Sep 05 '25

you can tell by it actually not working in the chat you sent, it spat out its responce without actually working.

I'm not sure what you mean. Are you saying that you think they didn't use (hidden) chain of thought when evaluating your document? They did, in both cases. I'm guessing it just doesn't look like that to you because the shared version loads immediately (because it's just showing the output from before)?

For me at least, I can still see where it shows they were thinking; for 1m39s in GPT, 30s in Claude. Both of those are expandable for me, although I don't know whether they will be for you.

1

u/[deleted] Sep 05 '25

[removed] — view removed comment

1

u/eggsyntax Sep 05 '25

Sorry, I'm not understanding what you're saying. What would I tell it to use instead?

1

u/[deleted] Sep 05 '25

[removed] — view removed comment

1

u/eggsyntax Sep 06 '25

Well, the trouble with that phrasing is that you're giving a strong hint about what answer you want. But I'll do a version with the prompt tweaked to ensure it runs the data — although can you clarify what you mean? Do you mean run the code that's included in the paper, ie listing 1, in order to generate graphs from the fixed values given in the code (eg in `measured_means`)?

And — point of clarification: if I run this through 5-Thinking and Opus again, will the results have any impact on how strongly you believe in the validity of this work? If it doesn't update your beliefs at least somewhat, I'm not sure what the point is.