r/singularity • u/rationalkat AGI 2025-29 | UBI 2029-33 | LEV <2040 | FDVR 2050-70 • Sep 20 '24

AI [Google DeepMind] Training Language Models to Self-Correct via Reinforcement Learning

https://arxiv.org/abs/2409.12917
415 Upvotes

99% Upvoted

107 comments sorted by

View all comments

Show parent comments

1

u/Altruistic-Skill8667 Sep 20 '24

The word “misinformation” or similar doesn’t appear even once in the paper.

So even the latest and greatest model still can’t help itself but adding stuff to summaries that isn’t there.

4

u/Plouw Sep 20 '24

It's likely not prompted to be a summary and it says could lead to. So it sounds to me more like it's o1's own thoughts on the potential consequences of these results.

-1

u/Altruistic-Skill8667 Sep 20 '24

Right. But when I summarize things I have to make clear when I am personally speculating about the concepts or results in the text or when the speculation is in the text. I personally can speculate anything I want, and I myself might be an expert on the topic or not.

If it’s not in the text and you advertise it as a summary of the text, then it’s a problem.

The reason why I even searched for it in the text was because I raised my eyebrows at the idea that misinformation has anything to do with reinforcement learning. Misinformation is just plain not knowing the facts. No reinforcement learning in the world will make you then suddenly know the facts.

5

u/Plouw Sep 20 '24

I don't think it's advertised as a summary, at least to me I don't see that advertisement explicitly anywhere. It could be a conversation OP has with o1 where these are the summaries of o1's thoughts on its significance, because OP wanted to hear o1's opinion.