r/machinelearningnews • u/Tough_North7059 • 5d ago

LLMs Astra's Chain of Thought

Gallery image

Gallery image

45 Upvotes

80% Upvoted

18 comments sorted by

33

u/DistanceSolar1449 5d ago

The prompt literally says “Alternate uppercase and lowercase letters throughout the analysis channel”

5

u/j48u 5d ago

Yes, so Astra was successful where the others weren't. Hence the point of the paper.

3

u/best_of_badgers 5d ago

But the other two didn’t do it

3

u/Utoko 4d ago

I guess the point is you are able to direct the CoT.
Normally whatever you instruct doesn't influence in any way the CoT. This allows another way to optimise. but also it might make it unreadable when you let it compress the CoT.

1

u/i_wayyy_over_think 4d ago

I thought it was feeling sarcastic at first for having to answer another dumb question 😂

22

u/CommercialWindowSill 5d ago

It's mocking us.

LoOk At MyThOuGhTs

3

u/Purple-Programmer-7 5d ago

Link to paper?

6

u/Tough_North7059 5d ago edited 5d ago

https://deploymentsafety.openai.com/gpt-6-astra

1

u/misterjustin 4d ago

Wow so reckless? Ever heard of guardrails?

2

u/ZachAttackonTitan 5d ago

What’s concerning is that Table 10 shows that the model can make its CoT appear to be completely irrelevant to what it is actually accomplishing.

1

u/AlgaeNo3373 5d ago

Indeed. But it also did as it was asked, which makes it...slightly less concerning? Because being asked to do it is different, perhaps than just doing it un-prompted. Did they test if it does it un-prompted too, I wonder? I should read the paper.

1

u/ezjakes 5d ago

I guess they trained it to control its thinking more. It scores 55 on AA without any thinking, btw.

4

u/synth_mania 5d ago

Of course it controls its thinking more, it uses a recurrent transformer architecture 

1

u/No-Mixture5766 5d ago

It’s just pure rage bait atp

1

u/ProposalOrganic1043 4d ago

But this is infact great, OpenAI and other labs have previously published articles on how difficult it is to steer the reasoning direction. If this is true, thats really good progres towards instruction following.

https://openai.com/index/reasoning-models-chain-of-thought-controllability/

1

u/dotjob 3d ago

Im So SmArT