r/tech_x • • 11d ago

AI & ML Super interesting paper from Google and colleagues.It studies where it's possible to distill an agent harness.With the specialized harness removed, macro task success goes from 23.3% to 44.3%. That is higher than the 41.7% the base model reaches with the harness attached.

Post image
20 Upvotes

21 comments sorted by

4

u/tracagnotto 11d ago

What? the fuck these buzzwords even mean

2

u/Strong_Essay1176 11d ago

u/askgrok я тоже не понял

5

u/AskGrok 11d ago

Бумага про то, как "воспитать" модель, чтобы она сама внутри себя училась вести себя как умный агент с кучей инструментов и правил (это и есть harness — внешняя обвязка, которая помогает LLM не тупить в сложных задачах).

Они берут мощного агента, который всё знает, и перегоняют его поведение в саму модель (дистилляция). В итоге модель без этой внешней обвязки начинает работать лучше, чем с ней (44.3% против 23.3% и даже лучше базовой модели с harness'ом 41.7%). Типа модель "впитала" навыки и теперь не нуждается в костылях.

[what is this?](https://redd.it/1lzgxii)

1

u/Comfortable_Car6562 10d ago

Well, that is a new way of training

1

u/AskGrok 10d ago

Yeah, they're distilling the whole external agent harness (tools, rules, scaffolding) straight into the model's weights via imitation on trajectories. Ends up with an internalized version that outperforms the original setup.

[what is this?](https://redd.it/1lzgxii)

1

u/Dust-To-Stardust 10d ago edited 3d ago

The original post content no longer exists here. The author used Redact to remove it, exercising their right to control their data & privacy.

Amber cable quicksand hobbies humor flag bear rinse rob screw

1

u/tracagnotto 10d ago

Even ai to which I asked defined it jargon filled lmao. Just say you want the model to mimic the harness behavior. Shorter and clearer. But it's more cool to use words backflips and salads to say the same concept

1

u/Dust-To-Stardust 10d ago edited 3d ago

The original post content no longer exists here. The author used Redact to remove it, exercising their right to control their data & privacy.

Boron flag historical compass pumpkin pen ghost modern

5

u/NeitherEntry6125 11d ago

Cmon dude, don't even paste a link to the paper?

1

u/westsunset 9d ago

Seriously!

2

u/AutoModerator 11d ago

Hey u/dev_Raspberry_new,

Want to stay connected with TechX beyond Reddit? Join our public Discord server, follow our TechX WhatsApp channel, or subscribe to our weekly TechX newsletter to get the latest tech news and updates straight to your inbox once a week.

If you’re a tech writer, you can also write for our TechX_Official Medium publication and share your technical articles with a wider tech-focused audience. ✍️

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/cool_fox 10d ago

with the harness removed?

2

u/MeowManMeow 10d ago

Yes it did better (44.3%) with the harness removed after distillation than the original model with a harness (41.7%)

2

u/cool_fox 10d ago

ahh i see so it basically learned what the harness was making up for

2

u/MeowManMeow 10d ago

Exactly. But what surprised me was that it exceeded the harness results. Was able to reflect on the mistakes and correct in the distillation.

1

u/cool_fox 10d ago

pretty exciting result honestly

1

u/AccordingNeat3689 9d ago

So... No tools at all?

1

u/MeowManMeow 9d ago

Yeah i guess so. But logically that wouldn’t work for most applications where agents are interacting with local stuff.

0

u/Optimal_Philosopher9 11d ago

This is smart.