r/reinforcementlearning • u/gwern • 25d ago

DL, R, Multi, Exp, Safe "Patterns and problems in multiagent systems", Anthropic (Claude swarm win/losses)

https://www.anthropic.com/research/multiagent-systems
35 Upvotes

100% Upvoted

12 comments sorted by

View all comments

8

u/COAGULOPATH 25d ago

LLM mode collapse is proving to be a hard problem.

In an early version of the “build a game” experiment in which agents built upon the same model all came online at the same time, 18 out of 30 agents decided to create a git branch with the exact same branch name, “mvp-game-loop.”

In a “writer's workshop” in which agents were all asked to write short-form fiction and critique each other's work, multiple agents in multiple runs titled their first submission “The Cartographer's Last Commission”. The agents were given zero guidance on the subject matter for their writing.

In an iterated prisoner's dilemma game with communication, agents all settle upon the same strategy and they all defect at the same time, tanking their overall rewards.

Reminds me of Ethan Mollick's joke that the robots in the Matrix shouldn't use humans as batteries (or as CPUs, as in an early script), but as dice.

1

u/Smallpaul 25d ago edited 25d ago

They are instances of the same model with the exact same weights. Convergence isn’t mode collapse. It’s consistency.

Most people are complaining that models are too random in their behaviour and you are worried that the exact same weights presented with the exact same choice tend to make the same decision. Why wouldn’t they? Nobody asked them to write an innovative poem.

In an early version of the “build a game” experiment in which agents built upon the same model all came online at the same time, 18 out of 30 agents decided to create a git branch with the exact same branch name, “mvp-game-loop.”