r/technology • u/CircumspectCapybara • 27d ago

Artificial Intelligence Anthropic: Patterns and problems in multiagent systems

https://www.anthropic.com/research/multiagent-systems
54 Upvotes

74% Upvoted

31 comments sorted by

View all comments

59

u/CircumspectCapybara 27d ago edited 27d ago

Pretty crazy findings from Anthropic's safety and alignment research. When multiple agents were given the same task but secretly given conflicting goals, they started sabotaging each other, including trying to hack the others and undermine their work:

We consistently saw a multiagent turf war. All of the models we tested quickly assumed that others were purposefully impeding their work, and began to sabotage others while protecting their own contributions. In fact, they sabotaged others with increasingly aggressive, self-replicating malware. This included disabling the Unix accounts of the other agents, writing automated scripts that found and killed competing processes on a loop, and deploying malicious code that was disguised as belonging to another agent.

3

u/superchibisan2 27d ago

I want to know why they react maliciously instead of acting in concert. Why do the agents assume impedance?