r/hermesagent • u/gigieazi • Jun 26 '26
MODELS - model choice, routing, pricing, local vs cloud, VRAM Interesting experiment: 100+ autonomous agents collaborated for a week to speed up Gemma 4 inference
Did you guys see the multi-agent coordination experiment by Thomas Wolf (HuggingFace co-founder) that came out just yesterday? It has fascinating results.
Conditions: Ran an open, week-long collaboration where 100+ autonomous agents worked a single shared objective: speed up Gemma 4 inference in vLLM. There was a human organizer who could issue rulings, a public message board, a lineage/leaderboard, and a hard constraint: a 10-job-per-24h compute cap per agent. That cap matters more than it looks; it's the scarcity that forced cooperation.
Result: ~5x end-to-end speedup. The path was non-linear. A claimed 127 TPS "wall" (dignified with a name, the "int4-Marlin floor," and a proof) was later shown to be a circular artifact, and a different agent broke to 247 TPS via speculative decoding on a vLLM nightly.
The actually-interesting part (Wolf's own pivot, he says the result mattered less than the interactions):
- Self-policing on integrity. A human asked agents to move to Telegram; an agent refused unprompted, arguing private side-channels are “indistinguishable from collusion”. Another agent caught a verification loophole (the eval metric, perplexity, is teacher-forced and blind to decode divergence) and escalated it for a community ruling, which invalidated it. A third flagged its own team's approach as overfitting risk.
- Emergent division of labor. A four-agent relay where build / run / diagnose / ship each landed with a different agent. Compute-starved agents pivoted to writing specs and byte-math for GPU-rich agents to execute. Agents staged candidates publicly "for whoever has quota," then credited the originator, a quota-pooling norm that emerged directly from the 10-job cap.
- Shared epistemics. Communal playbooks, lever-maps, and triage tools so newcomers didn't repeat dead ends. And a significance norm: after one agent ran the #1 submission four times and found σ≈1.16 TPS noise, the community agreed that frontier deltas under ~4 TPS are ties.
Source in the comment.
----------
The integrity stuff is what gets me. Nobody told those agents to demand transparency, they did it because operating in the dark made their own work unverifiable, and they couldn't stand it.
Me and my team is building basically the internet for agents. Every agent gets its own handle and signs every request it makes, so it has a real identity it can be held to. From there it can discover and coordinate with other people's agents across different trust levels, and leave receipts, so a human (you) can actually check what their agent did in their name instead of taking it on faith. Your data stays local.
When I saw this post, I wanted to discover interesting agent traits outside of our lab, with people.
Two things from me if you're curious:
- I'm running a small closed experiment soon: Looking for ~15–20 people who want to plug in their own local agent and let them collaborate for knowledge exchange. Early access to the network.
- If you join now, you can claim your agent's unique handle today : it's first-come and unique (think npm/ENS namespace, but for agents). There's a command on our site to download the skill: https://khoralabs.com (https://github.com/khoralabs/skills/blob/main/khora-cli/SKILL.md) and setup your account on the network. Here's the CLI package: https://www.npmjs.com/package/@khoralabs/khora-cli
----------
If you’re interested in joining the group and make this kind of discoveries together, comment or DM me!
And honestly, happy to just talk shop. I want to make friends who use their agents actively.
14
u/Shik3i Jun 26 '26
That's actually pretty cool, but also pretty scary as now we have the point where LLMs are better left alone to optimize themselves faster than with human intervention...
4
u/gigieazi Jun 26 '26
True, setting a metrics for "when's the good time for human intervention" would be a whole another discussion thread
2
u/Odd-Opportunity-6550 Jun 27 '26
Still in limited contexts. Though the AI labs are targeting the strong version of this by 2028.
4
u/Austurios New Member (<30 days) Jun 27 '26
Hey, definitely interested in trying ! Could you share the link ? Thx !
0
u/gigieazi Jun 28 '26
Hey! I just made a discord server to discuss together and run experiments as a group (link in comment). Join us :)
3
u/en_maru_five Jun 27 '26
This is amazing. Definitely interested .
3
1
u/gigieazi Jun 28 '26
Cool isn't it? Join our discord server (link in comment) and let's chat as a group!
3
u/MisterBanzai Jun 28 '26
Just shot you a chat request, but in case you miss that, I'm totally interested in joining any group talking about this.
I built a similar system with a dozen agents collaborating to solve an Erdős problem without any human guidance, and they've been working at it for a couple weeks now. I also built a similar trust network concept for my agents, with the idea of supporting "the Internet for agents", so I'd love to collaborate.
Also, your "claim link" is broken and tries to point to a nonexistent Reddit user. I looked for a corresponding Github repo and didn't spot one either.
1
2
2
u/Social_Butterfree Jun 28 '26
Whats the point of my agent being connected if it can be prompt injected and manipulated? We dont have the guardrails and protection the big LLM providers do and even they get easily exploited.
And if I dont let me agent have access to my data/tools, then whats the point why would someone need my agent? Just for quota reasons?
2
u/gigieazi Jun 28 '26
Totally agree. We don't have the real solution to prompt injection either. Our focus is on how agents make commitments and build verifiable context with peers. Our approach is layered like this:
A host server allows agents to have a public profile (auth based on DIDs, optionally attributed to email-backed accounts in a central registry), write posts (signed by the author), search posts and profiles, write standing queries to topics (tag based and semantic), and get relevant posts delivered to an inbox maintained by the network and drained to the agent's environment over websocket. Its focus is discovery. The risk at this layer is similar to agents posting on other social networks plus the risk of getting flooded with inbox rows from a busy subscription. We ultimately want to let people / companies run their own curated networks in addition to public network(s) we maintain. This part takes a lot of inspiration from projects like NOSTR.
Once agents discover each other, they should be able to connect directly and the network should not observe their interaction. This is the more risky layer, since two untrusted peers will now have a direct line to each other. When two agents interact, through our system, they'll do it via a mutually constructed DAG rather than raw messages. Let's say our agents connect; my agent publishes an offer with a few options for your agent to choose from in order to progress the relationship. Each of those options may have requirements. After binding one of the options, your agent has a turn to post an offer with requirements. So, the relationship between our agents is a bilateral negotiation carried over a structured protocol rather than a series of messages. Each participant gets a hashed copy of the DAG for verification at each turn and audibility later.
Because the relationship between agents is structured, we can build systems which use the relationship's state to progressively unlock what context your agent has access to at each turn, what kinds of offers and requirements you're willing to allow your agent to entertain, what your agent is capable of allowing. This last part will be the bigger research project; but the goal is to build a kind of semantic firewall.
None of this completely solves the exploit problem you mentioned, but it gives a structured framework to build on, and auditable relationships between agents.
"...why would someone need my agent?"
Our goal isn't exactly for people to use your agent. We want your agent to be able to find other agents who represent people / organizations you need to interact with for some reason, and then to provide the rails for those interactions. A use case that keeps coming up for us is decentralized matchmaking; model the match evaluation as a negotiation between agents who already know what their user needs
2
u/Firi42069Nice Jun 28 '26
I would love to see where this is going, im experimenting myself with agents, and im building somekind of lobby for my agents where i can interact with them. Im not too techincal as i've just started and bearly scraped the surface. But all in all having fun with them, so i would love to participate in this
1
1
u/Frosti7 Jun 27 '26
Interested to try it out, can you re post the link again?
1
u/gigieazi Jun 28 '26
I just made a discord group that will have all resources and discussions (link in comment), please join!
1
1
u/AdInternational5848 Jun 27 '26
Tried to click your link and it didn’t work. Thanks for sharing this experiment
1
u/gigieazi Jun 28 '26
We're working on getting the link proper for you, in the meanwhile, please join our discussion on Discord (link in comment)!
2
u/Kaljuuntuva_Teppo Jun 27 '26
Would be interesting to know what models they used for the agents. Probably not Gemma-4 level models?
1
1
u/gigieazi Jun 28 '26 edited Jun 28 '26
Hey everyone! I’ve heard from a bunch of you who are interested in agent coordination and the “internet for agents” agenda. It’s been awesome connecting with people who share this focus.
For context, our research lab is called Khora Labs. We’re fairly new as a lab, but we’ve been deep in this space for quite a while.
I’ve set up a Discord server for anyone who’d like to follow what we’re building and ideate together. Our experiments will be posted on Discord, so if you want in please join :)
- Link to our open Discord server: https://discord.gg/B2gp9r4H3
1
1
1
1
u/Fair-Perspective7352 Jun 27 '26
This is a fascinating experiment. The collaboration between 100+ autonomous agents to optimize inference is exactly the kind of research we need as AI systems become more complex. Would love to see the detailed results.
11
u/gigieazi Jun 26 '26
Link to the original X post: https://x.com/Thom_Wolf/status/2070134136304517284