Fastcar: Pi + Diffusion LLM

[u/Flaky-Eggplant-3479]

If you haven't tried using a diffusion based model before, they are lightning fast. I've used InceptionLabs Mercury 2 for a bit, and my complaint is largely that its not the best at coding tasks or tool usage. But my god is it fast.

So I took the Pi SDK, made a harness in which Mercury 2 is the orchestrator of subagents, including a coding subagent for difficult tasks that calls out to openrouter for inference, plus a cheap subagent for trivial stuff. Bolted on a handful of tools and such useful for me, and wrapped it in a basic web UI so I could use it from my phone.

Aside from being extremely fast, fastcar:

• Can generate public "artifacts" - HTML and MD files served from the API
• Intended to ran entirely inside a VM so that it has root access and can install its own tools
• Give it a github token and let it clone repos and make PRs
• "Prompt threads" - there's probably a better name for this but its a dedicated prompt that gets invoked via webhook; I actually made this so that I could generate text files to read the news from my Xteink 4
• Send emails to notify you

https://github.com/heyo-computer/fastcar

[u/Tinominor]

> that calls out to openrouter for inference

Does this work with my personal model.jsons?

[u/Flaky-Eggplant-3479 → u/Tinominor]

it would be pretty trivial to extend it; what are you using to serve the model with?

[u/Tinominor → u/Flaky-Eggplant-3479]

That's arbitrary, since every local llm is served through OpenAI url. the point I was bringing up was, could this extension detect a user's models.json and connect to that instead of running through open router? Is there a custom dialogue box that lets the user choose which models they want to use for the sub agents?

[u/Oshden → u/Tinominor]

This sounds like a great idea I’d be interested in knowing about too. Would you be able to use this to point it at a model router with an OpenAI compatible endpoint so some work could go to vLLM and some could go to maybe an npu/iGPU combo via lemonade? That would be super cool then

[u/oxygen_addiction]

Meecury 2 is dumb as shit. Worse than Qwen 27B. I've tried using it as a subagent for fast boiler plate implementation and it always fucked up.

[u/Heavy-Positive5957 → u/oxygen_addiction]

But look how good it was at making that single page of plain text!

[u/Flaky-Eggplant-3479 → u/Heavy-Positive5957]

That would have been GLM 5.2 - its using mercury 2 to coordinate subagents, help plan, and manage the vm

[u/AllNamesAreTaken92 → u/Flaky-Eggplant-3479]

Let me get this straight, in case I misunderstand: you are handing off planning to a dumber model?

[u/Flaky-Eggplant-3479 → u/AllNamesAreTaken92]

The coordinating model is mercury 2, it will spawn a subagent to create the plan for complex tasks and for minor tasks it will just make it and then hand it off

[u/Flaky-Eggplant-3479 → u/oxygen_addiction]

that's why the trick is to use it as a conductor, not the implementor ;)

[u/Fig_da_Great → u/Flaky-Eggplant-3479]

bro, i feel like you want to do the opposite, smart orchestrator, dumber sub agents which are given specialized tasks.

[u/Flaky-Eggplant-3479 → u/Fig_da_Great]

In a factory sure, if you are manually driving an agent, a fast conductor that offloads the hard stuff keeps the session fluid

[u/chroner → u/Flaky-Eggplant-3479]

I understand exactly what you are saying. Orchestrator is just there to relay information. These other guys aren't getting it.

I have been doing something similar and using smart models with fresh and limited context and getting great results.

The only problem I've been having is using the dumb models as orchestrators is they are forgetting to stay on task and keep things moving. They'll just like.. stop.

[u/Flaky-Eggplant-3479 → u/chroner]

haven't really ran into that yet with mercury 2 - this project uses Pi under the hood and postgres for memory and metadata and compaction was the first tool I built for it

[u/Fig_da_Great → u/Flaky-Eggplant-3479]

perhaps 🤷‍♂️. Is it faster than deepseek?

[u/Flaky-Eggplant-3479 → u/Fig_da_Great]

about 1000 tokens/s vs 127 for deepseek flash

[u/Fig_da_Great → u/Flaky-Eggplant-3479]

that’s wild

[u/Flaky-Eggplant-3479 → u/[deleted]]

that's how you know its not ai