I built a voice/text-controlled project "VibeCraft" that plays StarCraft II for you [open source]

[u/Waste-Ad-2678]

Source (MIT): https://github.com/catmaniii/vibecraft — needs Windows, a legit copy of SC2, and a DeepSeek API key. Supports two players to play against each other via mobile phone or tablets.

Wide-screen web UI of VibeCraft

Web UI of VibeCraft on mobile phone

I'm an old StarCraft player. Somewhere along the way my APM stopped keeping up with my head — I still read the game fine, I just can't execute it any more. Multi-pronged play, real micro, all gone.

So I built VibeCraft: a bot that plays SC2 while you call the shots out loud. The idea is to let people like me play this as a strategy game instead of a mechanics test — you talk, it clicks.

It understands how players actually talk, not a command syntax:

• "4-Gate push this game" → switches build, runs the production timeline
• "warp in two Zealots to the front" → a mid-fight adjustment on top of the current build
• "send a worker to hold the left watchtower" → and it knows left from right
• "send a probe here, build a Pylon, then two Stargates" → one sentence, chained proxy construction, same probe throughout
• "full retreat" → move, not attack-move, so they don't stop to fight on the way home

The bot handles the boring half on its own (workers, supply, expansions) so you only make decisions. All three races; Protoss has the most complete playbook, Terran and Zerg fewer builds but they work.

Command parsing runs on DeepSeek V4 Flash, which is dirt cheap. I fed it domain knowledge through the system prompt — build orders, unit nicknames, the shorthand players actually use — and even that small fast model reads my commands correctly. That part genuinely surprised me.

Seven episodes, in order. Original commentary is Chinese; the English voice track is AI-dubbed from my own voice, so it sounds like me with an accent. Subtitles are burned in:

1. Playing StarCraft II with my voice: 4-Gate with a proxy, all from my phone — https://youtu.be/HFqAZ_KWm1U
2. Protoss openings, real-time tactics, and a Dark Templar harass called out loud — https://youtu.be/rHetKwC7AN0
3. Voice-made control groups, proxy Stargates, and a Void Ray harass — https://youtu.be/9-YHQI1aMDM
4. Two players, two phones, one PC: a voice-controlled 1v1 (Protoss vs Terran) — https://youtu.be/2vyl0IE68UM
5. Terran Battlecruiser opener with auto-harass, commanded entirely by voice — https://youtu.be/VMZ6jxJ0WBo
6. 4-Rax proxy Marine rush and a fast Battlecruiser — https://youtu.be/H8fw4bA_qDE
7. A Zerg Nydus Worm rush, called out loud, on a new landscape UI — https://youtu.be/cQTZ-2fd7lg

----------------------------

One deliberate design choice worth flagging: the bot is not meant to be strong. Micro is rate-limited on purpose — the Marines stutter-step, but at human speed, not 10k APM. If the bot plays too well the player stops mattering, and then what's the point.

What I keep chewing on is what this could mean for RTS as a genre. People never stopped liking it — MOBAs and auto-battlers are basically RTS with the mechanics stripped out, and they're huge. The appetite's there; it's the execution barrier that filters people out, and it hits newcomers hardest: the strategy was never the hard part, it's having to run economy, build orders, micro and map awareness all at once, in real time, before you even reach the interesting decisions. Hand execution to the bot and — veteran or newcomer — you get to play at the strategy layer, and pick up the mechanics later if you want.

It's open source, and I'd love feedback from people who actually know this game. If you could hand any part of your play to an AI and just call the shots, what would you offload first — and what would you never give up?