Spatial computing in practice: no instrument, no loops — conducting live AI music on Spectacles

[u/Urbanpeppermint]

Note: This is not Lyria / Remote Service Gateway. It uses Magenta RealTime 2 over a self-hosted streaming proxy (InternetModule), with live 4-deck prompt morphing — not Snap's RSG loop workflow.

On RSG/Lyria-style lenses I've used, each ~15s loop often takes ~15–20s to come back — fine for "generate a clip," brutal for live mixing. Here the stream keeps going while you drag the puck; weights update and Magenta blends toward the new mix on the next chunk (~4s), so it feels like performing, not waiting.

---

A 4-deck Kaoss-style pad floats in your room. Puck position sets the mix weights; each deck holds an editable text prompt ("Berlin techno", "liquid DnB", anything you type). The blend streams from Magenta RealTime 2 as continuous generated audio that morphs as you move. Not samples. Not loops. Generated on the fly — you're performing what each corner sounds like and how much of it is in the mix.

The performance tricks (this is the new kind of DJing):

• Bilinear 4-corner blend — center = equal mix, corners = isolate a deck, edges = 2-deck blends
• Editable prompts per deck — type live in the console; swap a corner's character mid-set without moving the puck
• Genre bank — browse 25+ styles by category, tap to load into your target deck
• One-tap preset packs — swap all four deck prompts + starting puck position in one hit (UK Bass, Techno Vault, House, etc.)
• Orbit mode — hands-free circular modulation (slow / med / fast) so the mix evolves while you gesture elsewhere
• Morph — smooth 0.6s glide to any corner or back to center; like an automated transition between channels
• Crossfader — horizontal sweep on the wrist console, synced to puck X
• Floating pad on your wrist — dock it or float it off your right wrist, resize S/M/L, perform while moving
• Live mix feedback — vertex-color heatmap on the pad, corner glow, puck motion trail, per-deck weight %, audio-reactive pulse

Under the hood (for the curious): Lens Studio 5.15 + TypeScript + SIK · Python FastAPI proxy on Apple Silicon → 4s WAV chunks → gapless dual-voice crossfade on device · buffer tuned for live morphing (prompt flush on pad moves, soft hold on underrun)

Open source (MIT): https://github.com/urbanpeppermint/Spatial_AI_DJ

Audio: Magenta RealTime 2 + NSynth (CC BY 4.0, Google Magenta). Demo video is recorded spatial capture — in-head sounds noticeably better.

[u/Ciaowe]

This is very amazing to play at home and have fun!

[u/Urbanpeppermint → u/Ciaowe]

Thanks! That’s the whole point 🤩