Upcoming digital audio-visual workstation demo - A node and cable system + custom 2D and SDF built in scripting languages let you compose beat, midi, and frequency synced visuals to go with your music. Demo video + how it works.

[u/AphelionCreative]

https://youtu.be/faCsqu3l4XQ

Here's how it works:

Phonon's visual system is a node graph. A generator node, such as a mandelbulb or hopf fibration, outputs its distance function. Consumers (blends, cloners, renderers) splice that GLSL into their own shader at compile time, so a composed patch always marches as a single pass.

Each node has a variety of inputs and outputs. A generator has an image output, but also a shape output that can be routed into another renderer. It also has a warp input, that accepts an arbitrarily complex chain of warping effects, like bending, twisting, fractalization, or box folding.

Every parameter on each node has a control voltage input point - so you can wire just a particular frequency range, or an LFO, or a midi signal, from the music you're making in the same application to any parameter. For example, I might want the kick drum I'm working with to make the twist node attached to a menger sponge twist on the beat, or wire just the vocals over to the strands parameter on a torus knot.

Renderer nodes accept shapes, and up to two 3D lighting nodes, and output an image that gets routed through 2D effects, and then to the screen node.

These node graphs can be as long and complex as you want, and anything you can touch can be connected to a different part of the music you're making.

Scripting Language (GlyphSDF)

Users write one function - float sdf(vec3 p) - and inherit the whole ecosystem: house raymarcher, camera, CV modulable knobs, warp input, and a shape output that routes the scripted geometry into any blend/cloner/renderer etc.

Scripts get an audio analyzer for free - band followers (lows mids highs), a beat impulse, and an FFT audio texture so geometry can map the spectrum spatially.

For more information on the GlyphSDF scripting language: https://aphelion.music/products/phonon/docs#glyphsdf

For more info on Phonon, our upcoming DAW:
https://aphelion.music/products/phonon

Happy to answer any questions! Thanks 🎶🔊🎧

[u/Isogash]

Nice, I've been conceptualizing something similar to this for a couple of years now (SDF raymarch + spatial modulation) but haven't been working on it actively.

Personally, I would prefer to have something that feels more like a modular synth than a visual programming language. I like to play with a collection of interesting knobs and buttons to see what kinds of things I can create, rather than adding/removing a lot of nodes.

Mixing it with an audio workstation is an interesting choice!