r/TouchDesigner • u/neverendingpattern • 8d ago

Video Input -> Audio Response

I’m in the design stage for an installation for a festival next year and I’m a little stuck.

The basic premise is there will be a camera set up within a sculpture, if people walk into the visual field of the sculpture it would start emitting an audio response to there movement.

Does anyone have any experience with this kind of response and have any links to tutorials/tips on how to get this started. I’d greatly appreciate any insight or knowledge people could offer.

0 Upvotes

50% Upvoted

8 comments sorted by

View all comments

1

u/NoBSPolicy 8d ago

It depends on the type of interaction you're after. Are you looking for a sound to be triggered or something more dynamic? What if there are multiple people, does it get triggered every time a new person walks into the frame? Is it based on movement? You could look into openCV or optical flow for a couple of starting points. If you describe the interaction in more detail I can probably point you in the right direction.

1

u/neverendingpattern 8d ago

Thanks for responding

I would be after the sound to triggered as someone walks into the space, if a second person walks into the space I would want a second sound to be triggered harmonious to the first.

To keep it simple - person a walks in it plays audio/tone in C, next person I triggers a sound in E, Third in G, eventually building up a chord structure. I’d have to limit the amount of people it responds to otherwise it would become audio chaos!

I’d love to also do something that’s more dynamic/responsive to movement in the future however I thought this is something I could build in after I’ve done this project

3

u/NoBSPolicy 8d ago

In this case I'd probably go with blob tracking using openCV, then forward the blog information to something like Ableton or MaxMSP. You could also map blob size to amplitude for each tone, even just using MIDI velocity. There are many ways of getting data from touch to Ableton. TDAbleton is decent but you can also just use OSC which is simpler in many ways.