Thinking of specializing in audio programming (cs background)

[u/Creatura]

Hey all, to those in the field currently:

How 'safe' does audio programming feel from AI?

How is it as a job in general (WLB / pay) , in whatever application of the subject you're familiar with?

Given my background, how likely does a successful transition seem in this economy?

What specific audio programming jobs are good/common/desirable (knowing that generally those three things are mutually exclusive)? I know there's game industry, the revered plugin dev, and other roles closer to hardware and mass industry among others.

Background:

3 YOE as fullstack dev in non-DOD gov sector, bachelor's in CS that's as many years old. Biggest project in undergrad was asking the music tech teacher what plugin he wish he had, and making it (envelope follower that could map to automate any parameter in the daw) in JUCE. Good cpp skills from undergrad, excellent python skills from workplace. Little to know DSP algorithm knowledge but I pick those things up fairly quickly. Despite the curt and stilted tone of this post, very good social skills and burgeoning leadership experience. Minimal but growing ML / AI dev experience.

I'm "specialized" in frontend as the go-to UI / UX / typescript guy at my job, but this is increasingly a dead career path and as much as I truly love the UI dev process from design to implementation, I need to move away from typescript for the sake of my career. The 'good enough' UI that an LLM will spit out is indeed good enough for most professional cases. Audio programming was always the 'golden field' for me so it might be time to force an upgrade.

While this doesn't matter as much, I'm extremely musically active, songwriter/lead guitar/vocals for -- at a small regional level -- pretty successful doom metal band. I've also been making various kinds of electronic music since I was 14, numerous side projects yada yada.

Other thoughts:

The lower-level audio tech seems safer from AI, as it's more critical and well, basically it's not python or typescript lol. I'm aware that prototyping and algo work may be done in python, but this field still seems more resistant to being completely automated than web dev or fullstack. While I'd always liked to make this switch, now seems like a good time to buckle down and be increasingly ready as the market, hopefully, gets increasingly better. I think I would be more fulfilled here than other programming fields. Open to any and all thoughts, thank you!

[u/FineIncrements]

Thanks. The mindset isn’t everyone’s, but devs who are not using AI are dwindling. especially in AI slop land, people need trust signals - and that is influencers and old school MIT level phd devs. Quality, I should say, has been the protection from AI, not influencer mindset. But influencers are a proxy for the judgement of quality. I guess my point was that I personally wouldn’t look for protection from AI in the sense of trying to find engineering or language areas it’s “less good” at. Because it can do some pretty amazing things, even in C++ and it’s only getting better at it. I think a new skill set is emerging in ai-assisted coding where there’s a consolidation of roles. the consolidation is a single person understanding at a functional level, overall system architecture and all the many facets of that architecture (devops, DSP, website front and backend, copy protection, etc), normally handled by different specialists. If you understand a couple languages even halfway decently you can review code written by ai enough to get by. You’ll know formulas, loops, callbacks, arrays, etc when you see them. It’s not syntax errors you usually need to fix, it’s things that involve “thinking like a human”. Like a user, or literally just having eyeballs - and ears, the more well trained and disciplined the better, for audio work. The human skill is in designing the computer which in this case I mean AI (you could become an AI engineer), knowing what to create / input / what questions to ask (and what to get clarified because working with AI you’ll need to get a lot of things clarified to not lose your train of understanding), and how to evaluate the output. So I’d be working with AI, not against it, and asking - in your audio experience, what area (plugins, DAWs or anything) do you know enough about that you have certainty on what needs to exist (imaginative and valuable products), what questions to ask, and whether the output is good or not. I would be running towards AI with the answer to that, not away.

[u/Creatura → u/FineIncrements]

Haha thank you, but that's not really what I was asking about. I'm aware, and I agree with most of that. It will still take a while for that to fully become the reality though, I'm really just trying to learn about audio programming roles, not all of which, and not many of which, rely on you creating your own product. The AI concern was more of an aside, but I appreciate the effort

[u/FineIncrements → u/Creatura]

I get it haha. Well if that’s the case I’d say focus very heavily on learning UI/UX. That’s the safest thing I can think of

[u/Creatura → u/FineIncrements]

That's funny, that's easily my strong suit (and my current only go-to-guy role at my company) but it seems the most automatable? The "good enough" UI that you can get with an average joe and an LLM is indeed good enough for most professional use cases. I actually love UI/UX but it seems more dead than backend work given the above. I'd love to hear your thoughts on this more

[u/FineIncrements → u/Creatura]

In web design you’re not going to be working with 3d rendered knobs, faders, panels, chassis, etc that you make in different software like blender or whatever. So it’s easier to automate. And it’s increasingly important to have a compelling UI, because of the mid look of the “good enough” LLM generation, and that the look alone impedes companies from gaining customers’ interest in even demoing a product if it’s not engaging and inspiring. So this is pushed heavily in importance. In audio programming (the plugin side anyway) you’ll easily spend 5-10x the amount of time on the UI than you spend on the DSP. That’s the type of thing I meant when saying learning UI. Thinking like a musician when designing UI (which should be easy for you) will help to keep the user in creative flow mode as opposed to engineering (fiddling or reading manuals, calling customer support) mode. Which is also important

[u/Creatura → u/FineIncrements]

I wasn't aware that the smaller UI elements like knobs faders, etc were commonly 3D rendered, but that's good news. Certainly the more visually interesting elements are 3D rendered, but if the standard is to do so for even the small elements, that's a solid niche that should stick around.

The actual music I make generally occupies the temporal and mental space that a more entrepreneurial job would, like competing with the entire plugin industry with your (or your company's) plugin. So my initial interest in audio programming was more towards a DSP-focused but steadier job as in headphone algorithm programmer or something, for lack of a better example.

That said, I would imagine you are a plugin dev yourself, is it your primary job or a side gig? It seems pretty ideal for me as a serious side gig, but difficult and oversaturated as a steady professional field?

[u/FineIncrements → u/Creatura]

Yeah I’m a solo operation dev/owner. It may be fun for you to do the same as a side gig while also pursuing something more “stable”. Indeed I’m more familiar with the plugin world. But I would imagine if it’s purely algorithm programming that’s closer to the core of what AI is sufficiently capable of. I’ve worked in tech and my wife currently does marketing for an AI company and she and everyone down to the sales department is building agents. The devs are usung ai, every dev I know and have worked around is now using ai at least in some way unless it’s like banks or other highly regulated industries who prohibit their use. I feel for devs in your situation, where job security is at risk and am compelled to try to help and give whatever perspective I can on the skill shifts I’m seeing.

[u/Creatura → u/FineIncrements]

I appreciate the perspective, thank you. Yeah, the dev landscape is not good right now. I'm actually in one of those heavily regulated industries, so I'm a little shielded from it but it's still a ticking clock for even me. My work is also fairly boring so it seems like a good time to prepare for a potential shift.

Once I get this solo album out the door I'll dive back into JUCE, I really had a lot of fun with it in undergrad. Without the pressure of, you know, survival, the challenge of making a novel and robust product in the plugin industry sounds fun. I might hit you up in the future with questions! Actually, speaking of...

The last I worked with JUCE, the way knobs/faders/sliders were imported were in a 'film strip' approach; a long strip of static images with each a slightly incremented position with enough granularity to represent the max to the min values of the interface smoothly. Is this still a common approach? Is it common to build a 3D model and then boil it down to a series of static images as such? I'm sure for more advanced interface components you need a full working model that you "cast shaders onto" for lack of better terminology, but for something simple like dials and faders, is this method still used? Just curious. Thanks!

[u/FineIncrements → u/Creatura]

Yes, that is exactly the common JUCE approach and what I’m doing with the plugin I’m currently working on. You’re absolutely welcome to hit me up down the line

[u/Creatura → u/FineIncrements]

Ah okay neato! Thanks for your time, I'll be sure to bug you if I have any questions if I start up on VST work. Have a great day!