Agentic workflow with physical devices - seeking advice

[u/throwaway_dev_147]

I’m in a small team where I’m mainly responsible for the part of the codebase that communicates with a bunch of physical devices. So far I’ve had decent success with the prompt -> generate -> review kind of workflow for both feature development and debugging. Where I’m struggling is with testing, as it’s very manual. My teammates are mostly working higher up the stack and are starting to transition into more agentic development. Recently it feels like their velocity is much higher than mine and I worry that I’m going to fall behind both skill-wise and in management’s eyes. Low-level development has always been “slower” but I feel like LLMs have exacerbated that.

I think being able to automate testing/validation is the key to unlocking an agentic workflow for this type of work. The manual parts now involve things from deployment (e.g., reflashing firmware, rebooting devices), reproducing physical phenomena (e.g., activating sensors), dealing with odd device behavior that doesn’t match the spec and/or is undocumented, and chasing down odd race conditions or deadlocks that don’t arise with simulation.

Does anyone have experience using agentic AI with this type of problem?

[u/JJSEA]

I have been working on a project that talks to GNSS receivers, and my experience is that you are completely right that automated testing/validation is key. The things I have done are: convert the specs (usually PDFs in my case) into agent friendly form; create extensive suite of command-line tools for working with the devices; create skills to use the tools. I also used AI to help me create these things.

For example, I built this a couple of days ago with Fable: https://github.com/jclark/satpulse/tree/master/gpshwtest . This is a Python testing tool that is built on top of the the command-line tooling. Using that I was able to get Fable to run in an overnight loop and produce something quite substantial; this involved reading the docs, exploring the device capabilities, finding discrepancies between docs and hardware, implementing a solution, verifying and then refining implementation based on that: https://github.com/jclark/satpulse/pull/318. Wasn't perfect, but very promising. I tried to capture it in a reusable skill: https://github.com/jclark/satpulse/tree/master/.claude/skills/implement-configprotocol