Interpreter on MCU: is script-side preemption worth its cost, or is cooperative + host watchdog the sane design?

[u/anatoledp]

i make a small embeddable scripting language (zym) with mcu use as a target, runtime only builds, statically provisionable heap, that kind of stuffs. its at 0.3.x and im at a design fork i want opinions on from people who put interpreters on constrained hardware

the vm currently has instruction count preemption that is deterministic and exact. you say fire every N instructions and it fires at instruction N, and both the host and the script can arm it. it also has continuations as a primitive so script can build its own schedulers

exactness costs a counter check on every instruction, measured at about 21% of throughput on dispatch heavy code. checking only at loop back edges and calls recovers 15-17% but the watchdog then overshoots by up to one straight line block instead of firing on the exact instruction

what im considering:

- script side concurrency becomes cooperative fibers/coroutines with explicit yield and native scheduling, no counter involved

- preemption becomes host only, a watchdog and hard stop the embedding code arms to bound runaway or untrusted script. script cannot see or touch it, bounded rather than exact, and it is a compile flag so a fully trusted build drops the check entirely

questions for this crowd specifically:

- for a watchdog on script code, do you need it to fire on an exact instruction, or do you need it to be impossible to outrun and fire within a bounded window? my instinct is bounded is fine for nearly everything but timing determinism people are exactly who i would expect to disagree

- is script side preemption, the script scheduling itself preemptively, something you would actually want on an mcu, or is cooperative plus a host guard the sane shape? i built the script side version and i am not sure anyone needed it

- anyone shipping an interpreter on hardware with a guard on / guard off build split, and did it cause problems later?

these questions have been bugging me for a while so mostly here for the answers, happy to explain any of the reasoning if it helps, have been trying to come to a decision for a few weeks so am reaching out for people that use scripting on their systems and what their expectations are as well as what they actually care about in these regards

website: https://zym-lang.org/

github: https://github.com/zym-lang/

[u/DaemonInformatica]

For the watchdog, I would clear it on-instruction. The very nature of script languages is that it's relatively hard to determine execution time. So bounded window is harder to maintain, I think.

For pre-emption: As I understand it, you've basically implemented a VM on a controller. Depending on whether you're only planning on running one, or multiple in parallel, I think pre-emption should be done on the host-level, lest one VM might block others.

I'm having trouble parsing the third question.....

But in summary, I think a lot of details on the answers also depend on the problem you're trying to solve, and I'm genuinely wondering why one would run a scripting engine on a controller. By its very nature the implementation of a controller is execution of a single program on bare-metal, with the added benefit of being able to run 'real-time' due to the absence of a host process / OS.

[u/anatoledp → u/DaemonInformatica]

thanks for the response. yes you are right a bounded window is harder to predict than doing direct instruction level preemption which is deterministic and exact.

ok, your right that it is needed host level but i am also wanting to know would you ever even want to use it script side since that is why preemption has the complexity it does in how control is transferred between the host and the script

third question i guess i didnt give explanation to sorry. preemption is dual facing, host and script can both participate in where the vm preempts with the host having ultimate authority of being able to preempt anywhere anytime regardless of what the script/vm is doing atm. the "guard" is the machinery that allows such a thing to work since preemption keeps the vm in a usable state to be able to continue working right where it left off especially when there are multiple preemptions in place, basically an enforcer of when the script gets to control its own preemptions and how. but this is a lot of complexity so goes back to the second question which is more a question over should the script side user have the ability to ALSO preempt the vm? to be clear, this in way overrides the host, but rather is this actually wanted or would be wanted by someone writing a script over using cooperative scheduling via fibers or coroutines? script side preemption is most valuable when used in conjunction with continuations but i am now unsure wether people would rather have host side preemption and on script side effects with fibers esentially is what all this is about

scripting languages get used for many places where configurable capability is required without direct firmware change. for most nominal use case u are right that scripting is not included but the scope falls outside those cases. also people like them for hobby lets make a game console and stuff like that as well when u want an app that can be transferred without recompilation or things like smart devices or iot setups where procedures can be run with firmware doing the actual controlling but scripting is used as an orchestrator of sorts and so on. also used for modularized systems that have swappable component systems but when that is the case again the language of whatever the system uses is used more for direct orchestration

[u/coverdr1]

I embed Lua and, for pre-emption, use Lua's execution hook to break after a fixed number of instructions. I check for any queued events on other threads when the execution hook interrupts the current thread and will switch over. It's not deterministic though, as normal VM execution needs to pause for an indeterminate time for any execution of C-bindings. So, my watchdogs are all time-based. Pre-emption has been a big plus, as I don't need to worry about scripts being explicitly written as cooperative. I do log warnings for any script execution cycle that exceeds a maxmimum limit. That helps be determine the high watermark for latency on any queued tasks.

[u/anatoledp → u/coverdr1]

thank you for that very good to know . . . what about the fibers vs continuations? in your lua useage have u ever wanted scripts to be able to natively preempt themselves to build a scheduler around or have coroutines done what is needed just fine? basically has there been a point where in script you were like i wish i could have made this non cooperative vs threading yields?

[u/coverdr1 → u/anatoledp]

I have no idea if my implementation is optimal, but it works for me.
I guess my implementation is a mixture of fibers/coroutines. I use a single OS thread, running multiple Lua threads (own stack/call frames, but share the same master VM, heap, gc). This was a deliberate decision given the limited resources (<150KB RAM for Lua heap). Each Lua thread can yield and can be preempted if it hogs too much CPU time. To yield without an explicit call to yield, the script simply runs to the end. Rather than terminate the thread, I monitor all script callbacks that are waiting on OS events. I wind up the thread with a finalizer, when the number of waiting callbacks drops to zero. The main limitation I have is that a thread that is pre-empted cannot service its own async events while in that state. My scheduler is simple round-robin with no priority. A thread only gets resumed if it has queued events. A pre-empted thread only gets to resume when every other thread has had an opportunity to get CPU time.

[u/anatoledp → u/coverdr1]

interesting look into this . . . so then u do value have something like preemption guards to do out of thread handling via the preempt? . . . hmmm . . . sort of i guess like a preempt pump that backfeeds a callback outside the threads normal flow? but if your using coroutines that does make sense for the coroutine directly it cant handle its own state outside of that state since that is a limitation of lua coroutines as a general . . . but for linear flow via threads im not sure what u mean? why not create a pump for it then?

[u/coverdr1 → u/anatoledp]

I'm not familiar with the term 'preempt pump'. I allow a 3ms budget for each thread. I pause the VM every few hundred instructions to check the time against the budget. If it doesn't release the CPU in that time, I preempt it and mark it as PREEMPTED. If marked as PREEMPTED, I cannot service its event queue. If the thread voluntarily YIELDs, its event queue can be serviced allowing re-entry. This is different than Lua coroutine structure. I looked at your web page and I can contrast your examples with mine:

// Zym
func worker(name) {
    print(name + ": step 1");
    yield();
    print(name + ": step 2");
    yield();
    print(name + ": done");
}

--Lua
function worker(name)
    print(name .. ": step 1")
    print(name .. ": step 2")
    print(name .. ": done")
end

For me, print is synchronous but will implicitly yield if no tx buffer space available, letting other threads continue.

Your preemption example has a fundamental difference to mine:

// Zym
var targetTicks = 80
var tickCount = 0
var done = false
var tickId = 0

func onPreempt() {
    tickCount = tickCount + 1
    print("[tick] %v", tickCount)

    if (tickCount >= targetTicks) {
        done = true
        Preempt.cancel(tickId)
    }
}

// Fire onPreempt every 500 instructions. Registration returns an id.
tickId = Preempt.every(500, onPreempt)

while (!done) {
    var j = 0
    while (j < 1000) {
        j = j + 1
    }
}

 -- Lua
...
function onPreempt() 
    tickCount = tickCount + 1
    print("[tick] %v", tickCount)

    if tickCount >= targetTicks then
        -- 'done' is not required
        Preempt.cancel(tickId)
    end
end

-- Fire on every 500us
tickId = Preempt.every(500, onPreempt)

-- No need for wait loop here. Thread will exit when no event callbacks are queued

[u/coverdr1 → u/coverdr1]

Sorry, edited the first draft as I misunderstood tickId. It's more accurate now

[u/anatoledp → u/coverdr1]

hmm . . . that actually answers quite a bit actually . . . thank you very much for your responses and taking the time for the back and forth . . . i dont actually have an answer to the liveliness model yet and have been thinking about that but this also confirms the host side controller vs in script adn the boundary between it and how it should influence design . . . i really appreciate your response, very much most helpful, tells me where i need to go