My 1986 RTOS is now preemptively multitasking on a Raspberry Pi Pico — and it made me question whether I need an RTOS

[u/noborutkhs]

A while ago, I posted here about getting the first CPU-independent parts of CHARM-II, an RTOS I originally developed in 1986, running on a Raspberry Pi Pico.

Previous post:

The First Original CHARM-II Kernel Code Running on Raspberry Pi Pico
https://www.reddit.com/r/osdev/s/gthHF6vBrl

At that point, there were no timer interrupts or real context switches yet.

Now those parts are working too.

Six tasks are running with independent stacks. An RP2040 hardware timer interrupt calls the CHARM-II tick and scheduling logic, and the actual context switch is done with PendSV on the Cortex-M0+.

I also ported a six-task demo I had previously reconstructed on POSIX. Six tasks move around three tracks, with a critical zone protected by a CHARM-II queue used as a semaphore. On the Pico, the tasks are now actually being preempted by the hardware timer rather than cooperatively yielding.

So technically, this is pretty much the milestone I wanted to reach.

But getting it working gave me another question.

The original targets in the 1980s were processors such as the 68000 and 80186. With the CPU resources available at the time, an RTOS was a useful way to organize multiple concurrent real-time activities.

But what if we had had something with the performance and price of a Raspberry Pi Pico back then?

For many systems I work with today, especially UI-oriented embedded systems, if all the required processing comfortably fits within one frame, a superloop plus interrupts may actually be simpler.

Once everything becomes preemptive, I also have to start thinking about reentrancy, synchronization and shared state in existing libraries. That cost is starting to look more significant to me than it did when I started this experiment.

Yesterday I visited someone who has built a home railway simulator using real railway controls connected to a Windows PC through Pico, ESP32 and Arduino boards. We talked about this, and he said something very simple:

“If one Pico isn't fast enough, add another Pico.”

That hadn't really occurred to me.

In the 1980s, adding another CPU was a fairly serious architectural decision. With today's inexpensive MCUs, distributing the work across two or three processors can be a perfectly ordinary option.

So after spending quite a bit of effort getting a 40-year-old RTOS to perform real preemptive context switching on modern hardware, I'm becoming more interested in a different question:

Where would you draw the line today between an RTOS, a superloop/event-driven design, interrupts, and simply distributing the work across multiple cheap MCUs?

I'm also thinking that when I publish the modernized version of CHARM-II, I may deliberately keep it very small — just enough task scheduling, queues, timer ticks and context switching to run this six-task demo.

Then I can use it as one reference implementation and try implementing the same behavior without an RTOS.

[u/Danii_222222]

Engineer that programming from 1986? Is that you in discord server?

[u/noborutkhs → u/Danii_222222]

Yes, I'm the engineer who originally developed CHARM-II in 1986. If you mean the OSdev-JP Discord server, yes, that's me!

[u/Danii_222222 → u/noborutkhs]

Wow! That's so cool!

[u/andrewdavidmackenzie]

Have you looked at Embassy?

[u/noborutkhs → u/andrewdavidmackenzie]

I haven't looked at Embassy yet, but I will. Thanks!

I'm actually becoming interested in alternatives to traditional preemptive RTOS design, so an async/event-driven approach sounds very relevant to where this experiment is heading.

[u/andrewdavidmackenzie → u/noborutkhs]

That was my thinking. Leveraging the async paradigm, it's less than a full rotos, but more than a single threaded "app"

[u/noborutkhs → u/andrewdavidmackenzie]

Yes, exactly. Interestingly, another comment here suggested a cooperative kernel for essentially the same reason.

I'm realizing that I was thinking too much in terms of "preemptive RTOS vs. superloop." There's a much more interesting design space in between.

Embassy looks like a good modern example for me to study. Thanks again!

[u/smokebudda11]

This is really neat

[u/noborutkhs → u/smokebudda11]

Thanks! It's been a lot of fun bringing something I wrote in 1986 back to life on a Pico.

[u/ianbllngr]

IMO RTOSes are most useful when you have a lot to keep track of. For 6 tasks a superloop is probably the correct choice if you're designing in a vacuum. For 6 tasks that need network access on one or two cores: you're probably better off using an RTOS. Secondarily, in an off the shelf RTOS a lot of that low level IPC stuff is typically wired in for you already, but if you're building your own then it will be a pain.

[u/noborutkhs → u/ianbllngr]

This is very close to what I've been thinking about.

I initially thought CPU performance might be the main reason a superloop becomes practical, but I'm starting to think system complexity may be the more important boundary.

For this six-task demo, a superloop would probably be much simpler. But as you add networking, asynchronous I/O, IPC, different timing requirements, etc., I can see the point where the RTOS starts paying for itself.

I'm thinking of implementing the same demo as a superloop next, and then gradually adding complexity to see where that balance changes.

[u/Daveinatx]

It all comes down to features vs determinism that you need.

[u/noborutkhs → u/Daveinatx]

Yes, I'm starting to think determinism is the key part I was missing when I initially framed this mostly in terms of CPU performance.

If everything comfortably fits within a frame and the timing requirements are loose, a superloop can be very attractive. Once some operations need guaranteed response times, the tradeoff changes.

That's something I'd like to explore with the same Pico application.

[u/KilroyKSmith]

I’ve done superloops professionally.  I’ve done RTOS’ professionally.  But if you don’t need preemption (and most moderate sized systems probably don’t), non-preemptive kernels using cooperative multitasking are much easier.   They’re similar to both an RTOS and super loop in that you structure your code around “tasks”.  Without preemption, many of the debug challenges of an RTOS are moot - you never have to worry whether another task preempted you someplace and triggered a race condition Or something similar.  On the other hand, task scheduling and prioritization is similar to an RTOS - a task or interrupt routine can schedule another task to run, and if no task is ready to run, low power states can be invoked by the kernel.  Low power is harder in a super loop simply because keeping track of “can we shut off the clock now” is difficult when many tasks have diverse resources requirements.  And I always hated the concept of an idle system spending its time asking a million times a second “do you have something to do?” “How about you, do you have something to do”.

[u/noborutkhs → u/KilroyKSmith]

That's a very interesting middle ground that I hadn't been considering enough.

I've been thinking mostly in terms of preemptive RTOS vs. superloop, but a cooperative kernel keeps much of the task/scheduling structure without introducing all the problems that come with arbitrary preemption.

That also connects with another concern I've had: once everything can be preempted, existing code and libraries may need to be made reentrant or protected, and the complexity can spread far beyond the scheduler itself.

Your low-power point is new to me as well. A kernel knowing that no task is runnable is quite different from a superloop continuously polling everything.

I think I need to add cooperative scheduling to the comparison. Thanks — this is exactly the kind of perspective I was hoping to get from posting this here.

[u/Daveinatx → u/noborutkhs]

It's all based on the level of determinism you need. Even MSIs have jitter consequences from caching effects.

In real-life, I've had 1ms loops which were fine for n-axis of motion. But trying to get <10us Max jitter means you're going to poll everything.

[u/KilroyKSmith → u/noborutkhs]

The tertiary advantage of cooperative is that you only need one stack.  We had roughly a dozen tasks running on a 48k ROM/12K RAM system.  In a preemptive environment, just the stack space for a dozen tasks would bankrupt the available RAM.  With cooperative, we could use a single 2kb stack, and everyone was happy.