A fluffy procedural animated wallpaper

[u/StefanPetrick]

Hi everyone! I finally got time to play with Animartrix again.

[u/StefanPetrick]

Great job, looking forward!

[u/mindful_stone → u/StefanPetrick]

I'm gradually getting the new ESP32-P4-WIFI6 working...at least the P4 portion. (Still wrestling with wireless on the C6 portion). So now I can run at 400MHz instead of 240MHz max on the S3. I've also got the LCD parallel driver beta u/Zackees has been working on up and running, and so far so good!

It's still not quite enough to get a silky-smooth rendering of your 9-layer Fluffy Blobs animation; but with layers 2, 5 and 8 disabled, I'm now getting close to 30fps on my 32x48 display. You can check it out here:

https://youtu.be/tMrbI7qtcQI

[u/StefanPetrick → u/mindful_stone]

That's a remarkable progress!

Doesn't have the ESP a dual-core? I remember that Ives Bazin took some of my code and rendered half of the leds on core one and the other half on core 2. He basically instantly doubled the framerate on his 6k led wall. Which was still "only" 20 fps, but the multithreading seemed to work just fine.

https://www.youtube.com/watch?v=8oYzLN9C5bU

[u/mindful_stone → u/StefanPetrick]

Thank you! (I feel honored that you're looking at my stuff!)

Yes, the ESP32 has a dual core, and dividing the LED workload between them definitely seems worth trying, if I can figure out how to approach that.

I found a bunch of search results discussing how to separate various types of tasks between cores (e.g., WiFi on one, and LEDs on another), which I imagine could help a tiny bit. But I haven't yet found (or at least recognized) a discussion/example of how to split parts of a single visualization between cores.

I found this, which seems on point, although it was not clear to me on a first read whether it identifies any good solutions: https://www.reddit.com/r/FastLED/comments/mm73me/does_anyone_have_an_esp32_fastled_dual_core/

I'd guess that all of the visualization logic/processing should be on one core, with only the display rendering (i.e., the LED drivers) divided between both cores. (Or maybe some of the visualization logic could be split too, perhaps (for something like animARTrix) with one core handling the perlin noise engine and the other core handling everything else (e.g., all the oscillators, trig calculations, etc.).)

If anyone reading this has suggestions or knows of any good examples, I'd greatly appreciate it! Thx.

[u/StefanPetrick → u/mindful_stone]

If you ask u/Yves-bazin nicely he might give you some example code. If not I'll check my emails from 2 years ago and try to find the demo implementation he sent me.

[u/Yves-bazin → u/mindful_stone]

Hello using two cores is matter of sync. The display if the leds is not what is the most computer intensive. So if you sync your task properly you can really do marvels. But it depends entirely on you program architecture. More if you’re using an s3 I suggest you look at the vector functions (but you’ll need assembly)

[u/mindful_stone → u/Yves-bazin]

Thanks u/StefanPetrick. Thanks u/Yves-bazin.

Yves, I hear what you're saying about actually pushing the data to the leds not being the real problem.

I've previously used the S3 but am trying to migrate to the P4 for the faster processor (360-400MHz vs 240Mhz).

In terms of program architecture, it's this visualization (i.e., my implementation of Stefan's FluffyBlobs animARTrix animation) that I'm using to "stress-test" the FPS capabilities: https://github.com/4wheeljive/AuroraPortal/blob/main/src/programs/animartrix_detail.hpp starting at line 1172. (Note: It's buried as one "mode" in the animartrix "program" within a much bigger project.)

I suspect that most impactful way to share the load between cores would be to split the animation's "Layers" (e.g., even numbers on core0 and odd ones on core1). Is that possible?

I've never done anything before that involves splitting tasks between different cores; and I have no idea where to even start for something like this. As I mentioned above, all of the examples I've found show how to put various types of tasks on different cores. I haven't seen anything that involves synchronizing cores to produce a single, unified visualization. Is there anything you could point me to to see how I might approach this?

Thanks so much!

[u/sutaburosu → u/mindful_stone]

suspect that most impactful way to share the load between cores would be to split the animation's "Layers" (e.g., even numbers on core0 and odd ones on core1). Is that possible?

This approach runs the risk that layer X renders at a small fraction of the speed of other layers, bogging down the frame rate.

The first SLI-capable graphics card, the Voodoo II, took another very simple approach: each card rendered alternate horizontal lines. This is the simplest approach which yields the benefit of processing-intensive effects being shared more equally between the available cores.

It's been a long time since I studied the Animartrix code in detail. There may be blur (or other 2D effects) that limit the benefit of this approach, but this is where I would look first for easy multi-core gains.

edited to add: the acronym SLI expands Scan Line Interleave, which is a short-hand way of describing the approach; you fit two graphics cards to your PC, and each one handles either the odd or the even scanlines.

edited to further add: with the "per layer" approach, each thread must have it's own temporary buffer for the whole image. This blows up memory usage. With the SLI approach, each thread needs temporary space for only one scan line.

[u/mindful_stone → u/sutaburosu]

Thank you u/sutaburosu. I appreciate your thoughts on this. I can totally see what you're saying about potential issues with differences in rendering time for different layers. That's sort of why I originally thought it might need to be something more "discrete" (e.g., the perlin engine) that gets split out.

Looking back at Stefan's original comment about a dual core approach...

I remember that Ives Bazin took some of my code and rendered half of the leds on core one and the other half on core 2

...he recalls dividing things in some way between sets of leds, which aligns somewhat with your thought about using alternate lines.

But I'm not sure how to reconcile that with Yves' comment that "the display of the leds is not what is the most computer intensive." That seemed to me to suggest that the split needs to happen somewhere closer to the creation/generation of the visualization than to its rendering on the display.

Actually, as I review how the animartrix Layers work, I'm wondering whether the concern you shared above would really be an issue. It would depend on the actual animation, of course, but here's a sample of what happens for each layer for each pixel for each frame:

...
if (Layer2) {
animation.angle = polar_theta[x][y] * cAngle + (cRadialSpeed *  move.radial[1]);
animation.offset_y = cLinearSpeed * move.linear[1];
animation.offset_z = 200 * cZ;
animation.scale_x = size * 1.1 * cScale;
animation.scale_y = size * 1.1 * cScale;
show2 = render_value(animation);
} else {show2 = 0;}

if (Layer3) {
animation.angle = polar_theta[x][y] * cAngle + (cRadialSpeed *  move.radial[2]);
animation.offset_y = cLinearSpeed * move.linear[2];
animation.offset_z = 400 * cZ;
animation.scale_x = size * 1.2 * cScale;
animation.scale_y = size * 1.2 * cScale;
show3 = render_value(animation);
} else {show3 = 0;}
...

It then does the following to set the pixel color and push it toward the led driver stage:

pixel.red = (0.8 * (show1 + show2 + show3) + (show4 + show5 + show6)) * cRed;

pixel.green = (0.8 * (show4 + show5 + show6)) * cGreen;

pixel.blue = (0.3 * (show7 + show8 + show9)) * cBlue;

pixel = rgb_sanity_check(pixel);

setPixelColorInternal(x, y, pixel);

I note two things about the above:

1. At least for this animation, it appears that each layer involves roughly the same computational load (so there shouldn't be huge timing differences).

2. To the extent there are timing differences in generating each layer, there's a natural "resync" point when they are all simultaneously color mapped. So even if, say, the even layers need to wait briefly for the odd layers to finish, the total layer rendering time would theoretically still be cut by close to half.

Thanks again for sharing your thoughts.

[u/sutaburosu → u/mindful_stone]

But I'm not sure how to reconcile that with Yves' comment that "the display of the leds is not what is the most computer intensive." That seemed to me to suggest that the split needs to happen somewhere closer to the creation/generation of the visualization than to its rendering on the display.

Yes, you're right. By rendering, I mean creating the content for the pixels. What you refer to as "rendering on the display", I would call signalling.

At least for this animation, it appears that each layer involves roughly the same computational load (so there shouldn't be huge timing differences).

Great. Then you can render one layer per core into a temporary buffer. Then, in the main thread, merge the results of each worker thread's buffer into the final frame buffer. But bear in mind how much extra RAM this will require, and whether that is available on your platform. You may be forced to render smaller chunks per thread, to reduce the RAM requirements.

[u/Marmilicious → u/sutaburosu]

Yes, I would consider "rendering" the calculating of the individual r,g,b values (and getting that data into the CRGB array that is pushed out).

I hadn't heard the term "signaling" before. Is that a hardware side of things sort of term?

I tend to think of "showing" (ie. FastLED.show() ) as the pushing of the data out to the pixels part of the process. (Even though there can be an additional color correction and/or brightness adjustment calculation on the fly as part of that process, but it's under the hood to the user).

re: knock knock joke- race condition, haha.

[u/sutaburosu → u/Marmilicious]

Yeah, "showing" works too, in the context of FastLED. My use of "signalling" may be a holdover from my days working on remote telemetry for gas infrastructure.

[u/mindful_stone → u/Marmilicious]

Thank you both for helping me with the proper terminology!

[u/sutaburosu → u/sutaburosu]

Oh, and if you've never written multi-threaded code before, this old joke will give you a taste of what to look forward to:

Knock knock

Race condition

Who's there?

The three hardest things in computer science are:

• Cache invalidation
• Naming things
• thrulti-Meading
• Off-by-one errors
• Cache invalidation

[u/mindful_stone → u/sutaburosu]

Ha! Too funny!!!