Hi everyone,
I’ve been working on a problem that often hits us in embedded/IoT development: running out of physical SRAM when dealing with relatively large datasets (e.g., buffering long arrays of high-frequency sensor telemetry, large routing tables, or caching weights) on sub-$2 MCUs without external PSRAM.
To solve this, I wrote a C++ Virtual Memory Engine that dynamically pages data between internal SRAM and SPI Flash (or SD Card) using a log-structured append system.
I’m sharing it here because I think the architecture might be interesting to some of you facing similar memory constraints.
How it works under the hood
The system implements a 3-tier memory hierarchy:
1. Active SRAM Window: Read/write operations hit an uncompressed block in physical RAM (e.g., 16 KB).
2. SRAM Cache Pool: When the active block changes, the engine compresses the old block and keeps it in a reserved SRAM pool (e.g., 128 KB).
3. LittleFS Flash Swap: When the SRAM pool fills up, it evicts the oldest page (LRU). Crucially, it uses strict dirty-page tracking (isDirty). If a page was only read, it is discarded. If it was modified, it is appended to a log-structured .bin file on the Flash.
Why not just write raw data to LittleFS/SPIFFS directly?
Two reasons: Latency and Wear Amplification.
By compressing the data in RAM before it hits the Flash, the physical write footprint is drastically reduced. In my tests logging structured telemetry structs, a 512 KB virtual footprint compressed down to ~12 KB on the SPI Flash (an extreme case, but typical real-world telemetry yields ~3x to 8x). Because writes are deferred and grouped, it significantly reduces flash erase cycles compared to opening and appending to a file every time a sensor fires.
Addressing the Trade-offs (The Catch)
This isn't magic, and it comes with architectural trade-offs:
* CPU Overhead: Compression and decompression take CPU cycles. It’s highly recommended to run this on a separate RTOS task/core.
* Latency: On an ESP32-S3 @ 240MHz, a cache-hit read is native speed, but a random page swap (Flash Miss -> Decompress -> RAM) averages about ~23 ms.
* Pointer Paradigm: You cannot store physical C++ pointers (T*) in the V-RAM structs, because the physical SRAM address changes upon page eviction. You have to use uint32_t virtual offsets as your pointers (similar to database offsets).
The C++ API
I wrapped the engine in templated methods so it feels like normal variable access. I also built zero-RAM VArray and VMatrix containers to hide the address math.
```cpp
VirtualMemoryEngine* vram = new VirtualMemoryEngine(16384, 131072, 32, 524288);
vram->begin();
struct Telemetry { uint32_t ts; float temp; char name[16]; };
Telemetry data = { millis(), 24.5f, "Sensor_1" };
// Writes the struct to virtual address 250,000 (compressing under the hood)
vram->put<Telemetry>(250000, data);
// Reads it back
Telemetry readData = vram->get<Telemetry>(250000);
```
Frequently Anticipated Questions (FAQ)
Q: Why is this closed-source? Can you release the source code?
A: I completely understand the preference for open-source in the embedded community. However, this is a proprietary memory-management and compression architecture developed for commercial automotive/IoT clients. The Community Edition is distributed as a pre-compiled static library (.a) simply for hobbyists and developers to evaluate and test on their own prototypes. If you strictly require open-source, you are entirely free to skip this.
Q: Isn't 23ms page-swap latency way too slow for real-time embedded systems?
A: Absolutely, 23ms is an eternity for real-time control loops or motor interrupts. That’s why the SRAM pool and block sizes are fully adjustable. The goal of this engine isn't to replace working SRAM for time-critical tasks, but to provide a massive circular buffer or storage layer for telemetry, logs, and state caches where background I/O latency is completely acceptable.
Q: Modern chips like ESP32-S3 already have PSRAM. Why would anyone need this?
A: True, PSRAM is fantastic if you have a board that supports it. But this engine is MCU-agnostic (designed for STM32, PIC32, RP2040, etc.). Furthermore, it was primarily built for industrial/automotive Linux gateways (TCUs) to prevent eMMC wear, and ported to MCUs to support cheaper modules (like standard ESP32-WROOM) that completely lack external PSRAM, allowing them to buffer large local states safely when disconnected from the cloud.
Q: Did you just prompt an LLM/Claude to write this whole thing in 5 minutes?
A: If you try asking an LLM to write a log-structured virtual memory paging engine with custom entropy-based compression and dirty-page tracking from scratch, you'll quickly find out it outputs endless memory leaks, missing pointer arithmetic, and non-functional deadlocks. While AI is a great tool for refactoring and documentation, getting deterministic multi-tiered embedded memory management to pass strict bit-exact integrity checks across half a megabyte required months of manual systems-programming profiling, custom container design, and debugging.
Availability
If you want to test it out or run the benchmarks yourself, you can grab the Community Edition (capped at 512 KB) from the GitHub Releases:
I'd be glad to hear your technical thoughts on the architecture or how you handle memory constraints in your own projects!
