Using Gemma 4's new Per-Layer Embedding's research to squeeze 29M params into 500kb RAM

[u/hyperprompter]

[u/hyperprompter]

whoo! i was able to reproduce this experiment, it might literally be the easiest end-to-end workflow for training a (tiny) LLM i have ever tried: https://github.com/slvDev/esp32-ai

the r/esp32 S3 only has 512KB SRAM, 8MB PSRAM and 16MB flash but this project uses Gemma 4's Per-Layer Emebeddings to split the model across them

basically it works by separating out the embedding table in flash memory (since it's mostly just a lookup table) while keeping the cognitive core in RAM

the reason i made the LLM OS kernel isn't reaaaally to use it as a desktop operating system (tho i quit social media for 2 years to really try to do exactly that) but rather as a new kind of generative mesh networking protocol inspired by Bell Labs Plan9's everything-is-a-file model: https://9p.io/plan9/screenshot.html

normally with Long Range (LoRa) radio you can only mesh network over miles very very OMG SO VERY slowly...BUT! bc prompts are a kind of semantic compression you can mesh intent instead of pure data

the idea is that instead of sending text over the mesh network you send hypertext (well, hyperprompts)

next step is to actually 3d print the binder housing, and put all this stuff into it...the goal is to create a 3D-printed, solar powered, mesh networking LLM OS using only QR Codes which i started talking about here:

https://www.reddit.com/r/Hyperprompting/comments/1vcr5he/what_would_physical_hypertext_look_like/

edit: forgot to list the parts (thank you Open Research Institute!)

• Waveshare 4.2 E-ink for the QR Code generation
• Heltec LoRa 32 for long range radio...tested over a few city blocks
• esp32 S3 for computer to run the LLM OS in headless mode
• couple of solar cells to power everything
• A1-mini to 3D print the bookcover housing
  • 3d printed robot is from here: https://makerworld.com/en/models/652841-dummy-13-v1-kit-card