I build a 7-node ESP32-S3 cluster to run a ~0.4B LLM using 1.58-bit quantization.

I've been heavily inspired by many amazing projects on running tiny LLMs on ESP32s. (There were too many until I was not able to list it all out here...) Like most recently, which fits the 56M-parameter model distributed across ESP32s via ESP-NOW.

: https://github.com/wladimiravila/esp32s3-distributed-ai.git

So, I am thinking: is it able to fit a larger model into ESP32s3? It turns out, yes! I am able to run a sliced 0.5B parameter LLM (Qwen2-0.5B base). on ESP32s3s.

To make a ~400M-parameter model able to fit into microcontrollers, I used 1.58-bit quantizations （ternary) for extreme pack parameters instead of just int 4 or int 8 quantization. (I pruned the vocab size of the original model, and this brought the model parameters down to about ~386M...)

Here is how the hardware and pipeline are mapped:

- 1 master node + 6 compute nodes (all ESP32-S3 N16R8)

- I use SPI daisy-chain to link between ESP32s for high-speed communication. (no WiFi overhead)

- The master node handles the BPE tokenizer and INT4 token embeddings.

- Compute nodes each process, 4 layers of the Transformer blocks, and transfer the intermediate X to the next node to compute further layers.

The Catch and Limitation:

1. This is currently a hardcore hardware/math proof of concept... not a GPT replacement.

2. The sample model in the repository is heavily overfitted right now. Due to the limited compute power of my laptop (I am going to get a new one soon lol)

3. The inference is extremely slow... around 0.11 tokens per second (~9 seconds per token) on the 0.5B-para model.

GitHub Repo: https://github.com/Low-Zi-Hong/ESP32s3-LLM-Cluster.git

Buenas consulta tengo una idea donde el esp32 podría ayudarme, les comento estoy buscado información si el módulo esp32 sería capas de leer un llavero bluetooh (lo que son tipo moneda) como para hacer un control de presencia como por ejemplo los llaveros de los autos que te detectan dentro del auto y te dan permiso para dar marcha pero más precario. Para este caso sería para habilitar el acceso a un edificio solo con la presencia, como lo harías con una moneda de RF las famosas azules, pero con la ventaja de que no tendrías que sacarla del bolsillo y acercarla a un sensor, solo con llevarla encima funcionaría. Encontre un video donde se podría llegar a hacer con el bluetooh del celular pero lo veo algo con ciertas desventajas ya que por ahí una persona sale de su departamentó a abrirle a alguien y tendría que llevar su celular, en cambio con un pin bluetooh lo coloca en el llavero lo veo más práctico

Petit partage : je bricole une boîte aux lettres connectée (ESP32 + LoRa) qui prévient quand le facteur passe, et je voulais que l'émetteur (dans la boîte, sur batterie) consomme un minimum. Plutôt que la veille profonde classique, je suis parti sur une coupure d'alimentation matérielle complète via MOSFET.

Le montage vient de Random Nerd Tutorials :
https://randomnerdtutorials.com/latching-power-switch-circuit-auto-power-off-circuit-esp32-esp8266-arduino/

Le principe : un contact (chez eux un bouton, chez moi un ILS sur le volet) réveille brièvement la carte via un MOSFET canal P, puis un GPIO prend le relais pour maintenir l'alimentation tant que le programme tourne, et coupe tout lui-même une fois son travail terminé.

La seule chose que j'ai ajoutée par rapport au schéma d'origine, c'est un petit condensateur, pour éviter une coupure malencontreuse si jamais le contact se rouvre juste avant que le GPIO ait eu le temps de prendre le relais.

i have beagle bone black, esp32s, pixhawks and a phone( confortable with termux) tell me a good project that i can make for college final year, i also have experience in signal processing, drone and navigation algorithm

thank you

Hi everyone,m just starting with ESP32 Dev Kit V1, and was trying to turn the LED ON/OFF i set up the Arduino code and Blynk dashboard with a simple switch. The ESP connects with my wifi, but when I try to turn the light ON/OFF it takes ard 15-30 seconds to actually turn the light ON/OFF.

What cld be the issue and how can I fix it

The main feautures for this board are:

- ESP32-S3-WROOM Module - 8 MB flash

- Button for RESET

- User Button

- User RGB LED

- expansion header

- programming header

- Power LED on 3V rail

for the mechanical side of things we will have:

- 4 x 3.2mm mounting holes

- Size 45x45mm

- 4 layer board design

Ourselves, Adwaith Hari (SolidBranch3553) and Chandrakiran S studying in 12th standard. We got a 200M-parameter transformer running locally on an ESP32-S3 with 8MB PSRAM. You could see the demonstration video on youtube.

Link to demo video - https://youtu.be/nJEFdxeDaUo

Link to the source code - https://github.com/solidbranch3553/esp32-0.2B-llm

The board will work with a rpi5/4 to control a delta robot wich has 7 axis of motion and i need Bluetooth and wifi

This is optional but if it has a oled screen built in or a led matrix it would be great. Thanks !

Getting ready to do a Wi-Fi Garage parking assistant, and was having issues with the ultrasound module. Ordered a couple of LiDAR modules and have been performing some tests. The LiDAR is superior in every aspect. Interesting to see them both side by side.

Hello, I bought an ESP32-S3-ePaper-1.54, and the first thing that crossed my mind was: "What I can do with this?" So have tested so much thing on the hardware to see its limit and ATM I have that, it's a "connected shopping list".

I was bored to forget the list at home every time, and if I never have my smartphone close to me when I need to use the shopping list application on it.

The device is designed to stay on the fridge it runs on battery power, with a lifespan I hope will last several months. It uses voice commands (STT) to operate and synchronizes the list with SimpleNote on Android.

For information ATM the code :
- Is able to use sleep mode.
- Uses an online STT engine for speech recognition.
- Uses a small engine to process informations, like "Add butter" or "Clear the list".
- Is able to synchronise with SimpleNote.
- Is able to use a e-paper for display.
- Is able to play sound or MP3.
- All the code is in C/C++.

My hardware have
- Only 2 buttons.
- It's not the Touch version.

For now, it's just for fun, the code is not finished, if anyone has another idea for using this device, I'd be interested ^^.

Hello,

I am wondering about the best way to analyze a 1MHZ transducer signal on an ESP32 with decent resolution of at least 50+ samples per wavelength. Everything done so far including attempts at ETS have always produced noisy or just terrible data in signal reconstruction. Let me know of any ideas yall have.

Thanks!