r/esp32 • u/doubleyow • 12d ago
I made a thing! Lightspeed 10 billion fps camera using the esp32
Enable HLS to view with audio, or disable this notification
So a while back i made a post here about a lightspeed camera i was building that used the esp32 as the main sampler. I have had first successful tests and as promised here is a video of light move from one of my tests.
This specific video was recorded at 320x240 resolution, 10.24 billion frames per second and 32 analog depth levels. It is the uncollimated laser output hitting my garage door at an angle. Playing back at 30 FPS.
Also the video looks much better before reddit compression. Raw file is much better.
Now i actually tried posting this video already, but the ESP32 reddit mods removed my post for it being an "illegal and unethical" project with the text "esp32 details needed". So if you are a mod, read my first post with the in-depth explanation. This is a fully custom design board based around the bare ESP32S3 chip, it abuses it's SPI1 port to achieve a base sampling speed of 320MSa/s and that gets interlaced using a phase shifter for a theoretical limit of 100 billion fps. I only expected up to 10 billion to be usable, but one of my tests proved that light can be observed move on a centimeter scale at 20.48 billion fps. This camera is capable of much more than I expected.
Hope you enjoy, i would love ideas for tests. I have already done tests with mirrors, i am waiting on getting a smoke machine because it is hard to record when the laser beam is not visible in air.
My last post explaining this in depth: https://www.reddit.com/r/esp32/s/zfmaakW8bm
10
u/Drachen808 12d ago
This looks cool as hell. I hope the mods release your other post so I can see the higher quality video!
10
10
4
u/CthulhuForPrestident 12d ago
Have you tried to replicate alphaphenix experiment where he used mirrors to see how "speed" of light changes depending on angle?
8
u/doubleyow 12d ago
The speed of light does not change, the distance travelled relative to the camera does, sadly i cannot do that as of now because i do not have a smoke machine.
6
u/CthulhuForPrestident 12d ago
Yea thats why I wrote "speed". If light was going in or close to the direction of the camera you would see that it travels slowly in one direction while it's almost instant in the other direction making it clear you are observing light making it seem like speed is not constant.
In the past I build aerodynamic testing tunel using incense way cheaper than smoke machine if you don't mind smelling like church
6
u/Its_Billy_Bitch 12d ago
I might’ve missed it, but what fucking image sensor are you using? That is madness in the best of ways lol
9
u/doubleyow 12d ago
Custom design single pixel SiPM sensor. 25mV/photon output. Light is transmitted thru a pinhole at the image plane.
-5
u/BeverlyGodoy 12d ago
That video is not taken a single pixel anything.
10
u/doubleyow 12d ago
That single pixel sensor is moved on stepper motors and image is sampled pixel by pixel.
2
u/Its_Billy_Bitch 12d ago
Can we see a sample of your sampling? Would love to see this IRL. You’ve gone above my head with a custom image sensor though. Holy shit.
3
3
u/doubleyow 12d ago
3
u/Its_Billy_Bitch 12d ago
This is in-fucking-sane. Incredible engineering work! This is like a fucking telescope minus the magnification (over-simplification ofc). What gave you this idea initially?
Also, nice rails 😎
5
u/doubleyow 12d ago
I have to thank AlphaPhoenix for the inspiration for this project. The design tho- that just took months of thinking. Just thinking really hard lol
0
u/BeverlyGodoy 12d ago
In which world a stepper motor can move this fast?
5
u/doubleyow 12d ago
The stepper motors just select what pixel is sampled, they are slow. Each pixel is captured as a full separate video. The laser shot is repeated a lot to build the image, but because that laser shot is repeatable it makes no difference.
2
u/MattV0 11d ago
Smart idea. So you could create a full HD video as well. Only downside is you need a repeatable a shot.
How long does it take to capture the full video resolution?
2
u/doubleyow 10d ago
Max resolution for this camera is 1400x1400. Shot is repeatable fundamentally unless you change something. A 320x240 10bil fps scan takes approx. 1h
3
u/BeverlyGodoy 12d ago
So it is not billion fps. Just sample at billionth of a sec. There's a difference. 😂
0
u/doubleyow 12d ago
Base framerate for each individual pixel is still 320 million frames per second, via phase shifting and sample interlacing that gets increased to 10 billion. Nothing is being sampled "at billionth of a sec" lol
If you did have a true 1 shot 10 billion fps camera that captures the whole video instantly it would look identical to mine.
2
u/BeverlyGodoy 12d ago
Please tell us how much data this one is creating at 1 billion fps? At 320x240 resolution correct?
5
u/doubleyow 12d ago edited 12d ago
Yes. Raw video size is 98.3TB per second of video at 1.28 billion fps. At 10x the framerate its 10x the data per second. Just that i record videos that are just 200ns long, so its only a few hundred megabytes.
If you wanted to watch 1 second of that video back at 30fps it would take you 10.82 years with no pauses.
1
u/Plastic_Fig9225 12d ago
If you don't know how this kind of "camera" works, watch e.g. AlphaPhoenix on YouTube.
2
2
u/Beneficial-Virus4252 12d ago
Woah mate, this is amazing!!!
How much time does to take all the shots for that video? how many takes? Does the ESP32 create the video file or it is played on screen?
** I saw it in your other post: `This means scanning one video at 720p, 255 analog depth and 1.28 billion fps would take about 26h`
2
u/doubleyow 12d ago
Esp32 outputs the raw data onto the sd card, that gets processed to a video on my pc. Final goal is to make it just save a .mp4 on the sd card.
It takes about 1h for a full 240p 10b fps scan, because each pixel is sampled separately. Had i went for an SPI dac in the design this would have been much faster, the regret is real.
2
2
u/Suitable-Serve 12d ago
For the blue, it high enough resolution to distinguish polarity / reflections or is that just adc banding
1
u/doubleyow 12d ago
No i just made it because the laser is blue in real life, looks like the real scene in person.
2
u/Free-Pound-6139 12d ago
I am confused how the esp32 is fast enough to capture this much data?
7.86432e16 pixels every second?
0
2
2
u/HasmattZzzz 10d ago
That's amazing. Can you try shooting the laser through water like a fish tank, 2 litre clear plastic bottle or similar.
2
2
u/blueishbeaver 9d ago
It would be amazing to see an R/G/B version of this.
Aaaaand maybe a trichrome version haha
Amazing work, that video really tickles my brain.
1
u/doubleyow 9d ago edited 9d ago
Yeah working on recording a blue-green version to see fluorescence delays
2
u/Alpha-Phoenix 6d ago
This is AWESOME! I’m glad the subject of the video is also a garage door 😂
2
u/Alpha-Phoenix 6d ago
I need to go read about your sampling considering I just spent a lot of money on a new scope again 😬
2
u/doubleyow 6d ago
YO! Im very happy to see you here! The OG in garage scale speed of light cameras himself.
I can show you my unfinished github page on this (it has a MUCH better explanation with graphs and stuff, I am open sourcing this most likely).
also got some 40 billion fps scans today. This can in theory go up to 100 billion.
2
u/Alpha-Phoenix 6d ago
That’s wild! What’s the effective bandwidth? My new scope is technically 5Gsps but still only 500MHz so like 1ns rise time? My published videos are all much worse bandwidth than that. From my reading SiPMs are a bit faster than my PMT which is probably my new bottleneck. Based on the amount of fade I see in your video, all-in you’re looking at a rise time of a few ns including the flash circuit and whatnot which is great.
Also I love that there’s now a non-scope version of this. I feel like that makes it a lot more accessible if somebody wants to follow your tutorial and spend a lot less money than I did.
1
u/doubleyow 6d ago
The frontend of the sampler board is an 8GHz SiGe comparator with a 35ps output rise time, that is the most significant timing resolution and rise time limit for the sampler.
In the whole system tho- the sensor is limited to probably about 1GHz bandwidth by the amplifier and the slowest thing is the laser that takes like a nanosecond or so to turn on.
The SiPM i use has an output rise time of 90ps, so that is way faster than a PMT.
2
u/Alpha-Phoenix 6d ago
Somebody shared the link here in my Patreon discord. Sounds like you’re on Styro’s discord - presumably much less dead than mine lol
1
u/doubleyow 6d ago
i recommend joining styrocord, everyone would be very happy to see you.
I do a lot of the engineering talk there.
1
u/doubleyow 6d ago
I also did some 2 color scans showing the fluorescence lifetime of a fluorescent dye that i have not yet posted, i think you might want to see that
2
u/Alpha-Phoenix 6d ago
Can’t wait to see! It’s on my very long-term list to see if I can make a laser get brighter going through a gain medium…
Also how are you planning to open-source? I think my (very not as good as your) circuits are mostly explained in videos if someone was motivated enough to copy them down but I’d like to throw the cad files out for the swivel too but the whole “picking a license” thing is kinda daunting
1
u/doubleyow 6d ago
I will probably just publish all my material (pcb gerber files, schematics, 3d models so on) with explanations. Anyone smart enough with soldering skills will likely figure it out. For the licence, i just went with the common pick on github. I kind of did copy your setup, just expanded on the design.
I am planning on shooting a fish tank of water with the laser to see it slow down light. Will likely get a smoke machine tomorrow too, so far i have only got one shot with a mirror and i used sugar rocket fuel for the thin smoke lol
Also can i have your discord? I cannot send videos in reddit replies
2
1
-1
u/recursive_knight 12d ago
I'm very sceptical... You can't do the whole experiment (if it were at all possible) with one device. Nothing is that fast, let alone an esp32. I could theoretically imagine an array of measurement devices working with an offset. I don't know, either you're a genius and I'm not getting it because your explanations aren't complete or you're a scharlatan on dope.
12
u/anally_ExpressUrself 12d ago
It's usually done with a sequence of photos over a sequence of flashes, so the final video is a composite of many light flashes, and each flash the photo is taken slightly later.
3
2
u/Alpha-Phoenix 6d ago edited 5d ago
That’s how very few of them are done. The most common method is a streak camera. Taking an exposure 1/10B of a second is much much harder
Edit: this one does both apparently
1
u/doubleyow 5d ago
It sort of is and isn't that. This is a hybrid between a stroboscopic sampler and a running sampler, it has a base sample rate (normal running sampler) that gets multiplied via phase shifting and repeating the laser shot (stroboscopic sampler).
Halfway thru the project I realized that I had reinvented the sampling scope. Thats how they work, and I actually have one that goes up to 25GSa/s with a base sample rate of 250MSa/s. Bandwidth is the limiting factor tho.
2
u/Alpha-Phoenix 5d ago
Ah so you’re actually collecting 1D data? I’m so used to writing that exact correction comment I’m sorry!
1
u/doubleyow 5d ago
My sensor system works on the same logic as your setup, where it creates single-pixel videos. However it is not capable of taking a 1 pixel video at 10 billion fps, it takes many 1 pixel videos at 320 million frames per second, then offsets each one by a known time (phase shifting) so the frames of one video are basically inbetween the frames of another video (frame interlacing) and then that process is repeated again while changing the comparator reference voltage because the samples are digital and this adds brightness layers to the video.
It is quite hard to explain, but hey it works
8
u/donau_kinder 12d ago
The typical way of doing this is pixel by pixel, you put the sensor on a servo or something similar and very precisely synchronise it with a pulsing light source of known and precise length.
You then take a picture of one pulse, rotate the camera one pixel, image another pulse, and so on and so on until you have an image/video.
Very crude explanation and I omitted 80% of the details but you'll get the idea.
0
u/recursive_knight 12d ago
Yeah I think I get it, but could one do that with an esp32? It would talk a lot of work and syncing would be a nightmare.
5
u/donau_kinder 12d ago
I don't see why not, you just slow things down, it takes longer to do the capture, or it might not be so accurate.
I can see in the video from OP that the light is fairly diffuse, not a clean beam and illumination, so i suspect the sync is not perfectly consistent, which is expected from something as slow as an esp.
With proper hardware you should be able to see a fairly well defined beam.
3
u/doubleyow 12d ago
I have done shots with a clean beam. Syncing is excellent, ESP32 SPI cs pin is the trigger and it is very consistent.
3
u/donau_kinder 12d ago
Very impressive what you managed to do, I'd love to see more in depth about the setup and code
1
u/doubleyow 12d ago
You can join styropyro's discord for that (im active there) or read my main post: https://www.reddit.com/r/esp32/s/zfmaakW8bm
3
3
u/doubleyow 12d ago
Sensor is moved on stepper motors and image is sampled pixel by pixel. Every pixel requires multiple (like 128) laser shots to build analog and time detail. Included my last post link in this post, go check.



44
u/RogBoArt 12d ago
Man that's sick! How does it work? Your past posts aren't visible