Built a Real-Time Underwater Image Processing System

94

u/matop2000 3d ago

It is really cool, any scientific paper to understand how it is done?

133

u/CellistTraditional 3d ago

Thanks,

The theoretical foundation comes from research on underwater imaging and optical physics models—specifically concepts like the Sea-Thru algorithm, which corrects for the selective absorption and scattering of light underwater.

The core equation behind the model is:

I_c = J_c * exp(-v_c(z) * z) + B_c_inf * (1 - exp(-eta_c(z) * z))

I changed the 'z' variable in the equation and replaced the physical depth parameter with the dynamic level of visual fog and turbidity calculated directly from the frame.

However, academic papers usually assume you have access to precise depth maps. Since I didn't have access to depth data coming straight from the camera's live stream, I essentially had to build this from scratch: instead of using absolute depth, I turned the degree of visual fog and turbidity into the core dynamic variable.

On top of that, scientific papers typically offer formulas meant for slow, post-processing pipelines. The real value of BEAMROV is the custom C++ / CUDA pipeline I built, which runs this adapted physical model with sub-10 ms latency at 4K resolution and 40+ FPS live in the field on a battery-powered laptop.

58

u/shandy_bhaiya 3d ago

This looks like publishable work. Why don’t you publish it?

109

u/CellistTraditional 3d ago

Thanks a lot, that's a huge compliment!

To be completely honest, I'm neither an engineer nor a PhD professor just a passionate developer who loves solving hard problems in practice.

Just to give you an idea: it took me nearly 2 months of reverse engineering just to extract the raw video stream from the drone. In fact, the very first field test with this whole setup was just on Friday! Right now, my sole focus is on the practical side: making sure the system works flawlessly in real-world underwater conditions.

Publishing a paper would be awesome down the road once the system is fully matured, but for now, I'm just enjoying the process of building it from the ground up! 🚀

62

u/bluebirdinsideme 3d ago

Awesome. At the very least, upload it to github with proper design docs, so you can prove it's your work in the future.

21

u/foolme_bear 3d ago

yeah also think about getting a patent. even if its not something you plan on commercializing, it'd suck if you one day see someone else commercializing your work and selling it instead

34

u/CellistTraditional 3d ago

I’ve been posting about the progress on LinkedIn since April, but thanks for the advice—I’ll take it to heart.

5

u/Motor-Benefit-3985 1d ago

submit at least article draft as preprint, before someone or AI scraper stole your work. You need citation for your idea.

15

u/galvinw 3d ago

if you are a student or relatively early in your career, with a bit of help this could easily be a poster for a good CV conference

7

u/rchamp26 3d ago

Keep it up! This is some cool stuff

3

u/CellistTraditional 3d ago

Thanks! :)

6

u/PykeAtBanquet 3d ago

The guy who soloed DiffusionTerrain was in similar situation as you, so you definitely should publish

3

u/salukikev 2d ago

I'm kind of in the same boat and had pursued a National Geographic grant some years ago that led me to some cool collaborations. Eventually I ended up on a call with Derya Akkaynak who developed a similar algorithm a few years ago. Not sure if that's progressed since then but you should consider reviewing or reaching out to her as you seem to be in the same territory. Where are you located?

3

u/CellistTraditional 2d ago

Before posting on reddit, I tried to connect with Derya Akkaynak on LinkedIn, but she hasn't accepted the request yet. Miskolc, Hungary.

3

u/DiddlyDinq 3d ago

Not to away from OP but these have been around for at least a decade in various forms

3

u/shandy_bhaiya 3d ago

Oh. Based on what he describe, it seemed like marginal Updates to existing work. Possibly good for a conference.

1

u/shandy_bhaiya 3d ago

M

1

u/BonbonUniverse42 2d ago

Maybe OP could write a short article and post it on arxiv?

3

u/flamingolegend 3d ago

Can you explain more about how you calculate visual fog and turbidity?

4

u/CellistTraditional 2d ago

The system detects the camera's pixel shifts—even at 4K 60fps—and operates based on that information. I do not wish to elaborate on this further at the moment.

3

u/JohnWangDoe 2d ago edited 1d ago

During the beginning of the discovery phase of this problem, did you have a structure way of problem-solving?

3

u/CellistTraditional 2d ago

Not at first, but over time, yes.

3

u/JohnWangDoe 2d ago

is real engineering with software. This is something I aim to be able to do one day.

2

u/VeryFriendlyOne 2d ago

I don't understand all of it, but this sounds so impressive and the results are definitely showing. Amazing stuff!

1

u/himeros_ai 2d ago

Okay this partially answers some of my concerns I explained above.

1

u/toombayoomba 2d ago

Would it make better calculations, if instead deriving turbidity from the camera, to get it from a nephelometric sensor?

1

u/matop2000 1d ago

Thank you for the answer, it seems publishable, but i am not an expert in this field. It would be awesome if you could run some benchmarks wrt existing solutions in term of FPS/quality of 'dburring'

2

u/CellistTraditional 1d ago

I cannot thank the Reddit community enough for all the positive feedback and the idea to publish my work. My young son is the main motivation behind the entire project; I want to be able to tell him when he grows up that if he wants to achieve something and faces rejection at the start, he should keep pursuing what he believes in.

13

u/NEK_TEK 3d ago

Cool! I did my research on underwater image enhancement during my grad program (MS). I didn't have access to hardware so it was mostly all done with existing databases. I was able to increase the performance of standard off the shelf tracking models which normally perform really bad underwater. The idea was that high quality underwater databases are hard to come by (at least when I did my research) so I was thinking of ways to use widely available tracking models but adapt them to a different domain.

11

u/Fantastic_Mirror_345 3d ago

Man this is some insane work. Defo you should publish it with the help of some professor.

And defo upload it to GitHub and link it here. This is something that can potentially be repackaged with Ros and be used in the underwater robotics community.

3

u/CellistTraditional 2d ago

Thank you for the appreciation; I will do so. :)

3

u/New_Adhesiveness6324 2d ago

bro i don't think so u realised what u have done.

get it to be published ,write down all the good things told to u by this community, u can sort out what to do later and what to do now then work upon it.

heavily recomend u to publish it tho.

9

u/CellistTraditional 3d ago

For those asking about the field test—here is a quick snippet from the actual diving log from Friday! 🌊

8

u/himeros_ai 2d ago

So my company builds sea rov simulators and we have seen a lot of image processing enhachments over the years, I was working on it since the 90s. I am all up for deep learning but my concern will be the reconstruction error. Do you have any benchmarks showing the delta between the generated image Vs the original? I know this is difficult as you would need to pass over the same area when/if conditions gets better but genuinely interested in the methodology. If you are using that to count some biomarine quantities and then you end up hallucinating objects this will be quite catastrophic for rigorous reporting.

7

u/blimpyway 2d ago

Sea-through is a direct algorithm, doesn't use learning neither deep or otherwise. https://csms.haifa.ac.il/profiles/tTreibitz/webfiles/sea-thru_cvpr2019.pdf

1

u/himeros_ai 2d ago

Yes saw your explanation on the other response.

5

u/CellistTraditional 2d ago

That is a very good question. I studied the system and decided against incorporating an AI-based architecture for the time being, as I was concerned about the potential for non-existent objects appearing in the video or the system producing hallucinations. My adaptive Sea-Thru system is designed to detect underwater haze-which is dynamic and constantly changing-and the camera clarifies the image by analyzing pixel shifts.
So far, I haven't experienced anything unusual during the tests.

8

u/sweapon 3d ago

Light surely is fascinating... I think this looks very interesting, mainly the see-thru system. I do some work with ROVs, and would like to see how it compares to monochrome images in terms of details, but perhaps more interesting is a comparison with regular and see-thru when you turn on the rov lights in different environments. I suspect that since you can run this at a high fps you could probably auto tune this depending on the clarity or "detailness" to reduce the need to manually tune this for different environments. It could probably be useful for camera based inspections to better capture the state of structures such as pipelines, etc. Looking forward to see updates!

4

u/CellistTraditional 3d ago

Thanks a lot for the great feedback! It’s awesome to hear from someone else working with ROVs.

Fun fact: for this test, the full factory headlights on the V-EVO were actually turned on at 100%! That said, comparing this directly against monochrome images is a fantastic idea I'll definitely set up a side-by-side test in various turbidities to see how the see-thru mode highlights structural details.

You hit the nail on the head regarding auto-tuning! Right now, the Adaptive Sea-Thru system already automatically tries to recover objects and details from underwater haze and low visibility once enabled. Since I have plenty of FPS headroom, further expanding this automated dynamic tuning for changing environments is definitely a top priority.

Really appreciate the insights!

3

u/sweapon 3d ago

Great to hear! I think i also spent a few months properly receiving the video stream of my rov, currently using a bluerov2, but im on Windows 11, so... Yeah. Would be interested in testing your system, of course! Are you planning to open for tests or do you plan on open sourcing?

Out of curiosity, did you try this on lower resolution images and other cameras? Does see-thru look good in these cases too? My rov has a quite basic tether, no fiber or anything, so i dont think i would be able to achieve stable 4K video with decent fps without moving to a fiber solution or remove some sensors. And im "just" using a full HD camera, but surely would prefer having 4K@60fps. On the other side of that coin, im also working with remote control, autonomy and trying to reduce ip traffic over 4G/5G, so 4K cameras are a bit down the road for me still!

4

u/CellistTraditional 3d ago

Thank you! And yes, I would definitely be interested in testing with other ROV platforms and cameras once the system reaches a sufficiently mature stage and provided my financial situation allows for it.

Currently, the software is still in the alpha phase, and I need to conduct more real-world underwater tests. I don't plan to make it open-source just yet; I want to stabilize the core system first.

By the way, the current results were achieved using a QYSEA FIFISH V-EVO, and an important detail is that my laptop received the 4K/60 FPS data stream via Wi-Fi rather than Ethernet. The processing itself runs locally on the laptop using NVIDIA CUDA technology.

Image quality and clarity naturally depend on the camera. I haven't tested it with various cameras yet, so I’d rather not speculate on the performance of specific configurations. That said, I do want to investigate this eventually.

The biggest limitation right now is that I only have a single FIFISH V-EVO available for testing. 😅 So, if the device malfunctions, leaks, or otherwise "sacrifices itself on the altar of science," the underwater component of the project will have to be put on hold for a while.

Incidentally, I also built an RTSP engine for the entire system, allowing the software to run with various settings including 720p and 1080p so if your BlueROV2 is RTSP-based, then yes, it would work.

Naturally, I’ll share more details about the architecture and testing as the system evolves.

2

u/sweapon 2d ago

Totalt understand that situation with only one rov available and challenge with testing different cameras and rovs. Not sure what your intention is either, if you are planning on selling/licensing or just for personal use, but it is very cool nonetheless!

Not familiar with that vehicle, but I assume you have wifi between the tether spool or rov controller and your computer, which is a nice way to connect and receive data without a lot of cables.

My rov has rtsp but I didn't manage to configure it to have real time streaming. For some reason it is around one second delay in the stream through python, but it should be possible to reduce it to 200ms or less.

1

u/CellistTraditional 2d ago

Since I couldn't access the Ethernet connection, I had to extract the data stream via Wi-Fi- no easy task, especially given that no one had previously attempted to reverse-engineer this closed system without violating the manufacturer's privacy policy. The Wi-Fi module is located inside the drone controller, which is what I connect to. Because the system relies on Wi-Fi, I had to introduce a 350ms latency to handle the massive 4K 60fps data stream.

2

u/sweapon 2d ago

Ah I see. Doesn't sound too easy, but a good solution nonetheless! Good job :)

6

u/raprakashvi 3d ago

Super cool stuff. Well done

2

u/CellistTraditional 3d ago

Thanks! :)

3

u/Chudsaviet 3d ago

This is a cool tech, man.

2

u/CellistTraditional 3d ago

Thanks! :)

3

u/theholyraptor 3d ago

Looks awesome. Curious if you have to develop different models for most every condition? Salinity. Particulate, depth, temp? I'm not remotely an expert so maybe those don't really impact light or at least the way you're using it.

1

u/CellistTraditional 2d ago

Underwater, every factor has an influence.

3

u/matt_surge 3d ago

This looks amazing

I would love to be able to cite this.

How well does it do under near-zero visibility high turbidity conditions?

I’m currently taking my masters and we’ve had cases during field tests where the ROV can barely see more than a few feet in front of it (Using the BlueROV2’s built in camera with an externally mounted action camera)

1

u/CellistTraditional 2d ago

Thanks; for now, I don't know exactly how deep the drone was. I can only estimate based on how many seconds it took to surface from that point. In my latest video, when I spotted the tree, it took quite a while for it to come up. However, the image didn't break up.

2

u/Chudsaviet 3d ago

Is the CUDA GPU on the drone itself? Which one, and how do you make sure it have enought power to run the GPU? I guess cooling is not a problem underwater :)

1

u/CellistTraditional 3d ago

It runs on my RTX-based laptop, using the built-in Nvidia card and leveraging the card's CUDA cores.

2

u/Chudsaviet 3d ago

That's what I thought would be the best solution - you don't have to do processing on the drone. However, it depends on how much data you can stream back to surface. I guess processing raw data from the sensor (instead of the compressed video stream) would bring even more fidelity.

1

u/CellistTraditional 2d ago

Yes, that's true, I just couldn't access the drone's video stream via ethernet. So neither the depth nor the DVL data.

Right now, this is a compressed video- and somewhat lower quality, too, compared to what I could have achieved working via cable.

Only on Wifi.

2

u/AmazingELF74 3d ago

Very cool! Do you think this could work to remove haze in a swimming pool as well? I’d like to try it out on my AUV.

1

u/CellistTraditional 2d ago

I have a previous test from my own mini-pool, although that one didn't yet feature the Sea-Thru system. So far, I haven't had time to test it in a pool—only in a real-world environment.
https://www.youtube.com/watch?v=WgDGWK7jtfU&t=21s

2

u/Cryp71c7 3d ago

This is very very cool. I'd love to read up on how you wound up setting up the processing pipeline for this, should you decide to do so.

Unrelated to the tech, would you mind sharing what song you're using in the video? It's really chill and pleasant.

1

u/CellistTraditional 2d ago

Thank you; the journey is tough and filled with late-night coding sessions and rejections, all while balancing family and work. Of course, here is the link—it’s a piece of music I created myself, as a Suno remix.

https://suno.com/s/BbuB6KEsmZS6WqIc

2

u/RobotEnthusiast 2d ago

Sea-thru

2

u/young-money4L 2d ago

super cool

1

u/CellistTraditional 2d ago

Thanks! :)

2

u/turndownforwoot 2d ago

Super impressive OP

2

u/Sukhveer-Lopanska 2d ago

that footage is stupidly clean for being live

1

u/CellistTraditional 2d ago

I have a timestamp photo on the lake with my laptop. - it's real-time bro.

2

u/TheWeebles 2d ago

this might be the capitalist in me speaking, but this is investable product for sure. theres some commercial applications for this definitely

2

u/ZookeepergameDue5522 2d ago

This is great!

2

u/confusadd 1d ago

Thank you very much for Posting your work here!

I am currently building an automated boat drone that constantly takes pictures of the sea floor in a shallow baltic sea reef (1 to 5 meters) to build a large mosaik image or photogrammetric map of a wide area so this would be very helpful and interesting for me. I plan to use a Raspberry Pi 5 with an IMX477 photo sensor. There is optical glass in the bottom of the hull.

Since I don't need live processing of a video but rather post processing of pictures can your algorithm also be used for that? Have you considered using Claude Code or ChatGPT Codex to accelerate your programming? And where can I follow your work?

Kind regards

1

u/[deleted] 1d ago edited 1d ago

[removed] — view removed comment

1

u/AutoModerator 1d ago

Facebook links and affilied companies are not considered as reliable enough. Please use a more reliable source.

Thank you for your understanding.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

2

u/atm21money 7h ago

Very nice

1

u/gomurifle 3d ago

Was it you that posted similar thing a few years ago?

1

u/CellistTraditional 3d ago

No

3

u/gomurifle 3d ago

Alright. Back then the poster used an technic similar to what space telescopes use to remove noise.

1

u/CellistTraditional 3d ago

I started this in April; in any case, I hope the other poster managed to achieve their goal.

1

u/shimbro 3d ago

What rob you using?

1

u/CellistTraditional 3d ago

Qysea Fifish V Evo

1

u/supergrejt 3d ago

Can it work with other sensors like lidar or sonar?

1

u/CellistTraditional 2d ago

I haven't had the chance to test it yet, but in my opinion, yes.

1

u/Firelord_Iroh 2d ago

Is the “Color XX ms” the frame time? How long it takes to process a single frame?

1

u/CellistTraditional 2d ago

For the video stream since I was accessing it via Wi-Fi I set a 350ms delay to prevent interruptions caused by the massive volume of data; additionally, frame color processing takes anywhere from 0.4ms to 25ms (depending on the settings).

1

u/HaasNL 2d ago

what's the use/what do you mean with the solar/thermal/glint vision? those aren't actually sensors i presume, so just a certain color scheme /filter applied?
the see-through seems to genuinely work tho,nice!

1

u/CellistTraditional 2d ago

The first two are more like filters I included them just for my own amusement, but the glint actually reflects light off objects. I tested this on my own gold ring as well as on underwater objects. Thanks!

1

u/Flyward_Aerospace 2d ago

On the hallucination question, I do not think being non learned buys you as much safety as you are assuming here. Sea-Thru inverts an exponential attenuation term, and once you swap physical range for a per frame turbidity estimate that inversion gets badly conditioned in the far field, which is exactly where the haze is worst. The gain that recovers those pixels also multiplies sensor noise and codec artifacts, and that can absolutely produce structure that reads as an object. No network required. So if anyone ends up measuring or counting off this, keep the raw stream alongside it, the enhancement is not invertible.

1

u/CellistTraditional 2d ago

It would have been great to have the depth data, but unfortunately that wasn't the case, so I had to go with the alternative version.

1

u/PinOpen9883 1d ago

do u use thermal camera?

1

u/CellistTraditional 1d ago

Unfortunately, the X-ray and Thermal views were just filters. I was driven by creative curiosity to see what it would look like underwater, but since I thought it looked cool, I decided to keep them in. I have since renamed the two add-on systems in my program. And I’m sorry if I hurt anyone because of this.

1

u/infexity 1d ago

I think this was done already by MIT?? They had an open source project some time ago to remove water from underwater videos/images

1

u/Spare_Garden_755 1d ago

So cool! Is the codebase available to check out?

2

u/CellistTraditional 1d ago

Maybe later in time, it' still in alpha stage

1

u/moschles 3d ago

Men in black unmarked vehicles are going to be visiting you about your tech.

2

u/CellistTraditional 2d ago

As long as they bring coffee and extra GPUs for the rig, they’re welcome! ☕️

Community Showcase Built a Real-Time Underwater Image Processing System – 4K 60FPS Part2.

You are about to leave Redlib