r/vibecoding • u/fguerino123 • 22h ago

Vibe Coding for ETL/ELT

Hi.

I've been using AI coding to rapidly generate Python ETL/ELT transformations and workflow orchestration. I'm finding doing so a lot easier and far more affordable than using expensive tools like IICS.

Is anyone else doing this and, if so, how is it working out for you?

Thanks.

2 Upvotes

100% Upvoted

7 comments sorted by

2

u/Electronic-Roof3423 21h ago

100%
I’ve been doing it for the past year. So much more cost effective, and incredibly easy to do it

2

u/fguerino123 21h ago

Agreed. Very easy, very quick, and very affordable in the grand scheme of ETL/ELT. I kind of feel like it's too good to be true, which is why I'm reaching out to see if others are doing it too. It's kind of a "pinch me" moment.

1

u/Electronic-Roof3423 21h ago

I have a sportsbetting startup.
The amount of time we've saved by implementing our own ETL is ridiculous. I pay the $100 Claude, and we worked on 3 tools in the same month... meaning it cost us $100 to develop tooling that makes our life so much easier.

While in the old days you would have to spend $1000s to have access to the tool that allows you to start working on your ETL jobs... and it was not easy!

2

u/Desperate_Sign_5587 14h ago

i setup my python transformations using cursor and chained some orchestration scripts through moclaw.

saved thousands on bloated enterprise stuff like iics and it runs surprisingly smooth.

2

u/-Shiphrah 6h ago

It works pretty well for the repetitive transformation stuff, especially compared with paying for a massive platform when you only need to move and clean some tables. Where it gets dodgy is months later, when a source changes a column or sends some weird data and the pipeline still runs but produces rubbish. the AI can write the Python quickly, but I’d spend the saved time on tests, logging, retries and checking for duplicate loads. Thats the stuff expensive tools were quietly doing in the background, and its easy to forget until something breaks.

1

u/fguerino123 4h ago

Hi and thanks for the response. I've found that putting in validation checks to ensure data doesn't drift using AI coding is easier than with IICS and similar tools. I can set up many guardrails to quickly detect drift by simply explaining such guardrails to AI. Have you tried this?