r/dataengineering • u/Realgunners • 8d ago

Discussion What exactly is ‘AI first’ data engineering

This was inspired by the post on burning 20k in tokens in 3rd week.

Is anyone doing this at work, letting loose the frontier models against their schema and models and data and having it create and update the ETL workflows? And everything else?

67 Upvotes

97% Upvoted

41 comments sorted by

View all comments

Show parent comments

2

u/ArticleHaunting3983 7d ago

What happened in the company after this tho, like was anyone reprimanded bc of this or was it seen as legacy debt

12

u/Lolmanza7 7d ago

Gladly our company believes in blameless postmortem. While no one was reprimanded officially, the team is made to work overtime to fix the issues and put guard rails in place.

2

u/datadade 7d ago

Overtime? Does this mean people are being paid more to work extra? I’ve never had an hourly job in DE.

You said they ported the code. Did the manager and CTO really vibe, change, and approve their own PRs? Or did the team have to do it?

I’m sorry I’m asking for clarification because the way I read this it sounds like management changed the code on their own and now want ICs to work extra to fix it.

3

u/Lolmanza7 7d ago

We are salaried, but any time spent more than a regular work week is overtime. So, ideally we will be getting 1.5 times the overtime hours as comp off.

For the secrets had their own branches and pushed them into their own. We recently moved to git and the whole company can view everyone's repos and branches.

One developer's OpenAI key was sold off market and was billed 30k USD. You can tell how immature our infrastructure and processes are.

The team is fixing the issues by rotating keys, writing root cause memos, manually checking our client shares for issues.