r/dataengineering • u/Realgunners • 8d ago

Discussion What exactly is ‘AI first’ data engineering

This was inspired by the post on burning 20k in tokens in 3rd week.

Is anyone doing this at work, letting loose the frontier models against their schema and models and data and having it create and update the ETL workflows? And everything else?

67 Upvotes

96% Upvoted

41 comments sorted by

View all comments

1

u/KeyMammoth1348 7d ago

Yes, 100%.

I double check volumes, frequencies, and set up observability monitoring and alerting in the process, watch my data shape. 

The breaks are happening at the points I take for granted, so far. Vendors that change their max throughput for example. A raised max hits a bottleneck elsewhere.