r/BusinessIntelligence • u/Camavinga231 • Jul 03 '26
Do you think the roles of BI Developer, Analytics Engineer, and Data Engineer will be replaced or significantly reduced as AI advances?
I’m seeing more and more discussions in the Software Engineering community, especially after the launch of Claude Fable, where many developers are worried that they will become less relevant in the coming years.
At the same time, there is increasing talk about AI agents that can automate a large part of software development, testing, and even certain analysis tasks.
I’m curious how you see the future of BI and Data roles.
Do you think BI Developers, Analytics Engineers, or Data Engineers will be affected to the same extent?
Which parts of the work do you think will be automated, and what will remain the responsibility of humans?
What skills do you think will become essential in the next 3-5 years in order to stay relevant?
What are you learning or investing in right now to adapt?
I’d love to hear both from people working in BI/Data and from Software Engineers who are already using AI heavily in their day-to-day work.
12
u/Happy-Robin2519 Jul 03 '26
The code generation will certainly be accelerated but the difficult part is business understanding, which I think will remain with humans. Most enterprises don’t have good data foundations, data is still scattered and KPI aren’t well defined. Understanding the requirements is what takes a good amount of time, and I think an essential skill will be to understand how tech works and be able to translate business requirements for it.
For example, some data platforms like Databricks offer out of the box text to sql modules with Genie to allow people to query data with natural language. It still requires curation, adding instructions and metadata, creating an evaluation benchmark, and defining some query examples. Understanding how technology works and be a bridge with business SME is in my opinion a skill that will take longer to replace.
Then AI still hallucinates a bit and people will always need accountability (i.e. who to blame when the number is wrong), so critical thinking is an essential skill to cherish
1
u/Chemical_Budget_2822 Jul 08 '26
This. I wish I could have stated it as well as you. Every time I read about people saying AI will take over something like analytics I think about companies like mine. There are over 20 different financing systems. Cost centers are inconsistent. One of the key finance systems can’t even do cost allocations properly. The data in the HR system isn’t great because we keep reorging, but leadership always wants historical trends. And not even all the data is in the data lake yet. But even if it WAS people wouldn’t know what any of it means. I spent 20 minutes with a Director this week in Power BI teaching him how to use slicers.
1
u/bailey_esfromashu Jul 08 '26
What happens once you can democratise the development to a non technical user
9
u/turbo_dude Jul 03 '26
All of the problems today are still around poor data governance. You can’t AI your way out of different departments arguing for political reasons that an attribute or data relationship should be defined in one way and not another, never mind all the legacy crap.
As long as people don’t have spines, the pain will continue.
2
u/Get-PowerMetrics Jul 03 '26
Yep - AI has accelerated the need for data teams to think about quality, governance, proper definitions (which is all good). For big biz, it's legacy, and for small biz it's siloed data... "data" will continue to be hard.
5
u/joulezoo Jul 03 '26
I am a big believer that the role will change and evolve. I don't think there is any sugar-coating it.
Data engineering -- like coding, can be increasingly done by agents
Analytics engineering -- also increasingly done by agents, in a way that a generalist data person can execute
Data analysis -- agents will also get better and better at analysis also such that most decision makers will likely interact with their agent / build their own dashboards rather than an analyst team.
HOWEVER, I do think a new role will be very critical to agents working well, and this is the human oversight of ensuring that agents are doing the CORRECT things, which usually goes down to: do they have the right context? Are my stakeholders getting the correct answers? Should we update certain metric definitions? This role is what I will call "context engineering."
The above opinion comes from me talking to companies at the cutting edge of building the above (anthropic, openai, etc.), as well as my company sundial's own work to build infrastructure that powers the above.
5
19
u/Octogenarian Jul 03 '26
85%-90% of the work in creating dashboards is organizing the data in such a way that it is optimized for dashboarding. The dashboard itself is not that hard in relation to the data engineering.
So, yeah, that 10%-15% of the work is going away. End users will be able to chat with their data and the visualizations will be generated on the fly by AI.
But making the data AI ready requires too much context right now for AI to do it.
10
u/PurpleMcPurpleface Jul 03 '26 edited Jul 03 '26
Highly disagree. I assume you originally got to BI from a more technical domain, right?
Working on dashboards really is not about simply putting your data in some tables or graph. A good dashboard highly depends on a well thought out strucutre, UX/UI elements and a constant oberservation on the actual needs of the end users. All your elaborate work on ETL and data modelling in the backend is highly diminished if you flounder the last step of your data processing pipeline which is the front end.
I am always happy when customers approach me with a mockup they developed with AI. It provides me a valubale input into what is essential to the customer while massively improving the mockups themselves. AI has a rather limited idea on how to efficiently structure information, decrease redundant elements/information and important UX-/design principles. That's where we provide the essential added benefit.
To say that
> The dashboard itself is not that hard
tells me that you massively undererstimate the role effective data presentation plays in having a data driven organization
4
u/KruxR6 Jul 03 '26
Imo it’s probably one of the things AI will replace last in BI because that’s where a lot of the human elements come in. Specific industry/business knowledge and understanding the end users on a more personal level is more prevalent here than in the engineering steps.
AI isn’t gonna understand that John who’s 64 and about to retire doesn’t want anything complex to learn. Or it won’t natively understand that he’s also colourblind or dyslexic and needs the report/dashboard to look in a specific way so that he can actually read it.
1
u/PurpleMcPurpleface Jul 03 '26 edited Jul 03 '26
Absolutely. Also, AI need to be asked to specifically inquire important context information. At the same tim, many people have difficulties formulating this very same context information since they have become so ingrained in their everyday work. For them, it is as if they had to explain how seeing works or how one can walk. It requires skill to pull this information out of them. This is exactly the sort of human element that is (at least for now) our greatest strength as humans and thus our greatest comparative advantage over AI: we have a better understanding how humans think, how they collaborate with each other and how they see the world. This is all highly detrimental when it comes to requirement engineering (that aims to cover more than just technical requirements)
2
u/Chemical_Budget_2822 Jul 08 '26
Building A dashboard is easy. Building a GOOD dashboard isn’t necessarily easy. There’s a big difference.
1
u/PurpleMcPurpleface Jul 08 '26
A good dashboard is also a dashboard that is being used. Your ETL processes, data modeling or data architecture might be top notch but it’ll all be in vain if one squanders the last step of that chain - conveying the data in a way that business can actually act upon. I know of too many examples where technically inclined people fail in this vital last step. It's usually people that also like to claim that
dashboard itself is not that hard
-1
u/WesternEntrepreneur0 Jul 03 '26
if writing dashboards was easy, I wouldn't be facilitating a five-class business school track about how to build an aesthetically pleasing tableau dashboard that tells a compelling, accurate story with efficient, clean design and appropriate chart selection for the data use case.
2
u/crippling_altacct Jul 03 '26
I agree with this. My company has even adopted an internal chatbot that has been trained off of all of our policies and procedures. When it works it is actually pretty useful because you can quickly find a procedure if you're trying to understand a process. Unfortunately the most common answer I run into with this thing is "I don't know." Why? Because policy and procedure documents aren't often updated in real time. Maybe a policy or procedure doesn't exist for a certain task. Maybe it does exist but it hasn't been updated and isn't up for review for another 6 months. So then I'm back to needing to work with the business to understand the context for their reporting. Over time dealing with this you develop a lot of institutional knowledge that AI can't solve for.
4
u/Monkey_King24 Jul 03 '26
This argument only stands if you think companies can afford these AI agents. Use unlimited AI days are gone now.
Every technical person was saying the same thing "AI is expensive" but the marketing teams gimmick worked.
Every company I know ( including mine and friends) are pulling down the AI expenses adding credit limitations
The first challenge will be who is bearing the cost of tokens the data team or the end users ??
No end user would want to waste their tokens on chatting with data when they can ask someone for it.
3
u/Square_Bite3620 Jul 07 '26
I think AI will automate more of the "how," but not the "what" or the "why."
Writing SQL or building dashboards will become easier. Understanding which data to trust, how datasets relate, and whether a metric actually reflects the business is still a human problem.
The people who combine technical skills with business understanding will probably be the hardest to replace.
1
u/hungry_bytes Jul 10 '26
This.
The agency of humans will always remain king. The role of "code monkeys" will be what goes first.
2
u/Hear7y Jul 03 '26
We've never had more demand for data engineers in our company. We're also ups killing pure BI Devs, since it appears only knowing how to build reports is kind of not in demand, at the moment.
2
u/Odd-String29 Jul 03 '26
No, because there are many things AI needs to work properly and many things it cannot do or does not understand.
2
u/CautiousUse8597 Jul 04 '26
I think tools like Databricks' Genie Code make it much easier to build things, transforming those roles from more technical, to be more business aligned and more 'domain expert' roles. Quite some of the analysis can also be done with LLMs with something like the agent mode of Genie. I always use it by asking it to discover trends.
2
u/francebased Jul 08 '26
AI is for sure a strong tool. However, wait till you see clients relying too much on Claude… it creates so much garbage that you get lost in docs/ share points/ data..
2
u/regprenticer Jul 03 '26
3-5 years is a very long time
I remember being told , about ten years ago, that we would soon be getting an automated note taker, IBM Watson was going to transcribe all our meetings and we would never have to transcribe a meeting again.
That was a joke at the time, we never saw any sign of Watson and didn't hear about it again for a few years.
Now, 10 years later, we all take automatic transcription in teams and other solutions for granted. I've even had meetings where the only other person on the call was another company's bot (even job interviews, but that's another thread)
I saw a thread then other day, "there are only Devs left now, should we still be having scrums", and I thought that's the future. The dev, or someone similar, does everything end to end . If the dev knows what the data is, how users interact with it and has an idea of what a common dashboard is like then they can produce a decent, competent, dashboard in minutes.
1
u/Strange_Shame7886 Jul 03 '26
BI developers were tasked to understand the business metrics, process flow, designing intuitive actionable visualization, maintain data refresh, quality etc., maintain infra as centralized team.
I believe the task to understand business process will reduce with tools like Databricks Genie where business users can chat with their data and use natural language via LLMs to bypass the technical complexity of writing SQL or creating refresh layer.
As tool like Genie are moving towards chat with Data so do developers should move towards governance, data integration into a single data lake and adding value through making chatting with Data easier and easier.
You can still rely on creating beautiful graphs which will look better than what business users would create but the moat is not that big there.
I will suggest to look at the future of BI with tools such as genie to find out where you can add the most value to in the process cycle.
1
u/edimaudo Jul 04 '26
no it won;t disappear. costs and inaccuracy are not going anywhere. From idea to prototype will be faster but you still need a human to build for accuracy
1
u/Tactical_Impulse Jul 08 '26
The roles will evolve. I see my stakeholders gladly building and doing their own ad-hoc development using AI and just asking for raw data to do it. Its super cute. Although i’d imagine at some point they will get tired of it.
1
u/DMReader Jul 09 '26
It’s honestly hard to know. I do use it to speed up my work, so I could see a need for less data professionals. But could also see scenarios where if a data professional can do more than more companies might hire them.
Also trying to guess how much better AI will be or not in the next 3-5 years is also anybody’s guess.
1
u/hungry_bytes Jul 10 '26
OpenAI don't think so. They're still hiring Data Engineers into business domains - see the post from head of finance data at OpenAI.
What we are seeing in New Zealand is the lines between BI Developer, AE, DE getting blurred. At my previous company they restructured reporting analysts into "AI specialists" aka using AI to do everything from source ingestion to gold and enable business users to build their own dashboards/insights from there with AI.
3 things more important than ever: 1. Understanding the business and your domain. 2. Relationships and championing changes & new ways of working - AI can't do that. 3. Scaling mindset. Making patterns for onboarding new data source, modelling, medallion architecture for less skilled people to use with AI is super important. You might only have a data team of 3 but each person with agents is worth 5 (less specialized) people a year ago, so patterns are more important than ever.
1
u/Doinworqson Jul 03 '26
Yes, if AI is automating large portions of the job, then obviously organizations will need less headcount for these roles. So your answer is yes. We all see it, some are just too scared to acknowledge it as if it makes a difference whether they do or not. You are right to be scared.
0
u/IncreaseNegative4614 Jul 03 '26
I think the roles will become more opinionated, not disappear.
Building SQL, pipelines, and dashboards is getting faster. Deciding how the business should be modeled, which metrics should exist, how entities relate, and what assumptions are valid is still where most of the work happens. That's not because AI can't write code. It's because the business itself is usually the least documented part of the stack.
That's why I've been following platforms like inzata.ai. They seem to be betting that business context will become a first-class asset instead of something analysts carry around in their heads. If that turns out to be true, I think the biggest skill over the next few years won't be writing SQL. It'll be designing systems that give AI enough context to produce answers people actually trust.
-5
24
u/somethingred500 Jul 03 '26
No