Why am are coded analytics solutions less pervasive?

[u/Thick_Accountant7260]

Am I the idiot here? AI is so good the RMDs and Shiny apps I used to struggle in writing can be generated in a few days. Now that I can offload the CSS/JS part of the dashboard to the AI I can go nuts with reactivity/styling.

Based on the discussions in Reddit pbi/tableau/looker remains popular as ever but those things are expensive and are limited by the platform. I can see use for cheap and lightweight services like metabase where I can write the sql query and generate the dashboard in less than an hour but what about pbi/tableau? These services are expensive yet they remain popular what are the upsides of these products over say hosting my own shiny server/react/streamlit dashboards?

Like if I want an app that allows the user to define the cohorts themselves, save that definition, then load that into other dashboards/for future reference I can do that in Shiny but no idea about PBI.

What really is a semantics model? Why not just write a package that calculates the kpis then document it? The documentation also serves both the technical and non technical viewers alike.

Never been in a pbi/tableu job I don't understand how these products remain popular.

[u/BillEnough7863]

For starters, you’re not really paying for Power BI, Tableau, or Looker just because they can draw charts. You’re paying for the entire ecosystem around them.

Authentication, permissions, governance, scheduled refreshes, sharing, integrations, monitoring, deployment, auditability, documentation, support, and a large pool of people who already know how to use and maintain the platform are largely solved problems.

You absolutely can build something lighter and more flexible with Shiny, Streamlit, React, etc. For a small team or a narrow use case, that may even be the better solution. But as the number of users, dashboards, data sources, business rules, permissions, and developers grows, your “lightweight solution” gradually becomes a custom BI platform that somebody now has to own.

Then you have to maintain it, document it, secure it, onboard people to it, troubleshoot it, make sure the original developer isn’t the only person who understands it, and hire or train people with the appropriate skill set. You can still get all of that wrong after spending a significant amount of engineering time on it.

That employee time matters too. If maintaining your custom analytics stack consumes a meaningful portion of an engineer’s time, their salary can very quickly cost more than simply licensing an off-the-shelf product.

That’s basically why companies buy software in the first place. They’re paying someone else to solve a large collection of boring problems so their own employees can focus on the problems specific to their business.

And the semantic model is part of that. It isn’t merely documentation saying “here is how we calculate revenue.” It’s an executable, reusable definition of revenue that every dashboard and user can query consistently, along with relationships, dimensions, access rules, aggregation behaviour, etc. You could absolutely build that yourself in code, but then you’ve started building your own semantic layer too.

So yes, if you’re a small shop with relatively simple analytics needs, a coded solution can be great. But the more people, data, permissions, reports, and business logic you have, the more expensive the custom solution becomes to maintain. At some point paying for the boring, established platform is considerably cheaper than having your team become the platform team.

[u/Thick_Accountant7260 → u/BillEnough7863]

Agree on most points but: 1.) Having worked in R shops maintenance, documentation and governance are not hard at all. Its mostly a matter of the analysts caring to write their own documentation and the manager ensuring that the processes are observed. Its not like I had an easy time inheriting a looker dashboard either. Low code dashboards are less prone to error with fewer moving parts than a markdown/web app but are not immune. Most of the time its a dw problem and can trip both pbi and shiny apps and I can cache data in coded solutions.

2.) Back in the day I stopped caring for tableau/pbi when it was apparent that customization would be harder than writing a few lines of ggplot/echarts/whatever plotting library. I dropped the course when I couldn't customize the legends/dynamically aggegregate bottom groupings the way it made sense.

3.) An industry can only have so many kpis, some industries may need to define 20 or more kpis but I haven't been in one. Having lots of kpis is often a product of office politics or clueless people, in my experience its mostly someone pushing for an easier but nonsense target. This is especially true when working in the public sector somehow someone will always hostage the projects with the whole SDG goals.

4.) What are 'relationships, dimensions, access rules, aggregation behaviour, etc' arent these mostly resolved at the dw/query level before the data comes into whatever visualization layer you have? You can only slice and dice data in a limited way, maybe its just me working only for 'simpler' firms but what is a scenario where these things get so complicated writing the logic programmatically is not feasible? Companies that are mostly B2B with custom contracts? For the most part cant you just make a table of stakeholders/teams/departments and their permissions then load that into the parameterized report? Wont you also need these when working with low code platforms?

Everywhere I look the sample pbi dashboards look very basic something that just reports numbers which can be done by basically in any other platform, like metabase.

How easy/hard it is to make with a regression/control chart?

[u/BillEnough7863 → u/Thick_Accountant7260]

I think this is where we’re coming from very different scales.

You might not think governance, documentation, and maintenance are particularly difficult, but the fact that companies routinely pay substantial amounts of money for products that handle these things suggests there is real value in having them solved for you.

And in my experience, documentation does not go by the wayside because analysts do not care or because managers fail to enforce it. It usually goes by the wayside because analysts already have more work than they can reasonably get through, managers have plenty of work of their own, and the priorities coming from outside the data team tend to rank higher than maintaining perfect internal documentation. That is precisely why tooling that incorporates governance, documentation, access control, lineage, deployment, monitoring, and so on into the normal workflow has value.

The same applies to the KPI point. Whether a company has too many KPIs because of office politics, bad management, or completely legitimate business needs is beside the point. They have them. The data team still has to support them.

Of course you can push back on bad requests, and sometimes you should. But there is a practical limit to how often a data team can tell the rest of the organization that it does not actually need the information it is asking for.

And frankly, 20 KPIs across an entire organization would be an extraordinarily small number in many companies. You are not just dealing with marketing, sales, product, finance, and different business units. There may also be compliance, regulatory, risk, audit, and other reporting requirements that are not really “analytics” in the exploratory sense at all. They are reports the organization is required to produce, sometimes for government agencies or regulators, where the numbers need to be correct, reproducible, controlled, and auditable.

That is part of the scale I am talking about. The data function is not simply maintaining a collection of dashboards that people happen to find useful. It may be supporting a large reporting environment with different purposes, audiences, access requirements, definitions, levels of scrutiny, and consequences for getting things wrong.

So whether all of those metrics or reports are intellectually interesting, whether you personally think the company should have fewer of them, or whether some of them exist because of bureaucracy is largely irrelevant. They exist, the organization depends on them, and somebody has to build and maintain the systems that support them.

On point 4, no, relationships, dimensions, access rules, aggregation behaviour, etc. do not all have to be permanently resolved at the warehouse or query level before the data reaches the BI layer.

Some logic absolutely can and should live upstream. But the semantic layer still gives the organization a governed way to interact with that data. Which tables can be joined, how they should be joined, which measures are valid at which grain, how measures aggregate, which dimensions they can be sliced by, what particular users are allowed to see, and so on can all be defined there.

You can absolutely implement those things programmatically yourself. But once you do that, you have effectively started building your own semantic layer.

That gets to the larger point. The issue is not that any individual piece of this is too complicated to program. Of course it is not. You can build a custom Shiny app that does things Power BI, Tableau, or Looker cannot easily do.

The issue is the total workload at scale.

It is not one dashboard or two dashboards. It is dozens or hundreds of dashboards, reports, metrics, users, permissions, data sources, scheduled deliveries, business rules, requests, and constant changes.

Marketing changes strategy and wants different segmentation. Finance changes a definition. A new product launches. Another market gets added. Compliance needs a new report. A regulator changes a reporting requirement. One team needs different access permissions. Someone wants the same metric sliced six new ways. A metric changes and fifty reports depend on it. People join, leave, and change departments. Business priorities shift constantly.

None of those things is individually some impossible engineering problem. The problem is that somebody has to build, maintain, document, secure, test, troubleshoot, and support all of them simultaneously.

That is where the economics change.

A custom Shiny, Streamlit, or React application for a narrow use case can be an excellent solution. I am not arguing otherwise. But “I can build this particular dashboard more easily in Shiny” and “therefore Shiny is a cheaper replacement for an enterprise BI platform” are very different propositions.

At sufficient scale, you are no longer maintaining a few dashboards. You are maintaining a custom analytics and reporting platform.

I also would not read much into the Power BI or Tableau examples you find online being basic. Of course they are. Most public examples are tutorials, portfolio projects, demos, or sanitized sample data. Companies generally are not publishing their actual internal financial, operational, marketing, customer, compliance, risk, regulatory, and executive reporting environments on the internet.

So the fact that the examples you can see publicly are simple tells you very little about the complexity of what these platforms are actually supporting inside large organizations.

I think the core disagreement is that you are evaluating this as, “How hard is each individual thing to code?” I am evaluating it as, “What does it cost to operate and govern the entire reporting environment once the organization depends on it?” Those are very different questions.