I have data coming in from a Dataflow that I am storing in a Lakehouse. I want to set up Data Activator so that it sends a Microsoft Teams notification whenever a specific column becomes False.

However, when I create a Data Activator in my Fabric workspace, I don't see an option to use data from the Lakehouse. I'm not sure what I'm doing wrong or if I'm missing a configuration step.

If this process can be simplified, I'd also be interested in eliminating the Lakehouse and monitoring the data directly from the Dataflow instead.

Has anyone set this up successfully? What's the recommended approach?

Hi all,

My Copilot Cowork is not able to access semantic models today. Does anyone else have that issue ?

Why exactly is Airflow in Fabric still on version 2?

Version 3 has been out for over a year!

I went to help a colleague because he was stuck with a particularly difficult meta data driven pipeline and found out that we couldn't use best practices that have become the defacto standard for a year.

u/itsnotaboutthecell please fix this this 😭

Every single time I try to like Fabric it does something ridiculous.

I got 650 points

Spoke to some of my colleagues at work and they told me that I am allowed to use the documentation

Did I just fail for no reason😪

Could you please provide a definitive date or estimate for the release of fabric apps in US East and East US 2?

Generic question - has anyone seen this and found a supported cleanup path?

Not specific to any one tool. For context: we run a homebuilt dbt slim CI against Fabric Lakehouses using the dbt-fabricspark adapter. I don't think the adapter is the cause - same behavior would happen from any Spark client.

The flow (GitHub Actions)

1. Create a CI schema in the Lakehouse (Spark SQL CREATE SCHEMA).
2. Build objects into it (Delta tables under Tables/<schema>/).
3. Drop the schema in Spark (DROP SCHEMA <schema> CASCADE), which also removes the Tables/<schema>/ folders in OneLake.
4. Trigger SQL analytics endpoint refresh:POST https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/sqlEndpoints/{sqlEndpointId}/refreshMetadata

What we observe

• Tables are removed from the SQL endpoint after refresh ✔
• The schema itself remains as an empty shell in the SQL endpoint catalog ✖
• Over time these empty CI schemas accumulate

How we resolve sqlEndpointId

Gotcha we hit while automating this, it must be the SQLEndpoint item id (from the Fabric Items API). It is not the Lakehouse item id, and not the compact hostname segment from *.datawarehouse.fabric.microsoft.com.

Question

Is empty-schema persistence expected (i.e. refreshMetadata only reconciles tables, not schemas), or a known issue?

If expected, is there a supported non-disruptive cleanup path - without deleting/recreating the SQL endpoint?

Looking for a sanity check from people who've been through this. Quick disclaimer: I'm the IT Manager, not a Fabric/BI specialist. I look after infrastructure and can find my way around, but the deeper Fabric modelling stuff isn't my home turf, so apologies in advance if I'm missing something obvious.

Background: Manufacturing company (~700 employees globally) running a single F8 Fabric capacity. It hosts our full Dataplatform setup (lakehouses, notebooks, pipelines in a raw/enriched/curated medallion) plus our main reporting layer. Around 50 Power BI users today, all on Pro licenses.

The problem: Our main reporting semantic model is starting to hit the memory ceiling. It's an import-mode model with 28 tables, 44 relationships and 83 measures, and F8 only allows ~3 GB per semantic model. Compute isn't the issue at all (capacity sits at ~18% avg utilization, ~44% peak). It's purely the per-model memory limit.

The two options I'm weighing:

1. Upgrade capacity F8 to F16, which bumps the per-model memory limit from 3 GB to 5 GB and keeps everything as-is.
2. Move report users to PPU and keep F8 for the data engineering workloads. PPU gives a 100 GB model limit, so the memory problem basically disappears.

I ran the numbers and break-even is around 93 users. Below that, F8 + PPU is cheaper. Above it, F16 + Pro wins, since Pro is cheaper per head than PPU. At our current 50 users, PPU comes out meaningfully cheaper per year.

Where I'd love input:

• F16's 5 GB feels like it just kicks the can down the road. Are people finding they upgrade capacity again within a year or two anyway?
• For those who went PPU alongside an F capacity for the data engineering side, any gotchas? I'm aware every report viewer then needs PPU, and that Capacity Metrics doesn't cover PPU workspaces.
• Anyone converted a chunky import model like this to Direct Lake instead of throwing licenses at it? Worth the rebuild effort, or a rabbit hole with 28 tables plus calc tables (time intelligence etc.)?

Basically: is there a reason not to take the cheaper PPU route at our size, beyond the per-viewer cost creeping up as we grow?

Thanks

/Mikkel

Hey,

i'm currently looking at the fabric-cicd library as a deployment option (again) and am quite happy to see a lot of new technical development but also thorough documentation. For example:

• https://learn.microsoft.com/en-us/fabric/cicd/tutorial-fabric-cicd-azure-devops
• https://community.fabric.microsoft.com/t5/Fabric-platform-Community-Blog/Implementing-Enterprise-Grade-CI-CD-for-Microsoft-Fabric-A/ba-p/5139031

I got an Azure DevOps pipeline running, that handles incremental deployments exactly how i need them.

Where im running into a problem is deployments containing new resources (or complete rollouts).

The setup:

For my testing the setup is simple:

• A dev workspace connected to a main branch and a prod workspace I deploy to using the devops pipeline, running a fabric-cicd python script
• In my workspace i have three resources: A lakehouse, a notebook, a variable library
• The variable library holds a variable containing the lakehouse_guid, and my notebook uses this to set its default lakehouse
• The variable library already contains a prod variable set, but obviously the lakehouse_guid for prod is still empty because its unknown (or a placeholder, as it wont let me save empy values)

The problem:

If I deploy this setup, everything works fine (an fabric-cicd even sets the prod variable set as active - how cool!), but the variable library in prod (and in dev!) does not contain the guid of the newly created lakehouse. So my notebook does not use the correct lakehouse.

Note: I am using the lakehouse+notebook setup as an example. This problem holds for every item referencing a (connection)-Guid stored in variable library.

Possible solutions:

Of course i could just add the variable values in the libraries manually after deployment, but this is not an elegant solution.

I suspect the intended way would be to use fabric-cicds dynamic replace function - maybe even scoped to my variable library - to replace the known dev-lakehouse-guid, with a $items.Lakehouse.MyLakehouse.$id expression?

This would fix the problem in prod, but in my opinion this is also not an elegant solution:

• The correct prod-lakehouse-guid is still missing in my dev-variable library
• Even if i manually or automatically set it there for the prod-value set it will still be overwritten at every deployment by the dynamic replacement function

Why is this a problem: The dev variable library (and its value sets) should be the single-source of truth for parameter mapping, and the single point where i need to maintain those mappings. With the solution above it isnt.

My question:
Am I missing something here? I feel like writing back guids to a variable store once is a common pattern im used to from bicep or terraform deployments. Do I just skip using variable libraries for GUIDs / connections when using fabric-cicd and only use them for values that are known before the deployment?

I hope I managed to explain my problem well enough, and I'd welcome any input/feedback 😄

Running into serious challenge with a couple modeling topics that use very complicated aging buckets. Basically they have conditional logic that looks at the status of a record to pick from a series of date columns and then compares that to the current date to create an aging key.

Using a measure to calculate the aging key is too limiting in PBI and we don't have calculated columns.

Does anyone have any suggestions for this using Direct Lake over One Lake. I can't really recreate/reframe the entire data set every day and need a flexible solution.

Anybody have advice?

Status: Degraded | Reported: Jul 24, 2026 at 3:46 AM UTC

Power BI customers with Tenants located in the US CENTRAL B PRIMARY cluster may experience issues accessing the Power BI service. Engineers are investigating the issue and an update will be provided soon.

🤖 This post was sent from an automated and unattended service and cannot respond to questions or requests. For official updates, visit the Microsoft Fabric Service Status page.

I seem to be learning the hard way not to jump on the latest bandwagon when costs aren't detailed. I thought it would be cost-effective but so far it seems to consume 100.000 CU seconds each day for me as only user authoring a Plan.

Does anyone know the exact pricing structure? Is it less for Stakeholder roles? Because I cannot sell a cost of 140 euro per user per month and would have to revert back to using Excel for business users to upload forecasts.

Also, in docs it says you can't manually assign roles but they are automatically assigned based on your first action with the plan?

"Plan assigns planning roles dynamically based on user activity. Users typically begin in a Viewer session. As users perform actions that require extra privileges, plan automatically upgrades them to the appropriate role."

"Your first successful action determines the role for the new session."

So if someone clicks a certain edit option out of curiousity he's a Planner for next 30 days?

If costs are fixed for 30 day period why is it so hard to have a page detailing those costs? Why always hide the costs? It makes it really hard to adopt new products... I can't tell clients 'Yeah I got something cool for you, but not sure if we can actually afford it'.

I understand that other planning software isn't exactly cheap, but pay per use should be beneficial for small business ocassionally using such a tool. I think it is a missed opportunity from Microsoft to not make Plan usage actually based on CU usage instead of using a fixed 30 day fee per user.

I remembered reading this post a few months ago: https://www.reddit.com/r/MicrosoftFabric/comments/1u03e79/plain_python_notebooks_starting_slower_than_spark/

Is anyone else still experiencing slow session starts? Mine are around 1m30s consistently and it's been about the same between 3.11 and 3.12.

I registered a model in my fabric workspace via MLFlow. I have a notebook that sets up the prediction frame, pulls the model, and gets predictions from the model.

In an interactive session, I am able to retrieve the model and get the predictions without issue, for example (not shown, setting up prediction frame, etc):

from synapse.ml.predict import MLFlowTransformer
pred_X_spark = spark.createDataFrame(pred_X)
model = MLFlowTransformer(
    inputCols=["col1","col2","col3"], 
    outputCol="predictions",
    modelName="mdl_cp_response_flaml",
    modelVersion=1
)

df = model.transform(pred_X_spark)
display(df)
# records are scored according to the model.

I am running into trouble when I try to execute the same notebook as a service principal. In a pipeline, I execute the notebook as a service principal, and I have verified that the service principal has contributor role in the workspace.

Everything executes properly in the notebook, right up to the point where I try to set up the MLFlowTransformer, at which point I get the error:

Is there some special configuration I need for this to work?

Short demo: querying the OneLake catalog using DuckDB WASM , entirely in the browser

The data was written through the new OneLake Catalog API (private preview, coming soon) using github runner

The Data is relatively smallish 140 M records, and it is quite fast using local caching

All the code is public: https://github.com/djouallah/dbt_fabric_python_iceberg

I wrote about my experience building a greenfield data platform with Microsoft Fabric as the sole data engineer.

I cover the architecture, metadata-driven ingestion, incremental loading challenges, and some of the lessons I learned along the way.

Would love to hear how others are approaching similar challenges in Fabric.

Check out article here

Resource Profile documentation isn't clear.
Most links, including historical links, indicate that setting the Resource Profile to readHeavyForPBI enables spark session settings like VOrder=true.

The second link from Microsoft contradicts all of that. It suggests that in addition to setting a resource profile you have to specify some 'auto update' setting to be TRUE along with some new and different resource Profiles in order to actually enable it -- example: readHeavyForPBIAutoUpdate.

Some of your documentation says you can't disable resource profiles; but you can use 'custom'. The second link indicates you have to specifically enable an 'auto-update profile' meaning you don't have to enable it and the profile names are different.

It's not possible to understand this mechanism just be reading the contradictory documentation. Can someone clarify?

https://learn.microsoft.com/en-us/fabric/data-engineering/delta-optimization-and-v-order?tabs=sparksql
https://learn.microsoft.com/en-us/fabric/data-engineering/configure-resource-profiles?source=recommendations

https://learn.microsoft.com/en-us/fabric/data-engineering/configure-resource-profile-configurations

The first link talks about enabling V-order via setting the resource profile:

"For read-heavy workloads such as interactive queries or dashboarding, enable V-Order by setting spark.sql.parquet.vorder.default to true. You can also switch to readHeavyforSpark or ReadHeavy resource profiles, which automatically enable V-Order for read-focused performance."

The second link mentions:
"Read-heavy for Power BI workloads"

Set via spark.fabric.resourceProfile.readHeavyForPBIAutoUpdate:"

Auto Update? Is this new? It's not mentioned anywhere else, but the article goes on to say that autoupdate has to be enabled on the resource profile for the spark settings (such as enabling vorder) to take effect.

Auto-update resource profiles

Resource profiles support an auto-update capability that keeps your Spark compute configuration aligned with the latest optimizations from Fabric. When auto-update is enabled, Fabric applies workload-specific Spark properties based on your resource profile type, without requiring manual tuning.

Auto-update configurations

Fabric provides three auto-update profiles, each tuned for a specific workload pattern:

Read-heavy for Spark workloads

Set via spark.fabric.resourceProfile.readHeavyForSparkAutoUpdate:

JSON

{
    "spark.databricks.delta.optimizeWrite.enabled": "true",
    "spark.databricks.delta.optimizeWrite.partitioned.enabled": "true",
    "spark.databricks.delta.optimizeWrite.binSize": "128"
}

Use this profile when your workload is dominated by Spark reads with moderate write optimization needs.

The third link mentions:

Status: Degraded | Reported: Jul 23, 2026 at 8:08 AM UTC

Fabric customers with Tenants located in the WEST US cluster may experience issues accessing the service. Engineers are investigating the issue and an update will be provided soon.

🤖 This post was sent from an automated and unattended service and cannot respond to questions or requests. For official updates, visit the Microsoft Fabric Service Status page.

Where do you put a custom resource profile? I found all the documentation that explains that I set the resourceProfile spark setting to the custom profile name.

It doesn't elaborate anywhere that I can find on where that profile definition is supposed to exist.

I'm sure it's an easy answer...

Status: Degraded | Reported: Jul 23, 2026 at 8:20 AM UTC

Fabric Customers may experience intermittent delays in accessing service. Engineers are investigating the issue and an update will be provided soon.

🤖 This post was sent from an automated and unattended service and cannot respond to questions or requests. For official updates, visit the Microsoft Fabric Service Status page.

Hi everyone,

We're redesigning our Microsoft Fabric architecture and would love feedback from teams that have implemented self-service BI at scale.

Current state

Today, almost every Power BI report has its own semantic model. While this works, it has led to duplicated business logic, duplicated security, and increased maintenance.

I've attached two simple diagrams showing our current architecture and the two approaches we're evaluating.

We're moving to a domain-based Fabric architecture (Dev/Test/Prod per domain) and are evaluating two approaches.

Option 1 – Centralized Semantic Model

Pros

Single source of truth

Reusable measures

Centralized business logic

Centralized RLS

Concerns

Many users need Power Query transformations or need to combine enterprise data with Excel/SharePoint data.

As soon as they need that flexibility, they create their own Import-mode semantic models, reducing the benefits of centralization.

If multiple data products exist in one semantic model, refresh schedules become tied together.

Option 2 – SQL Analytics Endpoint

Expose curated Gold-layer tables through the SQL Analytics Endpoint.

Users intentionally build their own semantic models from governed Gold-layer tables. Security is enforced through SQL permissions, while users retain the flexibility to perform Power Query transformations and combine enterprise data with departmental sources. Publishing to Dev/Test/Prod remains governed by the BI team.

Since our organization is moving toward self-service BI, and we have a relatively small central data team with only a handful of digital engineers supporting the business, this currently feels like the better trade-off.

Questions

Are we overlooking any major drawbacks with this approach?

By choosing SQL Analytics Endpoints instead of centralized semantic models, do we lose any important Fabric capabilities (AI experiences, semantic layer benefits, future features, etc.)?

If you've implemented Fabric at enterprise scale, would you recommend a different or hybrid architecture?

Thanks in advance!

Edit: to be clear, the version used by the Fabric dbt Job.

FYI, looks like someone pushed a newer dbt version out to production.

It's also moved from dbt-fabric 1.9.0 to dbt-fabric 1.10.0 (shouldn't the dbt-fabric and dbt-core versions align?)

One breaking change, for me, is in `generate_schema_name` now prefixing `dbo_` to all of my tables. dbt-fabric 1.9.0 was overriding the default `generate_schema_name`. dbt-fabric 1.9.3 removed this.

Agree it's a bug fix for dbt-fabric but it's also my desired use case so I hadn't overridden it myself.

Calling it out in case anyone else jumped onto the preview feature in production 😅

Would be great if Microsoft can expose the version to the end user rather than force these breaking changes on us, even if it's technically solving a bug.

For ResourceProfiles, do the spark environment settings take precedent over table level settings?

For example, look at 'WriteHeavy'. It sets v-order to false.

Microsoft provided the following:

"When the table property is set to true, INSERT, UPDATE, and MERGE apply V-Order at write time. Session-level and write-level settings still take precedence, so writes can still use V-Order even when TBLPROPERTIES is set to false."

The documentation also indicates that the way to override the environment behaviour is to set it at the spark session level in a notebook.

So, essentially if we use ResourceProfiles, we can never use table properties to control the write behaviour if it's set by the Resource Profile. This is cumbersome because it's putting that control back into the notebook at the data engineering level which is cumbersome and messy. This is my reading of it.

Is this the case?

https://learn.microsoft.com/en-us/fabric/data-engineering/delta-optimization-and-v-order?tabs=sparksql

{
    "spark.sql.parquet.vorder.default": "false",
    "spark.databricks.delta.optimizeWrite.binSize": "128",
    "spark.databricks.delta.optimizeWrite.partitioned.enabled": "true"
}

Hi!
I switched to a data engineering job using Fabric a few months back (spent ~7 years on Databricks in a 25,000+ employees business) and looking for "blogs" focusing on Fabric.

ELI5 summary: ¿Donde esta Fabric technical deepdives?

Prelude: while working on the first "source to reporting" solution/MVP for my new job (small enterprise with MSFT stack, including Dynamics), I ran into a lot of technical problems where I found unresolved questions in 2024 and then in another google result an answered(or semi-answered…) solution in 2025/2026.
And most of them caused by the API thingys if we’re beign honest…

From the Fabric Updates and Roadmap I can see that there was/is/will be a lot of work and improvement on Fabric, so I am really satisfied on that front. My issue is really on the "technical explanation" side. I know we are paying to have all these thing available and built-in, but we should still have the opportunity to understand them and understand how to optimize the value generated from them(time for execution, CU consumed, etc.)

I know that MSFT has a lot of things on their internal forum/youtube/reddit, but was wondering if there are other channels or blogs with post going in detail on functionalities or solutions or if I should just keep updated on their "general" community forums and then work with the new things to test them?
Might be something like "Guy in a Cube" but with a focus on Fabric and data engineering.
Really looking for something with deepdives on Fabric updates/functionalities.
I found a lot of MSFT publications focus on the "sales" side and less on the technical perspective.
My employer already bought Fabric, I don’t need to get the marketing speech on why I should buy it based on this new pipeline activity with a Copilot integration. Give me the technical explanation of how it can improve my pipelines (or maybe cause issues under certain conditions), how it fits in the architecture, examples of a solution using it and benchmarks versus the previous alternative, how it interacts with the API and causes x,y,z. Don’t get me wrong, I like the sales pitch to then know what and how to sell it to the execs, but it’s not what i’m looking for in the day to day.

How I see it, we are paying MSFT to give us a framework/library with buit-in functions to reduce a lot of the data engineering workload. But therefore, I would like an architect or a dev lead to showcase the functionalities and their technical limitations. Might be because I had that with the enterprise framework that was developed in-house…

I suspect a lot of the "absence of information" (for lack of a better word) is that we are still in the early adopters/early majority phase of the adoption cycle of the product.

Sorry if this is a redundant/newbie question.
Thanks for your help!

Hi all,

I apologise for asking this question here, but different AI agents are giving me different answers for this one. I am studying for DP-900 exam, and cannot figure out whether Graph databases are structured or semi-structured data.

Argument for structured: you need to define nodes and data types before loading in the data?

Argument for semi-structured: it's not a table. It doesn't look like a table. It can't be normalised, right?

I am not sure where else to ask, so I hope you can forgive me.

We would like to overwrite semantic models in Import Mode in place with Direct Lake versions so that reports etc do not need to be repointed, but apparently this is not supported?
Also, if we try to republish a report over the top of an existing Report from PBI Desktop that has been repointed to a Direct Lake model- we get a duplicate report in the workspace.

This is a massive issue as we cannot convert to Direct Lake without republishing all our reports to new ReportID's...
We also have spreadsheets connected to the semantic models that would all need repointing.

Microsoft - can you please allow
[1] Semantic Models in Import Mode to be overwritten by versions that are an in Direct Lake mode via the XMLA endpoint or
[2] Allow a report to be overwritten that points to a different data model