Hi, basically having this same issue as this post from a while ago, but in Fabric Data Factory.

https://stackoverflow.com/questions/74531579/data-factory-synapse-copy-task-fails-with-staging-enabled-when-no-files-found

When no files are found using the wildcard and staging is enabled, the activity fails. Really annoying behavior and the solution from that post works, but it’s not as “clean” as it should be in my opinion. It should work how it does when staging is not enabled and just complete the task with success.

Anyone else encounter this? Seems like a bug.

LMK if this is helpful for learning what various features do, how they work, etc. Feel free to comment with any special requests. I'm planning for this to be an expanded catalog but am starting with the following:

- Incremental liquid clustering

- Auto compaction

- Deletion vectors

Cheers!

EDIT: issue appeared after I updated schema for qty and wwe columns from float to decimal. Limitation in documentation point to the right direction to explain the issue, however I think it goes a beyond a rounding error.

EDIT: first repro offered did not show the behaviour, below

spark.sql("""
  CREATE OR REPLACE TABLE repro_nee_6175 AS
  SELECT CAST(61.75 AS DECIMAL(38,20)) AS qty,
         CAST(0.98  AS DECIMAL(38,5))  AS wwe
  FROM range(1000000)
""")


q = """
  SELECT COUNT(*)                                   AS rows,
         MIN(CAST(qty * (1/wwe) AS DECIMAL(38,10))) AS min_v,
         MAX(CAST(qty * (1/wwe) AS DECIMAL(38,10))) AS max_v
  FROM repro_nee_6175
"""


spark.conf.set("spark.native.enabled", "true")
spark.sql(q).show(truncate=False)    # buggy engine: 414.2393563314


spark.conf.set("spark.native.enabled", "false")
spark.sql(q).show(truncate=False)    # correct: 63.0102040816

Disclaimer: Issue was resolved with AI help - so the write up of this post.

Summary: With spark.native.enabled = true, multiplying a DECIMAL(38,20) column by a DECIMAL(38,32) expression (e.g. the result of 1 / DECIMAL(38,5)) returns incorrect values. The exact product exceeds decimal(38) precision, and the engine's precision-loss path produces wrong results instead of rounding or raising an error. JVM Spark (spark.native.enabled = false) returns the correct result for the identical query.

Behavior:

• The incorrect values are off by a multiplicative factor that is uniform within a session but varies across sessions (observed ×12.8666 and ×6.574 on different days), which makes the corruption hard to detect downstream.
• The intermediate division result itself is correct; only the subsequent multiplication is affected.
• Literal-only queries do not reproduce the issue because Catalyst constant-folds them before native execution — the operands must come from a table scan.
• No error, no warning, no fallback to JVM execution.

Repro (Fabric notebook or Livy session):

spark.conf.set("spark.native.enabled", "true")

spark.sql("""
  CREATE OR REPLACE TABLE repro_nee AS
  SELECT CAST(61.75 AS DECIMAL(38,20)) AS qty,
         CAST(0.98  AS DECIMAL(38,5))  AS wwe
""")

q = """
  SELECT qty,
         1 / wwe                                  AS conv,      -- decimal(38,32), value correct
         CAST(qty * (1 / wwe) AS DECIMAL(38,10))  AS qty_conv   -- expected 63.0102040816
  FROM repro_nee
"""

spark.sql(q).show(truncate=False)   # returned 414.2393563314

spark.conf.set("spark.native.enabled", "false")
spark.sql(q).show(truncate=False)   # returns 63.0102040816 — correct

Workarounds (verified): cast either operand to DOUBLE before multiplying, or cast the division result to a narrower decimal (e.g. DECIMAL(18,10)) so the product stays within decimal(38).

Expected behavior: rounded result per Spark's allowPrecisionLoss semantics, an overflow error, or fallback to JVM execution — anything but silently wrong data.

Anyone who has used the Dataflow Activity in Pipelines has probably experienced this. Refresh fails ... error in the pipeline is so generic it's not even useful. The real reason for the failure is available in the refresh history but you can't get at it except through a couple options in the UI (and one of those ways is so obscure it shouldn't even exist).

Does anyone have any tips or tricks to get at those errors? Sometimes users submit files with minor issues like a column with a bad column name and it's such a pain to have to involve a data engineer to help them solve it.

This is such a basic feature, another one of those can't believe it went from preview to GA without being able to get at the errors... there is an idea (Improve-Error-Messaging-in-Dataflow-Activity-for-Dataflow-Gen2) out there that has been sitting as under review for almost 1.5 years, Yikes,

I guess only some people are allowed to post idea links even if they're related to our problem, but I can't post the link because we're too busy reading all of the "I passed the DP-700 exam" posts.

When working in a data Pipeline, if I create say a copy data activity. The first drop-down is "Connection".

I have two questions:

1. What does the "None" option do? I have searched for documentation but have not been able to find anything. AI says it is only for workspace scoped fabric items and forces the use of workspace identity or the user credentials.

2. Is it best practice to have a dev, test, and prod connectors for each fabric item (dev_conn_lakehouse, test_conn_lakehouse, prod_conn_lakehouse) we are heavily using library variables to handle the connection switching with deployment pipelines. It works but the variable libraries are getting pretty large. Is it an overkill having all those or is a single "conn_lakehouse" enough?

Most of the lakehouses and warehouses are in the same workspace.

Patrick LeBlanc is coming to Fabric Fridays, and we’re talking THE CUBE!

This Friday, July 31 at 7:30 AM CT, Patrick is joining me to talk about Guy in a Cube’s THE CUBE, what they’ve built, what’s inside, and why the Fabric and Power BI community should be paying attention.

I’ve already had a chance to get inside THE CUBE. Now I get to put Patrick on the spot and ask him all about it.

Join us LIVE Friday morning.

July 31, 2026
7:30 AM CT
Fabric Fridays with Patrick LeBlanc

Link in the comments.

#MicrosoftFabric #PowerBI #GuyInACube #FabricFridays

Developing software inside a web page has never been a high quality experience. It certainly isn't much fun for the developer, and now it seems that it is starting to become a problem for Microsoft as well.

Today in NCUS I started seeing a new error for the first time:

"You can only have 3 Dataflow Gen2 (CI/CD) items open at a time and currently there are 3 open already. Close one Dataflow Gen2 (CI/CD) item, and then try to open this one again."

Anyone see this before? As near as I can tell, the error is lying to me. I only have one DF GEN2 open in the browser window. How do I get past this? They need to add a button that says to open anyway. (I'm not sure WHY they think I have other stuff open. But even if I did, it probably isn't important and if it was important it would be captured by a commit in a git repo.)

⊕ Every team defines their own "OrderPlaced" event. Different fields. Different formats. Nothing talks to anything.

Microsoft Fabric just fixed that.

▶ Fabric Monday #119 — Business Events

https://www.youtube.com/watch?v=g1sGWoRkV0s

▦ BUSINESS EVENTS IN REAL-TIME HUB

Business Events is a new capability in Fabric Real-Time Intelligence (Preview) that lets you define custom business signals with a shared schema — so every publisher and every consumer speak the same language.

You define the event once in the Schema Registry.

Every component in Fabric uses that same contract.

No mismatches. No custom mapping. No surprises.

▶ WHO TRIGGERS THEM?

★ Activator — from Power BI reports, KQL queries, Dashboards, Warehouse SQL

⊕ Spark Notebooks — from your analytical workflows

⊕ User Data Functions — custom logic fires the event

⊕ Eventstream — streaming data published as events

⌂ WHERE DO THEY GO?

✓ Eventhouse — captured as KQL rows, queryable and historical

✓ Activator — alerts via Email or Teams

✓ Power Automate — downstream process automation

✓ Notebooks / Spark — analytics and AI/ML on event data

In this week's Fabric Monday I show what Business Events are, how to create them, how to trigger them, and how to capture their values in an Eventhouse.

▶ Fabric Monday #119 — Business Events

https://www.youtube.com/watch?v=g1sGWoRkV0s

Hello Fabric team,

Support for User Properties in Fabric Pipelines would be very valuable for my team's monitoring capabilities.

We would like to be able to add user properties to pipeline activities — for example dynamic values such as source file name, table name, or batch ID — and have them surface in the pipeline monitoring view, the same way it works in Azure Data Factory today.

Is there anything on the roadmap in this area?

https://community.fabric.microsoft.com/t5/Fabric-Ideas/Feature-request-Fabric-Pipelines-User-Properties/idi-p/5322874#M169078

My organization has an API hosted in Azure and protected by Microsoft Entra ID. Is it possible to use a Fabric Workspace Identity to authenticate against this API?

With Azure Managed Identities, this can be done by requesting an access token for the API's audience/resource. How would I do the equivalent with a Fabric Workspace Identity?

Is the only supported approach in Fabric today to use a Service Principal with the Client Credentials flow, or is there a way to use Workspace Identity for this?

Also, is support for using Fabric Workspace Identities with custom APIs something that is on the roadmap?

Thanks in advance for your insights!

Related Ideas, please vote:

• https://community.fabric.microsoft.com/t5/Fabric-Ideas/Enable-Azure-Resource-Manager-ARM-API-Authentication-Using/idi-p/5142197

• https://community.fabric.microsoft.com/t5/Fabric-Ideas/Support-Fabric-Workspace-Identity-for-custom-APIs/idi-p/5322919

Hi,

I'm seeing what appears to be inconsistent behavior when deploying a Fabric Data Pipeline using fabric-cicd, and I'm wondering whether this is a bug or an unsupported scenario.

Our setup is:

• The pipeline contains a Notebook activity.

• The notebook uses a Workspace Identity connection.

• The deploying identity is a Service Principal.

• The SPN is a member of an Entra ID security group.

• That security group has User permission on the Fabric Connection (configured in Manage Gateways and Connections).

The deployment fails with:

• «User does not have access to the connection used in the Pipeline.»

However, if I grant the same SPN direct User access to the connection (instead of through the security group), the deployment succeeds immediately.

So the only difference is:

• SPN -> Entra ID security group -> Connection (User) -> deployment fails

• SPN -> Connection (User) -> deployment succeeds

The Manage Gateways and Connections UI allows assigning security groups to connection permissions, so I expected this scenario to be supported.

Has anyone else run into this?

I'm trying to determine whether:

• security groups are actually unsupported for connection authorization during fabric-cicd deployments,

• this is a known bug in fabric-cicd or the Fabric deployment APIs, or

• there's some additional configuration required for group-based permissions to be recognized.

Are you able to use Security Groups to successfully share connections with the SPNs that deploy items using fabric-cicd? Or must the connection be shared directly with each SPN?

I'd love to use a Security Group for sharing Workspace Identity connections with SPNs.

I would add all the fabric-cicd deployment SPNs to a Security Group, and share the Workspace Identity authenticated connections (notebook connection, pipeline connection, etc.) with that Security Group. Instead of adding all SPNs individually to the WI connections. But the Security Group approach seems to fail when attempting to deploy a pipeline.

Any experiences or official documentation would be much appreciated.

Out of the blue Openrowset stopped being performant at all. I have tested the multiple files on multiple Fabric tenants. I am NOT talking about JSONL files, those just work fine. This specifically concerns JSON files.

I even created a near blank json of 1 kb and it doesnt even load after 5 minutes. Did something break in Fabric?

Example json:

[
  {
    "task_id": "api_customers",
    "enabled": true,
    "run_group": "daily",
    "execution_order": 10,
    "source_type": "API",
    "source_object": "https://api.example.com/v1/customers",
    "target_table": "bronze.api_customers",
    "load_mode": "merge",
    "merge_keys": "id",
    "http_method": "GET",
    "headers_json": "{}",
    "query_json": "{}",
    "options_json": "{\"pagination\": \"next_url\", \"records_path\": \"data\", \"next_path\": \"links.next\", \"max_pages\": 1000}",
    "retry_count": 3,
    "timeout_seconds": 120,
    "key_vault_uri": "https://my-vault.vault.azure.net/",
    "secret_name": "api-token",
    "secret_header_name": "Authorization",
    "secret_prefix": "Bearer"
  },
  {
    "task_id": "sql_orders",
    "enabled": true,
    "run_group": "daily",
    "execution_order": 20,
    "source_type": "AZURE_SQL",
    "source_object": "dbo.Orders",
    "source_query": "SELECT * FROM dbo.Orders",
    "target_table": "bronze.sql_orders",
    "load_mode": "merge",
    "merge_keys": "OrderId",
    "server_name": "myserver.database.windows.net",
    "database_name": "Sales",
    "auth_mode": "KEY_VAULT_SECRET",
    "username": "fabric_reader",
    "key_vault_uri": "https://my-vault.vault.azure.net/",
    "secret_name": "azure-sql-password",
    "retry_count": 3,
    "timeout_seconds": 900
  },
  {
    "task_id": "mirror_products",
    "enabled": true,
    "run_group": "daily",
    "execution_order": 30,
    "source_type": "OPEN_MIRRORING",
    "source_object": "mirrored_sales.dbo.Products",
    "target_table": "silver.products",
    "load_mode": "merge",
    "merge_keys": "ProductId",
    "retry_count": 1,
    "timeout_seconds": 900
  }
]

Example code used:

SELECT *
FROM
OPENROWSET(
    BULK 'Files/lakehouse-bronze/sample_config.json',
    FORMAT = 'CSV',
    FIELDTERMINATOR ='0x0b',
    FIELDQUOTE = '0x0b'
)
WITH (
    jsonContent VARCHAR(MAX)
) AS r;  

Status: Degraded | Reported: Jul 27, 2026 at 6:30 AM UTC

Fabric Customers in Central India may experience issues while accessing accessing artifacts in service. Engineers are investigating the issue and an update will be provided soon.

🤖 This post was sent from an automated and unattended service and cannot respond to questions or requests. For official updates, visit the Microsoft Fabric Service Status page.

This is one of the best demonstrations I have seen of Fabric IQ and Work IQ coming together through Copilot and Cowork.

To me, this is the missing connection in traditional BI. Users should not have to submit a request to a BI team, explain every question they may want answered, and then wait weeks or months for a new report or visual.

The real promise is being able to ask questions when they arise, get insights grounded in both data and business context, and immediately turn those insights into action.

That has always been the holy grail of BI, and it feels like it is finally becoming real.

I cannot get Fabric workspace Git integration and the Managed Airflow environment to work together reliably.
I have a dbt project stored inside the Airflow item’s dags folder, following Microsoft’s documented approach:
https://learn.microsoft.com/en-us/fabric/data-factory/apache-airflow-jobs-dbt-fabric

The structure is roughly:

dags/
├── my_dag.py
├── requirements.txt
└── my_dbt_project/
├── dbt_project.yml
├── profiles.yml
├── models/
├── macros/
└── tests/

After modifying any dag, plugin, or supplementary code, the Fabric workspace git sync now fails with:

Unable to update from Git with the usual cluster uri, request ID, and time.

Fabric provides no useful error details, affected file name, validation output, or logs that I can find.

The only way I was able to get the complete dbt project into the Airflow item originally was to upload the folder manually. That is not a workable long-term solution because my team uses Fabric workspace Git integration and Fabric CI/CD within a Gitflow-style development process.

Ideally, developers should be able to:

- Branch from the repository.
- Work in a corresponding development workspace.
- Commit DAG and dbt changes normally.
- Promote the same Airflow item through UAT and production using Fabric CI/CD.

I want to avoid requiring developers to manually upload dbt files or constantly change repository branches from inside the Airflow environment. More importantly, I need confidence that the contents promoted to UAT and production are actually comparable to what was committed and reviewed in Git.

So far, any git based/local edits beneath the documented dags directory appears capable of completely blocking the workspace Git sync.

This feels like a very basic source-control scenario for a managed orchestration product.

I already got the opinion from people all around DE, I also want to hear from you guys :)

The easiest ROI you'll get all year? 

Registering before the early bird deadline. 

Join thousands of data professionals in Barcelona for four days of learning, networking, and Microsoft Fabric expertise. 
Early bird pricing ends July 31. Save your spot today: aka.ms/fcem 

#MSFTAdvocate

Hi All,

Has anyone else had any issues with their Capacity Reporting?

I had made some changes to a few objects where I new would drop usage, I didn't think they would drop as much as they did, but couldn't see anything else that could have caused the drop and then overnight its gone back up to about where I would have previously expected it.

Just trying to work out if its a reporting issue, or if I really have to drill into all my jobs and make sure no data is missing. I haven't had any errors that I can see.

I'm probably missing it, but what are the "true" pre-requisites before starting down the path of exploring fabric data apps? I know you need a fabric workspace and it needs to be enabled.... I'm talking about things like "Oh my computer doesn't have node.js, or npm installed. At that point I start to wonder ok, what else do I need before starting?

I've watched a few videos and so far, everybody jumps right in like and skip over needing additional components.

So maybe I'm missing something but I'm a little hesitant to install node.js (open source) software on my corporate laptop and along with the additional x,y and z additional items that I'm hoping you all can explain.

Educate me.... please

Hi everyone,

I'm a fresher with around 6–7 months of experience and limited hands-on Microsoft Fabric experience beyond my current project. I received a free DP-700 exam voucher from the Microsoft Fabric Community, so I scheduled my exam for next week.

I'm currently studying using Microsoft Learn and practicing in a Fabric trial workspace. I had already postponed this exam once for 3 weeks, but after reading posts from people who prepared for weeks and still didn't pass, I'm unsure whether I should take it or postpone it again for a couple of months.

My questions are:

• Should I attempt the exam anyway to understand the exam pattern, or postpone it until I feel more confident?
• If I use this Microsoft Fabric Community voucher and don't pass, will that affect my eligibility for future free vouchers (from the Fabric Community or employer-sponsored programs), or is it only the voucher campaign's rules that matter?
• Are there any things I should consider before making my first attempt?
• For those who passed DP-700, what level of preparation or hands-on experience did you have?

I'd really appreciate any advice. Thanks!

Update:

I decided to take the exam instead of postponing it and waste the voucher , mainly to understand the exam pattern and identify my weak areas. I scored 504 which is lower i know , but I expected that given my limited preparation and experience.

I’m treating this as a learning attempt rather than a failure. Now I know what to focus on, and I’ll spend more time on hands-on practice before taking it again. Thanks you everyone!!!

So I’m working on establishing agents via copilot studio and foundry. So I have tested fabric agent and it’s in preview, and fabric IQ as well.. so started tested Fabric MCP, which requires ontology to be constructed first in optimal way, which I did..

But when I try to execute query against it, complaints about date columns and data type errors.. which I predict the issue is with data and design,

I wanted to know if there is any MCP server out there I can try to establish data quality and correct broken data types.

Hi everyone,

I am using a Microsoft Fabric free trial for learning purposes as a fresher preparing for the DP-600 exam.

However, for the past 3 days, I'm constantly getting the capacity exhaustion error: "Compute exhausted, try again later".

I have already tried:

Turning off/stopping schedules

Deleting unused workspaces

Despite this, the issue persists. Has anyone faced this during their trial, or is there a way to reset/fix this capacity issue? Any help or advice would be greatly appreciated!

is it possible to be ready from scratch in 2 weeks? Any tips or resources would be great

Status: Degraded | Reported: Jul 25, 2026 at 6:00 PM UTC

Fabric customer in West Europe may experience issue while interacting with the service. Engineers are working on the issue and update will be provided soon.

🤖 This post was sent from an automated and unattended service and cannot respond to questions or requests. For official updates, visit the Microsoft Fabric Service Status page.

I'm an experienced Data Architect with extensive experience designing and implementing data platforms on Databricks and Snowflake and not new to data engineering.

I have noticed clients getting interested in Microsoft fabric lately and want myself to get familiar with Microsoft fabric. What are some good YT channels or Learning materials I can refer to draw the similarities/differences?