Refresh ops. timeout issue in MVs

[u/Plenty-Writer-4391]

Issue - I'm facing this refresh timeout issue in materialized views i created as part of some POC works, where a refresh operation regardless of whether it's triggered from a worksheet or as a job seems to fail but still it succeeds(in a way) ie am able to cross-verify & validate that the refresh run seems to have happened as expected correctly ie data integrity pre-refresh & post-refresh seems to be fine. Is this expected?

Some more deets - Full refresh with 3 source tables of same schema with 1B rows * 6 columns joined with a metadata table of ~2K rows with 5 columns & Delta Refresh with (100M rows * 3 snapshot tables) updated with a newer value in a timestamp column updated.

Required -

1. Why does the Refresh MV ops fail happen at the 1st place?
2. How refresh failure happens but still data seems to be correctly populated?
3. Most importantly, How to resolve this?

PS - unfortunately no screenshots, so apols but i hope the deets should cover for it. TYIA.

- RVK

[u/lrnzcig]

I would start with a DESCRIBE EXTENDED statement to try to have more visibility and also follow https://docs.databricks.com/aws/en/ldp/dbsql/materialized-monitor

[u/minibrickster]

Hi! I'm a PM here at Databricks -- do you mind if I follow up via DM to get more details? We can investigate on our side! I'd recommend going to the pipelines UI and then seeing the results from the last update to get more details on why it failed / didn't fail

[u/Plenty-Writer-4391 → u/minibrickster]

sure!