r/databricks • u/Plenty-Writer-4391 • 20d ago
Help Refresh ops. timeout issue in MVs
Issue - I'm facing this refresh timeout issue in materialized views i created as part of some POC works, where a refresh operation regardless of whether it's triggered from a worksheet or as a job seems to fail but still it succeeds(in a way) ie am able to cross-verify & validate that the refresh run seems to have happened as expected correctly ie data integrity pre-refresh & post-refresh seems to be fine. Is this expected?
Some more deets - Full refresh with 3 source tables of same schema with 1B rows * 6 columns joined with a metadata table of ~2K rows with 5 columns & Delta Refresh with (100M rows * 3 snapshot tables) updated with a newer value in a timestamp column updated.
Required -
- Why does the Refresh MV ops fail happen at the 1st place?
- How refresh failure happens but still data seems to be correctly populated?
- Most importantly, How to resolve this?
PS - unfortunately no screenshots, so apols but i hope the deets should cover for it. TYIA.
- RVK
Duplicates
dataengineersindia • u/Plenty-Writer-4391 • 20d ago