How to Safely Decommission a Databricks Table in Hybrid Enterprise Environments
Removing a database table in enterprise settings is rarely straightforward, as downstream reports, ETL pipelines, and SQL Server-linked objects often depend on data without being formally documented. Engineers working in hybrid Databricks and SQL Server environments should first audit query history and cluster logs over the past 30–90 days to confirm whether the table is still actively used. Hard dependencies such as stored procedures, views, synonyms, and linked server objects must then be identified using tools like sys.sql_modules in SSMS and Unity Catalog lineage in Databricks. Rather than dropping the table immediately, a safer approach involves renaming it with a deprecation suffix, revoking write access while preserving read access temporarily, and notifying all teams with known access. Monitoring query logs during this short deprecation window helps catch any missed dependencies before the table is permanently deleted.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in