As we speak marks an essential milestone for open requirements and the broader knowledge group. We’re working with Microsoft to combine the info managed each in Unity Catalog and OneLake utilizing open requirements. The Unity Catalog Open API specification supplies an interface to trade knowledge each methods, zero copy, to open up entry to the complete lakehouse ecosystem for engines in both platform.
For almost a decade, Databricks and Microsoft have partnered to assist organizations unlock the worth of their knowledge by means of the openness and suppleness of the lakehouse. As we speak, that partnership takes one other step ahead, giving prospects extra alternative in how they entry and govern knowledge throughout platforms.
The Path to Open Interoperability
As open desk codecs like Delta Lake and Iceberg gained adoption, new silos emerged on the catalog-level. To deal with this, Databricks launched Unity Catalog Open APIs, establishing an open normal for unified governance throughout knowledge and AI, whereas additionally absolutely supporting the Iceberg REST catalog API for tabular knowledge. This dual-API strategy empowers organizations to undertake best-in-class options and keep away from vendor lock-in whereas seamlessly sharing knowledge throughout many knowledge platforms and SaaS Purposes. We additionally launched catalog federation to offer a single ruled view throughout all knowledge platforms. This mannequin, open APIs powering catalog federation, has grow to be the business normal for interoperability.
As we speak, we’re excited to see this open imaginative and prescient validated by our companions at Microsoft.
Microsoft provides assist for Unity Catalog Open APIs
As we introduced with our companions at Microsoft at present, our collaboration is deepening.
Microsoft has already added assist for studying from Azure Databricks with OneLake Mirroring, permitting Cloth engines to entry the entire knowledge managed in Unity Catalog seamlessly. In contrast to different OneLake integrations with proprietary platforms, this makes all knowledge in Unity Catalog – together with the best efficiency tables – accessible in OneLake with out copying, altering code, or extra prices.
As we speak, Microsoft and Databricks are extending their openness by integrating much more natively. With Microsoft OneLake including assist for the Unity Catalog Open API normal, Azure Databricks customers will be capable to seamlessly learn knowledge from OneLake.
With OneLake including assist for open catalog APIs together with Iceberg REST and Unity Catalog Open APIs, Microsoft is making OneLake structured and unstructured knowledge from companies like Dynamics, Dataverse, Blob retailer, and many others. natively accessible to the complete open lakehouse ecosystem, together with Databricks.
Databricks Builds on the Open Basis
Leveraging this expanded functionality, Databricks is including assist to make all OneLake knowledge seamlessly accessible in Unity Catalog.
This permits zero-copy studying of OneLake knowledge from inside Databricks making certain seamless interoperability and entry throughout each platforms, giving Databricks prospects entry to all their knowledge within the Microsoft ecosystem to energy their most superior AI, machine studying, and large-scale knowledge engineering workloads.
Catalog Integration
- Studying from Unity Catalog in Cloth – Typically Obtainable Now
Microsoft has launched Azure Databricks mirroring. Prospects can mirror Azure Databricks knowledge into OneLake, by means of Unity Catalog making certain that ALL knowledge managed by Azure Databricks is immediately accessible throughout Microsoft Cloth workloads. Each platforms can work over the identical copy of information with no knowledge motion. - Studying from OneLake in Azure Databricks – EOY 2025
This new functionality, permits Azure Databricks to learn knowledge from OneLake through their Unity Catalog Open APIs and permits all your OneLake knowledge to be leveraged seamlessly in Databricks. Prospects can create Unity Catalog Connections to OneLake based mostly on Managed Identities from EntraID, and might seamlessly mount knowledge objects from any Cloth workspace, immediately as a catalog in Unity.
Connecting to OneLake knowledge in Azure Databricks
The Future is Open
This milestone displays how far the business has come towards an open, interoperable knowledge ecosystem. Prospects now not want to decide on between innovation and interoperability—they’ll have each.
With Microsoft now embracing the open catalog normal, we’re excited to proceed constructing the way forward for knowledge and AI on this basis. Keep tuned for extra bulletins and roadmap updates, together with assist for Azure Databricks writing knowledge immediately in OneLake, as we deepen our integration.