We’re excited to announce the Normal Availability of Lakehouse Federation for Google BigQuery and the Public Preview for Oracle and Teradata. Now, you’ll be able to join, uncover, govern, and question information from these sources via Unity Catalog—with out migration or ETL. This makes information entry simpler whereas making certain an open, interoperable lakehouse structure.
Unify your information and governance throughout distributed platforms with the Lakehouse Federation
Lakehouse Federation allows customers to question and analyze information throughout disparate databases, information warehouses and catalogs with out duplicating or transferring information. By integrating exterior information sources straight into the Unity Catalog, organizations can:
- Achieve a unified view of your whole information property: Mechanically classify and uncover each structured and unstructured information in a single platform, empowering your group to securely entry and discover all out there information—wherever it resides.
- Question and analyze information seamlessly with a single engine: Allow quick, advert hoc evaluation and prototyping throughout all of your information, analytics, and AI workloads with out the necessity for ingestion. A unified engine optimizes question efficiency with superior question planning, caching, and cross-source execution, permitting you to effectively mix information from a number of platforms in a single question.
- Guarantee constant information safety and governance: Apply a unified permission mannequin to implement entry guidelines throughout all information sources. Implement row- and column-level safety, tag-based insurance policies, and centralized auditing whereas sustaining full visibility into information utilization. Meet compliance necessities effortlessly with built-in information lineage and auditability.
Lakehouse Federation helps a various set of platforms, together with MySQL, PostgreSQL, Teradata, Oracle, Amazon Redshift, Salesforce Knowledge Cloud, Snowflake, Microsoft SQL Server, Azure Synapse (SQL Knowledge Warehouse), Google BigQuery, and Hive metastore, with extra connectors coming quickly.
Normal Availability of Lakehouse Federation for BigQuery
With the GA launch of Lakehouse Federation for Google BigQuery, customers can now join BigQuery information to Databricks with out the necessity for information motion or duplication. This permits them to run workloads straight on BigQuery information from their Databricks atmosphere on GCP, Azure or AWS, whereas leveraging Unity Catalog’s fine-grained governance for safe and environment friendly information administration.
This integration permits for:
- Unified information administration: Seamlessly uncover and govern BigQuery information alongside different sources inside Unity Catalog, making certain constant entry management and governance.
- Optimized question efficiency: Make the most of BigQuery’s compute energy whereas benefiting from Databricks’ superior question optimization for quicker insights.
- Improved collaboration: Empower groups to work throughout cloud environments effectively with a unified interface for querying and analyzing information throughout a number of platforms.
Lakehouse Federation for BigQuery has seamlessly linked our Google Cloud and Azure environments. With out the necessity for information motion or duplication, we are able to now leverage Databricks on Azure to run Python and SQL workloads straight on BigQuery information, all whereas benefiting from Unity Catalog’s fine-grained governance. It is a game-changer for our multicloud technique, streamlining operations and enhancing analytics
— Lisa Fiege, Knowledge Science Engineer, Taylor Farms
The Google BigQuery connector is now typically out there beginning with DBR model 16.1 and can quickly be out there in Databricks SQL.
Public Preview for Teradata and Oracle Connectors
Along with the GA announcement for BigQuery, Databricks can be introducing Teradata and Oracle connectors into Public Preview. These new connectors will enable customers to increase the attain of Lakehouse Federation to much more information sources, additional unifying the information estates.
Get Began
Learn our documentation to get began with Lakehouse Federation connectors for Google BigQuery, Oracle and Teradata: