Unity Catalog<\/a>.<\/p>\n\u201c[Lakehouse Federation] is this effort to expand our platform to easily manage and query data in other systems as well,\u201d Databricks co-founder and chief technologist Matei Zaharia told me. One of the core features of this new capability is query federation, he explained, which allows users to connect different data sources and query them efficiently, all while essentially seeing that as a standard database inside of Databricks.<\/p>\n
Often, a company may have real-time data in a PostgreSQL database that powers an app, but an analyst may want to combine this with historical data from a data warehouse and query across both systems. Using Lakehouse Federation, Databricks can now handle the query planning for this (and cache data as needed to keep the system performant).<\/p>\n
Ideally, of course, Databricks would like everyone to use its platform, but the reality is that even though enterprises want to simplify their infrastructure, it\u2019s very hard to move data platforms. \u201cThis allows you to at least have a single interface for users and a single place to manage that,\u201d Zaharia explained. Often, companies try to build a system like this in-house, which tends to be costly and complicated (and often fails).<\/p>\n
Zaharia also noted that Databricks has an interesting advantage here because its product is built on Apache Spark \u2014 and the Spark open-source ecosystem includes a wide variety of connectors, which Databricks can then use to build a product like Lakehouse Federation without having to rebuild many of the core integration tooling.<\/p>\n
One advantage here is that Databricks is also layering its data governance features on top of this, allowing companies to more easily manage access to their data across platforms. That\u2019s something Microsoft has long bet on with its Purview governance solution, too, for example. Now more than ever, data governance is something enterprises are keenly focused on.<\/p>\n
\u201cWe\u2019re giving organizations access to all of the data they need through one system, which will lead to more innovation \u2014 and the best part about that innovation is that it doesn\u2019t sacrifice security. By enabling customers to easily apply the rules consistently across platforms and track data usage, we\u2019ll help them meet compliance requirements while pushing their businesses forward,\u201d said Zaharia.<\/p>\n<\/p><\/div>\n
<\/br><\/br><\/br><\/p>\n
Science, Tech, Technology<\/br>
\n<\/br>
\nSource:https:\/\/techcrunch.com\/2023\/06\/28\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/<\/a><\/br><\/br><\/p>\n","protected":false},"excerpt":{"rendered":"Source:https:\/\/techcrunch.com\/2023\/06\/28\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/ Databricks builds a data mesh with the launch of Lakehouse Federation 2023-06-29 22:50:59 Databricks today launched what it calls its Lakehouse Federation feature at its Data + AI Summit. Using this new capability, enterprises can bring together their various siloed data systems and discover, query and govern their data across a wide variety of […]<\/p>\n","protected":false},"author":1,"featured_media":36257,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"nf_dc_page":"","om_disable_all_campaigns":false,"pagelayer_contact_templates":[],"_pagelayer_content":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[26,17,8],"tags":[],"class_list":["post-36256","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-science","category-tech","category-technology"],"yoast_head":"\n
Databricks builds a data mesh with the launch of Lakehouse Federation - Science and Nerds<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Databricks builds a data mesh with the launch of Lakehouse Federation - Science and Nerds\" \/>\n<meta property=\"og:description\" content=\"Source:https:\/\/techcrunch.com\/2023\/06\/28\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/ Databricks builds a data mesh with the launch of Lakehouse Federation 2023-06-29 22:50:59 Databricks today launched what it calls its Lakehouse Federation feature at its Data + AI Summit. Using this new capability, enterprises can bring together their various siloed data systems and discover, query and govern their data across a wide variety of […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/\" \/>\n<meta property=\"og:site_name\" content=\"Science and Nerds\" \/>\n<meta property=\"article:published_time\" content=\"2023-06-29T22:50:59+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-29T22:51:00+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/scienceandnerds.com\/wp-content\/uploads\/2023\/06\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation_649e0ad3b629a.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"600\" \/>\n\t<meta property=\"og:image:height\" content=\"400\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/\",\"url\":\"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/\",\"name\":\"Databricks builds a data mesh with the launch of Lakehouse Federation - Science and Nerds\",\"isPartOf\":{\"@id\":\"https:\/\/scienceandnerds.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/06\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation_649e0ad3b629a.jpeg?fit=600%2C400&ssl=1\",\"datePublished\":\"2023-06-29T22:50:59+00:00\",\"dateModified\":\"2023-06-29T22:51:00+00:00\",\"author\":{\"@id\":\"https:\/\/scienceandnerds.com\/#\/schema\/person\/ea2991abeb2b9ab04b32790dff28360e\"},\"breadcrumb\":{\"@id\":\"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/#primaryimage\",\"url\":\"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/06\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation_649e0ad3b629a.jpeg?fit=600%2C400&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/06\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation_649e0ad3b629a.jpeg?fit=600%2C400&ssl=1\",\"width\":600,\"height\":400},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/scienceandnerds.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Databricks builds a data mesh with the launch of Lakehouse Federation\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/scienceandnerds.com\/#website\",\"url\":\"https:\/\/scienceandnerds.com\/\",\"name\":\"Science and Nerds\",\"description\":\"My WordPress Blog\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/scienceandnerds.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/scienceandnerds.com\/#\/schema\/person\/ea2991abeb2b9ab04b32790dff28360e\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/scienceandnerds.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/7e6e14fc6691445ef2b2c0a3a6c43882?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/7e6e14fc6691445ef2b2c0a3a6c43882?s=96&d=mm&r=g\",\"caption\":\"admin\"},\"sameAs\":[\"https:\/\/scienceandnerds.com\"],\"url\":\"https:\/\/scienceandnerds.com\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Databricks builds a data mesh with the launch of Lakehouse Federation - Science and Nerds","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/","og_locale":"en_US","og_type":"article","og_title":"Databricks builds a data mesh with the launch of Lakehouse Federation - Science and Nerds","og_description":"Source:https:\/\/techcrunch.com\/2023\/06\/28\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/ Databricks builds a data mesh with the launch of Lakehouse Federation 2023-06-29 22:50:59 Databricks today launched what it calls its Lakehouse Federation feature at its Data + AI Summit. Using this new capability, enterprises can bring together their various siloed data systems and discover, query and govern their data across a wide variety of […]","og_url":"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/","og_site_name":"Science and Nerds","article_published_time":"2023-06-29T22:50:59+00:00","article_modified_time":"2023-06-29T22:51:00+00:00","og_image":[{"width":600,"height":400,"url":"https:\/\/scienceandnerds.com\/wp-content\/uploads\/2023\/06\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation_649e0ad3b629a.jpeg","type":"image\/jpeg"}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/","url":"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/","name":"Databricks builds a data mesh with the launch of Lakehouse Federation - Science and Nerds","isPartOf":{"@id":"https:\/\/scienceandnerds.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/#primaryimage"},"image":{"@id":"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/06\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation_649e0ad3b629a.jpeg?fit=600%2C400&ssl=1","datePublished":"2023-06-29T22:50:59+00:00","dateModified":"2023-06-29T22:51:00+00:00","author":{"@id":"https:\/\/scienceandnerds.com\/#\/schema\/person\/ea2991abeb2b9ab04b32790dff28360e"},"breadcrumb":{"@id":"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/#primaryimage","url":"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/06\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation_649e0ad3b629a.jpeg?fit=600%2C400&ssl=1","contentUrl":"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/06\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation_649e0ad3b629a.jpeg?fit=600%2C400&ssl=1","width":600,"height":400},{"@type":"BreadcrumbList","@id":"https:\/\/scienceandnerds.com\/2023\/06\/29\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/scienceandnerds.com\/"},{"@type":"ListItem","position":2,"name":"Databricks builds a data mesh with the launch of Lakehouse Federation"}]},{"@type":"WebSite","@id":"https:\/\/scienceandnerds.com\/#website","url":"https:\/\/scienceandnerds.com\/","name":"Science and Nerds","description":"My WordPress Blog","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/scienceandnerds.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/scienceandnerds.com\/#\/schema\/person\/ea2991abeb2b9ab04b32790dff28360e","name":"admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/scienceandnerds.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/7e6e14fc6691445ef2b2c0a3a6c43882?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/7e6e14fc6691445ef2b2c0a3a6c43882?s=96&d=mm&r=g","caption":"admin"},"sameAs":["https:\/\/scienceandnerds.com"],"url":"https:\/\/scienceandnerds.com\/author\/admin\/"}]}},"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/06\/databricks-builds-a-data-mesh-with-the-launch-of-lakehouse-federation_649e0ad3b629a.jpeg?fit=600%2C400&ssl=1","_links":{"self":[{"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/posts\/36256","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/comments?post=36256"}],"version-history":[{"count":1,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/posts\/36256\/revisions"}],"predecessor-version":[{"id":36258,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/posts\/36256\/revisions\/36258"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/media\/36257"}],"wp:attachment":[{"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/media?parent=36256"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/categories?post=36256"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/tags?post=36256"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}