Announcing the Looker Integration for Datafold Cloud

Datafold is excited to introduce Datafold Cloud’s new Looker integration, serving as the initial safeguard against bad data entering the tools that matter to you and your stakeholders.

With the Looker integration in Datafold Cloud, your team can gain unparalleled visibility into Looker assets—including Explores, Views, Dashboards, and Looks—that are potentially impacted or changed by your dbt code updates. Equipped with this valuable information—easily accessible through Datafold Cloud's comprehensive column-level lineage and the insightful Datafold PR impact analysis comment—your team can identify data quality problems before your stakeholder does.

Evolving lineage and closing the gap between dbt and your BI tools

When you make big (or small) changes in your dbt project, you do your due diligence:

  • Running dbt tests
  • Performing spot check queries
  • Confirm with another team member that the changes look good before merging them in

It’s usually only after the code changes have been merged in when you learn that a dashboard has now been broken and the “I think something’s wrong with the data” DMs start.

At no point in the process above is the impact to your BI tools—the bloodline of reporting and decision making for your business—considered in full. And your data team reverts some work, business users are frustrated, and your team is (again) spending valuable time putting out a fire.

This chain of events is so timeless for one primary reason: BI tools have not easily integrated with dbt or other data transformation tools, so identifying the lineage between the two has been difficult (or impossible). dbt exposures, too, currently exist as a way to connect your dbt project and data app assets, but require manual effort and tracking that does not scale with your data and organization. As a result, data teams often have little visibility into how their dbt models are being used downstream in the tools of the businesses, make retroactive fixes in BI tools after dbt code changes upset things, and ultimately create this cyclical workflow between dbt development and BI reporting.

At Datafold, we want to close that gap.

We believe that data teams should not only know how their code changes will impact the data (before it happens), but deserve to know exactly what is impacted—from downstream dbt models to the dashboards that power your business. No dbt code change should result in inaccurate data entering a dashboard anymore.

Which is why we’re excited to bring evolved data lineage and a bridge between your dbt project and BI tool: the Datafold Cloud Looker Integration.

The new Datafold Cloud Looker Integration specifically enables:

  • Looker Explores, Views, Dashboards, and Looks to be accessible in Datafold Cloud’s column-level lineage, so you can trace your data throughout your data’s entire ecosystem.
  • Potentially impacted Looker assets are automatically added to Datafold Cloud’s data diff PR comment, so you and your team know exactly which BI assets will change with your code updates.
Datafold Cloud's column-level lineage explore showing Looker assets downstream from Snowflake objects
Looker assets potentially changed by your dbt code update automatically detected in Datafold's PR comment

Now, when you make changes to your dbt model and test them using Datafold Cloud, you know exactly which BI assets are going to be changed, so you, your fellow data team members, and end data consumers catch potential data quality issues before they enter the most important decision making tool of your business.

And as we all know, happy business users = happy data team 😊

📽️ Demo

Watch Datafold Solutions Engineering Sung Won Chung demo the functionality of the Looker integration in Datafold Cloud.

For the future: Lineage and impact analysis at the deepest granularity

As we continue to develop Datafold Cloud into a tool that enables you to have the greatest visibility into your data’s ecosystem, movement, and impact, expect new innovation around:

  • More BI tool integrations like Tableau and Power BI, so different internal teams with different tooling all have the same understanding of data lineage and impact.
  • A robust notification system over Slack and email that allows stakeholders to automatically be notified about the changes in dbt that will impact their Looker assets.

Getting started

If you’re ready to stop receiving those “these numbers look off” DM from your stakeholders, check out the following resources:

Have questions about this integration? Please reach out to us directly or in the #tools-datafold channel in the dbt Community Slack.

Happy diffing!

Datafold is the fastest way to validate dbt model changes during development, deployment & migrations. Datafold allows data engineers to audit their work in minutes without writing tests or custom queries. Integrated into CI, Datafold enables data teams to deploy with full confidence, ship faster, and leave tedious QA and firefighting behind.

Datafold is the fastest way to test dbt code changes