Datafold Blog - The Latest from the Datafold Team

Data Quality Best Practices, Data Observability, dbt

July 25, 2022

The day you stopped breaking your data

A guide on how to build data systems that do not break.

Data Quality Best Practices, Data Observability, dbt

Gleb Mezhanskiy

July 25, 2022

The day you stopped breaking your data

A guide on how to build data systems that do not break.

Data Quality Best Practices

November 11, 2022

Understanding your Data: The Process that Makes it Work

To deliver high quality data quickly teams need principles to guide how they move, update, and manage data while keeping their stakeholders informed.

Data Quality Best Practices

Emilie Schario

November 11, 2022

Understanding your Data: The Process that Makes it Work

To deliver high quality data quickly teams need principles to guide how they move, update, and manage data while keeping their stakeholders informed.

SQL

November 10, 2022

SQL Tools on the Command Line

A review SQL tools for performing data analysis from the command line covering ease of installation, supported data and file formats, SQL functionality, and benchmarking on a common query.

SQL

Matthew David

November 10, 2022

SQL Tools on the Command Line

A review SQL tools for performing data analysis from the command line covering ease of installation, supported data and file formats, SQL functionality, and benchmarking on a common query.

Company News, Modern Data Stack

October 11, 2022

Datafold Partners with Snowflake and Achieves Select Tier Partner Status

Datafold has officially launched its partnership with Snowflake, the Data Cloud company by providing proactive testing solutions to help joint customers trust and depend on their data.

Company News, Modern Data Stack

Richie Corcoran

October 11, 2022

Datafold Partners with Snowflake and Achieves Select Tier Partner Status

Datafold has officially launched its partnership with Snowflake, the Data Cloud company by providing proactive testing solutions to help joint customers trust and depend on their data.

Data Quality Best Practices, Modern Data Stack

October 7, 2022

Understanding your Data: The People Behind it

Many teams create data, many teams consume data, and the data team is responsible for facilitating.

Data Quality Best Practices, Modern Data Stack

Emilie Schario

October 7, 2022

Understanding your Data: The People Behind it

Many teams create data, many teams consume data, and the data team is responsible for facilitating.

dbt, Data Quality Best Practices, Data Testing

October 3, 2022

How to Setup dbt Development Environments

Learn the difference between development vs production environments and what you should consider for your own development environment in dbt.

dbt, Data Quality Best Practices, Data Testing

Kyle McNair

October 3, 2022

How to Setup dbt Development Environments

Learn the difference between development vs production environments and what you should consider for your own development environment in dbt.

Data Quality Best Practices

October 1, 2022

Improving the Modern Data Stack - Recap of Monday Morning Data Chat

In this episode of Monday Morning Data Chat, Gleb, Matt and Joe discuss improving the Modern Data Stack.

Data Quality Best Practices

Matthew David

October 1, 2022

Improving the Modern Data Stack - Recap of Monday Morning Data Chat

In this episode of Monday Morning Data Chat, Gleb, Matt and Joe discuss improving the Modern Data Stack.

dbt, Data Monitoring

September 27, 2022

How to Create dbt Freshness Alerts

Learn how to turn dbt freshness tests into production monitoring alerts using Datafold.

dbt, Data Monitoring

Kyle McNair

September 27, 2022

How to Create dbt Freshness Alerts

Learn how to turn dbt freshness tests into production monitoring alerts using Datafold.

dbt

September 21, 2022

Running dbt with Airflow

How to setup and run dbt with airflow on your local machine.

dbt

Forrest Bajbek

September 21, 2022

Running dbt with Airflow

How to setup and run dbt with airflow on your local machine.

dbt, Data Quality Best Practices

September 8, 2022

Getting Started with dbt Core on an M1 Mac

Overview on getting started with dbt Core via CLI on an M1 Mac.

dbt, Data Quality Best Practices

Kyle McNair

September 8, 2022

Getting Started with dbt Core on an M1 Mac

Overview on getting started with dbt Core via CLI on an M1 Mac.

Data Quality Best Practices, Data Testing

September 7, 2022

The Best Data Contract is the Pull Request

Data contracts can help us prevent data quality issues by formalizing interactions and handovers between different systems (and teams) handling data.

Data Quality Best Practices, Data Testing

Gleb Mezhanskiy

September 7, 2022

The Best Data Contract is the Pull Request

Data contracts can help us prevent data quality issues by formalizing interactions and handovers between different systems (and teams) handling data.

Data Quality Best Practices

August 29, 2022

Github Best Practices for Analytics Engineering

Github offers many different features such as protected branches, pull requests, and code reviews. It is best practice to take advantage of all of these in your analytics workflow and use them to maintain the integrity of your data models.

Data Quality Best Practices

Madison Schott

August 29, 2022

Github Best Practices for Analytics Engineering

Github offers many different features such as protected branches, pull requests, and code reviews. It is best practice to take advantage of all of these in your analytics workflow and use them to maintain the integrity of your data models.

Data Monitoring, Data Observability, Data Testing

August 9, 2022

Why create an open source extension of Data Diff?

Gleb and Simon discuss the Data Diff backstory including key design decisions, open sourcing, and how Data Diff works in practice.

Data Monitoring, Data Observability, Data Testing

Matthew David

August 9, 2022

Why create an open source extension of Data Diff?

Gleb and Simon discuss the Data Diff backstory including key design decisions, open sourcing, and how Data Diff works in practice.

Modern Data Stack, Data Quality Best Practices

July 28, 2022

Hightouch Integration

Datafold now seamlessly integrates with Hightouch to show how changes to a dbt model will impact Hightouch models and syncs.

Modern Data Stack, Data Quality Best Practices

Matthew David

July 28, 2022

Hightouch Integration

Datafold now seamlessly integrates with Hightouch to show how changes to a dbt model will impact Hightouch models and syncs.

Data Quality Best Practices, Data Testing, Modern Data Stack

July 19, 2022

How to Onboard Analytics Engineers

A guide on how to onboard Analytics Engineers to your company and data stack.

Data Quality Best Practices, Data Testing, Modern Data Stack

Kenny Ning

July 19, 2022

How to Onboard Analytics Engineers

A guide on how to onboard Analytics Engineers to your company and data stack.

DATA TESTING, DATA QUALITY BEST PRACTICES, DBT

June 28, 2022

Different ways to diff data

Data diffing is the process of comparing two datasets. See various ways to compare data at different levels of complexity.

DATA TESTING, DATA QUALITY BEST PRACTICES, DBT

Kenny Ning

June 28, 2022

Different ways to diff data

Data diffing is the process of comparing two datasets. See various ways to compare data at different levels of complexity.

COMPANY NEWS, DATA MIGRATION, DATA TESTING

June 22, 2022

Open source data-diff

Open source data-diff automates data quality checks for data replication and migration.

COMPANY NEWS, DATA MIGRATION, DATA TESTING

Gleb Mezhanskiy

June 22, 2022

Open source data-diff

Open source data-diff automates data quality checks for data replication and migration.

DBT, DATA TESTING

June 15, 2022

7 dbt Testing Best Practices

Learn best practices for how to write and manage dbt tests in your organization.

DBT, DATA TESTING

Matt David

June 15, 2022

7 dbt Testing Best Practices

Learn best practices for how to write and manage dbt tests in your organization.

DBT, MODERN DATA STACK, COMPANY NEWS

June 2, 2022

Datafold for Everyone

Datafold has launched new pricing to make data quality more accessible for analytics engineers and data engineers.

DBT, MODERN DATA STACK, COMPANY NEWS

Gleb Mezhanskiy

June 2, 2022

Datafold for Everyone

Datafold has launched new pricing to make data quality more accessible for analytics engineers and data engineers.

Data Observability, Data Quality Best Practices, Data Testing

May 16, 2022

Why Data Lineage Matters and Why it’s so Challenging

Data lineage tools provide visibility into how data is connected upstream and downstream within a database.

Data Observability, Data Quality Best Practices, Data Testing

Alex Morozov

May 16, 2022

Why Data Lineage Matters and Why it’s so Challenging

Data lineage tools provide visibility into how data is connected upstream and downstream within a database.

Company News

May 2, 2022

Announcing Datafold's SOC2 Type II Report

It's official Datafold is now SOC2 Type II compliant. We follow a security by design approach to our software development process and are focused on keeping our customers' data safe.

Company News

Gerard Toonstra

May 2, 2022

Announcing Datafold's SOC2 Type II Report

It's official Datafold is now SOC2 Type II compliant. We follow a security by design approach to our software development process and are focused on keeping our customers' data safe.

DBT, MODERN DATA STACK, COMPANY NEWS

March 30, 2022

Datafold + dbt: Ship Better Data

Datafold has partnered with dbt Labs and has launched an integration with dbt to deliver column-level lineage, data diff, and shareable impact reports for analytics engineers.

DBT, MODERN DATA STACK, COMPANY NEWS

Gleb Mezhanskiy

March 30, 2022

Datafold + dbt: Ship Better Data

Datafold has partnered with dbt Labs and has launched an integration with dbt to deliver column-level lineage, data diff, and shareable impact reports for analytics engineers.

Company News

December 15, 2021

The best of Datafold in 2021

2021 was a big year for Datafold. We reflect on top feature updates, blogs, and major company announcements from the past year.

Company News

Datafold Team

December 15, 2021

The best of Datafold in 2021

2021 was a big year for Datafold. We reflect on top feature updates, blogs, and major company announcements from the past year.

Events, Data Quality Best Practices

November 17, 2021

Data Quality Meetup #6

Get an overview of the Data Quality Meetup #6. With speakers from Yelp, Patreon, Convoy, and Lightdash, the event included lightning rounds on data quality best practices and approaches from leading data-driven companies.

Events, Data Quality Best Practices

Datafold Team

November 17, 2021

Data Quality Meetup #6

Get an overview of the Data Quality Meetup #6. With speakers from Yelp, Patreon, Convoy, and Lightdash, the event included lightning rounds on data quality best practices and approaches from leading data-driven companies.

Company News

November 9, 2021

Datafold: From Breaking Data to Series A

Datafold Founder and CEO, Gleb Mezhanskiy, shares what prompted Datafold's creation, how it has grown, and plans for the future.

Company News

Gleb Mezhanskiy

November 9, 2021

Datafold: From Breaking Data to Series A

Datafold Founder and CEO, Gleb Mezhanskiy, shares what prompted Datafold's creation, how it has grown, and plans for the future.

Data Testing, Data Migration, Data Quality Best Practices

October 26, 2021

What to Look for with Data Diffs

What should you be looking for when doing data QA with Data Diff? There are three core checks that can help prevent surprises in production dashboards, and this blog walks you through what you're looking for in each step.

Data Testing, Data Migration, Data Quality Best Practices

Sarah Krasnik

October 26, 2021

What to Look for with Data Diffs

What should you be looking for when doing data QA with Data Diff? There are three core checks that can help prevent surprises in production dashboards, and this blog walks you through what you're looking for in each step.

Data Observability, Data Migration, Data Testing

October 13, 2021

How to Trace PII with Column-level Lineage

There are plenty of rules around PII, but you can stay on top of where your sensitive data is flowing in your pipelines with column-level lineage.

Data Observability, Data Migration, Data Testing

Datafold Team

October 13, 2021

How to Trace PII with Column-level Lineage

There are plenty of rules around PII, but you can stay on top of where your sensitive data is flowing in your pipelines with column-level lineage.

Data Quality Best Practices

October 5, 2021

Enterprises Whose Bad Data Cost Them Millions: Lessons from Samsung and Uber

Bad data cost Samsung and Uber ridiculous sums of money with issues that could have been averted if they had been invested in data quality management. Read about their mistakes, and see how you could avoid doing the same.

Data Quality Best Practices

Datafold Team

October 5, 2021

Enterprises Whose Bad Data Cost Them Millions: Lessons from Samsung and Uber

Bad data cost Samsung and Uber ridiculous sums of money with issues that could have been averted if they had been invested in data quality management. Read about their mistakes, and see how you could avoid doing the same.

Data Observability, Modern Data Stack

September 28, 2021

Lineage Metadata Where You Need It - Datafold’s GraphQL API

If you want column-level lineage but you prefer tools like Amundsen or Data Hub, Datafold's GraphQL API lets you bring your metadata with you.

Data Observability, Modern Data Stack

Datafold Team

September 28, 2021

Lineage Metadata Where You Need It - Datafold’s GraphQL API

If you want column-level lineage but you prefer tools like Amundsen or Data Hub, Datafold's GraphQL API lets you bring your metadata with you.

Data Quality Best Practices

September 22, 2021

Why Conduct Data Quality Post-mortems?: Lessons learned from GitLab, Google, and Facebook

Without proactive data quality management, mistakes will happen. What you do can help improve your data quality in the future. Data quality post-mortems are a valuable tool for building improved processes and systems, plus rebuilding stakeholder trust.

Data Quality Best Practices

Datafold Team

September 22, 2021

Why Conduct Data Quality Post-mortems?: Lessons learned from GitLab, Google, and Facebook

Without proactive data quality management, mistakes will happen. What you do can help improve your data quality in the future. Data quality post-mortems are a valuable tool for building improved processes and systems, plus rebuilding stakeholder trust.

Data Quality Best Practices, Events

September 15, 2021

Data Quality Management According to Lyft, Shopify, and Thumbtack

It can be hard to even answer the question "is our data in good shape?" but these teams have gone on a journey towards improved data quality management. Here's how.

Data Quality Best Practices, Events

Datafold Team

September 15, 2021

Data Quality Management According to Lyft, Shopify, and Thumbtack

It can be hard to even answer the question "is our data in good shape?" but these teams have gone on a journey towards improved data quality management. Here's how.

Events, Data Quality Best Practices

September 7, 2021

Data Quality Meetup #5

Doordash, Truebill, Appfolio, Evidently.ai, and Narrator share valuable insights at the fifth Data Quality Meetup hosted by Datafold.

Events, Data Quality Best Practices

Datafold Team

September 7, 2021

Data Quality Meetup #5

Doordash, Truebill, Appfolio, Evidently.ai, and Narrator share valuable insights at the fifth Data Quality Meetup hosted by Datafold.

Company News

September 2, 2021

Datafold is SOC 2 compliant - What it means for you

SOC 2 compliance is a major step on our security journey. Here are some lessons we learned, as well as what Datafold's compliance means for your business.

Company News

Gerard Toonstra

September 2, 2021

Datafold is SOC 2 compliant - What it means for you

SOC 2 compliance is a major step on our security journey. Here are some lessons we learned, as well as what Datafold's compliance means for your business.

Data Quality Best Practices, Modern Data Stack

August 25, 2021

Proactive Data Quality on the Data Engineering Podcast

In July 2021, Datafold co-founder and CEO Gleb Mezhanskiy went on the Data Engineering Podcast to share his thoughts about a proactive approach to data quality management.

Data Quality Best Practices, Modern Data Stack

Datafold Team

August 25, 2021

Proactive Data Quality on the Data Engineering Podcast

In July 2021, Datafold co-founder and CEO Gleb Mezhanskiy went on the Data Engineering Podcast to share his thoughts about a proactive approach to data quality management.

Modern Data Stack

August 17, 2021

The Modern Data Stack: Open-source Edition

If you're looking to build the ideal modern data stack for analytics using only open-source options, this is the blog for you. Find all the best open-source alternatives to your favorite paid tools.

Modern Data Stack

Gleb Mezhanskiy

August 17, 2021

The Modern Data Stack: Open-source Edition

If you're looking to build the ideal modern data stack for analytics using only open-source options, this is the blog for you. Find all the best open-source alternatives to your favorite paid tools.

Data Quality Best Practices, Modern Data Stack

August 3, 2021

9 Best Tools for Data Quality in 2021

Data quality is increasingly a top KPI for data teams, even as multiple sources of data are making it harder to maintain data quality and reliability. These tools can facilitate quality data at every step.

Data Quality Best Practices, Modern Data Stack

Datafold Team

August 3, 2021

9 Best Tools for Data Quality in 2021

Data quality is increasingly a top KPI for data teams, even as multiple sources of data are making it harder to maintain data quality and reliability. These tools can facilitate quality data at every step.

Data Quality Best Practices, Modern Data Stack

July 28, 2021

Why Looker was and Lightdash is a big deal for BI

Lightdash is an open-source alternative to Looker that natively integrates with dbt. It may not be as mature as other open-source products like Metabase, Querybook, or Superset, but it is different in a few essential ways.

Data Quality Best Practices, Modern Data Stack

Gleb Mezhanskiy

July 28, 2021

Why Looker was and Lightdash is a big deal for BI

Lightdash is an open-source alternative to Looker that natively integrates with dbt. It may not be as mature as other open-source products like Metabase, Querybook, or Superset, but it is different in a few essential ways.

Data Monitoring, Data Observability, Data Quality Best Practices, Data Testing

July 20, 2021

4 Data Quality Best Practices to Help You Get to Data ROI Faster

Learn what steps your team needs to take to improve data quality and get the most out of your data.

Data Monitoring, Data Observability, Data Quality Best Practices, Data Testing

Zoe Hawkins

July 20, 2021

4 Data Quality Best Practices to Help You Get to Data ROI Faster

Learn what steps your team needs to take to improve data quality and get the most out of your data.

Data Quality Best Practices, Modern Data Stack

July 13, 2021

The State of Data Quality in 2021

Data quality is always evolving, so where is it in 2021? We asked and you answered - here are the results.

Data Quality Best Practices, Modern Data Stack

Datafold Team

July 13, 2021

The State of Data Quality in 2021

Data quality is always evolving, so where is it in 2021? We asked and you answered - here are the results.

Data Testing, Data Quality Best Practices

June 28, 2021

Data Quality In Data Engineering Workflows

Learn what steps your team needs to take to improve data quality and get the most out of your data.

Data Testing, Data Quality Best Practices

Sarah Krasnik

June 28, 2021

Data Quality In Data Engineering Workflows

Learn what steps your team needs to take to improve data quality and get the most out of your data.

Data Quality Best Practices, Events

June 22, 2021

Data Quality Meetup #4

Lyft vs. Shopify in testing ETL at scale, using fake data to align your stakeholders, and how to avoid nuclear meltdowns in your data platform.

Data Quality Best Practices, Events

Datafold Team

June 22, 2021

Data Quality Meetup #4

Lyft vs. Shopify in testing ETL at scale, using fake data to align your stakeholders, and how to avoid nuclear meltdowns in your data platform.

Data Quality Best Practices

April 14, 2021

Good Data: How Spotify, Shopify & Lyft approach data quality

Data Quality Best Practices

Datafold Team

April 14, 2021

Good Data: How Spotify, Shopify & Lyft approach data quality

Events, Data Quality Best Practices

April 7, 2021

Data Quality Meetup #3

Why implement regression testing for ETL code changes, how to align data producers and consumers, and what Data teams at Carta, Thumbtack, Shopify & Clari do to solve data quality.

Events, Data Quality Best Practices

Datafold Team

April 7, 2021

Data Quality Meetup #3

Why implement regression testing for ETL code changes, how to align data producers and consumers, and what Data teams at Carta, Thumbtack, Shopify & Clari do to solve data quality.

dbt, Data Testing, Data Observability, Data Migration, Modern Data Stack

February 5, 2021

Datafold + dbt: The Perfect Stack for Reliable Data Pipelines

Take your ETL workflow to the next level with Datafold and dbt integration that automates data testing and provides column-level data lineage

dbt, Data Testing, Data Observability, Data Migration, Modern Data Stack

Gleb Mezhanskiy

February 5, 2021

Datafold + dbt: The Perfect Stack for Reliable Data Pipelines

Take your ETL workflow to the next level with Datafold and dbt integration that automates data testing and provides column-level data lineage

Data Quality Best Practices

November 30, 2020

The Data Quality Flywheel

The more people that are looking at the data, and the more apps that are using the data, the faster data quality issues will be identified and resolved.

Data Quality Best Practices

Michael Kaminsky

November 30, 2020

The Data Quality Flywheel

The more people that are looking at the data, and the more apps that are using the data, the faster data quality issues will be identified and resolved.

Events, Data Quality Best Practices

November 26, 2020

Digest of Data Quality Meetup #2

On the second Data Quality Meetup, we discussed three types of data testing and when to apply them, new-generation ETL frameworks and ROI of open-source data catalogs.

Events, Data Quality Best Practices

Gleb Mezhanskiy

November 26, 2020

Digest of Data Quality Meetup #2

On the second Data Quality Meetup, we discussed three types of data testing and when to apply them, new-generation ETL frameworks and ROI of open-source data catalogs.

Company News

November 19, 2020

Datafold raises a $2.1M Seed round led by NEA

Over the past 10 years, we've seen a great advancement in technologies and tools for analytics and machine learning: with today’s modern analytics stack, we have fast and scalable data warehouses, dirt-cheap data storage, capable ETL orchestrators, and powerful BI tools.

Company News

Gleb Mezhanskiy

November 19, 2020

Datafold raises a $2.1M Seed round led by NEA

Over the past 10 years, we've seen a great advancement in technologies and tools for analytics and machine learning: with today’s modern analytics stack, we have fast and scalable data warehouses, dirt-cheap data storage, capable ETL orchestrators, and powerful BI tools.