Data Contracts: Developing Production Grade Pipelines at Scale - Softcover

Sanderson, Chad; Freeman, Mark; Schmidt, B E

Softcover

ISBN 10: 109815763X ISBN 13: 9781098157630

Publisher: O'Reilly Media, 2025

Synopsis

Poor data quality can cause major problems for data teams, from breaking revenue-generating data pipelines to losing the trust of data consumers. Despite the importance of data quality, many data teams still struggle to avoid these issues--especially when their data is sourced from upstream workflows outside of their control. The solution: data contracts. Data contracts enable high-quality, well-governed data assets by documenting expectations of the data, establishing ownership of data assets, and then automatically enforcing these constraints within the CI/CD workflow.

This practical book introduces data contract architecture with a clear definition of data contracts, explains why the data industry needs them, and shares real-world use cases of data contracts in production. In addition, you'll learn how to implement components of the data contract architecture and understand how they're used in the data lifecycle. Finally, you'll build a case for implementing data contracts in your organization.

Authors Chad Sanderson and Mark Freeman will help you:

• Explore real-world applications of data contracts within the industry
• Understand how to apply each component of this architecture, such as CI/CD, monitoring, version control data, and more
• Learn how to implement data contracts using open source tools
• Examine ways to resolve data quality issues using data contract architecture
• Measure the impact of implementing a data contract in your organization
• Develop a strategy to determine how data contracts will be used in your organization

About this title

About the Author

Chad Sanderson is one of the most well-known and prolific writers and speakers on Data Contracts. He is passionate about data quality and fixing the muddy relationship between data producers and consumers. He is a former head of data at Convoy, a LinkedIn writer, and a published author. Chad created the first implementation of data contracts at scale during his time at Convoy, and also created the first engineering guide to deploying contracts in streaming, batch, and even oriented environments. He lives in Seattle, Washington, and operates the Data Quality Camp Slack group and the Data Products newsletter, both of which focus on data contracts and their technical implementation. Mark Freeman is a community health advocate turned data engineer interested in the intersection of social impact, business, and technology. His life's mission is to improve the well-being of as many people as possible through data. Mark received his M.S. from the Stanford School of Medicine and is also certified in Entrepreneurship and Innovation from the Stanford Graduate School of Business. In addition, Mark has worked within numerous startups where he has put machine learning models into production, integrated data analytics into products, and led migrations to improve data infrastructure.

Edition details

• Publisher: O'Reilly Media
• Publication date: 2025
• Language: English
• ISBN 10: 109815763X
• ISBN 13: 9781098157630
• Binding: Paperback
• Number of pages: 350