Cleaning Data for Effective Data Science

David Mertz

ISBN 10: 1801071292 / ISBN 13: 9781801071291

Published by Packt Publishing Limited, GB, 2021

About this Item

Think about your data intelligently and ask the right questionsKey FeaturesMaster data cleaning techniques necessary to perform real-world data science and machine learning tasksSpot common problems with dirty data and develop flexible solutions from first principlesTest and refine your newly acquired skills through detailed exercises at the end of each chapterBook DescriptionData cleaning is the all-important first step to successful data science, data analysis, and machine learning. If you work with any kind of data, this book is your go-to resource, arming you with the insights and heuristics experienced data scientists had to learn the hard way. In a light-hearted and engaging exploration of different tools, techniques, and datasets real and fictitious, Python veteran David Mertz teaches you the ins and outs of data preparation and the essential questions you should be asking of every piece of data you work with. Using a mixture of Python, R, and common command-line tools, Cleaning Data for Effective Data Science follows the data cleaning pipeline from start to end, focusing on helping you understand the principles underlying each step of the process. You'll look at data ingestion of a vast range of tabular, hierarchical, and other data formats, impute missing values, detect unreliable data and statistical anomalies, and generate synthetic features. The long-form exercises at the end of each chapter let you get hands-on with the skills you've acquired along the way, also providing a valuable resource for academic courses.What you will learnIngest and work with common data formats like JSON, CSV, SQL and NoSQL databases, PDF, and binary serialized data structuresUnderstand how and why we use tools such as pandas, SciPy, scikit-learn, Tidyverse, and BashApply useful rules and heuristics for assessing data quality and detecting bias, like Benford's law and the 68-95-99.7 ruleIdentify and handle unreliable data and outliers, examining z-score and other statistical propertiesImpute sensible values into missing data and use sampling to fix imbalancesUse dimensionality reduction, quantization, one-hot encoding, and other feature engineering techniques to draw out patterns in your dataWork carefully with time series data, performing de-trending and interpolationWho this book is forThis book is designed to benefit software developers, data scientists, aspiring data scientists, teachers, and students who work with data. If you want to improve your rigor in data hygiene or are looking for a refresher, this book is for you.Basic familiarity with statistics, general concepts in machine learning, knowledge of a programming language (Python or R), and some exposure to data science are helpful.

Seller Inventory # LU-9781801071291

Bibliographic Details

• Title: Cleaning Data for Effective Data Science
• Publisher: Packt Publishing Limited, GB
• Publication Date: 2021
• Language: English
• ISBN 10: 1801071292
• ISBN 13: 9781801071291
• Binding: Paperback
• Condition: New

About this title

Synopsis

A comprehensive guide for data scientists to master effective data cleaning tools and techniques

Key Features

• Think about your data intelligently and ask the right questions
• Master data cleaning techniques using hands-on examples belonging to diverse domains
• Work with detailed, commented, well-tested code samples in Python and R

Book Description

In data science, data analysis, or machine learning, most of the effort needed to achieve your actual purpose lies in cleaning your data. Using Python, R, and command-line tools, you will learn the essential cleaning steps performed in every production data science or data analysis pipeline. This book not only teaches you data preparation but also what questions you should ask of your data.

The book dives into the practical application of tools and techniques needed for data ingestion, anomaly detection, value imputation, and feature engineering. It also offers long-form exercises at the end of each chapter to practice the skills acquired.

You will begin by looking at data ingestion of a range of data formats. Moving on, you will impute missing values, detect unreliable data and statistical anomalies, and generate synthetic features that are necessary for successful data analysis and visualization goals.

By the end of this book, you will have acquired a firm understanding of the data cleaning process necessary to perform real-world data science and machine learning tasks.

What you will learn

• Ingest and work with common tabular, hierarchical, and other data formats
• Apply useful rules and heuristics for assessing data quality and detecting bias
• Identify and handle unreliable data and outliers in their many forms
• Impute sensible values into missing data and use sampling to fix imbalances
• Generate synthetic features that help to draw out patterns in your data
• Prepare data competently and correctly for analytic and machine learning tasks

Who this book is for

This book is designed to benefit software developers, data scientists, aspiring data scientists, and students who are interested in data analysis or scientific computing.

Basic familiarity with statistics, general concepts in machine learning, knowledge of a programming language (Python or R), and some exposure to data science are helpful.

The text will also be helpful to intermediate and advanced data scientists who want to improve their rigor in data hygiene and wish for a refresher on data preparation issues.

Table of Contents

1. Data Ingestion – Tabular Formats
2. Data Ingestion - Hierarchical Formats
3. Data Ingestion - Repurposing Data Sources
4. The Vicissitudes of Error - Anomaly Detection
5. The Vicissitudes of Error - Data Quality
6. Rectification and Creation - Value Imputation
7. Rectification and Creation - Feature Engineering
8. Ancillary Matters - Closure/Glossary

About the Author

David Mertz, Ph.D. is the founder of KDM Training, a partnership dedicated to educating developers and data scientists in machine learning and scientific computing. He created a data science training program for Anaconda Inc. and was a senior trainer for them. With the advent of deep neural networks, he has turned to training our robot overlords as well.

He previously worked for 8 years with D. E. Shaw Research and was also a Director of the Python Software Foundation for 6 years. David remains co-chair of its Trademarks Committee and Scientific Python Working Group. His columns, Charming Python and XML Matters, were once the most widely read articles in the Python world.