Practical Data Analysis Cookbook

Key Features

• Clean dirty data, extract accurate information, and explore the relationships between variables
• Forecast the output of an electric plant and the water flow of American rivers using pandas, NumPy, Statsmodels, and scikit-learn
• Find and extract the most important features from your dataset using the most efficient Python libraries

In Detail

Data analysis is the process of systematically applying statistical and logical techniques to describe and illustrate, condense and recap, and evaluate data. Its importance has been most visible in the sector of information and communication technologies. It is an employee asset in almost all economy sectors.

This book provides a rich set of independent recipes that dive into the world of data analytics and modeling using a variety of approaches, tools, and algorithms. You will learn the basics of data handling and modeling, and will build your skills gradually toward more advanced topics such as simulations, raw text processing, social interactions analysis, and more.

First, you will learn some easy-to-follow practical techniques on how to read, write, clean, reformat, explore, and understand your data-arguably the most time-consuming (and the most important) tasks for any data scientist.

In the second section, different independent recipes delve into intermediate topics such as classification, clustering, predicting, and more. With the help of these easy-to-follow recipes, you will also learn techniques that can easily be expanded to solve other real-life problems such as building recommendation engines or predictive models.

In the third section, you will explore more advanced topics: from the field of graph theory through natural language processing, discrete choice modeling to simulations. You will also get to expand your knowledge on identifying fraud origin with the help of a graph, scrape Internet websites, and classify movies based on their reviews.

By the end of this book, you will be able to efficiently use the vast array of tools that the Python environment has to offer.

What you will learn

• Read, clean, transform, and store your data usng Pandas and OpenRefine
• Understand your data and explore the relationships between variables using Pandas and D3.js
• Explore a variety of techniques to classify and cluster outbound marketing campaign calls data of a bank using Pandas, mlpy, NumPy, and Statsmodels
• Reduce the dimensionality of your dataset and extract the most important features with pandas, NumPy, and mlpy
• Predict the output of a power plant with regression models and forecast water flow of American rivers with time series methods using pandas, NumPy, Statsmodels, and scikit-learn
• Explore social interactions and identify fraudulent activities with graph theory concepts using NetworkX and Gephi
• Scrape Internet web pages using urlib and BeautifulSoup and get to know natural language processing techniques to classify movies ratings using NLTK
• Study simulation techniques in an example of a gas station with agent-based modeling

About the Author

Tomasz Drabas is a Data Scientist with over 10 years of experience in various analytical roles across the airline, consulting, telecommunications, and technology industries. He holds a PhD in Operations Research from the University of New South Wales. Currently, Tom works as Data Scientist for Microsoft Corporation in Redmond, WA.

Tomasz Drabas is a Data Scientist working for Microsoft and currently residing in the Seattle area. He has over 12 years' international experience in data analytics and data science in numerous fields: advanced technology, airlines, telecommunications, finance, and consulting. Tomasz started his career in 2003 with LOT Polish Airlines in Warsaw, Poland while finishing his Master's degree in strategy management. In 2007, he moved to Sydney to pursue a doctoral degree in operations research at the University of New South Wales, School of Aviation; his research crossed boundaries between discrete choice modeling and airline operations research. During his time in Sydney, he worked as a Data Analyst for Beyond Analysis Australia and as a Senior Data Analyst/Data Scientist for Vodafone Hutchison Australia among others. He has also published scientific papers, attended international conferences, and served as a reviewer for scientific journals. In 2015 he relocated to Seattle to begin his work for Microsoft. While there, he has worked on numerous projects involving solving problems in high-dimensional feature space.