Learning Spark 2e: Lightning-Fast Data Analytics - Softcover

Damji, Jules; Lee, Danny; Wenig, Brooke; Das, Tathagata

Softcover

ISBN 10: 1492050040 ISBN 13: 9781492050049

Publisher: O′Reilly, 2020

Synopsis

Data is bigger, arrives faster, and comes in a variety of formatsâ and it all needs to be processed at scale for analytics or machine learning. But how can you process such varied workloads efficiently? Enter Apache Spark.

Updated to include Spark 3.0, this second edition shows data engineers and data scientists why structure and unification in Spark matters. Specifically, this book explains how to perform simple and complex data analytics and employ machine learning algorithms. Through step-by-step walk-throughs, code snippets, and notebooks, youâ ll be able to:

• Learn Python, SQL, Scala, or Java high-level Structured APIs
• Understand Spark operations and SQL Engine
• Inspect, tune, and debug Spark operations with Spark configurations and Spark UI
• Connect to data sources: JSON, Parquet, CSV, Avro, ORC, Hive, S3, or Kafka
• Perform analytics on batch and streaming data using Structured Streaming
• Build reliable data pipelines with open source Delta Lake and Spark
• Develop machine learning pipelines with MLlib and productionize models using MLflow

About this title

About the Author

Jules S. Damji is an Apache Spark Community and Developer Advocate at Databricks. He is a hands-on developer with over 20 years of experience and has worked at leading companies, such as Sun Microsystems, Netscape, @Home, LoudCloud/Opsware, VeriSign, ProQuest, and Hortonworks, building large-scale distributed systems. He holds a B.Sc and M.Sc in Computer Science and MA in Political Advocacy and Communication from Oregon State University, Cal State, and Johns Hopkins University respectively. Denny Lee is a Technical Product Manager at Databricks. He is a hands-on distributed systems and data sciences engineer with extensive experience developing internet-scale infrastructure, data platforms, and predictive analytics systems for both on-premise and cloud environments. He also has a Masters of Biomedical Informatics from Oregon Health and Sciences University and has architected and implemented powerful data solutions for enterprise Healthcare customers. His current technical focuses include Distributed Systems, Apache Spark, Deep Learning, Machine Learning, and Genomics. Brooke Wenig is the Machine Learning Practice Lead at Databricks. She guides and assists customers in implementing machine learning pipelines, as well as teaching Distributed Machine Learning & Deep Learning courses. She received an MS in Computer Science from UCLA with a focus on distributed machine learning. She speaks Mandarin Chinese fluently and enjoys cycling. Tathagata Das is an Apache Spark committer and a member of the PMC. He's the lead developer behind Spark Streaming and currently develops Structured Streaming. Previously, he was a grad student in the UC Berkeley at AMPLab, where he conducted research about data-center frameworks and networks with Scott Shenker and Ion Stoica.

Edition details

• Publisher: O′Reilly
• Publication date: 2020
• Language: English
• ISBN 10: 1492050040
• ISBN 13: 9781492050049
• Binding: Paperback
• Edition number: 2
• Number of pages: 300