Master end-to-end data engineering on Azure Databricks. From data ingestion and Delta Lake to CI/CD and real-time streaming, build secure, scalable, and performant data solutions with Spark, Unity Catalog, and ML tools.

Key Features

Build scalable data pipelines using Apache Spark and Delta Lake
Automate workflows and manage data governance with Unity Catalog
Learn real-time processing and structured streaming with practical use cases
Implement CI/CD, DevOps, and security for production-ready data solutions
Explore Databricks-native ML, AutoML, and Generative AI integration

Book Description

"Data Engineering with Azure Databricks" is your essential guide to building scalable, secure, and high-performing data pipelines using the powerful Databricks platform on Azure. Designed for data engineers, architects, and developers, this book demystifies the complexities of Spark-based workloads, Delta Lake, Unity Catalog, and real-time data processing.

Beginning with the foundational role of Azure Databricks in modern data engineering, you’ll explore how to set up robust environments, manage data ingestion with Auto Loader, optimize Spark performance, and orchestrate complex workflows using tools like Azure Data Factory and Airflow.

The book offers deep dives into structured streaming, Delta Live Tables, and Delta Lake’s ACID features for data reliability and schema evolution. You’ll also learn how to manage security, compliance, and access controls using Unity Catalog, and gain insights into managing CI/CD pipelines with Azure DevOps and Terraform.

With a special focus on machine learning and generative AI, the final chapters guide you in automating model workflows, leveraging MLflow, and fine-tuning large language models on Databricks. Whether you're building a modern data lakehouse or operationalizing analytics at scale, this book provides the tools and insights you need.

What you will learn

Set up a full-featured Azure Databricks environment
Implement batch and streaming ingestion using Auto Loader
Optimize Spark jobs with partitioning and caching
Build real-time pipelines with structured streaming and DLT
Manage data governance using Unity Catalog
Orchestrate production workflows with jobs and ADF
Apply CI/CD best practices with Azure DevOps and Git
Secure data with RBAC, encryption, and compliance standards
Use MLflow and Feature Store for ML pipelines
Build generative AI applications in Databricks

Who this book is for

This book is for data engineers, solution architects, cloud professionals, and software engineers seeking to build robust and scalable data pipelines using Azure Databricks. Whether you're migrating legacy systems, implementing a modern lakehouse architecture, or optimizing data workflows for performance, this guide will help you leverage the full power of Databricks on Azure. A basic understanding of Python, Spark, and cloud infrastructure is recommended.

The Role of Azure Databricks in Modern Data Engineering
Setting up an End-To-End Azure Databricks Environment
Data Ingestion Strategies for Azure Databricks
Data Engineering with Apache Spark
Building Real-Time Data Pipelines
Working with Delta Lake: ACID Transactions and Schema Evolution
Automating Data Systems with Lakeflow Spark Declarative Pipelines
Orchestrating Data Workflows: From Notebooks to Production
CI/CD and DevOps for Azure Databricks
Optimizing Query Performance and Cost Management
Security, Compliance, and Data Governance
Machine Learning and AI on Databricks

"Synopsis" may belong to another edition of this title.

About the Author

Dmitry Foshin is a Business Intelligence team leader focused on delivering business insights to the management team through data engineering, analytics, and visualization. He has led and executed complex full-stack BI solutions (from ETL processes to building DWHs and reporting) using Azure technologies, Data Lake, Data Factory, Data Bricks, MS Office 365, Power BI, and Tableau. He has also successfully launched numerous data analytics projects – both on-premises and in the cloud – that help achieve corporate goals for international FMCG companies, banks, and manufacturing companies.

Dmitry Anoshin is a data-centric technologist and a recognized expert in building and implementing big data and analytics solutions. He has a successful track record of implementing business and digital intelligence projects across retail, finance, marketing, and e-commerce. Dmitry possesses in-depth knowledge of digital/business intelligence, ETL, data warehousing, and big data technologies. He has extensive experience in data integration and is proficient in various data warehousing methodologies. Dmitry has consistently exceeded project expectations across the financial, machine tool, and retail industries. He has completed a number of multinational full BI/DI solution life cycle implementation projects. With expertise in data modeling, Dmitry also has a background and business experience in multiple relational databases, OLAP systems, and NoSQL databases. He is also an active speaker at data conferences and helps people to adopt cloud analytics.

Tonya Chernyshova is an experienced Data Engineer with over 10 years in the field, including time at Amazon. Specializing in Data Modeling, Automation, Cloud Computing (AWS and Azure), and Data Visualization, she has a strong track record of delivering scalable, maintainable data products. Her expertise drives data-driven insights and business growth, showcasing her proficiency in leveraging cloud technologies to enhance data capabilities.

Sergii Volodarskyi is a Data Engineer working daily on the Databricks ecosystem, across both platform and product data engineering. His expertise spans the full spectrum of modern data platform delivery, from designing lakehouse architectures and building CI/CD pipelines to extracting data from APIs and shipping analytical products that drive business decisions. This book is a reflection of experience and best practices built up across real projects. He actively shares his knowledge with the engineering community and is a builder with a deep interest in the intersection of data, AI, and software engineering.

"About the title" may belong to another edition of this title.

PBShop.store US

Wood Dale, IL, U.S.A.

5-star seller

AbeBooks seller since April 7, 2005

Shipping rates within U.S.A.

Item	7 to 14 business days	7 to 14 business days
First item	£ 0.00	£ 0.00

Delivery times are set by sellers and vary by carrier and location. Orders passing through Customs may face delays and buyers are responsible for any associated duties or fees. Sellers may contact you regarding additional charges to cover any increased costs to ship your items.

Payment methods

Store description

We first started out as “The Paperback Exchange,” a chain of physical bookstores where we would part exchange your beloved books for new stories to transport you to faraway places. However, as shopping started to evolve to online shops and marketplaces, we bid our stores goodbye to become “PBShop.” This transition has only allowed us to blossom as we now ship thousands of titles to book lovers across the globe. We pride ourselves in being a community of local book lovers which allows our passion and devotion to shine in everything we do. In 2020 we not only celebrated our 20th birthday but our 1st birthday as being completely employee owned after becoming an E.O.T in September 2019. We are proud to be different and embrace standing apart on a book mountain by working from a virtual inventory which allows us to provide thousands of books that may be difficult to get for your bookshelf or your studies. Working with a number of different suppliers allows us to explore other avenues such as puzzles, sheet music and even stationery so we really do have something for everyone. Life is about being versatile in all realms of existence. If this is your first visit or you are a returning customer, we would like to welcome you to the PBShop family, for there is no friend as loyal as a book. We are a company who put our customers at the centre of everything we do as we understand the importance of reading because once you learn to read, you will forever be free. There are a whole lot of things in this world of ours that we are yet to explore, which is why we will forever inspire curious minds.…

Specialty

Hardbacks, Paperbacks

Seller's business information

Pbshop.co.uk Ltd

Unit 22 Horcott Industrial Estate, Horcott Road
Fairford, United Kingdom GL7 4BX

Contact sellerView this seller's itemsView seller's storefront

Terms of sale

Returns Policy
We ask all customers to contact us for authorisation should they wish to return their order. Orders returned without authorisation may not be credited.
If you wish to return, please contact us within 14 days of receiving your order to obtain authorisation.

Returns requested beyond this time will not be authorised.

Our team will provide full instructions on how to return your order and once received our returns department will process your refund.
Please note the cost to return any unwanted order to us is borne by the buyer.

Should your order arrive damaged, not as advertised or faulty, we must be advised of this within 14 days of delivery. Please contact us so we can find the best solution for you.

Our Customer Care Team can be contacted via emailing paperback-us@paperbackshop.co.uk, or by calling our UK Office on +441285 712 917. We are available 9:00am till 5:30pm GMT weekdays and 9:00am till 1:00pm GMT on Saturdays.

Right of withdrawal

If you are a consumer you can withdraw from the contract in accordance with the following. Consumer means any natural person who is acting for purposes which are outside his trade, business, craft or profession.

Information regarding the right of withdrawal

Statutory right to withdraw

You have the right to withdraw from this contract within 14 days without giving any reason.

The withdrawal period will expire after 14 days from the day on which you acquire, or a third party other than the carrier and indicated by you acquires, physical possession of the last good or the last lot or piece.

To exercise the right of withdrawal, electronically fill in and submit a clear statement on our website, under "My Purchases" in "My Account". We will communicate to you an acknowledgement of receipt of such a withdrawal on a durable medium (e.g. by e-mail) without delay.

To meet the withdrawal deadline, it is sufficient for you to send your communication concerning your exercise of the right of withdrawal before the withdrawal period has expired.

Effects of withdrawal

If you withdraw from this contract, we will reimburse to you all payments received from you, including the costs of delivery (except for the supplementary costs arising if you chose a type of delivery other than the least expensive type of standard delivery offered by us).

We may make a deduction from the reimbursement for loss in value of any goods supplied, if the loss is the result of unnecessary handling by you.

We will make the reimbursement without undue delay, and not later than 14 days after the day on which we are informed about your decision to withdraw from this contract.

We will make the reimbursement using the same means of payment as you used for the initial transaction, unless you have expressly agreed otherwise; in any event, you will not incur any fees as a result of such reimbursement.

We may withhold reimbursement until we have received the goods back, or you have supplied evidence of having sent back the goods, whichever is the earliest.

You shall send back the goods or hand them over to PBShop.store US, Springfield Gardens, New York, U.S.A., +44 1285 712917, without undue delay and in any event not later than 14 days from the day on which you communicate your withdrawal from this contract to us. The deadline is met if you send back the goods before the period of 14 days has expired. You will have to bear the direct cost of returning the goods. You are only liable for any diminished value of the goods resulting from the handling other than what is necessary to establish the nature, characteristics and functioning of the goods.

Exceptions to the right of withdrawal

The right of withdrawal does not apply to:

The delivery of newspapers, journals or magazines with the exception of subscription contracts; and
The supply of digital content which is not supplied on a tangible medium (e.g. on a CD or DVD) if you accepted when you placed your order that we could start to deliver it, and that you could not withdraw once delivery had started.

Shipping terms

Books are shipped from our US or UK warehouses. Delivery estimates allow for delivery from either location.

Data Engineering with Azure Databricks

Item description from seller

Bibliographic details

Synopsis

About the title