All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Group Product Manager: Ali Abidi
Publishing Product Manager: Tejashwini R
Book Project Manager: Farheen Fathima
Content Development Editor: Priyanka Soam
Technical Editor: Kavyashree K S
Copy Editor: Safis Editing
Proofreader: Safis Editing
Indexer: Manju Arasan
Production Designer: Alishon Mendonca
DevRel Marketing Coordinator: Vinishka Kalra
First published: December 2016
Second edition: December 2018
Third edition: Jan 2024
Production reference: 1120124
Published by Packt Publishing Ltd.
Grosvenor House
11 St Paul’s Square
Birmingham
B3 1RB, UK
ISBN 978-1-83763-630-3
I have dedicated many books to many loved ones in the past, and for this edition, I want to dedicate this work to the people of Packt Publishing, who not only gave me my first chance at writing a book when I was early in my career but have stuck by me and continued to release editions with me since.
Thank you to everyone at Packt Publishing for all of your hard work, patience, and dedication to my work!
– Sinan Ozdemir
Contributor
About the author
Sinan Ozdemir is an active lecturer on large language models and a former lecturer of data science at Johns Hopkins University. He is the author of multiple textbooks on data science and machine learning, including Quick Start Guide to LLMs. Sinan is currently the founder of LoopGenius, which uses AI to help people and businesses boost their sales, and was previously the founder of the acquired Kylie.ai, an enterprise-grade conversational AI platform with RPA capabilities. He holds a master’s degree in pure mathematics from Johns Hopkins University and is based in San Francisco.
About the reviewer
Jigyasa Grover, a 10-time award winner in AI and open source and the coauthor of the book Sculpting Data for ML, is a powerhouse brimming with passion to make a dent in this world of technology and bridge the gaps. With years of machine learning engineering and data science experience in deploying large‐scale systems for monetization on social networking and e‐commerce platforms, she primarily focuses on ad prediction, sponsored content ranking, and recommendation. She is an avid proponent of open source and credits her access to opportunities and career growth to this sphere of community development. In her spirit to build a powerful community with
a strong belief in the axiom, “We rise by lifting others,” she actively mentors developers and machine learning enthusiasts.
Table of Contents
Preface
1
Data Science Terminology
What is data science?
Understanding basic data science terminology
Why data science?
Example – predicting COVID-19 with machine learning
The data science Venn diagram
The math
Computer programming
Example – parsing a single tweet
Domain knowledge
Some more terminology
Data science case studies
Case study – automating government paper pushing
Case study – what’s in a job description?
Summary
2 Types of Data
Structured versus unstructured data
Quantitative versus qualitative data
Digging deeper
The four levels of data
The nominal level
Measures of center
The ordinal level
The interval level
The ratio level
Data is in the eye of the beholder
Summary
Questions and answers 3 The Five Steps of Data Science
Introduction to data science
Overview of the five steps
Exploring the data
Guiding questions for data exploration
DataFrames
Series
Exploration tips for qualitative data
Summary
4
Basic Mathematics
Basic symbols and terminology
Vectors and matrices
Arithmetic symbols
Summation
Logarithms/exponents
Set theory
Linear algebra
Matrix multiplication
How to multiply matrices together
Summary
Impossible or Improbable – A Gentle
Introduction to Probability
Basic definitions
What do we mean by “probability”?
Bayesian versus frequentist
Frequentist approach
The law of large numbers
Compound events
Conditional probability
How to utilize the rules of probability
The addition rule
Mutual exclusivity
The multiplication rule
Independence
Complementary events
Introduction to binary classifiers
Summary
Advanced Probability
Bayesian ideas revisited
Bayes’ theorem
More applications of Bayes’ theorem
Random variables
Discrete random variables
Continuous random variables
Summary
7
What Are the Chances? An Introduction to
Statistics
What are statistics?
How do we obtain and sample data?
Obtaining data
Observational
Experimental
Sampling data
How do we measure statistics?
Measures of center
Measures of variation
The coefficient of variation
Measures of relative standing
The insightful part – correlations in data
The empirical rule
Example – exam scores
Summary
8
Advanced Statistics
Understanding point estimates
Sampling distributions
Confidence intervals
Hypothesis tests
Conducting a hypothesis test
One-sample t-tests
Type I and Type II errors
Hypothesis testing for categorical variables
Chi-square goodness of fit test
Chi-square test for association/independence
Summary
Communicating Data
Why does communication matter?
Identifying effective visualizations
Scatter plots
Line graphs
Bar charts
Histograms
Box plots
When graphs and statistics lie
Correlation versus causation
Simpson’s paradox
If correlation doesn’t imply causation, then what does?
Verbal communication
It’s about telling a story
On the more formal side of things
The why/how/what strategy for presenting
Summary
How to Tell if Your Toaster is Learning –
Machine Learning Essentials
Introducing ML
Example – facial recognition
ML isn’t perfect
How does ML work?
Types of ML SL UL RL
Overview of the types of ML
ML paradigms – pros and cons
Predicting continuous variables with linear regression
Correlation versus causation
Causation
Adding more predictors
Regression metrics
Summary
Predictions Don’t Grow on Trees, or Do They?
Performing naïve Bayes classification
Classification metrics
Understanding decision trees
Measuring purity
Exploring the Titanic dataset
Dummy variables
Diving deep into UL
When to use UL
k-means clustering
The Silhouette Coefficient
Feature extraction and PCA
Summary
12 Introduction to Transfer Learning and PreTrained Models
Understanding pre-trained models
Benefits of using pre-trained models
Commonly used pre-trained models
Decoding BERT’s pre-training
TL
Different types of TL
Inductive TL
Transductive TL
Unsupervised TL – feature extraction
TL with BERT and GPT
Examples of TL
Example – Fine-tuning a pre-trained model for text classification
Summary
13
Mitigating Algorithmic Bias and Tackling Model and Data Drift
Understanding algorithmic bias
Types of bias
Sources of algorithmic bias
Measuring bias
Consequences of unaddressed bias and the importance of fairness
Mitigating algorithmic bias
Mitigation during data preprocessing
Mitigation during model in-processing
Mitigation during model postprocessing
Bias in LLMs
Uncovering bias in GPT-2
Emerging techniques in bias and fairness in ML
Understanding model drift and decay
Model drift
Data drift
Mitigating drift
Understanding the context
Continuous monitoring
Regular model retraining
Implementing feedback systems
Model adaptation techniques
Summary
14 AI Governance
Mastering data governance
Current hurdles in data governance
Data management: crafting the bedrock
Data ingestion – the gateway to information
Data integration – from collection to delivery
Data warehouses and entity resolution
The quest for data quality
Documentation and cataloging – the unsung heroes of governance
Understanding the path of data
Regulatory compliance and audit preparedness
Change management and impact analysis
Upholding data quality
Troubleshooting and analysis
Navigating the intricacy and the anatomy of ML governance
ML governance pillars
Model interpretability
The many facets of ML development
Beyond training – model deployment and monitoring
A guide to architectural governance
The five pillars of architectural governance
Transformative architectural principles
Zooming in on architectural dimensions
Summary 15
Navigating Real-World Data Science Case Studies in Action
Introduction to the COMPAS dataset case study
Understanding the task/outlining success
Preliminary data exploration
Preparing the data for modeling
Final thoughts
Text embeddings using pretrainedmodels and OpenAI
Setting up and importing necessary libraries
Data collection – fetching the textbook data
Converting text to embeddings
Querying – searching for relevant information
Concluding thoughts – the power of modern pre-trained models
Summary Index
Other Books You May Enjoy
Preface
Principles of Data Science bridges mathematics, programming, and business analysis, empowering you to confidently pose and address complex data questions and construct effective machine learning pipelines. This book will equip you with the tools you need to transform abstract concepts and raw statistics into actionable insights.
Starting with cleaning and preparation, you’ll explore effective data mining strategies and techniques before moving on to building a holistic picture of how every piece of the data science puzzle fits together. Throughout the book, you’ll discover statistical models with which you can control and navigate even the densest or sparsest of datasets and learn how to create powerful visualizations that communicate the stories hidden in your data.
With a focus on application, this edition covers advanced transfer learning and pre-trained models for NLP and vision tasks. You’ll get to grips with advanced techniques for mitigating algorithmic bias in data as well as models and addressing model and data drift. Finally, you’ll explore medium-level data governance, including data provenance, privacy, and deletion request handling.
By the end of this data science book, you’ll have learned the fundamentals of computational mathematics and statistics, all while navigating the intricacies of modern machine learning and large pre-trained models such as GPT and BERT.
Who is this book for?
If you are an aspiring novice data scientist eager to expand your knowledge, this book is for you. Whether you have basic math skills and want to apply them in the field of data science, or you excel in programming but lack the necessary mathematical foundations, you’ll find this book useful. Familiarity with Python programming will further enhance your learning experience.
What this book covers
Chapter 1, Data Science Terminology, describes the basic terminology used by data scientists. We will cover the differences between often-confused terms as well as looking at examples of each term used in order to truly understand how to communicate in the language of data science. We will begin by looking at the broad term data science and then, little by little, get more specific until we arrive at the individual subdomains of data science, such as machine learning and statistical inference. This chapter will also look at the three main areas of data science, which are math, programming, and domain expertise. We will look at each one individually and understand the uses of each. We will also look at the basic Python packages and the syntax that will be used throughout the book.
Chapter 2, Types of Data, deals with data types and the way data is observed. We will explore the different levels of data as well as the different forms of data. Specifically, we will understand the differences between structured/unstructured data, quantitative/qualitative data, and more.
Chapter 3, The Five Steps of Data Science, deals with the data science process as well as data wrangling and preparation. We will go into the five steps of data science and give examples of the process at every step of the way. After we cover the five steps of data science, we will turn to data wrangling, which is the data exploration/preparation stage of the process. In order to best understand these principles, we will use extensive examples to explain each step. I will also provide tips to look for when exploring data,
including looking for data on different scales, categorical variables, and missing data. We will use pandas to check for and fix all of these things.
Chapter 4, Basic Mathematics, goes over the elementary mathematical skills needed by any data scientist. We will dive into functional analysis and use matric algebra as well as calculus to show and prove various outcomes based on real-world data problems.
Chapter 5, Impossible or Improbable – A Gentle Introduction to Probability, focuses heavily on the basic probability that is required for data science. We will derive results from data using probability rules and begin to see how we view real-world problems using probability. This chapter will be highly practical and Python will be used to code the examples.
Chapter 6, Advanced Probability, is where we explore how to use Python to solve more complex probability problems and also look at a new type of probability called Bayesian inference. We will use these theorems to solve real-world data scenarios such as weather predictions.
Chapter 7, What Are the Chances? An Introduction to Statistics, is on basic statistics, which is required for data science. We will also explore the types of statistical errors, including type I and type II errors, using examples. These errors are as essential to our analysis as the actual results. Errors and their different types allow us to dig deeper into our conclusions and avoid potentially disastrous results. Python will be used to code up statistical problems and results.
Chapter 8, Advanced Statistics, is where normalization is key. Understanding why and how we normalize data will be crucial. We will cover basic plotting, such as scatter plots, bar plots, and histograms. This chapter will also get into
statistical modeling using data. We will not only define the concept as using math to model a real-world situation, but we will also use real data in order to extrapolate our own statistical models. We will also discuss overfitting. Python will be used to code up statistical problems and results.
Chapter 9, Communicating Data, deals with the different ways of communicating results from our analysis. We will look at different presentation styles as well as different visualization techniques. The point of this chapter is to take our results and be able to explain them in a coherent, intelligible way so that anyone, whether they are data-savvy or not, may understand and use our results. Much of what we will discuss will be how to create effective graphs through labels, keys, colors, and more. We will also look at more advanced visualization techniques such as parallel coordinates plots.
Chapter 10, How to
Tell if Your Toaster is Learning – Machine Learning
Essentials, focuses on machine learning as a part of data science. We will define the different types of machine learning and see examples of each kind. We will specifically cover areas in regression, classification, and unsupervised learning. This chapter will cover what machine learning is and how it is used in data science. We will revisit the differences between machine learning and statistical modeling and how machine learning is a broader category of the latter. Our aim will be to utilize statistics and probability in order to understand and apply essential machine learning skills to practical industries such as marketing. Examples will include predicting star ratings of restaurant reviews, predicting the presence of disease, spam email detection, and much more. This chapter focuses more on statistical and probabilistic models. The next chapter will deal with models that do not fall
