DATA WRANGLING TRANSFORMING RAW DATA INTO ANALYSIS-READY DATA Presented by: Rachit Joshi
WHAT IS DATA WRANGLING? • Data wrangling is the process of transforming and mapping raw data into a more useful format. • It is also known as Data Munging. • It makes data suitable for data analysis and other downstream purposes. • It improves the quality, structure, and usability of data. • Common activities include: •
Extracting data
•
Parsing data
•
Sorting data
•
Transforming data
•
Storing data
IMPORTANCE OF DATA WRANGLING • Makes raw data usable for downstream analysis. • Brings data from different sources into a centralized location. • Organizes raw data according to the required format and business context. • Cleans and converts data into a standard format. • Removes noise, flawed data, and missing elements. • Acts as a preparation stage for data mining. • Helps business users make concrete and timely decisions.
DATA WRANGLING PROCESS Discovery → Organization → Cleaning → Data Enrichment → Validation → Publishing Discovery • Define objectives. • Understand what you want to achieve with the data. Organization • Structure the collected raw data. • Handle different data types and sources. Cleaning • Remove outliers. • Handle null values. • Eliminate duplicate data. Data Enrichment • Determine whether enough data is available. • Ensure sufficient data for meaningful insights. Validation • Apply validation rules. • Check consistency, quality, and security. Publishing • Prepare data for future use. • Provide documentation and access to users/applications.
USE CASES OF DATA WRANGLING 1. Fraud Detection • Identify unusual behavior. • Support data security. • Standardize data for accurate and repeatable modeling. • Help maintain industry and government compliance. 2. Customer Behavior Analysis • Reduce time spent preparing data. • Understand the business value of customer data. • Allow analytics teams to use customer behavior data. • Discover trends through data discovery and visual profiling.
DATA WRANGLING TOOLS • Spreadsheets / Excel Power Query — Basic manual data wrangling. • OpenRefine — Automated data cleaning. • Tabula — Data wrangling for different data types. • Google DataPrep — Explores, cleans, and prepares data. • Data Wrangler — Data cleaning and transformation. • Plotly — Useful for maps and chart data with Python. • CSVKit — Converts data.
BENEFITS & DATA WRANGLING FORMATS Benefits •
Data Consistency
•
Improved Insights
•
Cost Efficiency
•
Improved data usability
•
Easier creation and automation of data flows
•
Integration of different information sources
•
Ability to process large volumes of data.
Data Wrangling Formats •
Transactional Data
•
Analytical Base Table (ABT)
•
Time-Series Data
•
Document Library
DATA WRANGLING EXAMPLES Examples: • Merge multiple data sources into one dataset. • Identify missing or empty cells and fill or remove them. • Delete irrelevant or unnecessary data. • Identify severe outliers and explain or remove them. • Detect corporate fraud. • Support data security. • Analyze customer behavior. • Discover data trends.
CONCLUSION • Data wrangling is an important data-preparation process that transforms raw data into useful and reliable information. • By organizing, cleaning, enriching, validating, and publishing data, organizations can improve data quality and make better use of their data for analytics and decision-making.