“`html

Data Cleaning with Pandas: A Practical Python Tutorial

Raw data is messy. It contains missing values, duplicates, and incorrect data types that can ruin your analysis. Pandas, Python’s most popular data library, provides simple and efficient methods to clean datasets in just a few lines of code.

This tutorial assumes you have a DataFrame named df. Start by importing pandas with import pandas as pd. We’ll explore four essential data cleaning tasks you’ll use almost every day.

1. Handle Missing Values

Missing values appear as NaN. First, detect them with df.isnull().sum(). Then decide whether to remove or fill them:

• df.dropna() — remove all rows containing missing values
• df.fillna(0) — fill missing values with 0
• df['price'].fillna(df['price'].mean()) — fill a column with its mean

2. Remove Duplicate Rows

Duplicates skew your counts and averages. Check them with df.duplicated().sum(). Remove them with df.drop_duplicates(). To target specific columns, pass a subset:

df.drop_duplicates(subset=['email'], keep='first')

3. Convert Data Types

Wrong types cause errors and slow down analysis. Convert them using astype() and pd.to_datetime():

• df['age'] = df['age'].astype(int)
• df['price'] = df['price'].astype(float)
• df['date'] = pd.to_datetime(df['date'])

4. Clean Columns and Filter Rows

Rename columns for consistency, drop unused ones, and keep only the rows you need:

• df.rename(columns={'old': 'new'}, inplace=True)
• df.drop(columns=['unused'], inplace=True)
• df_filtered = df[df['age'] > 18]
• df['name'] = df['name'].str.strip().str.title()

Conclusion

Data cleaning doesn’t have to be painful. With pandas, you can handle missing values, duplicates, type issues, and column cleanup in a few commands. Master these basics, and your data will be analysis-ready in no time.

“`