Mastering Data Manipulation with Pandas: A Practical Tutorial
Pandas is the go-to Python library for data manipulation and analysis. With its intuitive DataFrame structure, you can clean, filter, transform, and summarize datasets efficiently. This tutorial covers the essential operations you need to handle real-world data.
Start by loading your data. Use pandas.read_csv() to import a CSV file into a DataFrame. For example, df = pd.read_csv('data.csv') creates a table-like object with rows and columns. You can quickly inspect the first few rows with df.head().
Filtering and Selecting Data
Select specific columns by passing a list of column names: df[['col1', 'col2']]. Filter rows using boolean conditions:
df[df['age'] > 30]— rows where age exceeds 30.- Combine conditions with
&(and) or|(or):df[(df['age'] > 30) & (df['city'] == 'NYC')].
For positional indexing, use df.iloc[row_indices, col_indices]; for label-based indexing, use df.loc[row_labels, col_labels].
Handling Missing Values
Real datasets often contain NaN values. Detect them with df.isna(). Remove rows with any missing values using df.dropna(). Fill them with a specific value, e.g., df.fillna(0) or with the column mean: df['col'].fillna(df['col'].mean()).
Grouping and Aggregating
Summarize data by categories using groupby(). For instance, compute average sales per region:
df.groupby('region')['sales'].mean()
Apply multiple aggregation functions with .agg(['sum', 'mean', 'count']). To reset the result into a flat DataFrame, chain .reset_index().
Creating New Columns
Derive new columns from existing ones. Example: df['total'] = df['price'] * df['quantity']. Use df.assign() for multiple new columns in one step.
Pandas makes data manipulation intuitive and fast. Practice these core techniques—loading, filtering, cleaning, grouping, and transforming—to turn raw data into actionable insights.