Introduction to Pandas

What Is Pandas?

Pandas is the standard Python library for data manipulation and analysis. It provides two core data structures — Series and DataFrame — that let you load, clean, transform, aggregate, and export structured data with an expressive, readable API.

The name comes from “Panel Data” — a term from econometrics for multi-dimensional datasets. Pandas was created at AQR Capital Management in 2008 by Wes McKinney to solve the problem of manipulating financial time series data in Python.

When to Use Pandas

Use Pandas when you need to:

• Load and inspect CSV, Excel, JSON, or SQL data
• Clean messy real-world data (nulls, wrong types, duplicates)
• Reshape, filter, group, and aggregate tabular data
• Merge and join multiple datasets
• Prepare features for machine learning
• Analyze time series

Installation

pip install pandas

Your First DataFrame

import pandas as pd

# Create a DataFrame from a dict — each key is a column name
sales = pd.DataFrame({
    "product":  ["Widget A", "Widget B", "Widget C", "Widget A", "Widget B"],
    "region":   ["North", "North", "South", "South", "North"],
    "quantity": [120, 85, 200, 150, 95],
    "revenue":  [1200.0, 1020.0, 2400.0, 1800.0, 1140.0],
})

print(sales)
#    product region  quantity  revenue
# 0  Widget A  North       120   1200.0
# 1  Widget B  North        85   1020.0
# 2  Widget C  South       200   2400.0
# 3  Widget A  South       150   1800.0
# 4  Widget B  North        95   1140.0

# Basic info
print(sales.shape)          # (5, 4) — 5 rows, 4 columns
print(sales.dtypes)         # column data types
print(sales.info())         # non-null counts, dtypes, memory
print(sales.describe())     # statistics for numeric columns
print(sales.head(3))        # first 3 rows
print(sales.tail(2))        # last 2 rows

Series

A Series is a 1-D labeled array — the building block of a DataFrame.

import pandas as pd
import numpy as np

# Create a Series with an explicit index
monthly_sales = pd.Series(
    [42000, 38000, 51000, 47000, 55000],
    index=["Jan", "Feb", "Mar", "Apr", "May"],
    name="revenue"
)

print(monthly_sales)
# Jan    42000
# Feb    38000
# ...

# Access by label
print(monthly_sales["Mar"])     # 51000

# Access by position
print(monthly_sales.iloc[0])    # 42000

# Boolean indexing
print(monthly_sales[monthly_sales > 45000])
# Mar    51000
# Apr    47000
# May    55000

# Arithmetic — aligns by index
q2_bonus = pd.Series([1000, 1000, 1000], index=["Apr", "May", "Jun"])
combined = monthly_sales + q2_bonus   # NaN where indices don't overlap
print(combined)

Loading Data from Files

import pandas as pd

# CSV — the most common format
df = pd.read_csv("data.csv")

# With options
df = pd.read_csv(
    "data.csv",
    sep=",",               # delimiter (use '\t' for TSV)
    header=0,              # row number for column names
    usecols=["id", "name", "revenue"],   # only load these columns
    dtype={"id": int, "revenue": float}, # enforce types
    na_values=["N/A", "null", "-"],      # treat these as NaN
    parse_dates=["created_at"],          # parse to datetime
)

# Excel
df_excel = pd.read_excel("report.xlsx", sheet_name="Sheet1")

# JSON
df_json = pd.read_json("events.json")

# SQL
import sqlite3
conn = sqlite3.connect("database.db")
df_sql = pd.read_sql("SELECT * FROM orders WHERE status = 'shipped'", conn)

# Inspect what you loaded
print(df.shape)
print(df.head())
print(df.dtypes)
print(df.isnull().sum())   # count nulls per column

Key DataFrame Operations at a Glance

import pandas as pd

df = pd.DataFrame({
    "name":    ["Alice", "Bob", "Carol", "Dave", "Eve"],
    "dept":    ["Eng", "Eng", "Sales", "Sales", "Eng"],
    "salary":  [95000, 88000, 72000, 76000, 105000],
    "years":   [5, 3, 7, 4, 8],
})

# Select a column — returns a Series
print(df["salary"])

# Select multiple columns — returns a DataFrame
print(df[["name", "salary"]])

# Filter rows
senior = df[df["years"] >= 5]
print(senior)

# Add a computed column
df["monthly_salary"] = df["salary"] / 12

# Sort
print(df.sort_values("salary", ascending=False))

# Groupby and aggregate
dept_summary = df.groupby("dept")["salary"].agg(["mean", "min", "max", "count"])
print(dept_summary)

# Rename columns
df.rename(columns={"dept": "department"}, inplace=True)