Exercise to practice Pandas Groupby

exercise-pandas-groupby.md

Practice: groupby()

Interation 1

Load the Titanic dataset (from seaborn).

#
# Note: you may need to install seaborn
#
# e.g. "pip install seaborn", or "conda install seaborn"
# 

import seaborn as sns
df = sns.load_dataset("titanic")

Iteration 1

• Find the average age for each passenger class.

Iteration 2:

• Find the average fare and survived rate grouped by class, using agg() to get both in one call.

Iteration 3:

• Group by class and sex together, and find the survival rate (survived mean) for each combination. Use as_index=False.

Bonus

Bonus 1: For each class, find the min, max, and mean fare, all in one agg() call.

Bonus 2: Group by embark_town and find the average fare. Then, identify which embark_town has the highest average fare (research a way to do this without manually scanning the output).

Bonus 3: This dataset has missing data (for example, there's many rows where age is NaN). Do some research: how did pandas handle missing values when you calculated the average age?

Bonus 4: Check for missing data using df.isna().sum(). Which columns have missing values, and what percentage of rows is missing for each? One of the columns with missing data is age. Drop all rows where age is missing (df.dropna(subset=["age"])). How many rows were removed?

---

Solutions

Solution: Iteration 1

df.groupby("class")["age"].mean()

Solution: Iteration 2

df.groupby("class").agg({"fare": "mean", "survived": "mean"})

Solution: Iteration 3

df.groupby(["class", "sex"], as_index=False)["survived"].mean()

Solution: Bonus 1

df.groupby("class")["fare"].agg(["min", "max", "mean"])

Solution: Bonus 2

avg_fare = df.groupby("embark_town")["fare"].mean()
avg_fare.idxmax()

Solution: Bonus 3

By default, pandas aggregation functions (mean, min, max, sum, etc.) skip NaN values automatically (skipna=True by default).

So in df.groupby("class")["age"].mean(), rows with missing age are just excluded from that group's average — they don't cause an error and don't count as 0.

Solution: Bonus 4

# Check missing data
missing_counts = df.isna().sum()
print("\n\nMissing data...\n")
display(missing_counts)

# Calculate the percentage of missing data
missing_pct = (df.isna().mean() * 100).round(2)
stats_df = pd.DataFrame({"missing_count": missing_counts, "missing_pct": missing_pct})
print("\n\nMissing data & percentage of rows affected...\n")
display(stats_df)

# Drop rows where 'age' is missing
df_clean = df.dropna(subset=["age"])

# Rows removed
print("\n\nNumber of rows removed...\n")
print(len(df) - len(df_clean))