Heatmap

Definition

Purpose

1. Correlation Analysis: Quickly spot highly correlated variables ($r>0.8$ or $r<-0.8$).
2. Missing Data Patterns: Visualize where NaN values occur (are they random or structural?).
3. Clustering Results: When rows/cols are reordered by similarity (Clustered Heatmap), groups emerge.
4. Time Series: Weekday vs Hour of Day grids (Traffic patterns).

When to Use

Key Types

1. Correlation Heatmap

• Input: $N\times N$ Correlation Matrix.
• Use: Feature selection (drop redundant features).

2. Clustered Heatmap (Hierarchical)

• Input: Raw Data Matrix ($N\times P$).
• Action: Reorders rows and columns so similar items are next to each other.
• Use: Gene expression analysis, Customer segmentation.

Assumptions

• Normalization: If comparing raw values, variables should be on the same scale (or standardized).
• Color Scale: The midpoint (usually 0) must be neutral (white/grey) for diverging data (Correlation -1 to +1).

Python Implementation

import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

# Mock Data
data = pd.DataFrame(np.random.rand(10, 10), columns=[f'Var{i}' for i in range(10)])
corr = data.corr()

# 1. Basic Correlation Heatmap
plt.figure(figsize=(10, 8))
sns.heatmap(corr, 
            annot=True,      # Show numbers
            fmt=".2f",       # 2 decimal places
            cmap='coolwarm', # Diverging colormap
            center=0,        # Center at 0
            vmin=-1, vmax=1) # Fixed range for correlation
plt.title("Feature Correlation Matrix")
plt.show()

# 2. Clustermap (Reorders rows/cols)
sns.clustermap(data.iloc[:20, :], cmap='viridis', standard_scale=1)

Limitations

Related Concepts

• Pearson Correlation - The metric usually visualized.
• Principal Component Analysis (PCA) - Alternative for high-dim data.
• Confusion Matrix - Often visualized as a heatmap.