Matrix Multiplication

Definition

Core Statement

Matrix Multiplication is a binary operation that takes a pair of matrices and produces another matrix. If $A$ is an $m \times n$ matrix and $B$ is an $n \times p$ matrix, their product $C = A B$ is an $m \times p$ matrix where each element $c_{i j}$ is the dot product of the $i$ -th row of $A$ and the $j$ -th column of $B$ .

Purpose

Linear Transformations: Rotating, scaling, or shearing space (Computer Graphics, Data Preprocessing).
System of Equations: Representing $A x = b$ efficiently.
Neural Networks: The "Forward Pass" is essentially a series of matrix multiplications (Weights $\times$ Inputs).
Covariance Calculations: $X^{T} X$ constructs correlations.

The Rules

1. Dimension Compatibility

To multiply $A \times B$ :

Columns of $A$ Must Equal Rows of $B$ .
$(m \times n) \cdot (n \times p) \to (m \times p)$
If inner dimensions don't match, multiplication is undefined.

2. The Dot Product Operation

For element $c_{i j}$ :

c_{i j} = {Row}_{i} (A) \cdot {Col}_{j} (B) = \sum_{k = 1}^{n} a_{i k} b_{k j}

Worked Example: Shopping Bill

Problem

Prices (Matrix P): Apple=$1, Banana=$0.5.
Quantities (Matrix Q):

Person A: 2 Apples, 4 Bananas.
Person B: 1 Apple, 0 Bananas.

Calculate Total Cost for each.

Setup:

$Q$ (Quantities) is $2 \times 2$ : $[\begin{matrix} 2 & 4 \\ 1 & 0 \end{matrix}]$
$P$ (Prices) is $2 \times 1$ : $[\begin{matrix} 1 \\ 0.5 \end{matrix}]$

Calculation ( $Q \times P$ ):

Person A: $(2 \times 1) + (4 \times 0.5) = 2 + 2 = 4$ .
Person B: $(1 \times 1) + (0 \times 0.5) = 1 + 0 = 1$ .

Result:

[\begin{matrix} 4 \\ 1 \end{matrix}]

Person A spends $4, Person B spends $1.

Geometric Intuition

Matrix multiplication $A B$ can be seen as:

Transformation: Columns of $A$ specify where the basis vectors ( $\hat{i}, \hat{j}$ ) allow the input vectors ( $B$ ) to land.
Composition: Doing transformation $B$ followed by transformation $A$ .

Python Implementation

import numpy as np

# Define A (2x3)
A = np.array([[1, 2, 3],
              [4, 5, 6]])

# Define B (3x2)
B = np.array([[7, 8],
              [9, 1],
              [2, 3]])

# Multiply
C = np.dot(A, B)
# OR ideal modern syntax:
C_modern = A @ B

print(f"Result (2x2):\n{C_modern}")

Limitations & Pitfalls

Pitfalls

Non-Commutative: In general, $A B \neq B A$ . Order matters! (Rotation then Shear $\neq$ Shear then Rotation).
Element-wise confusion: Matrix multiplication is NOT just multiplying corresponding elements (that is the Hadamard Product).
Broadcasting: In Python/NumPy, A * B often does element-wise multiplication. You usually want A @ B or np.dot().

Eigenvalues & Eigenvectors - What vectors stay the same after multiplication?
Principal Component Analysis (PCA) - Uses correlation matrix ( $X^{T} X$ ).
Multiple Linear Regression - Solution involves $(X^{T} X)^{- 1} X^{T} y$ .