NumPy Broadcasting

A Scalar Is the Simplest Case

import numpy as np

prices = np.array([9.99, 14.99, 24.99, 4.99])
print(prices * 1.08)
print(prices > 10)

$ python scalar.py
[10.7892 16.1892 26.9892  5.3892]
[False  True  True False]

The single number 1.08 is combined with each of the four prices. That is broadcasting: a scalar acts like an array of shape (), which is compatible with any shape.

The Rules, Checked by NumPy

NumPy compares shapes from the last axis backwards; lengths must be equal or one of them must be 1.

import numpy as np

pairs = [((4, 3), (3,)), ((4, 1), (3,)), ((4, 3), (4, 1)),
         ((2, 1, 5), (3, 1)), ((4, 3), (4,)), ((4, 3), (3, 4))]
for a, b in pairs:
    try:
        print(a, "+", b, "->", np.broadcast_shapes(a, b))
    except ValueError as e:
        print(a, "+", b, "-> ValueError:", e)

$ python rules.py
(4, 3) + (3,) -> (4, 3)
(4, 1) + (3,) -> (4, 3)
(4, 3) + (4, 1) -> (4, 3)
(2, 1, 5) + (3, 1) -> (2, 3, 5)
(4, 3) + (4,) -> ValueError: shape mismatch: objects cannot be broadcast to a single shape.  Mismatch is between arg 0 with shape (4, 3) and arg 1 with shape (4,).
(4, 3) + (3, 4) -> ValueError: shape mismatch: objects cannot be broadcast to a single shape.  Mismatch is between arg 0 with shape (4, 3) and arg 1 with shape (3, 4).

(4, 3) and (3,) line up on the last axis (3 = 3), and the missing leading axis counts as 1. (2, 1, 5) with (3, 1) stretches both size-1 axes to give (2, 3, 5). The last two fail: (4, 3) against (4,) compares 3 with 4, and (3, 4) compares 3 with 4 on both axes.

Centering Columns with a Row Vector

import numpy as np

rng = np.random.default_rng(42)
data = rng.normal(loc=[10, 50, 100], scale=[1, 5, 20], size=(6, 3))

means = data.mean(axis=0)
stds = data.std(axis=0)
z = (data - means) / stds

print(data.shape, means.shape)
print(z.mean(axis=0).round(10))
print(z.std(axis=0))

$ python standardize.py
(6, 3) (3,)
[-0. -0. -0.]
[1. 1. 1.]

data is (6, 3) and means is (3,), so each row has the three column means subtracted. After dividing by the column standard deviations every feature has mean 0 (to 10 decimals) and standard deviation 1, however different the original scales were. The -0. is a tiny negative rounding residue that round(10) turned into negative zero.

Rows Against Columns with np.newaxis

import numpy as np

row = np.array([1, 2, 3, 4])
col = np.array([10, 20, 30])[:, np.newaxis]

print(row.shape, col.shape)
print(row + col)
print(np.outer([1, 2, 3], [1, 10]) == np.array([1, 2, 3])[:, None] * np.array([1, 10]))

$ python row_col.py
(4,) (3, 1)
[[11 12 13 14]
 [21 22 23 24]
 [31 32 33 34]]
[[ True  True]
 [ True  True]
 [ True  True]]

[:, np.newaxis] turned col into shape (3, 1). Combined with (4,), both stretch to (3, 4): every column value is added to every row value. The last line shows that an outer product is exactly this pattern with *; None is a shorter spelling of np.newaxis.

Pairwise Distances

import numpy as np

pts = np.array([[0., 0.], [1., 0.], [0., 1.], [1., 1.]])

diff = pts[:, np.newaxis, :] - pts[np.newaxis, :, :]
dist = np.sqrt((diff ** 2).sum(axis=-1))

print(diff.shape)
print(dist.round(3))

$ python pairwise.py
(4, 4, 2)
[[0.    1.    1.    1.414]
 [1.    0.    1.414 1.   ]
 [1.    1.414 0.    1.   ]
 [1.414 1.    1.    0.   ]]

(4, 1, 2) minus (1, 4, 2) gives (4, 4, 2): the coordinate difference between every pair of points. Summing squares over the last axis leaves a (4, 4) distance matrix — zero on the diagonal, 1 between neighbours, √2 ≈ 1.414 across the diagonals of the unit square.

Broadcasting Doesn’t Copy

import numpy as np

b = np.array([1, 2, 3])
big = np.broadcast_to(b, (1000, 3))

print(big.shape, big.strides)
print(np.shares_memory(big, b))
try:
    big[0, 0] = 99
except ValueError as e:
    print("ValueError:", e)

$ python no_copy.py
(1000, 3) (0, 8)
True
ValueError: assignment destination is read-only

The stretched array reports shape (1000, 3) but a stride of 0 along the first axis: moving to the next row moves zero bytes, so all 1000 rows are the same three numbers in memory. That is why the view is read-only — writing one “row” would change them all.

The Column Mistake

import numpy as np

A = np.ones((4, 3))
b = np.array([1, 2, 3, 4])

try:
    A - b
except ValueError as e:
    print("ValueError:", e)

print(A - b[:, np.newaxis])

$ python column_mistake.py
ValueError: operands could not be broadcast together with shapes (4,3) (4,) 
[[ 0.  0.  0.]
 [-1. -1. -1.]
 [-2. -2. -2.]
 [-3. -3. -3.]]

b has one value per row, but broadcasting aligns from the right, so NumPy tried to match its 4 against A’s 3 columns. Reshaping to (4, 1) says what was meant: subtract b[i] from every element of row i.

Practice

1. Predict the broadcast shape of (5, 1, 4) and (3, 1), then check with np.broadcast_shapes.
2. Build a 10 × 10 multiplication table from a single np.arange(1, 11).
3. Scale each row of np.ones((3, 4)) by [1, 2, 3] respectively.

import numpy as np

print(np.broadcast_shapes((5, 1, 4), (3, 1)))

$ python practice_shape.py
(5, 3, 4)

import numpy as np

n = np.arange(1, 11)
table = n[:, None] * n
print(table[:3])
print(table.shape)

$ python practice_table.py
[[ 1  2  3  4  5  6  7  8  9 10]
 [ 2  4  6  8 10 12 14 16 18 20]
 [ 3  6  9 12 15 18 21 24 27 30]]
(10, 10)

import numpy as np

M = np.ones((3, 4))
print(M * np.array([1, 2, 3])[:, None])

$ python practice_rowscale.py
[[1. 1. 1. 1.]
 [2. 2. 2. 2.]
 [3. 3. 3. 3.]]