How Computers See Images
To a computer an image is a table of numbers. Convolution — a small weighted sum slid across it — turns those numbers into edges, textures and eventually objects.
What you will be able to do
- Explain pixels, channels and an image tensor's shape
- Compute one convolution by hand
- Explain why a CNN learns its kernels
The photo on your phone is, to a computer, a table of numbers. A 28 × 28 greyscale image is 784 numbers between 0 (black) and 255 (white). A colour image adds a third dimension: three channels — red, green, blue — so its shape is H × W × 3.
Convolution: a sliding weighted sum#
Take a small kernel, say 3 × 3 numbers. Place it over the top-left corner of the image, multiply each kernel number by the pixel beneath it, and add them up — that is one output pixel. Slide one step and repeat. The result is a new image that lights up wherever the kernel finds what it is "looking for".
- x
- the input image
- k
- the kernel (filter)
- y
- the output feature map
One kernel, by hand#
The image below is dark on the left and bright on the right — a single vertical edge. The kernel [−1, 0, 1] on every row computes "right minus left", so it is large exactly at the edge and zero over flat areas.
import numpy as np image = np.array([[0, 0, 0, 10, 10]] * 5) # 5 x 5, edge in the middlekernel = np.array([[-1, 0, 1]] * 3) # 3 x 3, vertical edge out = np.zeros((3, 3), dtype=int)for i in range(3): for j in range(3): out[i, j] = np.sum(image[i:i+3, j:j+3] * kernel) print(out)[[ 0 30 30]
[ 0 30 30]
[ 0 30 30]]Output size#
- n
- input width (or height)
- k
- kernel size
- p
- padding on each side
- s
- stride — the step size
From hand-made filters to learned ones#
For decades, computer vision researchers designed kernels like Sobel by hand. The central idea of the convolutional neural network (CNN) is to let the numbers in the kernel be weights learned by gradient descent. The first layer of a trained CNN typically learns something resembling edge and colour detectors; deeper layers combine those into textures, object parts and then whole objects.
| Property | Why it matters |
|---|---|
| Weight sharing | The same kernel is used across the whole image, so the parameter count is small — 3 × 3 × 3 = 27 weights (plus a bias) per filter on an RGB image, whatever the image size. |
| Locality | Each output pixel looks only at a small neighbourhood, which matches how visual patterns are built. |
| Translation equivariance | Shifting an object in the image shifts the filter's response by the same amount. |
A 32 × 32 image is convolved with a 5 × 5 kernel, padding 0, stride 1. What is the output size?
Why does a CNN need far fewer parameters than a fully connected network for the same image?