
- Backpropagation: the method for propagating errors backward through a network to update weights.
- Chain rule: the calculus rule used to compute how each weight affects the final output.
- Workflow: forward pass → compute loss → backward pass (apply chain rule) → update weights (optimizer).
- Applications: classification, recommendation systems (e.g., Netflix suggestions), regression, and many other tasks in deep learning.
Intuition: Netflix recommendations
Imagine you finish watching a series and Netflix starts suggesting shows you will likely enjoy. Early recommendations may feel random; over time the system learns from your choices and adjusts model weights to reduce errors. That adjustment process is backpropagation: it uses gradients computed via the chain rule to move weights in directions that reduce prediction error.
Neural network structure (simplified)
- Inputs: features describing the last show (e.g.,
genre,setting). - One hidden layer (for simplicity) with a single neuron.
- Output: a prediction score, interpreted as probability.
- Inputs:
genre = 4,setting = 2 - Initial weights:
w_genre = 0.2,w_setting = 0.6 - Weighted sum into neuron: s = 4 * 0.2 + 2 * 0.6 = 0.8 + 1.2 = 2.0
t = 0.65. The model overestimates the probability, so weights must be adjusted to reduce the prediction. Backpropagation tells us how to change each weight to reduce the loss.
Why the chain rule matters
The network output y depends on s, and s depends on the weights w. Symbolically: y = f(s(w)). To know how a small change in a weight w changes y, we compute dy/dw using the chain rule:
dy/dw = (dy/ds) * (ds/dw)
dy/dsis the derivative of the activation function with respect to its inputs.ds/dwis the derivative of the weighted sum with respect to the weightw(usually the corresponding input value).
- For sigmoid: dy/ds = sigmoid(s) * (1 - sigmoid(s)).
- For s = 4 * w_genre + 2 * w_setting, ds/dw_genre = 4.
t = 0.65:
- The update moves the weight opposite to the gradient (hence the minus sign) to reduce loss.
- Apply the same process to every weight (e.g.,
w_setting) and update them together. - Repeating over many examples and epochs is how backpropagation trains the network.
z(y) = y^3 + 3 Combined: z(x) = (3 x^2 + 1)^3 + 3 To compute dz/dx: dz/dx = dz/dy * dy/dx
- dy/dx = 6 x
- dz/dy = 3 y^2

- y(x) = x^3 - 2 # dough quality depends on yeast x
- z(y) = 2 - y^2 # bread quality depends on dough quality
- dy/dx = 3 x^2
- dz/dy = -2 y
- y = 1^3 - 2 = -1
- dz/dx = -6 * 1^2 * (-1) = 6

- Forward pass: compute s = sum(inputs * weights) and y = activation(s).
- Backward pass: compute derivatives using the chain rule: dE/dw = (dE/dy) * (dy/ds) * (ds/dw)
After computing
dE/dw for every weight, the optimizer (gradient descent, SGD, Adam, etc.) adjusts weights accordingly.
A visual recap: the chain rule is applied layer-by-layer to propagate error derivatives back to earlier weights, enabling each weight to be updated according to its contribution to the error.

Backpropagation uses the chain rule to compute gradients efficiently. In practice, modern frameworks like TensorFlow and PyTorch compute these derivatives automatically via autodiff, but understanding the chain-rule flow is essential for debugging, optimizing architectures, and designing custom layers or loss functions.
- Composite functions are the mathematical form of layered neural networks.
- The chain rule multiplies derivatives of inner and outer functions to determine how small changes propagate.
- Backpropagation repeatedly applies the chain rule across layers to compute gradients and update weights, enabling networks to learn from data.

- Calculus: Chain Rule (Khan Academy)
- Backpropagation and neural networks (Wikipedia)
- TensorFlow documentation — gradients and autograd
- PyTorch Autograd: automatic differentiation