Skip to content
Datamewby Intura
Start free
Introduction to Deep Learning & Neural Networks

Training Deep Networks

Optimizers, learning-rate schedules, initialisation, normalisation and regularisation — what makes training actually converge.

Level
Intermediate
Lessons
4
Total time
1h 5m
Simulations
4
Included in Introduction to Deep Learning & Neural Networks · $9
Tools you'll use
  • Python
  • PyTorch
  • NumPy
Skills
  • SGD, Momentum, Adam
  • BatchNorm & LayerNorm
  • Dropout & weight decay
Try it now — free

This is the real simulation from the module. No account needed.

What you will be able to do

  • Choose an optimizer and learning-rate schedule for a new problem
  • Diagnose vanishing and exploding gradients

Before you start

A suggestion, never a lock — you can start here.

  • neural-network-foundations
Modules that help first

Syllabus

4 lessons · 1h 5m

Every lesson's title and summary is public. The lessons themselves open once you own the track.

Every lesson in Introduction to Deep Learning & Neural Networks

  1. Optimizers: SGD, Momentum, Adam

    18 min

    What Adam actually does, and when plain SGD still wins.

    Included with the track Simulation

  2. Learning-Rate Schedules and Warmup

    15 min

    Cosine decay, warmup, and finding a learning rate in ten minutes.

    Included with the track Simulation

  3. Initialisation and Normalisation

    16 min

    Why Xavier and He initialisation exist, and what BatchNorm and LayerNorm fix.

    Included with the track Simulation

  4. Regularisation: Dropout, Weight Decay, Early Stopping

    16 min

    Three ways to stop a network memorising, and how to choose between them.

    Included with the track Simulation
Lifetime access
$9
Buy the track
More in Introduction to Deep Learning & Neural Networks
Module · Beginner

Neural Network Foundations

What a neural network actually computes, why the nonlinearity is the whole trick, and how the weights learn.

  • 4 lessons
  • 1h 20m
  • 7 simulations
Included in Introduction to Deep Learning & Neural Networks· $9
Module · Intermediate

PyTorch for Deep Learning

Tensors, autograd, nn.Module, DataLoader and GPU training — enough PyTorch to work on your own.

  • 4 lessons
  • 1h 14m
  • 2 simulations
Included in Introduction to Deep Learning & Neural Networks· $9
Module · Intermediate

Convolutional Neural Networks

Convolution, pooling and CNN architectures — why weight sharing makes images learnable.

  • 3 lessons
  • 54 min
  • 4 simulations
Included in Introduction to Deep Learning & Neural Networks· $9
Module · Intermediate

Sequence Models and Transformers

From RNNs and LSTMs to self-attention — the architecture behind every modern LLM.

  • 3 lessons
  • 56 min
  • 3 simulations
Included in Introduction to Deep Learning & Neural Networks· $9