Skip to content
Datamewby Intura
Start free
Introduction to Deep Learning & Neural Networks

Sequence Models and Transformers

From RNNs and LSTMs to self-attention — the architecture behind every modern LLM.

Level
Intermediate
Lessons
3
Total time
56 min
Simulations
3
Included in Introduction to Deep Learning & Neural Networks · $9
Tools you'll use
  • Python
  • PyTorch
  • NumPy
Skills
  • RNNs & LSTMs
  • Self-attention
  • Transformers
Try it now — free

This is the real simulation from the module. No account needed.

What you will be able to do

  • Explain self-attention in terms of Q, K and V
  • Read a transformer architecture diagram

Before you start

A suggestion, never a lock — you can start here.

  • training-deep-networks
Modules that help first

Syllabus

3 lessons · 56 min

Every lesson's title and summary is public. The lessons themselves open once you own the track.

Every lesson in Introduction to Deep Learning & Neural Networks

  1. RNNs and LSTMs

    16 min

    Processing sequences one step at a time, and why that is hard to train on long sequences.

    Included with the track

  2. The Attention Mechanism

    20 min

    Query, key, value — every token decides which other tokens it needs to look at.

    Included with the track Simulation

  3. The Transformer Architecture

    20 min

    Multi-head attention, positional encodings, and the block stacked inside every LLM.

    Included with the track Simulation
Lifetime access
$9
Buy the track
More in Introduction to Deep Learning & Neural Networks
Module · Beginner

Neural Network Foundations

What a neural network actually computes, why the nonlinearity is the whole trick, and how the weights learn.

  • 4 lessons
  • 1h 20m
  • 7 simulations
Included in Introduction to Deep Learning & Neural Networks· $9
Module · Intermediate

Training Deep Networks

Optimizers, learning-rate schedules, initialisation, normalisation and regularisation — what makes training actually converge.

  • 4 lessons
  • 1h 5m
  • 4 simulations
Included in Introduction to Deep Learning & Neural Networks· $9
Module · Intermediate

PyTorch for Deep Learning

Tensors, autograd, nn.Module, DataLoader and GPU training — enough PyTorch to work on your own.

  • 4 lessons
  • 1h 14m
  • 2 simulations
Included in Introduction to Deep Learning & Neural Networks· $9
Module · Intermediate

Convolutional Neural Networks

Convolution, pooling and CNN architectures — why weight sharing makes images learnable.

  • 3 lessons
  • 54 min
  • 4 simulations
Included in Introduction to Deep Learning & Neural Networks· $9