Winter 2027
Deep Learning:
Architectures and Representations
For most of the history of computing, the way to make a computer do something was to write down the rules. Take translation between two languages as an example. Early translation programs were built from hand-written grammar rules and dictionaries, and their output was stiff and often wrong. In 2016 Google switched its translator to a neural network that learned from millions of example sentences, and translation errors dropped by about 60%.[1] This is not special to language translation, neural networks, or deep learning, now dominate the entire field of artificial intelligence. Their success is due to their ability to build up a hierarchy of useful representations of the data. A network that recognizes cats, for example, might first learn to detect edges and color gradients, then combine those into textures and shapes, then into eyes, whiskers and fur, and finally into the concept "cat." This course is about how that works and why it works so well.
We start from the basics. The simplest single-layer neural network is nothing but a logistic regression model. With just one layer, the model can only weigh up the raw inputs it is given, and has no way of building more abstract concepts out of them. We will see that adding hidden layers with a nonlinearity gives the network that ability, letting each layer build new concepts out of the ones before it. We then derive gradient descent and backpropagation, the engine that trains every model in this course. From there we talk about the architectures behind the headline results of the last decade: convolutional networks for images and recurrent networks for sequences. We also discuss the now very popular transformer architecture that powers large language models like ChatGPT and Claude. We then turn to generative models (VAEs, GANs and diffusion) that create new images, videos or text. Along the way we implement and train each of them in PyTorch.
The same hierarchy of representations that makes these networks powerful also makes them hard to understand. There is no line of code we can read to find out why the model called the picture a cat. We close the course with tools developed to answer questions like "why did the model make this classification?" or "does the model actually know what a cat is?" These tools include saliency maps, integrated gradients and linear probes. Methods like these also highlight failure modes of AI systems. One striking example: add a carefully chosen pattern of noise to a photo of a panda, too faint for any human to notice, and a state-of-the-art classifier will confidently call it a gibbon.[2] This is very important for AI safety. Failures like this, along with models that latch onto shortcuts or are confidently wrong on unfamiliar data, matter more and more as deep learning moves into medicine, law and self-driving cars.
Tentative lectures
From logistic regression to neural networks: backpropagation and gradient descent
Making deep networks train: initialization, regularization and hyperparameter search
Convolutional neural networks
Deeper vision: AlexNet, residual connections and transfer learning; words as vectors
Recurrent neural networks and attention
Transformers and language models
Generative models I: latent spaces and variational autoencoders
Generative models II: GANs and diffusion
What does a network learn?
AI safety: adversarial attacks and how networks fail
Prerequisites: Calculus, probability, and linear algebra (matrix multiplication, dot products). Problem sets require Python programming. We will quickly review basic concepts of supervised learning (loss functions, gradient descent, training), but familiarity with these concepts is a plus.
This class will meet virtually via Zoom on Wednesdays from 4:30 to 6:00 PM Pacific time and Thursdays from 4:30 to 6:30 PM Pacific time from January 6 to March 11, 2027.
Applications for Winter 2027 are due November 22, 2026. After that, we will continue to accept applications on a rolling basis while spots remain. Click here to apply!
[1] Wu et al. (2016), Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation.
[2] Goodfellow, Shlens, Szegedy (2014), Explaining and Harnessing Adversarial Examples.