A neuron multiplies each input by a weight, adds them up, and squashes the total into a yes or no.
The neuron takes two inputs (where a point sits across and up), multiplies each by a weight, adds a bias, then passes the total through a squashing function that turns it into a number between 0 and 1. Above 0.5 means blue, below means amber.
Move the sliders and watch the dividing line turn and slide. Hover a point to see the actual arithmetic on the right. Then press Let it learn and gradient descent, from module 01, finds the weights for you.
You have seen this before
This single neuron is exactly the logistic regression from module 02. A neural network is what you get when you wire many of them together.
One neuron can only draw a straight line. Most real problems are not straight.
Here blue sits in two opposite corners and amber in the other two. Press Train one neuron: however long it trains, it gets stuck around half right, because no single straight line separates them.
Now press Train a small network: four neurons in a hidden layer feeding one output. Each hidden neuron draws its own line, and the output neuron combines them into a shape. It solves the pattern in seconds.
The bend matters
The squashing step is called the activation function. Without it, stacking neurons would just add straight lines into another straight line, however many layers you used. The bend is what lets layers build curves. Modern networks mostly use ReLU, which simply turns negatives into zero: cheap to compute and easy to train.
Pick a shape, design a network, and watch it learn.
Dots are training data; hollow squares are test data the network never trains on.
The diagram is the network. Each small square is one neuron, painted with the pattern it responds to across the whole map. Lines are weights: blue pushes up, amber pushes down, thicker means stronger.
Try the spiral with 1 layer of 2 neurons: it cannot do it. Give it 3 layers of 6 and it can. Try a step size of 0.3 and watch training become jumpy. And look at the neurons: the first layer only learns straight edges, and later layers combine those edges into curves.
How does each weight know which way to move? The error is passed backward, layer by layer.
This network has one input, one hidden neuron and one output, with two weights and two biases. Its job: when the input is 1, answer 1.
Press the button to walk through one training round:
Forward: the input flows through, and the network makes a guess. Loss: measure how wrong it is. Backward: starting at the output, work out how much each weight contributed to the error, using the chain rule from calculus. Each layer's share depends on the layer after it, so the blame flows backward. Update: nudge every weight against its gradient.
Why it matters
This is backpropagation. It computes the gradient for every weight in one backward sweep, whether the network has 4 weights or 400 billion. It is what makes training deep networks possible at all.
Each layer builds on the one before. That is what "deep" learning means.
| Network | Parameters |
|---|---|
| Your playground network | - |
| The Lab's mini language model (module 06) | 867,072 |
| GPT-2 (2019) | 1.5 billion |
| GPT-3 (2020) | 175 billion |
| Llama 3.1, largest version (2024) | 405 billion |
These are the neurons from the network you trained in step 3, layer by layer. The first layer can only split the map with straight edges. The next layer mixes those edges into corners and curves. The last layer assembles the final shape.
Real networks do the same thing at a vastly larger scale. In an image model, early layers detect edges, middle layers detect textures and parts, and late layers detect whole objects. In a language model, early layers handle spelling and grammar and later layers handle meaning.
Same parts, more of them
Every model in the table is built from the units on this page: weights, sums, activation functions, trained by backpropagation and gradient descent. The difference is the number of parameters, the amount of data and the computing power, which is what the AI infrastructure track is about.
Next in the course: how networks see, in module 04.