Story

How AI guesses the next word

Five slides, one cloud of points. Scroll through how a language model turns words into numbers, numbers into a guess, and a wrong guess into a better one.

Chat assistants feel like they think. Underneath, they do one small thing over and over. Scroll slowly.

  1. 1 / 5 Words become numbers. Each word is stored as a long list of numbers, a point in a space with hundreds of directions. Training pushes words used in similar ways close together, so “cat” lands near “dog”, and “king” near “queen”.
  2. 2 / 5 A neuron is tiny arithmetic. It multiplies each input by a weight, adds them up, and passes the total on if it's big enough. Positive weights (blue) push the answer up, negative ones (pink) pull it down.
  3. 3 / 5 Stack thousands of neurons in layers and you get a network. The numbers for your words flow in on the left and move through layer after layer. Each layer mixes the signals from the one before.
  4. 4 / 5 Learning runs the other way. The network's guess is compared with the real next word, and the error is sent backward. Every weight gets nudged a little in the direction that would have made the guess better. Repeat billions of times.
  5. 5 / 5 What comes out is not an answer but a ranking: a probability for every possible next word. The model picks one, adds it to the text, and runs the whole thing again for the word after that. That's all a reply is.

What you just scrolled through

Embeddings. The list of numbers for each word is called an embedding. Nobody writes these by hand. They start random and drift into place during training, until distance in that space roughly tracks difference in meaning.

Weights are the knowledge. A model’s “size” is its count of weights. Everything it seems to know, grammar, facts, style, lives in those numbers and nowhere else.

Backpropagation. Sending the error backward through the layers is called backpropagation. It works out, for every single weight, which way to turn it to shrink the error. Training is this step, repeated over a huge pile of text.

One word at a time. Models actually work with tokens, which are often pieces of words. A long answer is hundreds of these guesses in a row, each one fed back in as input for the next.