Every word gets its own list of numbers. Similar words get similar lists.
Each stripe is one word's embedding: 50 numbers, shown as blue (positive) and amber (negative) cells. No single number means anything you could name. The meaning is in the overall pattern.
Compare two words and the similarity score measures how closely their patterns line up, from 100% (pointing the same way) down to zero or below (unrelated). Try the presets, or type any common English word.
A surprise
happy and sad score as quite similar. The embeddings were learned from which words appear near each other in text, and "happy" and "sad" show up in the same kinds of sentences. Similar here means "used in similar ways", which is close to meaning, but not the same.
Lay the words out by their numbers and meaning turns into geography.
No one told the model which words are countries, animals or colours. Yet when the words are placed according to their embeddings, they gather into neighbourhoods on their own. Cities sit near their countries, sports near each other, numbers in a tight cluster.
The real embeddings have 50 dimensions; this map squashes them into 2, so distances on it are only approximate. A searched word is drawn at the middle of its closest neighbours on the map.
Because meaning is stored as numbers, you can add and subtract it.
Take the numbers for king, subtract man, add woman, and find the word whose numbers are closest to the result. The answer is queen. The step from "man" to "woman" points in roughly the same direction everywhere in the space, and so does the step from a country to its capital, or from a country to its currency.
Try the presets or type your own. It does not always work; it is a side effect of learning from word patterns, not a rule anyone programmed.
The uncomfortable one
Try doctor − man + woman. The top answer is "nurse". The model learned that from millions of real sentences, so it absorbed the stereotypes in them too. Bias in AI often starts exactly here, in the data, which is why module 14 covers how it is measured and reduced.
Keyword search needs the same words. Embedding search needs the same meaning.
The 12 help articles are written as examples for this page. Each one, and your question, is turned into a single embedding by averaging the embeddings of its words.
A client asking about moving a pension after quitting their job shares no words with the article titled "rolling over a retirement plan from a former employer", so keyword search finds nothing. Embedding search turns both into numbers and finds that they point the same way.
This is the retrieval step behind tools that "chat with your documents", called RAG and covered in module 12. Averaging word embeddings, as this page does, is crude and makes mistakes. Real systems use models built to embed whole sentences and paragraphs, but the idea is identical.
Words, sentences, images, songs, customers: anything can be embedded.
| Embedding | Numbers each | Used for |
|---|---|---|
| This page (GloVe) | 50 | Words |
| The Lab's mini language model (module 06) | 128 | Its 512 word pieces |
| CLIP (2021) | 512 | Images and text in one shared space |
| BERT-base (2018) | 768 | Sentences, search ranking |
| OpenAI text-embedding-3-large (2024) | 3,072 | Documents, search, RAG |
Word embeddings: GloVe 6B by Jeffrey Pennington, Richard Socher and Christopher D. Manning (Stanford NLP), made available under the Public Domain Dedication and License v1.0. This page uses the 50-number version for the 8,000 most common words plus a few finance terms.
The embeddings on this page come from GloVe, trained in 2014 at Stanford on 6 billion words of Wikipedia and news. It learned purely from which words appear near which. Because it stopped learning in 2014, it has never heard of "bitcoin"; try it in step 2.
Modern models embed whole passages, and some put images and text into the same space, so a photo of a dog lands near the words "a dog". The first layer of every language model, including the one in module 06, is an embedding table like this one.
Where you meet them
Search engines, Spotify and Netflix recommendations, fraud detection (transactions that sit far from a customer's usual neighbourhood) and every "chat with your files" tool. The specialised databases that store and search them are called vector databases.
This completes the Deep learning track. Next in the course: how an LLM works.