A language model does not keep a definition of a word. It keeps a position: a list of a few hundred numbers, placed so that words used in similar ways land near each other. That list is an embedding, and “near” is the whole trick behind search, recommendation and “find me something like this”. Type a few words or sentences and watch a small model place them, in this tab.
cooking · work in progressEach line becomes 384 numbers. That is a point in a space with 384 directions, which nobody can draw, so the map below keeps the two directions along which your set spreads out the most and drops the rest. Distance on the map is a shadow of the real distance. Click a point to measure from it.
Waiting for the model.
The number people have heard of and never seen is the cosine similarity: 1.0 means the two lists of numbers point the same way, 0 means they have nothing in common, and in practice the model rarely goes below 0. Measured from the highlighted line to every other.
One row per line, one cell per number: warm is positive, cool is negative, pale is near zero. No single cell means anything on its own. Two rows that look alike are close; that is all “meaning” is, in here.
The famous party trick: take king, subtract man, add woman, and look for the nearest word. It is searched against 2,268 common English words placed by the same model file, in advance. The three words you typed are listed but greyed, because in the original 2013 paper they were quietly excluded from the answer, and you should see why.
Claude ran the pairs below through the same model file this page ships, and I read the results. Everything on this page is counted, not judged; the numbers just say what a small embedding model does and does not notice.
Hot and cold are used in the same sentences, about the same things, so they land close. The model learned how words are used, not what they mean, and opposites are used identically. Try the Opposites preset.
I like it and I don't like it share every word but one. I hate it is measurably closer to I like it than the negation is, which is exactly backwards from what a reader would say.
Alice paid Bob and Bob paid Alice come out almost identical, and so do the cat and the mat. The model averages the words in a sentence; averages do not keep order.
An invoice for $1,284 and one for $12,840 are neighbours; 4 pm and 9 pm too. That is a feature for “find the invoice email” and a hazard for anything where the number is the point.
The same sentence in Spanish or German scores near zero against its English original, because this model was trained on English. Multilingual models exist; this one is small and honest about its limits.
Tokyo − Japan + Germany finds Berlin and actor − man + woman finds actress. Paris − France + Italy finds Athens first and Rome second. The trick was discovered on word vectors trained for it; a sentence model does it by accident, and the accident is only sometimes in your favour.
The model is all-MiniLM-L6-v2 from the sentence-transformers project, six layers and about 22 million weights, quantised to 8-bit by the Xenova conversion, Apache 2.0. It reads up to 256 tokens of a line and averages what it finds; longer lines are cut. The map is a two-axis projection recomputed for whatever is in the box, so adding a line can move every point. The word list behind the arithmetic was placed in advance with the same file, stored as 8-bit integers, in Node rather than a browser; a word typed live lands within about 0.01 of its stored copy, and most of that gap is the two runtimes rounding differently, not the 8-bit storage. This is one small model, and the search engine you use has a bigger one that is wrong in some of the same places.