What Are Embeddings?: How AI Knows a Cat and a Kitten Are Related ⭐

This is a very simplified version of what an embedding is and is meant for beginners. Felt that I had to put a disclaimer here for the ai bros.

The question embeddings answer

How does a computer "know" that a cat and a kitten are related, that a dog is closer to a cat than to a car, and that a car is closer to a truck than to either a cat or a dog?? Computers don't understand meaning, they only understand numbers. So the trick is turning the meaning of text into numbers that math can work with.

An embedding is simply a long list of numbers that represents the meaning of a piece of text. Modern embeddings often contain hundreds or even thousands of numbers, but you don't need to understand each number individually. Pieces of text with similar meaning end up close together in this mathematical space.

A map of meaning

Imagine every word or phrase being placed on a giant map. Similar meanings end up close together, while unrelated end up far away. A model places words near each other based on how they are used in huge amounts of text, so "cat" and "kitten" end up near each other, "dog" and "puppy" are also close together and nearby to cats too, and something unrelated like a "car" sits in a completely different region.

A practical guide explaining what embeddings are, how they work, and how they're used for semantic search, recommendations, and retrieval.