What embeddings are, explained by building one

Embeddings are behind search, recommendations, and most of modern AI, and they are usually explained with intimidating diagrams. The core idea is simple and worth building yourself: an embedding turns a thing (a word, a document, a product) into a list of numbers (a vector) so that similar things end up close together in that number space.

Why turn things into vectors

Computers cannot compare meaning directly, but they can compare vectors. If "king" and "queen" are nearby points, and "king" and "banana" are far apart, then "closeness of vectors" becomes a usable stand-in for "similarity of meaning." Once your items are vectors, search and recommendation become geometry: find the nearest points.

Measuring closeness

The standard measure is cosine similarity, the angle between two vectors. Identical direction scores 1, unrelated scores near 0, opposite scores -1.

import math



def cosine(a, b):

    dot = sum(x * y for x, y in zip(a, b))

    na = math.sqrt(sum(x * x for x in a))

    nb = math.sqrt(sum(y * y for y in b))

    return dot / (na * nb)

With just this, you can already build a tiny semantic search: embed your documents, embed the query, and return the documents with the highest cosine similarity.

A first embedding you can build by hand

You do not need a neural network to feel the idea. A simple bag-of-words vector already places similar documents near each other:

def embed(text, vocab):

    counts = {w: 0 for w in vocab}

    for w in text.lower().split():

        if w in counts:

            counts[w] += 1

    return [counts[w] for w in vocab]

Two documents about the same topic share words, so their vectors point in a similar direction, so cosine similarity is high. That is the whole mechanism, in miniature. Real embeddings (word2vec, or the ones inside large language models) learn far richer vectors where direction captures meaning, not just word overlap, but the principle is identical: similar things, nearby vectors.

Why building it matters

Once you have built a vector space and searched it by cosine similarity, the buzzwords resolve: a "vector database" is a store of these vectors with fast nearest-neighbor search; "semantic search" is exactly what you just did; retrieval for AI is embedding your documents and finding the closest ones to a question. You will understand the systems instead of trusting them.

Build it for real

The AI and Deep Learning track builds embeddings from scratch, from counting vectors to learned representations and the attention that powers transformers, all graded in your browser. The first project is free.

Turn things into vectors, and a huge amount of modern AI becomes geometry you can reason about.