Applied AI / Forward Deployed Engineer · LLM Foundations · Lesson 2 of 5
Embeddings & vector space
An embedding is a vector (a list of numbers, e.g. 1536-dim) that represents the meaning of a piece of text. Texts with similar meaning land close together in this high-dimensional space; unrelated texts land far apart. "dog" and "puppy" are near; "dog" and "tax law" are far.
- You compute embeddings with an embedding model (a separate, cheaper model from the chat LLM).
- Similarity is measured with cosine similarity (angle between vectors) — 1 = identical direction, 0 = unrelated.
- Embeddings are the foundation of semantic search and RAG: embed a query, find the nearest document chunks by vector distance.
AnalogyThink of embeddings as GPS coordinates for meaning. Every sentence gets a location; "nearby" means "similar in meaning." Retrieval = "find the documents closest to where my question lands."
Interview trapDon't confuse embeddings (meaning vectors, used for search/RAG) with fine-tuning (changing the model's weights). And embeddings are not the same as tokens — tokens are input units; embeddings are learned vector representations.
◆ Lock it in
- An embedding = a vector encoding meaning; similar meaning → nearby vectors.
- Cosine similarity measures closeness; it powers semantic search and RAG.
- Embedding models are separate and cheaper than chat models.
Feynman drill — say it out loudExplain how you'd find the most relevant paragraph to a user's question using embeddings — no code.
Step 1 rate your confidence · Step 2 pick your answer
Two sentences have an embedding cosine similarity of 0.92. What does that imply?
Step 1 — how sure are you?