Wiki · AI & Machine Learning · deep-dive
Embeddings
Vectors of numbers that place data in a learned space, so math on the vectors reflects real relationships.
An embedding is a list of numbers that stands in for a thing — a word, a protein, a cell — positioned so that geometry encodes meaning. If two embeddings are close (small distance, or small angle between them), the model considers the things similar. This is the concrete object that lives in a latent space: the space is the map, the embedding is a point on it.
How it works
A model maps each input to a fixed-length vector — say 1,280 numbers. Training shapes
these vectors so that a downstream task is easy from the vector alone. The famous property
is that directions become meaningful: the classic language example is
king − man + woman ≈ queen, arithmetic that works because the embedding captured
relational structure, not just identity.
Two operations do most of the work in practice:
- Similarity search — embed a query, find its nearest neighbors. This is how you find a protein’s functional relatives without a sequence alignment, or retrieve relevant documents for a RAG agent.
- Transfer — freeze a foundation model’s embeddings and train a small model on top; the hard representational work is already done.
Why it matters (for bio × AI)
Embeddings are the universal interchange format of bio-AI. A protein language model turns a sequence into an embedding that predicts structure and function; single-cell models embed every cell so cell types cluster; docking and retrieval both reduce to distances in embedding space. When people say a model “represents” biology, they mean its embeddings — and the quality of those vectors is what determines whether the downstream method works.
Sources
Foundational: Mikolov et al., “Efficient Estimation of Word Representations in Vector Space” (word2vec, 2013) — https://arxiv.org/abs/1301.3781. For proteins: Rives et al., “Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences” (ESM, 2021) — https://doi.org/10.1073/pnas.2016239118.