Wiki · Bio × AI Methods · concept
Single-cell foundation model
A transformer pretrained on millions of single cells that turns a cell's gene expression into a reusable representation.
A single-cell foundation model is a transformer pretrained on millions of single-cell gene-expression profiles. It treats each cell like a sentence — the set of genes a cell expresses, and how strongly — and learns to read that sentence the way a language model reads text. The payoff is a reusable representation: hand it a cell and it returns a vector capturing what kind of cell it is, which you can then use for many downstream tasks without training from scratch.
How it works
Each cell is turned into tokens: its expressed genes, tagged with expression level. The model is then pretrained on huge public atlases — tens of millions of cells — using a masked objective: hide some of a cell’s genes and make the model predict them from the rest. To do that well it has to learn which genes travel together and what defines a cell state. Out come two kinds of embeddings: a vector per cell and a vector per gene. You then fine-tune the pretrained model for specific jobs — labelling cell types, predicting how a cell responds to a perturbation, or integrating datasets collected on different machines so batch effects wash out.
Why it matters (for bio × AI)
This is an early step toward a virtual cell: one model that has seen enough biology to reason about a cell it was never explicitly shown. Those cell representations feed perturbation-prediction — guessing what a drug or gene knockout will do before running the experiment. The best-known exemplars are scGPT (Bo Wang’s lab) and Geneformer (from the Ellinor/Theodoris groups). One honest caveat: whether these models genuinely beat much simpler baselines is actively debated, and several careful comparisons have found the gains thinner than the headlines suggest.
Related
See foundation model for the pretrain-then-adapt recipe, embeddings for the cell and gene vectors these models produce, and transformer for the architecture underneath.