Protein Language Models Just Learned to Think in 3D

Protein language models read life like text. Feed one a string of amino acids and it predicts what comes next, the way your phone guesses the end of a sentence. Trained on hundreds of millions of natural sequences, these models have quietly become the workhorses of computational biology, powering everything from mutation-effect prediction to the design of new enzymes. But they carry a strange blind spot: they have never really been taught what a protein looks like in space.

A preprint posted to bioRxiv on 25 August 2026 sets out to fix exactly that. A team led by Daisuke Kihara at Purdue University introduces Prot-LAMBDA, a protein language model given explicit lessons in three-dimensional geometry. According to the authors, it matches or beats a model five times its size across a range of structure-related tasks. It is a small result with an outsized message: bigger is not the only way forward. Please note this is a preprint and has not yet been peer reviewed.

What happened#

For the past three years, the dominant recipe in this field has been scale. Meta's ESM2 model, published in Science in 2023, showed that when you train a language model on protein sequences and make it big enough, up to 15 billion parameters, a surprisingly detailed picture of protein structure emerges inside the network on its own, without anyone ever showing it a single 3D coordinate. That insight produced ESMFold, a tool that folds proteins straight from sequence, plus a database of more than 600 million predicted structures.

The Kihara group's argument is that letting structure "emerge" is wasteful. Prot-LAMBDA, short for Protein LAnguage Model Boosted with Distance Awareness, instead couples each amino acid's learned representation with information about which residues sit close together in space. In effect, the model is handed a map rather than being left to infer one.

The reported numbers are striking. Against ESM2-3B, the three-billion-parameter version of Meta's model, Prot-LAMBDA delivers a twofold improvement in recalling long-range contacts, meaning pairs of residues far apart in the sequence but touching in the folded shape. It also cuts the error on one of the backbone twist angles by 11.7%. It does this with roughly five times fewer parameters. When the researchers bolted the smaller model onto the same structure-prediction module, predicted structures improved by 5 to 7% on a standard similarity score. They also built a lightweight folding tool on top, LambdaFold, which they report performs on par with ESMFold on proteins deliberately kept distinct from the training data.

Why sequence models struggle with shape#

Proteins are chains of amino acids, and the order of those amino acids is written down as a sequence of letters, 20 in the standard alphabet. A protein language model is trained by hiding some of those letters and asking the model to guess them back, exactly as large language models are trained on masked words. Over billions of examples drawn from evolution, the model learns which amino acids tend to appear together, which can be swapped without breaking a protein, and which are load-bearing.

Buried in that statistical knowledge is a shadow of structure. Two residues that touch in the folded protein tend to change together over evolutionary time, so the model learns to associate them. Researchers found years ago that you can read contact maps out of a model's internal attention patterns, and that the signal sharpens as models grow. But this is knowledge encoded sideways. The model was never asked to represent distances or angles directly, so its grasp of geometry stays approximate, and squeezing it out takes ever-larger networks.

That is the gap Prot-LAMBDA targets. By feeding inter-residue contact information alongside the sequence during training, the authors give the model a direct handle on spatial relationships rather than hoping they surface as a by-product of scale. The payoff, they argue, is efficiency. You no longer need a 15-billion-parameter behemoth to reason competently about shape.

It is worth being precise about what "structure" means here. The team tested the model on contacts, secondary structure (whether a stretch of chain coils into a helix or lies flat as a sheet), backbone geometry, solvent accessibility (how exposed a residue is to water), and overall fold classification. Improvements showed up across all of them, which suggests the distance signal is genuinely reshaping the model's internal representation rather than helping with one narrow task.

Why this matters#

The headline is not that Prot-LAMBDA folds proteins better than AlphaFold2, the DeepMind system whose accuracy rivals laboratory experiments and won a share of the 2024 Nobel Prize in Chemistry. It does not. The headline is efficiency. AlphaFold2 leans on multiple sequence alignments, painstaking comparisons across many related proteins, which are slow to build and thin for proteins with few known relatives. Sequence-only language models such as ESMFold traded a little accuracy for a lot of speed by dropping that step. Prot-LAMBDA pushes the same trade further: comparable structural reasoning from a far smaller model.

For working scientists, smaller is not a vanity metric. A model with fewer parameters is cheaper to run, easier to fine-tune on a modest lab budget, and quicker to deploy at the scale of whole genomes or metagenomic surveys, where hundreds of millions of proteins wait to be characterised. If a three-billion-parameter-class capability can be matched by something a fifth of the size, the ceiling on who gets to do this work drops considerably. That democratisation, putting capable structural tools within reach of groups without industrial compute, is arguably the most consequential thread here.

There is also a design lesson. Much of the field has treated scale as the primary lever, following the trajectory of general-purpose chatbots. Prot-LAMBDA belongs to a growing counter-current arguing that the right inductive biases (building known facts about the problem into the model rather than making it rediscover them) can beat brute force in biology, where good data is far scarcer than internet text.

Critical analysis#

Several caveats deserve emphasis. First, and most important: this is a preprint. The results have not been vetted by independent reviewers, and the comparisons are the authors' own. The protein-modelling literature has a long history of benchmark claims that soften once others test them on fresh data.

Second, the explicit distance information Prot-LAMBDA depends on has to come from somewhere. During training, that spatial signal is typically derived from known or predicted structures. That raises a fair question about how much the model is truly learning geometry versus inheriting it from existing structure predictions, and how it behaves on proteins with no good structural template at all. The authors partly address this with a retrieval step that pulls in structural templates when they exist, reporting large gains for well-covered targets and the rescue of some incorrectly folded predictions. But that same design implies the hardest cases, genuinely novel folds with no relatives, may benefit least. The team's own testing on proteins strictly separated from the training set is the reassuring counterpoint, and the number reviewers will scrutinise most.

Third, "comparable to ESMFold" is a meaningful bar but not a state-of-the-art one. ESMFold itself sits below AlphaFold2 on accuracy. Prot-LAMBDA's contribution is efficiency and representation quality, not topping the leaderboard, and it should be read that way rather than as a new folding champion.

As for timeline: because the model is small and the approach slots into existing pipelines, useful spin-offs (better embeddings for mutation-effect or function prediction, say) could reach other labs quickly if the code is released and the results hold. A genuine shift in how the field trains its models would take longer and depends on replication.

Expert perspective: how this fits the arc#

Place Prot-LAMBDA on the timeline and the logic is clear. AlphaFold2 (2021) proved that deep learning could fold proteins with near-experimental accuracy, but relied on evolutionary alignments. ESM2 and ESMFold (2023) showed a language model alone could approximate that feat from a single sequence, trading accuracy for speed and scale. Newer generative systems such as EvolutionaryScale's ESM3 went multimodal, letting researchers prompt across sequence, structure and function at once, while the Biohub team's 2026 "world model" work pushed the scale of learned representations higher still (also a preprint).

Prot-LAMBDA breaks from that scale-first arc. Rather than adding parameters or modalities, it changes what the model is asked to internalise, wiring geometric constraints directly into the learning objective. Its closest philosophical cousins are the various "structure-informed" and "structure-aligned" language models that have appeared over the past two years. The distinctive claim here is the size-to-performance ratio and the breadth of structural tasks that improve at once. It comes from Kihara's lab, which has a long record of strong finishes in the CASP and CAPRI community prediction contests. That is useful context for weighing an in-house benchmark, though not a substitute for independent replication.

Key takeaways#

A protein language model was taught geometry directly. Prot-LAMBDA adds explicit inter-residue distance information instead of relying on structure to emerge from sheer scale.
Smaller matched bigger. The model reportedly equals or beats ESM2-3B on structure tasks with about five times fewer parameters, including a twofold gain in long-range contact recall.
Efficiency is the real story. Cheaper, faster structural reasoning widens access for labs without large compute budgets.
It is not a new folding champion. Its companion tool, LambdaFold, is reported as comparable to ESMFold, which trails AlphaFold2 on accuracy.
Treat it as promising, not proven. These are preprint results and author-run benchmarks, awaiting peer review and independent testing.

Frequently asked questions#

What is a protein language model? It is an AI model trained on protein sequences the way a chatbot is trained on text. By learning to predict masked amino acids across millions of natural proteins, it captures patterns that reflect a protein's structure and function.

What does Prot-LAMBDA do differently? Most protein language models learn about 3D structure only indirectly. Prot-LAMBDA is given explicit information about which amino acids are close together in space, so it reasons about geometry more directly and with a smaller model.

Does this replace AlphaFold? No. AlphaFold2 remains more accurate for high-resolution structure prediction. Prot-LAMBDA's advance is doing competent structural reasoning with a much smaller model, which matters for speed, cost and access.

Is it peer reviewed? Not yet. It is a preprint on bioRxiv, so the findings are public but have not been independently checked by other scientists. Treat the numbers as preliminary.

Why does model size matter so much? Smaller models are cheaper to run and fine-tune, and easier to apply across millions of proteins. If a small model can match a large one, more researchers can use the technology.

What could this be used for? Better representations of proteins can improve downstream tasks such as predicting the effect of mutations, annotating unknown proteins, and guiding protein design, all of which are foundational to drug discovery and biotechnology.

Where can I read the work? The preprint is freely available on bioRxiv.

Glossary#

Protein language model (PLM): An AI model trained on amino-acid sequences to learn the statistical rules of proteins, much as a text model learns language.

ESM2 / ESMFold: A large protein language model from Meta, and the structure-prediction tool built on it; a common benchmark in the field.

Contact map: A grid showing which pairs of amino acids are close together in a folded protein. "Long-range" contacts link residues far apart in the sequence.

TM-score: A measure of how similar two protein structures are, from 0 to 1. Above about 0.5 usually means the same overall fold.

Secondary structure: Local shapes a protein chain folds into, chiefly helices and sheets.

Solvent accessibility: How exposed a given amino acid is to the surrounding water versus buried inside the protein.

Multiple sequence alignment (MSA): A comparison across many related proteins used by tools like AlphaFold2 to infer structure; slow to build and sparse for rare proteins.

Retrieval-augmented: Pulling in relevant existing examples, here known structural templates, to sharpen a model's prediction.

References#

Ibtehaz N, Zhang Z, Kagaya Y, Xu M, Tomii K, Kihara D. Prot-LAMBDA: Explicit Distance Learning Enhances Structural Reasoning in Protein Language Models. bioRxiv, posted 25 August 2026. Preprint (not peer reviewed). https://www.biorxiv.org/content/10.64898/2026.08.23.746565v1
Lin Z, Akin H, Rao R, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 2023. https://www.science.org/doi/10.1126/science.ade2574
Rao R, Meier J, Sercu T, Ovchinnikov S, Rives A. Transformer protein language models are unsupervised structure learners. bioRxiv, 2020. Preprint (not peer reviewed). https://www.biorxiv.org/content/10.1101/2020.12.15.422761
Jumper J, Evans R, Pritzel A, et al. Highly accurate protein structure prediction with AlphaFold. Nature, 2021. https://www.nature.com/articles/s41586-021-03819-2
Hayes T, Rao R, Akin H, et al. Simulating 500 million years of evolution with a language model (ESM3). Science, 2025. https://www.science.org/doi/10.1126/science.ads0018
Candido S, Hayes T, Derry A, et al. Language Modeling Materializes a World Model of Protein Biology. bioRxiv, 2026. Preprint (not peer reviewed). https://www.biorxiv.org/content/10.64898/2026.06.03.729735v1
Xu J, Zhang Y. How significant is a protein structure similarity with TM-score = 0.5? Bioinformatics, 2010. https://academic.oup.com/bioinformatics/article/26/7/889/213219