A cofolding model ranks its own best pose badly

Protein–ligand structure models usually generate a correct pose somewhere among their candidates and then rank it badly. A linear read of the activations already inside the model orders those candidates better than the confidence score the model ships with, and the same direction, pushed into the sampler, changes what gets generated.

Pose selection

A pose is one predicted placement of a small molecule inside a protein's pocket, and it is what a medicinal chemist works from: which chemical group to modify, whether a series is worth pursuing, what to synthesise next. Cofolding models predict that geometry. Given a protein sequence and a small molecule, they fold the two together, emit many candidate poses, and rank them with a confidence score of their own.

Generating a correct pose and recognising which one it is are separate skills. Modern cofolding models are much better at the first. This page is about the second.

The setup: five general-purpose cofolding models, 100 poses per complex, and four benchmarks of protein–ligand complexes with experimentally solved structures (EV-A71 2A protease, 802 complexes; the SARS-CoV-2 Mac1 macrodomain, 349; MERS main protease, 96; the protein–ligand subset of FoldBench, 554, the most chemically diverse of the four). A pose counts as correct only if it sits within 2 ångströms of the experimental structure, reproduces the protein–ligand interface contacts, and passes standard physical-plausibility checks; that rule is stricter than the one used on published cofolding leaderboards, so these numbers do not belong beside those tables. GNINA, an external physics-aware docking program that knows nothing about the models' internals, supplies a second opinion.

The shipped confidence score ranks poorly

Boltz-2, given FoldBench complex 7FWO, human fatty acid binding protein 4 (FABP4) with its bound ligand, generates 100 candidate poses. The one its own confidence ranks first sits 5.067 ångströms from the experimental structure. The one it ranks hundredth of a hundred sits 0.986 ångströms away. GNINA, scoring the same 100 poses from outside, puts that pose first.

Rotating view of FoldBench complex 7FWO, human fatty acid binding protein 4, alternating between two
  predicted poses of the same molecule against the experimental structure: the pose Boltz-2
  ranks first, 5.07 ångströms away, and the pose GNINA ranks first, 0.99 ångströms away.
FoldBench complex 7FWO, human fatty acid binding protein 4 (FABP4). The same pocket, the same 100 candidates, two picks. Pale grey is the experimental structure. Pink is the pose Boltz-2's own confidence ranks first (rank 1 of 100, 5.07 Å away); orange is the pose GNINA ranks first (rank 100 of 100 in Boltz-2's ordering, 0.99 Å away). The view turns steadily while the molecule alternates between the two picks; the two are never drawn at the same time.

That extreme case, the model's own first pick more than 5 ångströms out while GNINA's first pick is correct, occurs 42 times in 8,707 model–complex pairs. The typical case is milder and costs search. On MERS main protease with OpenDDE, reading the model's own confidence order you have to open 59 of the 100 poses to reach 80 per cent of what the pool can deliver; reading GNINA's order, 21.

A linear probe on the model's own activations

GNINA is a separate program that has to be installed, configured and run. The cofolding model already contains the signal. Each candidate pose leaves a trace of 1,536 numbers in the model's confidence module, and a linear probe on that trace, one weight per number and a single dot product, ranks poses better than the model's own confidence head. No weights are touched and nothing is regenerated: the probe reads activations that were already computed and discarded.

Averaged over the five general-purpose models, out of 25 poses selected from 100, the probe beats the model's own confidence on all four benchmarks and beats GNINA on three of them. Across the selectors, correct poses per 100 span 8.80 to 13.72 on EV-A71, 49.84 to 53.33 on Mac1, and 18.99 to 25.48 on MERS. FoldBench is the exception: 21.65 to 21.93, against an unranked floor of 21.37, so nothing separates there.

Correct poses per 100 selected, by distance threshold

The four selectors on the same 25-of-100 budget, with the interface-contact and physical checks fixed and only the distance bar moving. The curves move together, so the probe's lead does not come from where the 2 Å line was drawn.

Benchmark

counts as correct within 2.00 Å

The more demanding comparison is against picking at random from the same 100 poses, a floor that can be computed exactly for a pool of 100. On EV-A71, the probe's single top-ranked pose is correct 4.13 times as often as a pose drawn at random, in all six checkpoints tested (the five cofolding models tested plus a version of OpenFold3 fine-tuned on that target). On MERS it is 2.65 times, in four of six; on Mac1, 1.22 times, in six of six; on FoldBench, 1.04 times, in one of six.

On EV-A71, for one of the five cofolding models tested, taking a single pose gives 17.63 per cent of complexes solved by random selection, 19.83 per cent by the model's own confidence, and 35.91 per cent by the probe. For two of the five general-purpose models the model's own confidence falls below what picking at random gives: OpenDDE 1.25 against 1.60 for random selection, base OpenFold3 1.37 against 3.25. For base OpenFold3 that shortfall survives correction for multiple comparisons (q = 0.0019, where q is the false-discovery-rate-adjusted p-value and 0.05 is the conventional bar).

Rotating view of Mac1 complex 7HQ2 showing the experimental
  structure in pale grey, OpenDDE's own top-ranked pose in pink 6.48 ångströms away, and
  the probe's top-ranked pose in green 0.44 ångströms away.
One complex, Mac1 7HQ2, with OpenDDE. Pale grey is the experimental structure. Pink is the model's own first choice, 6.48 Å out, in a different part of the site. Green is the probe's first choice, 0.44 Å out, a pose the model itself had ranked 21st of 100.

Transfer to an unseen benchmark

The probe above is fitted on complexes from the same benchmark it is then scored on, and every complex it is scored on was held out when it was fitted. Its ranking therefore holds up on ligands it never saw during fitting, provided some solved structures exist for that target. The harder test is an entirely new target, and that is what the frozen cross-benchmark probe measures: fit it on some benchmarks, freeze it, and apply it to a benchmark it has never seen.

Boltz-2+2.579-1.042-0.1805Chai-1+4.585+1.042+0.9025OpenDDE+1.146+14.58+1.805OpenFold3 base+2.579+10.42+0.722Mac1MERS MproFoldBenchHeld-out benchmarkCofolder+150-15Δ percentage points vs the model's built-in confidence
An internal probe trained on other benchmarks and applied frozen, measured against each cofolder's own confidence ranking, in percentage points of complexes solved. Warm is a gain, cool a loss. The best cell, OpenDDE carried to MERS, gains 14.58 percentage points on the model's own confidence.

Measured against each cofolder's own confidence ranking, the probe trained on other benchmarks and applied without adjustment solves more complexes in places: up to 14.58 percentage points more, for OpenDDE on MERS.

Steering the sampler

A probe is a direction in the model's activation space, so its gradient can be added to the diffusion sampler during generation, nudging the model toward poses the probe scores highly.

010203040506011.25No guidance46.94Probe guidance 0.0557.12Probe guidance 0.151.38Random direction 0.054.17xCorrect poses per 100 generatedhigher is better ↑
OpenDDE generating poses for 16 MERS complexes, 100 poses per complex per arm. Probe guidance at strength 0.05 takes correct poses from 11.25 to 46.94 per 100, a factor of 4.17, and strength 0.15 reaches 57.12. A random direction of the same size, applied the same way, produces 1.38.

Correct poses per 100 generated rise from 11.25 to 46.94, a factor of 4.17. The effect depends on which direction is applied, not on perturbing the sampler: a random direction of the same magnitude, which moves the poses by an indistinguishable amount, yields 1.38 per 100 and produces 22 physically valid poses out of 1,600 where the probe direction produces 775.

The gain is physical plausibility, not better placement. The closest pose to the experimental structure got farther away in 14 of the 16 complexes, and the set of complexes for which a correct pose existed anywhere in the pool did not change at all: the change in coverage is exactly zero. The guiding direction was fitted on the same 16 complexes it steered, so this shows the mechanism works, not that it would work on a new target.

Interpretation

The signal that orders poses was already inside the network, computed on every forward pass and discarded by the score the model reports. A single dot product over those activations recovers it, and the same direction pushed into generation changes what the model produces.

More generally, when a model is good at a task and bad at knowing when it has succeeded, its own report is not the best available estimate of its state.

Discussion

The probe generalises to ligands it never saw during fitting: every complex it is scored on was held out, so on a target with some solved structures it ranks new ligands' poses better than the model's own confidence does. What it needs is those solved structures, and an entirely different target is the harder test that the frozen cross-benchmark probe measures.

Steering buys physical plausibility rather than better placement: the closest pose receded in 14 of the 16 complexes and the set of complexes solved did not change, and the guiding direction was fitted on the same 16 complexes it steered, so the result shows the mechanism works and not that it generalises. One further caveat on reading the numbers: the correctness rule used here is stricter than the one behind published cofolding leaderboards, so these values should not be placed beside those tables.