One concept space for two vision models: they agree on positions inside it

Presence is the easy half of agreement

When people compare two vision models, they usually compare what the models notice. Both networks fire on fur, or on printed text, or on sky, in roughly the same places, so the two are said to have learned similar concepts. That is a coarse form of agreement. It says the models carve the visual world into similar categories, and nothing about how either one organises the variation inside a category.

Fur is not one thing. A patch of fur can be pale or dark, fine or matted, lit or in shadow, at the centre of an animal or at its silhouette. A model that represents fur usefully has to keep those apart, which means a concept is not a switch but a small space, and every patch that belongs to the concept has a position in it.

Whether two models agree on those positions decides what you can build across them. If the coordinate means the same thing in both, a direction found inside one model has a defined counterpart in the other, so a monitor, an edit, or a translation designed against one model has a stated target in the second. If the models agree only on presence, you have a shared vocabulary and no shared grammar, and every downstream use has to be rebuilt per model.

One code, two models, blocks instead of directions

The measurement comes from a study of DinoV2-B, trained on images alone without labels, and SigLIP-B, trained to match images against text. Both models were run over the same 50,000 ImageNet validation images, giving 9,800,000 patch activations per model, paired patch for patch.

Those paired activations were used to fit one shared sparse code with a per-model encoder and a per-model decoder on either side of it. The code has 512 blocks of 16 dimensions. A block is the unit that gets selected: when a block is active on a patch, it does not contribute one number times one fixed direction, it contributes a point in a 16-dimensional subspace. The block says which concept; the point inside it says where in the concept. Because the code is shared and only the encoders and decoders are per-model, a code read out of one model can be decoded back into either model's activation space.

Sparsity is set by how many blocks may be active on a patch at once. The study ran three settings, 8, 16, and 32 active blocks, which is 128, 256, and 512 active dimensions.

The two coordinates land in nearly the same place

The test is direct. Take a block that is active on the same patch in both models, take each model's 16-dimensional coordinate inside that block, and measure the cosine between them. At 16 active blocks that cosine averages 0.6738 over 52.1 million co-firing patches in 479 blocks.

Two reference levels make that number readable. Shuffle the pairing, so a patch's DinoV2 coordinate is compared against a different patch's SigLIP coordinate in the same block, and the average falls to 0.2342: that is what block identity alone buys you, with the patch thrown away. The same comparison run with both coordinates drawn from a single model reaches 0.9300, which stands in for the best agreement this measurement shows. The two models therefore cover 63.2% of the distance between those two levels: aligned well past the floor, a clear majority of the way to the reference, and not close to it.

The effect holds at every sparsity setting and gets weaker as the code gets denser: 72.2% of the gap at 8 active blocks, 63.2% at 16, 54.6% at 32. The block-level bootstrap interval on the observed-minus-shuffled difference excludes zero at all three settings: 0.3345 to 0.3773 at 8 blocks, 0.4179 to 0.4427 at 16, and 0.4217 to 0.4379 at 32. At 16 active blocks, 97.7% of individual blocks beat their own shuffled control after correcting for testing all of them.

One shared concept, seen from both models

Here is a single block to look at rather than average over. Block 51 is one of the strongest shared blocks in the 16-block code: it is active on 8.02% of patches, fires in both models on 786,302 of the same patches, and its cross-model coordinate agreement is 0.7554 against a shuffled value of 0.3923.

The cloud below holds 2,000 patches per model, all of them patches where this block fires, with both models' within-block coordinates projected into the same three-component basis. Every point's colour is its position inside the block, so colour is the quantity under test: if the two models place a patch in the same part of the concept, the DinoV2 point and the SigLIP point for that patch carry the same colour. Turn one model off and the other's cloud keeps the same shape and the same colour layout.

Blocks are multidimensional, and the two models use them differently

A block only earns its 16 dimensions if the patches that activate it actually spread out inside them. They do, partly. Measured as stable rank on each model's own activating patches at 16 active blocks, the median block uses 3.548 dimensions in DinoV2 and 4.294 in SigLIP, out of 16 available. So a block is genuinely more than a single direction, and it is also nowhere near a full 16-dimensional subspace.

The two models differ consistently. Across the 479 blocks powered in both, SigLIP uses more of the block than DinoV2 does, by a paired median of +0.622 dimensions with a 95% bootstrap interval of +0.538 to +0.749, and SigLIP is higher on 75.2% of them. The language-supervised model spreads a concept over slightly more internal degrees of freedom than the image-only one.

Shared blocks are also the normal case, not a selected minority. Of the 512 blocks in the 16-block code, 351 qualify as universal on all three criteria the study applies at once: the block's firing is split between the models rather than dominated by one, the two models fire it on enough of the same patches, and it fires often enough to measure. Only 35 blocks are DinoV2-specific and 7 are SigLIP-specific; the remaining 119 fall between the definitions.

The block structure is not free

Blocks buy the within-concept coordinate at a cost in reconstruction, and the cost is largest exactly where the alignment is strongest. Compared against a plain sparse autoencoder matched on active dimensions, the block featurizer reconstructs the other model's activations worse at the tightest budget: cross-model R² of 0.2123 against 0.2324 at 128 active dimensions. Give both methods more room and the deficit disappears: the two are within 0.0007 of each other at 256 active dimensions and 0.0015 at 512.

The setting with the best cross-model coordinate agreement is therefore also the setting where grouping dimensions into blocks costs the most reconstruction. The study does not resolve which side of that trade is worth taking, and neither does this report.

What this does not show

The measurement is observational throughout. Nothing here intervenes on a block, so the shared coordinate is not shown to drive either model's behaviour; it is shown to be present in both. The obvious next test is causal, and it is not in this result.

Three further limits. The agreement is real but well short of the same-model reference, so a coordinate carried from one model to the other is an approximation, not a translation. The whole result comes from one layer, one pair of models, and one dataset, and nothing here says how it behaves at other depths, on other architectures, or off ImageNet. And what the within-block coordinate means is untouched: the two models agree on where a patch sits inside a concept, without this report establishing what moving along one of those internal dimensions corresponds to visually.