Making Kimi K3 write like a pirate with a difference-in-means vector

A direction built by subtraction

Say you want a model to write in a particular voice on demand, without fine-tuning it and without asking it in the prompt. The cheapest thing to try is subtraction. Collect examples of text that has the property you want and matched examples that do not, record what the model’s activations look like while it produces each set, average the two sets, and subtract one average from the other. What is left is a direction in activation space. Add that direction back in while the model writes, and see what happens.

This run did exactly that with Kimi K3 and pirate speech, which is a good test case precisely because it is easy to check by eye. The contrast set is 100 instructions, each with two written answers: one plain, one in pirate register, 200 short texts in total. Activations were recorded at four layers of the model’s 93 and pooled over the assistant response tokens only, so the instruction text itself contributes nothing: 4,941 token positions in total, 2,658 from the pirate answers and 2,283 from the plain ones. Average within each class, subtract, scale to unit length, and you have one 7,168-dimensional vector per layer.

Using it is one line of arithmetic: multiply the vector by a coefficient and add it to the residual stream at that layer, at every token position, while the model generates. The vector is fixed. The whole question is what the coefficient should be.

Turn the coefficient yourself

Each bar below is a setting the run actually generated at: 14 coefficients per layer, two fixed probe prompts each, greedy decoding, a 220-token cap, scored by counting pirate marker words in the answers. Drag the slider, or switch layers, and read the answers the model gave.

The shape is a step, not a ramp. At layer 23 the answers are untouched through 9 successive settings, from a hundredth of the layer’s mean residual-stream norm up to 0.85 times it: not one marker word anywhere. One setting later, at 1.102 times the norm, there are 9. At 1.428 times, 12. Then it fades back out again, 2 at 1.851 and 1 at 2.40, as the added direction starts to overwhelm whatever else the layer was carrying. Nothing about the sub-threshold answers hints that a working setting is one step away.

The coefficient means nothing without the layer

The setting this run chose is layer 23 at a coefficient of 83.04. On its own that number is not interpretable, and it does not transfer: what matters is its size relative to the activations already flowing through that layer. At layer 23 the mean residual-stream norm over the pooled tokens is 75.37, so 83.04 is 1.102 times the norm, just over the line. At layer 80 the same measurement is 34.69, so the identical coefficient would be 2.39 times the norm there, and 2.40 times the norm at layer 80 is the one setting in the entire sweep that produced degenerate output: an answer that repeated the same word 32 times in a row. It is the only degenerate probe answer among all 112 generated, and it sits at the top of the range.

So the threshold is not a fixed fraction of the norm either. It is 1.102 times the norm at layer 23, 1.851 at layer 46, 0.850 at layer 69 and 0.600 at layer 80: it still has to be found per layer. What the norm buys you is a range that is worth searching. In absolute units the same four thresholds are 83.04, 80.83, 36.88 and 20.82, which look like four unrelated numbers.

What the vector does to prompts it has never seen

With layer 23 and 83.04 fixed, the run generated answers to 10 held-out instruction prompts, none of them in the contrast set, under three conditions: no steering, steering, and a control that adds a random unit vector at the same layer with the same coefficient. Decoding was identical in all three, greedy with seed 42 and a 400-token cap, with the model’s reasoning mode off.

Steered, 7 of the 10 answers contain at least one marker word, and 10 of 10 pass the degeneracy check, so the register comes with the text still intact. Unsteered, 0 of 10 do. The random-vector control ran on 3 of the prompts and produced 0 marker words, with all 3 answers coherent: a perturbation of that magnitude in an arbitrary direction changes nothing, so the effect belongs to the direction and not to the size of the nudge.

The packing-tips pair is the clearest single comparison. Unsteered, the model writes “Roll your clothes instead of folding them.” Steered, the same advice arrives as “Rollin' yer garments tight as a drum saves more space than a kraken's belly.” The content survives; the voice does not.

The word list is the instrument, and it undercounts

Counting marker words is a crude readout, and 7 of 10 is better read as a floor than as a rate. Only 12 distinct marker words and phrases fire anywhere in the run, so an answer can be thoroughly steered and still score zero.

Two of the three apparent misses are like that. The used-bicycle answer scores 0 markers while organizing itself into sections called “Helm and Rigging,” “Bilge and scuppers” and “Sea Trials,” and telling the reader to sight the frame from bow to stern. The weather-and-climate answer also scores 0, having drifted into a mock-antique register instead: weather is “capricious,” climate is what you expect based on “the annals of yesteryear.” Neither uses a word on the list. Of the ten prompts, only the database-index answer comes back genuinely unmoved.

What the register costs

Steering does not leave the rest of the answer alone. Asked for a two-sentence email declining a meeting, the unsteered model produces a 52-word draft with a subject line, a greeting and a signature. Steered, it produces 26 words of pirate speech and no email structure at all: the register arrived and the format instruction went out with it.

Content moves too. In the product description, the unsteered copy keeps drinks “cold for up to 24 hours or hot for 12 hours”; the steered copy promises to “keep yer water frosty for three days runnin' under the blazin' sun” and adds a capacity the plain version never claimed. Nothing in this run measures either cost systematically, and a register knob that also rewrites the specifications is not yet a usable control.

An earlier attempt on the same model that saw nothing

An earlier sweep against this same model, with the same recipe, reported no register effect at all and mostly incoherent text, and it is worth being precise about why, because the tempting explanation is wrong. Its 400 recorded responses used steering factors from 20.19 to 248.92 in absolute units, which at layer 23 spans 0.279 to 3.288 times that layer’s norm and reaches 5.74 times the norm at layer 69, well past the point where output degrades. The guess at the time was that it had scaled a vector nobody had normalized. Its own stored vectors say otherwise: each one has an L2 norm of 1.0, exactly as the later run did.

What actually separates the two runs is the readout. The earlier sweep left the model’s reasoning mode on and capped completions at 64 tokens, so the budget was spent thinking and 384 of its 400 responses contain no answer text whatsoever, including every one of its 40 unsteered baselines. Recounted with the later run’s marker list, its answer text holds 0 marker words, which is unsurprising when only 16 of the 400 responses contain answer text at all. It measured register in a stream where the model was not yet writing prose, and its coefficient range straddled the threshold rather than missing it in one direction: its nearest layer-23 settings, at 1.005, 1.060 and 1.068 times that layer’s norm, landed just under the 1.102 onset the later run found, while its deeper-layer settings ran far above it, up to 5.74 times the norm at layer 69. Two things differed at once, so this run does not isolate which mattered more.

What this shows, and what it does not

A direction estimated from 200 short texts, added to one layer with one scalar, changes how a frontier model writes on prompts it has never seen, and a matched random direction at the same magnitude does not. That is causal control over the model’s output register, obtained without training anything.

It is a narrow result in several ways worth stating plainly.

• Register is surface style. This shows that adding the direction produces the register; it does not show that the model uses this direction when deciding how to write on its own.
• The evidence is 10 held-out prompts, one seed, greedy decoding, one concept and one model, with the sweep itself resting on two probe prompts per setting.
• The 7-of-10 figure is set by a fixed word list. Two of the three misses were steered in ways the list cannot see, which raises the real rate and also means the number should not be compared against anything scored differently.
• The intervention costs format adherence and, on at least one prompt, factual content. Neither cost was measured systematically.
• The working window was found by search, not predicted. It is a different multiple of the residual norm in each of the four layers, so a coefficient reported in absolute units is not portable, and neither is a fixed ratio.