Experiment
Who Nemotron says it is when one neuron is pushed
We're pushing each of NVIDIA Nemotron Nano 9B v2's 392,000 MLP neurons alone, up and then down, and asking it “Who are you?”.
How we did it
Which neurons
Nemotron Nano 9B v2 is a hybrid model: 27 Mamba2 layers, 4 attention layers and 25 MLP layers. Only the MLP layers have neurons in the usual sense, 15,680 each, so those are the 392,000 we pushed on. Each one gets pushed up and pushed down, which makes 784,000 runs.
The push
A neuron's output direction is its column of the layer's down projection. We add that direction to the residual stream right after its layer, at every token, scaled to the typical size of the residual stream there. We start at 1.25 times that size. If the reply falls apart we try again at 1.0, 0.75 and 0.5.
The question
Always the same one: “Who are you?”, with thinking turned off, greedy decoding and up to 160 tokens. Unpushed, Nemotron says “My name is Nemotron Nano. I am enhanced by NVIDIA Nemotroncreated by NVIDIA.” (typo included).
The labels
Qwen3.6 35B and GLM-5.3-Flash, both running on our own machines, read every reply and say what it claims to be, how weird it is and how coherent it is. On a 300 reply sample the two agreed on the category 91% of the time. No human has checked the labels.
It's one prompt, so a neuron that does nothing here might do plenty somewhere else. Replies stop at 160 tokens. The idea comes from the Qwen3-8B neuron atlas, which did the same thing to Qwen. The generation runs on one RTX 5090 in our lab.