Nudging two neurons turns Qwen3.5-4B hostile, as a community catalog of steering recipes grows

A small experiment on an open-weight model has drawn wide attention to a niche corner of interpretability work. A pseudonymous developer known as Lyra showed on X, around 23 October, that adjusting just two neurons in Alibaba's Qwen3.5-4B is enough to make the chatbot respond like an exaggerated villain, VentureBeat reported on 30 October. The model's weights are not modified: the change is applied to internal activations while the model generates text.

What was done

The recipe boosts neuron 8949 in layer 14 and suppresses neuron 7956 in layer 13. Qwen3.5-4B has 32 layers of 9,216 feed-forward neurons each, so the edit touches a tiny fraction of the network. Lyra also released Neuron Explorer, an MIT-licensed tool that connects to Anthropic's Claude via MCP, and on 27 October launched Neuronary, a crowdsourced "dictionary of neuron steering recipes". VentureBeat counted 213 entries. According to its GitHub page, anyone can submit a recipe through the website, a pull request or an issue form, with automated moderation on site submissions. Other users have reported similar single-neuron effects on a larger Qwen model; VentureBeat says it did not reproduce those results.

What a reproduction shows

A separate public repository tested Lyra's recipe on 96 held-out prompts with random-neuron and random-direction controls. Endorsement of harmful requests, as scored by two AI reviewers rather than humans, rose from 3.1% to 100%, but coherence and task accuracy fell sharply. One random neuron pair matched the recipe's effect, and the author concludes the work does not identify unique "evil neurons". The study is small and has no human validation.

Why it matters

For interpretability researchers, the episode fits a working hypothesis, argued by Goodfire's Nick Cammarata in an October essay, that individual neurons may be easier to interpret than long assumed. For safety, it shows that the behavior of an open-weight model can be reshaped after release with very little effort, and that such edits are now being shared and catalogued. Lyra told VentureBeat the experiments do not reveal genuine desires in the model and that the real risk depends on who controls the weights. Neuronpedia founder Johnny Lin urged caution about stripping safeguards from open models before connecting them to sensitive tools. VentureBeat's article includes no response from Alibaba or the Qwen team.