Running experiments on a fly brain, in software

There is exactly one animal whose whole brain exists as a model you can run. The fruit fly. Its wiring diagram, 127,400 neurons and every connection between them, was finished by the FlyWire project in 2024. The same year, Shiu and colleagues turned it into a simulation: a simple model of each neuron, wired exactly as in the real fly. You can drive any neuron, silence any neuron, and rerun the experiment as often as you like.

That makes it the one place where we can test our thesis today, years before a mouse map exists. If a complete map lets you predict what a brain does, the fly is where it should show first. So in October 2026 we ran two tests on it, both with the rules frozen before the first run.

The answer key

We collected 259 published experiments from 24 papers, each of the form: drive these neurons, does that neuron respond? 71 said yes, 153 said no change, 35 said the readout was inhibited. 169 of the 259 came from the experiments Shiu's team used to validate their own model. The other 90, from 22 other papers, the model had never been checked against. Those 90 are the ones I care about most.

The score throughout is AUROC: how well a method ranks the experiments that produced a response above the ones that did not. A coin flip scores 0.50. Perfect is 1.00.

Test one: does the detail matter?

A question we get asked constantly: why map every synapse? Why not map which brain regions connect to which, the way an atlas does, for a tiny fraction of the cost?

We tested that. We took the real wiring and scrambled it at four levels of detail, keeping connection strengths, then ran the same model on the same 259 experiments. Here's what happened:

Blur the wiring to the region level and the model drops to chance. Activity still spreads through the brain. It almost never reaches the right neurons. The gap between real and regional wiring is +0.318 with a 95 percent interval of [+0.258, +0.375]. On the 90 independent items it is +0.163 [+0.045, +0.278].

Two honest notes. First, scrambling within cell type cost nothing, so a cell-type-level map was enough for these experiments. That map still requires tracing every neuron, since you cannot know a neuron's type without tracing it, but it is a real finding and we report it. Second, the ceiling is modest. The real model scores only 0.672 on the independent items. It is a crude model. Every comparison here is against that ceiling.

Test two: can an agent do better than the model alone?

The raw simulation gives you one number per experiment. A scientist does more: runs the experiment, silences intermediate neurons to see what carries the effect, checks whether the model is behaving sensibly, then decides. We asked whether AI agents could do that.

Here's how it works:

1. Arm S, the raw simulation. Run the model, read the number. Score 0.831.
2. Arm L, names only. A language model given the names of the neurons and the experiment's design, with no wiring, no model and no tools. This measures how much a model guesses from memory of the literature. Score 0.825.
3. Arm A, the agent lab. The same language model with tools over the wiring and the simulator. It queries connections, drives neurons, silences intermediates, and writes up what it found. Two independent runs per experiment, averaged. Score 0.902.

We set the pass bar before we started: the agent had to beat the raw simulation by +0.05 with an interval excluding zero. It did. +0.071 [+0.023, +0.120]. Each of the two runs passed on its own. A second lab's model, run through its own agent tool on the same experiments, scored 0.912, so this is not one model's quirk. The whole thing cost $101.62 in model calls.

That said, two things qualify the pass.

The first is statistical. With 24 papers, one of which supplies 127 experiments, the interval computed by resampling papers instead of experiments is [-0.006, +0.276]. It touches zero. The pass does not survive clustering by paper, and we said in advance that we would report both.

The second is the one that matters. Look at Arm L again. A model with no map and no simulator scored 0.825 from the names alone, and 0.786 on the 90 independent items, above the raw simulation's 0.672 there. The agent flagged that it relied on prior knowledge on 136 to 150 of the 259 experiments, depending on the run. On the independent items, its gain over the names-only arm is +0.057 with an interval of [-0.008, +0.128]. Most of what looked like reading the map was remembering the paper.

What survives

The cleanest evidence is where the literature helps least: the 169 experiments from Shiu's own screen, which test one obscure cell type after another. There the names-only arm scores 0.866, below the simulation's 0.914, and the agent still beats both: +0.030 [+0.001, +0.072] over the simulation, +0.078 [+0.006, +0.159] over the prior. Running experiments on the model adds something. It is a small something, and it is real.

The agent also caught the model making mistakes. In one set of seven experiments, the simulation fires a readout neuron for every input because activity runs away through local neurons the model signs as excitatory. The agent silenced neurons until it showed the response came from other cells, not the one being driven, and answered no. Correct. It also said it recognized the paper. So that correction is at once a real mechanistic finding and a memory, and we cannot pull them apart on a published experiment.

Which is the real result of this test. On experiments that are already in the literature, you cannot tell map from memory. The only clean test is an experiment nobody has run. We could not stop there, though, because we had to know how much of the gain was the map. So we hid the names and ran it again. That is the next post.

Dan