Work

Open vision-language models pick whichever frame looks right: the base Qwen3-VL-8B stays at chance (52%) across ten held-out rooms. A small LoRA fine-tune reaches 97.1%, beats all 149 retrieval rules I tested, and depends on the order of the walk.

Inside the fine-tuned model, copying activations from a view taken near the other option moves the decision 0.50 to 0.67 of the way there, so the location is carried and used. Its single units still fall short of a place code. And units that pass rodent place-cell criteria turn up in raw pixels 72% of the time, so on their own they mean little.

BIT is a brain-to-text decoder that reads attempted speech from intracortical recordings: a transformer encoder turns neural activity into tokens, and a language model turns the tokens into a sentence. It trains its encoder on phoneme labels first, and reports that skipping that stage fails.

The goal here is diffusion decoding. A diffusion language model sees the whole trial's neural activity before it writes anything, so it could fill in the confident words first and settle the uncertain ones from both sides, where a left-to-right decoder commits word by word. Skipping the phoneme stage is in service of that: I'm testing whether self-supervised pretraining, plus contrastive alignment to the language model's own representations, can connect the encoder to the language model directly.

Every run is scored by free generation on the Brain-to-Text '25 data. Without a phoneme stage, block-diffusion training cuts word error rate by about ten points. With a phoneme-trained encoder the two objectives tie, and contrastive alignment has not replaced the phoneme stage so far.

A reproduction of the isolated-word decoding analysis in Kunz et al. 2025 (Cell) from the public Dryad release. The three paper values I checked against all fall inside my confidence intervals.

The extension trains the same decoder on one behavior and tests it once on every other. A decoder trained only on listening decodes inner speech well above chance on several arrays, recovering roughly a third to a half of inner speech's own above-chance decodability. It survives stricter normalization and holds in interleaved blocks, where drift can't line up with behavior.

Tested on the Huth-lab story-listening data, with every study registered before its test scores were read. Ridge regression on raw BOLD beat every self-supervised encoder, and pretrained encoders were no better than random frozen ones.

The gains that replicated came from using the supervision better: dense training targets, and weighting each voxel by its training-only encoding score. For the first subject, test r rose from 0.122 to 0.230.

An offline reproduction of the brain-to-voice decoder of Wairagkar et al. 2025 (Nature) from their public code and data, followed by an attempt to make the same kind of decoder speak inner speech, using the public data of Kunz et al. 2025 (Cell).

The reproduction works. For inner speech, the sentence can be read and the voice cannot be understood: the decoder picks the right sentence out of 50 in 85% of trials (chance 2%), while a speech recognizer still gets 73% of the synthesized words wrong. Inner speech here carries the attempted-speech code at about a third of its strength.

Independent, exploratory work on public datasets: offline only, one session per participant, not peer reviewed.