Writing
Notes on decoding, interpretability, and results that didn't go my way.
2026
- A diffusion language model as a brain-to-text decoder Training the language model in a speech decoder by block diffusion, not left to right, cut word error rate by about ten points. The gain disappeared once the encoder had been trained on phonemes.
- Aligning a neural encoder to a language model, without phonemes Contrastive alignment taught a speech encoder to pick the right sentence out of 704 a third of the time, with no phoneme labels. It did not make the encoder a better starting point for decoding.
- Can a projector turn inner speech into attempted speech? Inner speech looks like attempted speech, shifted and quieter. Reversing that with a linear map made the sentence easier to identify and did nothing for the synthesized voice.
- Place cells are easy to find. That's the problem. Units that pass the rodent place-cell test turn up in raw pixels and in random networks. To find out whether a vision-language model knows where it is, I had to change its activations and watch the answer move.