Writing 6 min
Can a projector turn inner speech into attempted speech?
Inner speech looks like attempted speech, shifted and quieter. Reversing that with a linear map made the sentence easier to identify and did nothing for the synthesized voice.
Wairagkar et al. 2025 built a decoder that turns attempted speech into a synthesized voice. I reproduced it offline from their public code and data, then tried to make the same kind of decoder speak inner speech: silent, imagined speech, using the public data of Kunz et al. 2025.This is independent, exploratory work on public datasets. It is not peer reviewed, nothing was run live with a participant, and most results come from one recording session.
One session in the Kunz data makes a clean test possible. Participant T15 attempts a set of sentences and imagines the same ones, in paired blocks of 50 on the same day. No drift between days is involved.
The idea
In a PCA of that session, inner speech sits apart from attempted speech in two ways. It is lower in magnitude, and it is offset along what looks like a motor-intent direction.
So what if a projector reversed that? Shift and amplify inner speech until it looks like attempted speech, then hand it to a decoder that already works on attempted speech.
The projector’s idea, in three principal components. Inner speech (red) sits offset from attempted speech and is smaller. The projector shifts it to the attempted mean, then scales it up until the two clouds share a centre and a spread. The points are made up, and the size difference is drawn larger than it turned out to be.
I tested four linear maps and one control. Each is fitted on three block pairs and applied to the fourth, so the test sentences and their attempted partners are never used.
- Gain. Per channel, match the mean and spread of inner speech to attempted speech.
- Projector. Within the top 20 principal components, shift inner speech to the attempted mean and scale it up.
- Paired. Align each imagined trial in time to the attempted trial of the same sentence, then rotate and re-scale to match it.
- Paired, shuffled. The same, with each imagined trial paired to a different sentence. This is the control for the pairing.
- Learned. A new input layer trained on inner speech from the sentence text alone. It is a projector too, but a supervised one.
The first test looked promising
I put converted inner speech through the first-generation decoder and asked how well each decode matched the decoder’s own output for the attempted version of the same sentence, ranked among all 50.
| Transform | Own sentence’s rank of 50 | Top-1 |
|---|---|---|
| None | 21.84 | 7.0% |
| Gain | 15.95 | 14.0% |
| Projector | 19.04 | 11.5% |
| Paired | 14.81 | 10.5% |
| Paired, shuffled | 16.62 | 8.0% |
Chance is a rank of 25.5 and a top-1 of 2%. Every transform helped. Unconverted inner speech decoded as near silence, and every conversion made the decoder speak.
The PCA picture held only partly. The difference between the attempted and imagined means does lie 97 to 98% inside the top 20 components. The magnitude difference is small, a factor of 1.06 to 1.19, because the preprocessing already rescales each channel.
Then it didn’t replicate
That score compared the decoder’s output with its own output. Scored against synthesized speech of the actual sentences, no transform helped. None of the changes was significant, on the paired sentences or on a block of new ones, and word error rate went up.
Converting inner speech made the two decodes more alike, plausibly in timing and rhythm. It did not bring the output closer to the sentence’s sound. Converted inner speech follows attempted speech’s beat, not its words.
The second test, judged by the voice
Next I rebuilt the lab’s second-generation decoder, which has a phoneme output and a voice output and can adapt from text alone. I put the projection in front of it. This time the rule was written down first: a transform works if a speech recognizer’s word error rate on the synthesized inner-speech voice drops by at least five points.
| Transform | Sentence top-1 of 50 | Phoneme error | Voice WER | Told apart from attempted |
|---|---|---|---|---|
| None | 31.5% | 77.1% | 90.1% | 0.77 |
| Gain | 48.0% | 71.2% | 91.9% | 0.43 |
| Projector | 45.5% | 72.7% | 91.6% | 0.61 |
| Paired | 55.5% | 70.4% | 96.0% | 0.62 |
| Paired, shuffled | 51.0% | 71.3% | 91.6% | 0.62 |
| Learned | 74.5% | 59.5% | 82.7% | – |
The last column is how well a linear classifier separates the converted inner speech from attempted speech on held-out blocks, where 0.5 means it can’t.
- No geometric transform helps the voice. None meets the rule, and the paired one makes it worse.
- All of them help identification. The sentence is picked out of 50 more often. Most of that gain is generic: the shuffled pairing gets nearly as much.
- Looking the same isn’t enough. After the per-channel gain, a classifier can no longer tell inner speech from attempted speech. The voice still doesn’t improve.
- Only the supervised map helps both. A layer trained from the sentence text raises identification to 74.5% and lowers the voice’s word error rate by about six points per trial.
Matching the statistics of inner speech to attempted speech sharpens the phoneme evidence enough to pick the sentence. It does not give the voice output the input it needs.
Where it ended up
The premise was partly right. Templates built from attempted speech recognize T15’s inner-speech words exactly as well as templates built from inner speech itself: 14.0% against 14.0%, where attempted on attempted gives 34.3%. Inner speech here is the attempted-speech code, weaker and noisier, at about a third of its strength.
What a linear map can’t do is put back what is missing. Even when trained on inner speech directly, the decoder’s phoneme loss stays about seven times higher than on attempted speech.
The final recipe adapts the decoder on inner speech from text, and trains its first layer too.
| T15, same day | Sentence top-1 of 50 | Phoneme error | Voice WER |
|---|---|---|---|
| Inner speech, final recipe | 85.2% | 45.1% | 73.3% |
| Attempted speech | 100% | 10.9% | 31.9% |
The sentence can be read from inner speech. The voice still can’t be understood. Fixes aimed at the voice itself gave small gains at best, and none on the final recipe. The limit looks like the signal, not the voice model.
Listen
Numbers only say so much. Here are three of the 200 inner-speech trials from the final recipe, each next to the decoder’s voice for the same sentence when it was attempted.
I chose them by rule, not by ear: the 200 trials sorted by word error rate, then the first, the hundredth and the last.
Best of 200
It is right.
- Attempted The recognizer heard: “It is right.”
- Inner The recognizer heard: “It is right.”
Median
You have no computer.
- Attempted The recognizer heard: “They have now come here.”
- Inner The recognizer heard: “You are late.”
Worst of 200
Please tell my family I am coming.
- Attempted The recognizer heard: “What do you like to marry a half cousin?”
- Inner The recognizer heard: “This dinner, my family, a-a-a-al-al-al-al-al…”
Under each clip is what the speech recognizer heard. Before judging them:
- The voice is synthetic. The decoder was trained toward a text-to-speech voice. It is not the participant’s own voice.
- The sentences were cued from a fixed set built on 50 words. Nothing here is something the participant chose to think.
- Everything was decoded offline from the public recordings, and the audio is not cleaned up.
- Attempted speech misses too. In the middle pair the attempted decode is wrong as well. Across the day, about a third of attempted words come out wrong.
Limits
- Offline only. Nothing was run in closed loop.
- Closed sets. Identification is among 50 known sentences from a 50-word vocabulary. It is not open decoding.
- One session. T15’s inner speech is one day.
- The same 200 test trials throughout. Later choices were made after seeing earlier results on them, so the final numbers are somewhat optimistic.
- A recognizer, not listeners. Word error rate comes from Whisper, which has a language prior. No listening test was run.
- Inner speech may include faint attempted movement. The release has no audio or muscle recordings to rule that out.
Every experiment, including the ones that failed, is in the repo.