In theory, it’s simple. Our input is brain activity, and our target output is what the person was doing at the time – for example, what text the person wrote. Given the brain activity, we want to predict output that is semantically similar to what the person wrote.

To train models that can predict text given brain signals, we must apply the same lesson learned by those predicting text given speech audio, or text given preceding text: the bitter lesson. The lesson roughly states that you should throw more useful compute at your model, and your model will become better than any ingenious algorithm you could've hand-crafted. That means we must scale up our data collection by orders of magnitude beyond what has ever been done in academia.

It’s hard to collect enough data using invasive methods. Few people want a chip in their brain! But non-invasive methods are getting much better. The hardware is improving and getting cheaper, though I apologize for being vague about the particularities of our hardware.

As we're training on more data, the model is predicting text that is more semantically similar to the subject-written text. Yes, there's some irreducible error due to noise, but for most modalities we're not yet in a regime where we're pushing against that. Concretely, the scaling laws are looking good: the cosine similarity of our latent space predictions with the target latent spaces goes up as a straight line with respect to the logarithm of the number of hours of data. We're in the GPT-2 era.

We don’t need perfect decoding to be useful. Your thoughts will be like GPS in a city: a noisy GPS signal isn’t enough to determine your exact location. But combined with a map and a navigation route — equivalently, the LLM and context — it becomes remarkably accurate.

To read up on how we do data collection, check out Conduit's blog post. If you're more of the active learning type, come be a research participant!