Visual Reconstruction from EEG
Decoding what someone sees from their brain activity, using deep learning.
This project reconstructs images from EEG brain signals. A subject views an image while wearing an EEG cap, and a two-pathway neural network decodes the brain activity into a visual reconstruction, all from less than a second of signal. Built on the THINGS-EEG2 dataset.
How it works
The system uses two complementary pathways to go from raw EEG to an image:
- EEG Recording: Brain activity is captured from 63 electrodes across the scalp using a BrainVision actiCHamp device.
- Semantic Pathway: Maps EEG embeddings into CLIP embedding space, capturing high-level meaning (what the object is).
- Diffusion Prior: Translates the predicted embeddings into image space via Stable Diffusion, producing the final reconstruction.
Results
Demo
Select a sample to watch the visual cortex respond in real time. The waveform shows 8 electrodes over the back of the head, where the brain processes vision. Two moments are marked:
- P100 (~100ms) — The first major response. The brain detects basic visual features: edges, contrast, spatial layout.
- N170 (~170ms) — The brain recognizes what the object is. This is the signal the model relies on most for reconstruction.
Explore the Full Brain Response
The model uses all 63 EEG channels as input. Click any electrode on the head map to see that channel's recorded signal.