Jack L. Gallant: 2012 Allen Institute for Brain Science Symposium

Functional magnetic resonance imaging (fMRI) detects the location of functions in the brain better than any other method we have today. While localization is necessary, it is not sufficient for understanding how the brain works. Dr. Gallant suggests the reverse approach – to search for functional maps. That is, he uses brain activity to determine or reconstruct what a subject was looking at. To this end the Gallant lab has constructed the WordNet model, which is able to predict what an observer is seeing from 2,000 nouns and verbs. The process uses brain activity in fMRI to predict from semantic models while an observer watches a video, and the results are remarkably accurate. Dr. Gallant explains how encoding models, decoding models, and functional maps of the brain are all closely related. “Once you have encoding, you get decoding for free,” he proclaims.