Human “brain reading”: Bayesian approaches for reconstructing perceptual experiences from functional MRI by Jack L. Gallant from UC Berkeley
27 April 2009 Monday @ Boğaziçi
Kare Blok, EE Lounge
A recent paper from our laboratory demonstrated that functional MRI measurements of hemodynamic brain activity contain far more information than was believed previously (K. N. Kay et al., Identifying natural images from human brain activity. /Nature/, v.452, p.352-355, 2008.) In fact those data suggested that fMRI might provide enough information to permit true noninvasive reconstruction of perceptual experiences: a direct window into visual perception.
This talk will focus on a new Bayesian decoder that can reconstruct natural images seen by an observer from measured brain activity. The decoder combines three elements: a structural encoding model that characterizes signals from early visual areas; a semantic encoding model that characterizes signals from higher visual areas; and appropriate priors that incorporate statistical information about the structural and semantic content of natural scenes. By combining all these elements the decoder produces reconstructions that accurately reflect the distribution, structure and semantic category of the objects contained in the original image. These results help clarify why the brain contains many distinct representations of the visual world, and highlight the important role of prior knowledge in visual perception. This decoding framework might form the basis of practical new brain reading technologies that could reconstruct dynamic perceptual experiences or subjective mental states such as visual imagery and dreams. Such technologies could have wide application in medicine and psychology, as a new means of communication and as brain-computer interfaces.