In case you missed it, mind reading is no longer science fiction. In fact, it stopped being science fiction twenty-four years ago with the publication of a landmark paper in Science by Haxby et al., (2001). Titled Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex, it ushered in a paradigm shift by enabling researchers to extract and interpret hidden layers of information in neuroimaging data. Today, this revolutionary method is a staple of the cognitive neuroscientist’s toolbox, allowing to reliably detect shifts in mental states based on distributed patterns of neural activity. In this instalment of ZAGROSCIENCE, we will be exploring how this technique actually works and what it entails for the future of humanity.

Early in my PhD studies when my supervisor and I were discussing how to analyze the task-based functional magnetic resonance imaging (task-fMRI) data we had been acquiring for my memory project, he mentioned a statistical approach with a fancy name, multivariate pattern analysis (MVPA). Our goal had been to determine the behaviour of the brain during different phases of memory processing, including information encoding/retrieval, correct/false recall, and response modulation by cognitive load. While I never ended up implementing MVPA—the course of a PhD is marked by sharp angle turns and unexpected twists—I came to understand just how useful and powerful it really is in deciphering the functional encryption of the mind.

Haxby et al., (2001) was the first study to tap into the information-rich well of patterns of activity across multiple voxels. Specifically, it investigated how the ventral temporal cortex (VTC) can represent different categories of stimuli through configurational changes in its functional topography. The researchers hypothesized that each categorical representation would manifest as a distinct pattern of response in collections of voxels distributed across the VTC. They scanned six participants as they viewed faces, cats, houses, chairs, scissors, shoes, bottles, and nonsense pictures, splitting individual datasets into even and odd runs. To predict the category of the stimulus being viewed based on underlying VTC neural activity, they assessed the similarity of multi-voxel responses by comparing within- and between-category correlations (i.e., the association between faces on even and odd runs vs. the association between faces on even runs and cats on odd runs). Based on these pairwise comparisons, the authors were able to correctly identify the nature of the images with a 96% accuracy, speaking to the distinct and identifiable functional representation that each category evoked in the VTC. What this experiment showed is that by reverse-engineering how different cognitive states are functionally codified by the brain, we can predict what the contents of someone’s mind are by working out how well their underlying neural dynamics fit with pre-established functional templates.

To understand how MVPA works, we must first take a look at the analytical technique it is meant to complement—if not, supersede—that is, mass univariate analysis (MUVA). In a typical MUVA, brain activation maps are constructed by contrasting two or more experimental conditions using preprocessed task-fMRI data, which are functional time series that have been expunged of artifacts to increase signal-to-noise ratio (SNR). One could, for example, ascertain which voxels are more active during trials with faces compared to those with cats by running an independent statistical test for each voxel in individual subjects. This process, known as a 1^st level—or subject—analysis, examines task-induced cortical activations in single brain scans. To conduct a 2^nd level—or group—analysis, parameter estimates derived from these initial analyses can be compiled by projecting them onto a shared volumetric template of the brain, the most common one being MNI152 space, which was developed in the early 1990s at my alma mater, the Montreal Neurological Institute—hence its abbreviated name. Resulting group activation maps are then statistically thresholded to correct for probabilistic errors that accrue when performing numerous independent tests, otherwise known as multiple comparisons.