Multimodal imaging approach to information processing of the parietal lobe during a reaching task with delayed visual feedback

Multimodal imaging provides us a wealth of valid strategies. For example, simultaneous measurement of such as electroencephalography (EEG) and near-infrared spectroscopy (NIRS) can complement each temporal and spatial information which is missed by a single modality. Previous hemodynamic studies have shown the activity of parietal lobe when the conscious experience that “this action is caused by myself” is diminished due to delayed visual feedback. However, the temporal resolution was insufficient to reveal whether the activity was caused by the processing of multisensory inconsistency or the retrospective judgment of agent. By using EEG-NIRS measurement, we demonstrated the hemodynamic response of parietal lobe associated with the error-related EEG signal, but it did not show the correlation with the strength of sense of agency. Our results suggest that the parietal lobe receives online prediction error signals but does not directly represent the conscious experience of movement agency. In addition to this research, I will briefly introduce the ongoing multimodal imaging project which combines transcranial electrical stimulation and EEG. The combination of brain stimulation and imaging techniques has potential to indicate a causal relationship between brain activity and function. Through these studies, I would like to discuss validity and potential of multimodal imaging.

Cortical mapping of visual-semantic– and motor-related activities using electrocorticogram.

Electrocorticogram (ECoG) has high temporal resolution suitable to reveal cortical representation of various activities. In this talk, I will introduce following two studies of cortical mapping using ECoG. (1) Recent studies using functional magnetic resonance imaging revealed content of natural movies could be inferred from brain activity by use of the semantic space of the natural language processing model, called a skip-gram. ECoGs were recorded while participants watched movies consist of natural scenes. Still images were cropped from the movies to be annotated manually and to create vectorised representation of the annotations. By decoding analysis using ECoGs from each cortical region, the vectors could be inferred not only visual areas, but also temporal areas, showing response to the “semantic” aspects of the movies were successfully captured by the ECoGs. (2) Dynamic mode decomposition (DMD) is a method to reveal the spatiotemporal patterns in signals. We applied DMD to the ECoGs recorded while the participants moved their hand once at the timing of execution cue. The acquired spatiotemporal patterns were used to decode the movement type of the hand. The classification accuracies of three movement types was significantly higher than those using power features. Interestingly, the averaged magnitude of the decomposed modes within the high-gamma band peaked around hand motor area with narrower peak compared to powers in the same high-gamma band. It was shown that the DMD is promising approach to evaluate spatiotemporal pattern in the brain activity.

Experience of direct brain perturbation studies in humans at The University of Iowa

Brain is a complex system that is composed of numerous networks ranging from micro- (μm) to macro- (cm) spatial scale.  Better understanding of brain function needs the inference of these network’s property and ultimately knowledge of the causal relationship between neural activity and behavior, the information of importance yet difficult to obtain in humans. Toward this goal, we have been conducting perturbation study in meso- and macro-scale network in epilepsy patients who have intracranial electrodes implanted. Direct electrical stimulation and concurrent intracranial EEG (iEEG) recording, as well as BOLD fMRI (es-fMRI) acquisition have been utilized. Studies with concurrent iEEG recording during Transcranial Magnetic stimulation (TMS-iEEG) have also been underway. Perturbations were applied at three key nodes, Heschl’s gyrus, Amygdala and Dorsolateral prefrontal cortex (DLPFC) that are important in human cognitive pathways (auditory and emotional network) and depression. In this invited talk, I would like to share not only the results on these perturbation studies, specifically on the effective connectivity and property of network configuration, but also my experience of the general clinical-research environment at The University of Iowa.