Machine learning, meet human emotions: How to help a computer monitor your mental state

Researchers from Skoltech, INRIA and the RIKEN Advanced Intelligence Project have considered several state-of-the-art machine learning algorithms for the challenging tasks of determining the mental workload and affective states of a human brain. Their software can help design smarter brain-computer interfaces for applications in medicine and beyond. The paper was published in the

IEEE Systems, Man, and Cybernetics Magazine

.

A brain-computer interface, or BCI, is a link between a human brain and a machine that can allow users to control various devices, such as robot arms or a wheelchair, by brain activity only (these are called active BCIs) or can monitor the mental state or emotions of a user and categorize them (these are passive BCIs). Brain signals in a BCI are usually measured by electroencephalography, a typically noninvasive method of recording electrical activity of the brain.

But there is quite a long way from raw continuous EEG signals to digitally processed signals or patterns that would have the ability to correctly identify a user’s mental workload or affective states, something that passive BCIs need to be functional. Existing experiments have shown that the accuracy of these measurements, even for simple tasks of, say, discriminating low from high workload, is insufficient for reliable practical applications.

“The low accuracy is due to extremely high complexity of a human brain. The brain is like a huge orchestra with thousands of musical instruments from which we wish to extract specific sounds of each individual instrument using a limited number of microphones or other sensors,” Andrzej Cichocki, professor at the Skoltech Center for Computational and Data-Intensive Science and Engineering (CDISE) and a coauthor of the paper, notes.

Thus, more robust and accurate EEG classification and recognition of various brain patterns algorithms are badly needed. Cichocki and his colleagues looked at two groups of machine learning algorithms, Riemannian geometry based classifiers (RGC) and convolutional neural networks (CNN), which have been doing quite well on the active side of BCIs. The researchers wondered whether these algorithms can work not just with so-called motor imaginary tasks, where a subject imagines some movements of limbs without any real movement, but for workload and affective states estimation.

They ran a competition of sorts among seven algorithms, two of which the scientists designed themselves by further improving well-performing Riemannian methods. The algorithms were tested in two studies, one with a typical arrangement for BCIs where algorithms were trained on data from a specific subject and later tested on that same subject, and one that was subject-independent — a much more challenging setup, since our brain waves could be quite different. Real EEG data was taken from earlier experiments done by Fabien Lotte, a coauthor of the paper, and his colleagues, as well as from DEAP, an existing database for emotion analysis.

The scientists found, for instance, that an artificial deep neural network outperformed all its competitors quite significantly in the workload estimation task but did poorly in emotion classification. And the two modified Riemannian algorithms did quite well in both tasks. Overall, as the paper concludes, using passive BCIs for affective state classification is much harder than for workload estimation, and subject-independent calibration leads, at least for now, to much lower accuracies.

“In the next steps, we plan to use more sophisticated artificial intelligence (AI) methods, especially deep learning, which allow us to detect very tiny changes in brain signals or brain patterns. Deep neural networks can be trained on the basis of a large set of data for many subjects in different scenarios and under different conditions. AI is a real revolution and is also potentially useful for BCI and recognition of human emotions,” Cichocki said.

###