Effectiveness of multi-task learning for deep learning on the prediction performance of EEG-based cognitive state recognition

In neuroergonomics, Convolutional Neural Network (CNN)-based electroencephalogram (EEG) classifiers have been attracted to recognize cognitive states with their reliable performances. However, several restrictions include feature extraction/selection and scarcity of training data when CNN is applied to EEG classification for mental states. First, the hand-crafted feature extraction/selection can make it challenging to choose input features for the CNN that require lots of computation to select the best features. Second, the data scarcity can lead to the EEG classifier's poor performance due to overfitting to training data. To overcome those two limitations, we propose a novel framework combining CNN-based raw EEG classifiers and Multi-Task Learning (MTL) to minimize feature engineering and alleviate the overfitting problem. We utilized the state-of-the-art CNN-based EEG classifiers (e.g., EEGNet). We integrated those models with MTL, including subject identification as a sub-task that can cover inter-subject variability and the main task for a cognitive state. To validate the proposed framework's effectiveness, we used BCI competition data and compared the framework with Single-Task Learning (STL) models. In the results, we confirmed the proposed MTL was sufficient to improve the EEG classifiers' performances. In conclusion, the proposed MTL-based raw EEG classifiers can improve the EEG-based cognitive state recognition performance by mitigating data scarcity and applying inter-subject variability to the models. The proposed model would help recognize the human's mental states in any area related to applications of the EEG-based classification for neuroergonomics.