Early detection of dysphoria using electroencephalogram affective modelling

ABSTRACT

forced cheerfulness. Such a response is deemed void and useless to detect dysphoria because it may lead to the wrong conclusion. In addition, the language used (typically, the questionnaire is in English) can be wrongly construed by the respondent, who needs to become more familiar with English or the terms used. To date, no computational measurement tool currently is widely and readily available to the public to assess their dysphoria tendency except a questionnaire forcing the manual interpretation of the questionnaire. Such an approach is prone to human error and not economical in terms of effort and time. Hence, a new kind of genuine and 'pure' input is needed from the respondent to gain insight into their dysphoria level with the help of a computational tool to assist in dysphoria detection to complement the psychologists and psychiatrists is needed. The proposed approach is to supplement the role of psychiatrists and clinical psychologists. The knowledge and experience of psychiatrists and clinical psychologists are essential to diagnosing. However, because there are a limited number of trained psychiatrists and clinical psychologists and if available at private hospitals where the consultation fees are costly, it is hoped that the computational tool can provide early screening for the psychiatrists and clinical psychologists.
This project uses computational intelligence methods to quantify negative emotions, and a dysphoria model of affect (DMoA) is developed. Such a model can provide a datum for better comparative analysis and correlate it with the standard psychological instruments in detecting dysphoria. It also helps to complement the DASS-21 and NDS-24 reports by using the affective space model (ASM) for machine implementation. These will facilitate clinicians and psychologists for their decision support system (DSS) so that it can be used as a dysphoria indicator instrument, to be more precise. This tool can also measure stress in a workplace and educational institutions so that better quality of life can be achieved, and that early intervention can be planned and executed accordingly.

METHOD
The DMoA is developed using psychological and neurophysiological profiling, as presented in Figure 1. Figure 1 represents the block diagram of the framework of the proposed model. It provides actionable steps to complete the proposed solution and creates a shared understanding of the system. The framework comprises two perspectives: psychological dysphoria profiling and neurophysiological profiling using electroencephalogram (EEG) data.

5877
The psychological profiling is developed using the DASS-21 and the NDS-24 questionnaires. Such an approach gauges the fundamental symptoms of depression, anxiety, and stress. In addition, the latter instrument assesses confirmatory factors in dysphoria, such as irritability, discontent, surrender, and interpersonal resentment. Both questionnaires need to be answered before the EEG emotion data are collected. All the questionnaire results have been recorded for further analysis and correlation between the EEG signals. The EEG brain signals data collection is completed using the 19 channels EEG. Three phases involve pre-processing and artifact removal, bandpass filtering and feature extraction, and emotion classification. In the final stage, the results from both psychological and neurophysiological are compared for correlation analysis. Detailed descriptions of the data collection and processing are provided in the following subsections.

Participants and psychological profiling
Data collection involved nine male participants with a mean age of 21.2 years and a variance of ±18. 16. None of the participants had a history of acute depression, anxiety, or stress, and they were all deemed healthy. The experiments took place in the laboratory of the International Islamic University Malaysia (IIUM), and participants were provided with a briefing about the study before signing the consent form. The study followed the ethical guidelines set forth by the IIUM ethics committee, and its approval number is IREC 2017-064.
The participants involved in the study were in a neutral state, without any influence from drugs, alcohol, or medication. Prior to the experiment, each participant completed the DASS-21 [21] and NDS-24 [23] questionnaire tests to assess their psychological profile. Permission to use the questionnaires was obtained beforehand, and the participants were instructed to answer the questions truthfully and carefully, taking their time to select the most appropriate responses. Additionally, participants were encouraged to seek clarification if they required any help in understanding the meaning of any word in the questionnaire, to minimize the likelihood of misunderstandings leading to inaccurate responses.
The DASS-21 and NDS questionnaires are comprised of 21 and 24 questions respectively. The DASS-21 questionnaire assesses the participant's levels of depression, anxiety, and stress, and asks them to report any symptoms experienced in the past week. Each item is scored on a scale of 0 to 3, with 0 indicating that the item did not apply to the participant at all, and 3 indicating that it applied to them very much or most of the time. Moreover, the NDS-24 questionnaire is used to determine the severity of dysphoria. Like the DASS-21, the participants need to provide a rating of the frequency ranging from 0 (not at all) to 3 (most of the time). In order to create a psychological profile for each participant, they are required to respond to all questions on the questionnaire sheet by selecting their preferred choices. Once the task is completed, the collected questionnaire sheets will be analyzed based on the participants' answers. The DASS-21 and NDS-24 psychological interpretations will be used to construct a psychological profile for each individual.

EEG data collection
Brain signals manifest the neurons triggering in the brain in the form of electricity, and these electrical activations can be detected using the EEG. In this work, the raw 19 channels EEG signals are captured using an EEG device from Brain Marker, and the electrode placement is based on the standard 10-20 electrode placement system. However, only six channels are considered; namely, F3, F4, F7, F8, Fp1, and Fp2, to measure the negative emotions, as illustrated in Figure 2. These six channels are selected because signals from the frontal brain region are relevant to emotion processing [24]- [26]; hence, contains much emotional information. Figure 3 shows the different ways of using direct and cap-based electrode placements on the scalp.

Figures 3(a) and 3(b) depict the EEG with manual placement, and Figures 3(c) and 3(d)
show the cap-based EEG electrode placement. This work employs EEG cap-based electrode placement because it is convenient, practical, and economically fast. Low impedance data are collected to ensure that the contact between the electrodes and scalp is good. Once the data have low impedance and the quality of the recording is good, the real-time data from the brain signals are captured and stored in .tdms file format for further processing.
The participants are instructed to sit comfortably on a stationary chair, facing the laptop screen where the selected stimuli will be presented. Throughout the experiment, they are reminded to minimize physical movements, as any artifacts may disrupt the EEG signals. The electrodes are attached to the participant's scalp according to the international 10 to 20 electrode placement system, with conducting gel applied between the electrodes and the scalp to improve conductivity and minimize impedance. The experiment takes place in a laboratory setting with controlled environmental conditions to reduce background noise and distractions. Prior to capturing the EEG signals, an electrode placement check is performed to ensure that high-quality signals can be recorded.
Once the experimental setup is completed, participants are shown a set of stimuli as summarized in the protocol presented in Figure 4. Initially, the resting state of the participant is recorded. The resting state refers to the state before any stimuli are presented to the participants. The participants had to open their eyes for a minute and then close them for another minute. Then, the emotional test will be conducted. In this test, participants need to look at four emotional stimuli representing happiness, sadness, calm, and fear images in one minute each, respectively. The images are taken from the international affective picture system (IAPS) [27]. This dataset contains a standardized emotionally induced photograph that can evoke emotions from the viewer and is widely used by affective computing researchers in emotion elicitation. The pictures used in this work are adopted from the picture's subset selected by in few studies [28]. Subsequently, the video stimuli are presented for fear and sadness to gauge the video's effect on negative emotion. The data collection protocol concludes with a one-minute resting state, consisting of both eye-opening and eye-closing stages [29], [30], following the emotional test. The total duration of data collection for this study is 10 minutes, not accounting for the pre-EEG and post-EEG experiment tasks. The focus of this study is to analyze the brain signal outputs during the emotional test and resting state, prior to presenting any stimuli to the participants.

EEG data pre-processing and artifacts removal
After obtaining the raw EEG data, the EEG signals went through the pre-processing phase. In this phase, all the unwanted signals and artifacts have been removed to ensure the signals are considered clean signals containing the most relevant features. To ensure the participants are fully focused, the first 2 seconds of each signal are discarded. The signals are then normalized, filtered, and any artifacts are eliminated to obtain a clean EEG data. This pre-processing phase is crucial in ensuring the analyzed data is confined to the region of interest and free from any biases or distortions caused by artifacts and noise. Otherwise, the data would be deemed as unreliable and misleading.
Typically, the initial 2 seconds of each 1-minute data protocol comprises approximately 500 instances, which depend on the chosen window's size. This data is often disregarded since it may not provide any valuable information, given that the participant is still acclimating to differentiate emotions based on the presented stimuli. Subsequently, the time-frequency signals are translated into 19 channels, resulting in 19 channels x 14,500 instances. However, for this study, only six data channels related to the brain's frontal region were considered, as the study's sole focus is on detecting dysphoria, which is associated with emotional and executive brain functions. Thus, even though data was collected from all 19 channels, only six were used in this study. Then, the data from the six channels are normalized using norms(c) function of MATLAB software (The Math Works R2015a). During this process, it normalizes the sum of the squares of the elements in each column to have a value of 1. Since the raw EEG value instances are sorted according to Column 1 to Column 14,500, the normc(M) function is used for column normalization instead of the normr(R) function that normalizes the row.
After the normalization process, the signals undergo filtering using the Ellipord() function in MATLAB. This function determines the minimum order of an analog or digital elliptic filter that can satisfy a set of filter design specifications. The output includes the value of n, which represents the lowest order of the elliptic filter that maintains a passband ripple of no more than Rp (dB) and a stopband attenuation of at least Rs (dB), and a scalar (or vector) of corresponding cutoff frequencies, Wp. The output arguments n and Wp are then used in ellip. By using elliptical filters, the signals are cleaned of any high-frequency data (above 50 Hz), which are usually artifacts resulting from hand, eye, and muscle movements, before proceeding to the feature extraction process.

Bandpass filtering and feature extraction
Once the pre-processing phase is completed by removing artifacts and noise, the relevant features are extracted using the mel frequency cepstral coefficient (MFCC) feature extraction method. In this work, the window type used is the Hamming window with a 12 coefficient, producing 72 features cumulatively (12 features X 6 channels). Based on the preliminary experimental results [29], the window size of 1,024 yielded the optimum result resulting in 67 instances per participant/emotion. Therefore, the individual data is 268 instances * 72 features. First, the dataset architecture is presented in Figure 5. Then, the extracted features were used with a selected classifier to classify emotion.

Emotion classification
A simple multi-layer perceptron (MLP) with one hidden layer of 10 neurons with a learning rate of 0.01 is used for classification. The reason for such implementation is to show the simplicity of the approach. The initial step in the experiment is to train the multi-layer perceptron neural network using four basic emotions, which is then utilized to recognize the participants' emotional states during resting conditions, such as eyes closed and eyes open. This recognition of emotional states is then linked to the psychological dysphoria analysis using DASS-21 and NDS-24. As the identification of standard emotion pictures is used to categorize each emotion, it is highly probable that the emotion detected during the resting state is similar to the standard emotions. The experimental outcome and a comprehensive discussion are provided in the subsequent section.

Psychological profiling
The results from the DASS-21 and NDS-24 questionnaires are extracted and inferred from the standardized DASS scale. Table 1 presents the raw result of each participant and the summary of the participants' state. The obtained values are presented in a tabular form to facilitate analysis. The severity/frequency scales range from 0 to 3 and are used by the participants to rate the extent to which they experienced depression, anxiety, and stress in the past week. Based on the DASS severity/frequency scale in Table 1, all participants fall within the normal range, with none of them scoring in the mild, moderate, severe, or extremely severe states. Although the depression scale assesses dysphoria, hopelessness, devaluation of life, self-deprecation, lack of interest/involvement, anhedonia, and inertia, all participants show similar results, making it difficult to determine whether they have dysphoria or not. Participants were asked to rate their state of mind on a scale from 0 to 3 using the NDS-24 questionnaire during the experiment. The NDS-24 results are computed using item means, standard deviations, item-total score correlations, and a pattern matrix of principal axis factoring solution with Promax rotation constrained to four factors: irritability, discontent, surrender, and interpersonal resentment [23]. O'Connor's SPSS syntax was used to perform the calculations, and the factor scores are displayed in Table 2. The results show that only three participants (Subject3, Subject7, and Subject9) reported severe dysphoria, while the others reported mild to moderate levels. From the results, it can be observed that NDS-24 allows more insights harnessed on dysphoria compared to the DASS-21 questionnaire. In the DASS-21 result, all participants are identified as normal. Thus, it is difficult to perform any correlation to the emotions identified in the EEG emotion identification experiment. The NDS questionnaire score reveals that the participants have different ranges of dysphoria. Subject1, Subject2, and Subject8 are experiencing only mild dysphoria, while Subject4, Subject5, and Subject6 are experiencing moderate dysphoria. In addition, it is also observed that Subject3, Subject7, and Subject9 are experiencing severe dysphoria. Therefore, the result in the NDS can be used for further exploration.

EEG emotion identification
The purpose of conducting the EEG emotion identification experiment is to evaluate the classifier's ability to anticipate a specific emotion from a varied emotion dataset. The dataset is structured to ensure that each emotion has an equal number of instances, with 67 instances per emotion, obtained from the participants' frontal EEG signals of F3, F4, F7, F8, FP1, and FP2, as depicted in Figure 5. Figure 6 displays a comprehensive accuracy report for emotion identification. The results in Figure 6 indicate that Subject9 achieved the highest accuracy in identifying fear, with an accuracy rate of 95%. Conversely, Subject5 achieved the lowest accuracy in identifying happiness, with an accuracy rate of 55.15%. The mean accuracy of the selected approach yielded between 63.14% and 78.9%. The results indicate that the EEG signals are discerned enough to segregate between four different emotions, even with a simple classifier. Figure 6. The EEG emotion identification accuracy

EEG emotion identification
Before the participants were presented with stimuli, the brain signals were captured when the eyes opened and closed because these states reflect the actual current state of the individual. Then, after the stimuli are presented, an individual's eye open and eye close states are also captured to contrast the states before and after the stimuli are presented. Finally, using MLP generated previously for emotion identification, the eye open and eye close data are used as testing data. Figure 7 shows the average performance of emotion identification before and after stimuli are presented. Based on the result, fear is identified as the highest percentage during the eye closed (before) state with an accuracy of 35.8%. Such a situation can be associated with the nervousness the participants experienced as they were unsure what would happen in the experiment. Once they opened their eyes, happiness was recorded as the highest emotion identified (31.1%). After the stimuli were given, happy is still dominated with an accuracy of 29.3% when eyes closed. It can be linked to the relief feeling because the experiment is ending. However, mixed results of calm and sad are recorded when the individual opens their eyes because the visual stimuli from the surrounding may influence their emotion after stimuli are given. Hence, it is sound to consider the result when the participants close their eyes.  Then, the details of each individual's experimental results are analyzed and presented in Table 3. It is noticeable that individuals experiencing intense dysphoria tend to exhibit high levels of fear. Specifically, Subject3, Subject7, and Subject9 consistently scored higher in fear compared to the other participants. The findings indicate a potential link between EEG emotion identification and NDS-24 scores. However, this trend is less noticeable in the remaining participants who reported only mild to moderate dysphoria severity in the NDS-24 survey. As a result, additional research utilizing more intricate and sophisticated methodologies is required.

CONCLUSION
The early identification of dysphoria is crucial to prevent it from worsening and to provide timely treatment. This study suggests the use of EEG signals and the affective space model mapping to detect dysphoria. This approach can supplement the NDS-24 questionnaire to provide objective insights that can aid psychologists and psychiatrists in assessing their patients. Additionally, the brain signals captured through EEG cannot be controlled, making it less susceptible to misleading responses that may occur when patients are in denial. Therefore, this method may increase the accuracy of assessments and lead to better treatment outcomes.
The psychological profiling using the DASS-21 questionnaire may not be able to detect well for dysphoria as all the participants scored in the range of normal for depression, anxiety, and stress scale. Interestingly, the NDS-24 questionnaire does provide a better measure for dysphoria because it shows the distinction between mild, moderate, and severe dysphoria levels. First, the EEG signals emotion identification experiments are conducted to evaluate the simple multi-layer perceptron ability to separate emotions. The experimental results show the average emotion identification accuracy ranging from 63.14% to 78.90%. Then, the dysphoria correlation experiments are conducted. Experiments using eyes opened and eyes closed before stimuli are presented and after stimuli are presented were also conducted. The actual current state of an individual can be captured with eyes closed and eyes open before the stimuli experiment. Interestingly, the experimental results of emotion recognition during eyes close before stimuli are more accurate than those of emotion recognition during eyes open before stimuli because the brain processes much information once eyes are opened, which may dilute performance. The eye open and eyes close after stimuli are presented are also captured to contrast the findings in the previous experiments. Subject3, Subject7, and Subject9 yielded severe dysphoria range, which seems to directly correlate with the EEG eyes close before stimuli emotion identification experimental results. Fear is highly recorded, with an accuracy of 70.9%, 63.6%, and 62.3%, respectively. However, the correlation with the other participants who scored mild and moderate dysphoria levels is not highly observable. Thus, the experimental results show the potential to explore further the correlation between emotion and dysphoria.