Traitement Du Signal: Received: 7 June 2020 Accepted: 13 October 2020
Traitement Du Signal: Received: 7 June 2020 Accepted: 13 October 2020
Traitement Du Signal: Received: 7 June 2020 Accepted: 13 October 2020
Low-Cost Brain-Computer Interface Using the Emotiv Epoc Headset Based on Rotating
Vanes
Mesut Melek1*, Negin Manshouri2, Temel Kayikcioglu2
1
Department of Electronics and Automation, Gumushane University, Gumushane 29100, Turkey
2
Department of Electrical and Electronics Engineering, Faculty of Engineering, Karadeniz Technical University, Trabzon
61080, Turkey
https://doi.org/10.18280/ts.370516 ABSTRACT
Received: 7 June 2020 In the brain-computer interface system (BCI), electroencephalography (EEG) signals are
Accepted: 13 October 2020 converted into digital signals and analyzed, allowing direct communication between humans
and the electronic devices around them. The convenience of the user and the speed of
Keywords: communication with the surrounding devices are the most important challenges of BCI
EEG, Emotiv Epoc headset, brain-computer systems. The Emotiv Epoc headset minimizes the discomfort of the user thanks to its wet
interface, rotating vanes, information electrodes and easy handling. In the continuation of our previous works, in this paper, we
transfer rate developed our BCI system based on the gaze at the rotating vanes using the inexpensive
Emotiv Epoc headset. In addition to user comfort, our design has an acceptable mean
accuracy rate (ACC) and mean information transfer rate (ITR) compared to similar systems.
831
system has not been computed. in front of a 32-inch monitor, and after a few seconds of rest,
The aim of this study was to develop a user-friendly, recordings commenced. The participants were asked to look at
completely independent BCI system by using only the brain each vane for 125 seconds. This process was started and
signals of the disabled, thereby increasing the users’ quality of finished with a beep sound. After hearing the second beep
life by enabling them to interact with the surrounding sound, and before starting watching the next vane, the
environment. Each of the common paradigms used in BCI participants were given a resting time of about 1 minute. In
systems has its own disadvantages. In the P300 and SSVEP this way, the data set was recorded from a total of five
paradigms, the user has to look at the lights constantly blinking. participants. Then, the data set was divided into 6-sec, 4-sec,
This is an irritating issue for long-term users and may even and 2-sec epochs, with four seconds, two seconds, and one
cause eye diseases. In the motor imagery paradigm, the user second of overlapping, respectively (Table 1). For example,
has to consider the movement of his/her limbs constantly. We the EEG signal in a single channel, which is 125 seconds for
proposed a BCI system based on rotating vanes in our previous each vane, was separated into 6-second segments by four
studies [18, 19] to solve these problems. However, the EEG seconds of overlapping, resulting in a total of 59 epochs.
device used in our studies was wired and with gel. In the
present study, we tested our system on the Emotiv Epoc device,
which has a lower quality signal and is cheap. The BCI system
based on rotating vanes with the Emotiv Epoc device will
provide more comfort for users.
The organization of this article is as follows: Materials and
methods are explained after the Introduction section. This
section involves data description, pre-processing, feature
extraction, classification, and performance metrics in BCI
systems, in that order. In the third section, the results are
described, and in the fourth and fifth sections, discussion and
conclusion are given, respectively.
Figure 1. Rotating vanes designed by Matlab 2014a [19]
2. MATERIALS AND METHODS Table 1. Description of the data set for one participant in an
EEG channel
2.1 Data description
Epoch Overlapping Epoch number for each
As mentioned earlier, in order to overcome the problems of duration (sec.) vane
popular BCI paradigms, we proposed a BCI system based on 6-sec 4 59
rotating vanes in our previous studies. Thanks to this system, 4-sec 2 60
the user can avoid looking at flashing lights or thinking of 2-sec 1 120
constantly moving limbs. We used the interface designed
through Matlab 2014a in our previous works. This interface is 2.2 Pre-processing
given in Figure 1. As it appears, four "A" letters were written
in white on a black screen, and a red vane was placed on top The amplitude of the signals can directly affect the
of each. The speed and direction of rotation of the vanes are as classification performance. Therefore, the min-max
follows: normalization method was applied in each epoch, as shown in
⚫ Vane 1 (top left) rotates in a counterclockwise manner Eq. (1), to reduce the effect of amplitude.
every five seconds.
⚫ Vane 2 (top right) rotates in a counterclockwise 𝑥 − 𝑚𝑖𝑛(𝑥)
𝑋𝑁 = (1)
manner every one second. 𝑚𝑎𝑥(𝑥) − 𝑚𝑖𝑛(𝑥)
⚫ Vane 3 (bottom left) rotates in a clockwise manner
every five seconds. Here, x and XN are the original epoch and normalized epoch
⚫ Vane 4 (bottom right) rotates in a clockwise manner indicator, respectively. Furthermore, processing of EEG
every one second. signals requires special approaches such as filtering [20]. After
In this way, the options presented according to the user's normalization, each epoch was passed through a 0.1-45 Hz
needs will be located under each vane and it will be sufficient bandpass Butterworth filter. The order of the filter was
for the user to look at the vane on the desired option. determined as three by trial and error, and Matlab filtfilt
EEG signals were taken by the Emotiv Epoc EEG device command was used to reduce the phase shift to zero.
from five participants (four men and one woman) aged 27 to
32 years at the Electrical and Electronics Engineering 2.3 Feature extraction
Department, Karadeniz Technical University. The
experiments were performed with the registration number The power spectrum density (PSD) analysis provides basic
24237859-640 according to the rules of the Faculty of information on how power is distributed as a function of
Medicine Ethics Committee, Karadeniz Technical University frequency. In this study, the Welch method was adopted to
(Turkey). The participants had no previous experience of using calculate PSD [21]. A short summary of the Welch method can
any BCI systems and could withdraw at any time during the be described as follows: In the first step, each epoch is divided
experiment without any pressure on them. The Emotiv Epoc into parts of a certain length with certain overlaps. In the
device sampled the EEG signal through 14 electrodes at 128 second step, by applying a window to each part, the relevant
samples/sec. The participants were seated at a distance of 1 m periodogram is calculated, i.e., a Fast Fourier Transform (FFT)
832
is applied to the windowed data. Finally, in the third step, the stability of their systems, and in this way, they obtain the
spectral density is estimated by averaging over the modified overall accuracy rate of the system by calculating the average
periodograms. The spectrum estimate obtained is scaled to of the calculated accuracy rates each time.
calculate the PSD. In our study, a smooth Hamming window
[22] w(n) is applied to each part of data based on Eq. (2). 2.5.2 ITR
The value of ITR is very influential, as BCI systems interact
W(n)=0.54-0.46 cos(2nπ/L) (2) directly with patients [31]. The ITR is the most comprehensive
performance metric of BCI systems. This metric derives from
where, n=0,1,2 … L-1, and L denotes the length of each part. the principles of information theory and summarizes the three
After calculating the PSD of each epoch, features were important parameters of the BCI system with a single value.
extracted by considering the five frequency rhythms of the These parameters are the accuracy rate of the system, the time
EEG signal [23]. For this, the areas formed between the PSD it takes for a selection to be recognized in the system, and the
graph and the x-axis for delta (0-4 Hz), theta (4-7 Hz), alpha number of tasks in each selection. According to Shannon's ITR,
(8-12 Hz), beta (13-30 Hz), and gamma (30-45 Hz) bands was ITR is defined by Eq. (4) [32].
calculated by Matlab trapz command, and thus 5 features were
1−𝑝
obtained from each epoch. A feature vector of an epoch was Bt= 𝑙𝑜𝑔2 𝐾 + 𝑝𝑙𝑜𝑔2 𝑝 + (1 − 𝑝)𝑙𝑜𝑔2 ( ) (4)
𝐾−1
created by bringing these features obtained from 14 channels
side by side.
where, K indicates how many rights the user has in each
selection, and p represents the system's ACC. The unit of Bt is
2.4 Classification
bits per trial and in minutes can be expressed by Eq. (5).
An algorithm trained with labelled training samples to 𝐵𝑡
identify unknown samples is called a classifier. The support ITR= 60 ∗ (5)
𝑇
vector machine (SVM) [24], linear discriminant analysis
(LDA) [25], k-nearest neighbors (k-NN) [26], and Ada boost where, T is the time it takes for a selection to be recognized in
[26] are classifiers used on Emotiv-based BCI systems. SVM the system. In our study, we have four vanes so K is 4. Also,
(linear, quadratic, cubic, medium Gaussian), linear T is 6, 4, and 2 when we used 6-sec, 4-sec, and 2-sec epochs,
discriminant, and ensemble subspace discriminant were respectively. For more details and proofs of ITR based on
employed in this study. information theory, could be referred to the studies [33, 34].
833
Table 2. Accuracy rates and ITRs for 6-sec epochs based on LOO-CV strategy; the highest value in each metric is indicated in
boldface
Table 3. Accuracy rates and ITRs for 4-sec epochs based on LOO-CV strategy; the highest value in each metric is indicated in
boldface
Table 4. Accuracy rates and ITRs for 2-sec epochs based on LOO-CV strategy; the highest value in each metric is indicated in
boldface
Table 5. Accuracy rates and ITRs for 6-sec epochs based on holdout strategy; the highest value in each metric is indicated in
boldface
834
Table 6. Accuracy rates and ITRs for 4-sec epochs based on holdout strategy; the highest value in each metric is indicated in
boldface
Table 7. Accuracy rates and ITRs for 2-sec epochs based on holdout strategy; the highest value in each metric is indicated in
boldface
In all of the sections of this study, the Linear discriminant successful in 4-sec epochs. The average accuracy rate and ITR
classifier, the covariance structure was selected as full. In of the system were calculated as 83.80% and 16.92 bits/min,
Linear, Quadratic, Cubic SVM classifier Box constraint level respectively. Successful in 6-sec and 2-sec epochs, the
was selected 1, and kernel scale mode was Auto. In the quadratic SVM followed this classifier with little difference
Medium Gaussian SVM classifier box constraint level was 1 (82.96% and 16.26 bits/min).
and the kernel scale was selected as 8.4. In all SVM classifiers, In most Emotiv-based BCI systems (such as [36] and [37]),
one vs. one method was used for the upgrade of the binary the system's accuracy rate and ITR were not computed. Still,
classifier to a multi-class classifier. In Ensemble Subspace extensive studies are available in the literature. In a study
discriminant number of learners was 30 and subspace based on the SSVEP paradigm, Brennan et al. [38] used
dimensions were selected as 35. These parameters were Emotiv Epoc and reached a 79.2% accuracy rate and 15.23
selected by the try and error method and are constant in all bits/min ITR. In a 2012 study [11], the accuracy rate of the
sections of this study. system in the offline mode was calculated as 82.99% ±4.98%.
In this SSVEP-based study, the speed of the system was given
as 28.06 ± 6.45 bits/min. Four participants contributed to this
4. DISCUSSIONS study. The accuracy rate of the system in the online mode was
95.83% ± 3.59% and the ITR was 20.97 ± 0.37 bits/min. The
A BCI system based on the rotating vanes with the Emotiv results of other studies are close to these values [14] or even
Epoc device was proposed and evaluated in 6-sec, 4-sec, and less [13, 15, 16]. When we compare the paradigm problems
2-sec epochs according to two strategies. According to the and the results of these studies with the proposed system's
LOO-CV strategy, the cubic SVM classifier gave better results paradigm and results, it becomes clear that the proposed
than the other five classifiers. The average accuracy rate system provides more comfort for users, and the design has an
obtained with this strategy for the 6-sec epochs was calculated acceptable ACC and ITR compared to similar systems.
as 93.72%, and the average ITR of the system was 15.79
bits/min. The average accuracy rates were 85.58% and 78.08%
in 4-sec and 2-sec epochs, and the ITR of the system was 5. CONCLUSION
computed as 17.88 and 26.95 bits/min, respectively.
In the results obtained with the hold-out strategy, which is In EEG-based BCI systems, EEG signals are analyzed and
more common in offline studies of BCI systems, quadratic the secret thoughts of the user are translated into certain
SVM is the most successful classifier in 6-sec and 2-sec commands; in this way, the user can communicate with his/her
epochs. The average accuracy rates obtained for the 6-sec and surroundings. Although BCI faces many challenges in terms
2-sec epochs were 91.14% and 76.95%, and the speed of the of applicability, usability, and accuracy rate, it is the future of
system equaled 14.50 and 25.90 bits/min, respectively. The the human-machine interface and the only communication
Ensemble Subspace discriminant classifier was more destination for paralyzed people. The convenience of the user
835
and the speed of communication with the surrounding devices [10] Masood, N., Farooq, H. (2017). Emotiv-based low-cost
are the most important challenges of BCI systems. The Emotiv brain computer interfaces: A survey. Adv. Intell. Syst.
Epoc headset minimizes the discomfort of the user with its wet Comput., Springer, Cham, pp. 133-142.
electrodes and easy usage. To continue our previous works, in https://doi.org/10.1007/978-3-319-41691-5_12
this article, we developed our BCI system based on rotating [11] Liu, Y., Jiang, X., Cao, T., Wan, F., Mak, P.U., Mak, P.I.,
vanes using the Emotiv Epoc headset. The system's offline Vai, M.I. (2012). Implementation of SSVEP based BCI
accuracy rate and speed were calculated as ~77%, 84%, 92%, with Emotiv EPOC. In: Proc. IEEE Int. Conf., Virtual
and 26, 17, 15 bits/min for the 2-sec, 4-sec, and 6-sec epochs, Environ. Human-Computer Interfaces, Meas. Syst., pp.
respectively. In the next study, we have planned to increase the 34-37. https://doi.org/10.1109/VECIMS.2012.6273184
accuracy rate of 2-sec epochs by testing different feature [12] Van Vliet, M., Robben, A., Chumerin, N., Manyakov, N.
extraction and classification methods, and to test the system in V., Combaz, A., Van Hulle, M.M. (2012). Designing a
a realistic environment by using the proposed BCI system as a brain-computer interface controlled video-game using
control terminal in the online mode. consumer grade EEG hardware. 2012 ISSNIP Biosignals
and Biorobotics Conf, pp. 1-6.
https://doi.org/10.1109/BRC.2012.6222186
REFERENCES [13] Tong, J., Peng, Z., Ran, X., Lei, D. (2015). The portable
P300 dialing system based on tablet and Emotiv Epoc
[1] Kaur, B., Singh, D., Roy, P.P. (2018). EEG based headset. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc.
emotion classification mechanism in BCI. Procedia EMBS, Institute of Electrical and Electronics Engineers
Comput. Sci., Elsevier B.V., 132: 752-758. Inc., pp. 566-569.
https://doi.org/10.1016/j.procs.2018.05.087 https://doi.org/10.1109/EMBC.2015.7318425
[2] Wolpaw, J.R., Birbaumer, N., Heetderks, W.J., [14] Soroush, P.Z., Shamsollahi, M.B. (2019). A non-user-
McFarland, D.J., Peckham, P.H., Schalk, G., Donchin, E., based BCI application for robot control. 2018 IEEE-
Quatrano, L.A., Robinson, C.J., Vaughan, T.M. (2000). EMBS Conf. Biomed. Eng. Sci., Sarawak, Malaysia, pp.
Brain-computer interface technology: A review of the 36-41. https://doi.org/10.1109/IECBES.2018.8626701
first international meeting. IEEE Transactions on [15] Mijani, A.M., Shamsollahi, M.B., Hassani, M.S.,
Rehabilitation Engineering, 8(2): 164-173. Jalilpour, S. (2019). Comparison between single, dual
https://doi.org/10.1109/TRE.2000.847807 and triple rapid serial visual presentation paradigms for
[3] Kelly, S.P., Lalor, E.C., Finucane, C., McDarby, G., P300 speller. 2018 IEEE Int. Conf. Bioinforma. Biomed,
Reilly, R.B. (2005). Visual spatial attention control in an Madrid, Spain, pp. 2635-2638.
independent brain-computer interface. IEEE https://doi.org/10.1109/BIBM.2018.8621505
Transactions on Biomedical Engineering, 52 (9): 1588- [16] Li, X.D., Wang, X.J., Wong, C.M., Wen, R.W., Wan, F.,
96. https://doi.org/10.1109/TBME.2005.851510 Hu, Y. (2019). Influence of stimuli color combination on
[4] Rafik, D., Larbi, B. (2019). Autoregressive modeling online SSVEP-based BCI performance. In: 2019 IEEE
based empirical mode decomposition (EMD) for Int. Conf. Comput. Intell. Virtual Environ. Meas. Syst.
epileptic seizures detection using EEG signals. Appl. CIVEMSA 2019 - Proc., Institute of Electrical and
Traitement Du Signal, 36(3): 273-279. Electronics Engineers Inc.
https://doi.org/10.18280/ts.360311 https://doi.org/10.1109/CIVEMSA45640.2019.9071623
[5] Valente, G., Kaas, A.L., Formisano, E., Goebel, R. [17] Zhang, X., Yao, L., Zhang, S., Kanhere, S., Sheng, M.,
(2019). Optimizing fMRI experimental design for Liu, Y.H. (2019). Internet of Things meets brain-
MVPA-based BCI control: Combining the strengths of computer interface: A unified deep learning framework
block and event-related designs. NeuroImage, 186: 369- for enabling human-thing cognitive interactivity. IEEE
381. Internet of Things Journal, 6(2): 2084-2092.
https://doi.org/10.1016/j.neuroimage.2018.10.080.Epub https://doi.org/10.1109/JIOT.2018.2877786
2018 Nov 1 [18] Maleki, M., Manshouri, N., Kayikcioglu, T. (2018). Fast
[6] Chen, C.H., Ho, M.S., Shyu, K.K., Hsu, K.C., Wang, and accurate classifier-based brain-computer interface
K.W., Lee, P.L. (2014). A noninvasive brain computer system using single channel EEG data. In: 26th IEEE
interface using visually-induced near-infrared Signal Process. Commun. Appl. Conf. SIU 2018,
spectroscopy responses. Neuroscience Letters, 580: 22- Institute of Electrical and Electronics Engineers Inc., pp.
26. https://doi.org/10.1016/j.neulet.2014.07.042 1-4. https://doi.org/10.1109/SIU.2018.8404376
[7] Stamps, K., Hamam, Y. (2010). Towards inexpensive [19] Maleki, M., Manshouri, N., Kaykcioglu, T. (2017). A
BCI control for wheelchair navigation in the enabled new brain-computer interface system based on
environment - A hardware survey. Lect. Notes Comput. classification of the gaze on four rotating vanes.
Sci. (Including Subser. Lect. Notes Artif. Intell. Lect. International Journal of Computer Science and
Notes Bioinformatics), Springer, Berlin, Heidelberg, pp. Information Security (IJCSIS), 15(2): 437-443.
336-345. https://doi.org/10.1007/978-3-642-15314-3_32 [20] Dimitrov, G.P., Kostadinova, I.S., Panayotova, G.,
[8] Alzu’bi, H.S., Al-Zubi, N.S., Al-Nuaimy, W. (2011). Petrova, P., Aleksiev, K., Oleksii, B., Shevchenko, V.
Toward inexpensive and practical brain computer (2019). Using mixed linear programming to classify BCI
interface. 2011 Developments in E-systems Engineering, signals submission. In: 2019 IEEE 60th Annu. Int. Sci.
Dubai, pp. 98-101. Conf. Power Electr. Eng. Riga Tech. Univ. RTUCON,
https://doi.org/10.1109/DeSE.2011.116 Riga, Latvia, pp. 1-5.
[9] Maskeliunas, R., Damasevicius, R., Martisius, I., https://doi.org/10.1109/RTUCON48111.2019.8982368
Vasiljevas, M. (2016). Consumer-grade EEG devices: [21] Welch, P.D. (1967). The use of fast fourier transform for
Are they usable for control tasks? PeerJ, 4: e1746. the estimation of power spectra: A method based on time
https://doi.org/10.7717/peerj.1746
836
averaging over short, modified periodograms. IEEE and Learning Algorithms. Cambridge University Press.
Transactions on Audio and Electroacoustics, 15(2): 70- [34] Cover, T.M., Thomas, J.A. (2005). Elements of
73. https://doi.org/10.1109/TAU.1967.1161901 Information Theory. Wiley.
[22] Xiong, Q., Zhang, X., Wang, W.F., Gu, Y.H. (2020). A [35] Cohen, J. (1960). A coefficient of agreement for nominal
parallel algorithm framework for feature extraction of scales. Educational and Psychological Measurement,
EEG signals on MPI. Computational and Mathematical 20(1): 37-46.
Methods in Medicine, 2020: 9812019. https://doi.org/10.1177/001316446002000104
https://doi.org/10.1155/2020/9812019 [36] Alshbatat, A.l.N., Vial, P.J., Premaratne, P., Tran, L.C.
[23] Kayikcioglu, T., Maleki, M., Eroglu, K. (2015). Fast and (2014). EEG-based brain-computer interface for
accurate PLS-based classification of EEG sleep using automating home appliances. Journal of Computers, 9(9):
single channel data. Expert Systems with Applications, 2159-2166. https://doi.org/10.4304/jcp.9.9.2159-2166
42(21): 7825-7830. [37] Alrajhi, W., Alaloola, D., Albarqawi, A. (2017). Smart
https://doi.org/10.1016/j.eswa.2015.06.010 home: toward daily use of BCI-based systems. in: 2017
[24] Wang, J.F. (2015). A Framework for The Integration of Int. Conf. Informatics, Heal. Technol., IEEE, pp. 1-5.
the Emotiv EEG system into the NeuCube Spiking https://doi.org/10.1109/ICIHT.2017.7899002
Neural Network Environment for Robotics Control. [38] Brennan, C., McCullagh, P., Lightbody, G., Galway, L.,
Auckland University of Technology. Feuser, D., González, J.L., Martin, S. (2015). Accessing
[25] Duvinage, M., Castermans, T., Petieau, M., Hoellinger, tele-services using a hybrid BCI approach. Advances in
T., Cheron, G., Dutoit, T. (2013). Performance of the Computational Intelligence, Springer, Cham, pp. 110-
Emotiv Epoc headset for P300-based applications. 123. https://doi.org/10.1007/978-3-319-19258-1_10
BioMedical Engineering Online, 12(1): 56.
https://doi.org/10.1186/1475-925X-12-56
[26] Pham, T.D., Tran, D. (2012). Emotion recognition using APPENDIX
the emotiv EPOC device. In: Lect. Notes Comput. Sci.
(Including Subser. Lect. Notes Artif. Intell. Lect. Notes
Bioinformatics), Springer, Berlin, Heidelberg, pp. 394-
399. https://doi.org/10.1007/978-3-642-34500-5_47
[27] Shi, M.H., Liu, X.Q., Zhou, C.L., Chao, F., Liu, C., Jiao,
X.Z., An, Y.F., Nwachukwu, S.E., Jiang, M. (2018).
Towards portable SSVEP-based brain-computer
interface using Emotiv EPOC and mobile phone. In: Proc.
- 2018 10th Int. Conf. Adv. Comput. Intell. ICACI 2018,
Institute of Electrical and Electronics Engineers Inc., pp.
249-253. https://doi.org/10.1109/ICACI.2018.8377615
[28] Ayado, V.G.B., Dungca, C.J.N., Lazam, S.A., Pereira, Figure A1. Cohen’s kappa coefficient and sensitivity of each
M.N.L., Tan, J.R.O., Prado, S.V. (2019). Development vane in 6-sec epochs based on the hold-out strategy
of an EEG-based motor imagery brain-computer
interface system for lower limb assistive technologies. In:
2018 IEEE 10th Int. Conf. Humanoid, Nanotechnology,
Inf. Technol. Commun. Control. Environ. Manag.
HNICEM 2018, pp. 1-6.
https://doi.org/10.1109/HNICEM.2018.8666350
[29] Nanthini, B.S., Santhi, B. (2017). Electroencephalogram
signal classification for automated epileptic seizure
detection using genetic algorithm. Journal of Natural
Science, Biology and Medicine, 8(2): 159-166.
https://doi.org/10.4103/jnsbm.JNSBM_285_16
[30] Cui, C.K., Bian, G.B., Hou, Z.G., Zhao, J., Zhou, H.
(2017). A multimodal framework based on integration of Figure A2. Cohen’s kappa coefficient and sensitivity of each
cortical and muscular activities for decoding human vane in 4-sec epochs based on the hold-out strategy
intentions about lower limb motions. IEEE Transactions
on Biomedical Circuits and Systems, 11(4): 889-99.
https://doi.org/10.1109/TBCAS.2017.2699189
[31] Sadeghi, S., Maleki, A. (2019). Accurate estimation of
information transfer rate based on symbol occurrence
probability in brain-computer interfaces. Biomedical
Signal Processing and Control, 54: 101607.
https://doi.org/10.1016/j.bspc.2019.101607
[32] Wolpaw, J.R., Ramoser, H., McFarland, D.J.,
Pfurtscheller, G. (1998). EEG-based communication:
Improved accuracy by response verification. IEEE
Transactions on Rehabilitation Engineering, 6(3): 326-
333. https://doi.org/10.1109/86.712231 Figure A3. Cohen’s kappa coefficient and sensitivity of each
[33] Mackay, D.J.C. (2003). Information Theory, Inference, vane in 2-sec epochs based on the hold-out strategy
837