Introduction about Emotion Detection

Human emotions are psychophysiological experiences that affects all parts of everyday lives. Emotions comprises number of components such as feelings, bodily changes, cognitive reactions, behaviours, and thoughts. Emotion detection or recognition can be based on physiological signals [1-4]. These signals are electrocardiogram (ECG), photoplethysmogram (PPG), electroencephalogram (EEG), electromyogram (EMG), electrooculogram (EoG), skin conductive resistance (SCR), galvanic skin response (GSR), blood volume pressure (BVP) and respiration rate (RR). These bio-signals are currently being used in clinical diagnostics and have been proven to be useful to convey information about emotional states.

ECG is a powerful diagnostic tool in medicine used to assess the functionality of the heart. Obtaining physiological signals from ECG is a conventional method for non-invasive interpretation of the electrical activity of the heart in real time. The ECG signal is recorded by attaching electrodes on the surface of the body. These currents stimulate the cardiac muscle and cause the contractions and relaxations of the heart. The electrical signals propagate through the electrodes to the ECG device to record the characteristic of waves. Different waves reflect the activity of different areas of the heart which generates the respective flowing electrical currents [5].

PPG is regularly used in clinical diagnostics to monitor heart-induced changes in blood volume in microvascular beds at peripheral body locations such as wrist, finger, nose, earlobe and toe. The pulsating element of a PPG signal is associated with modifications in the quantity of blood within the arteries and is synchronous with heartbeat, whereas the non-pulsating element depends on the blood volume, breathing, sympathetic nervous system, and thermoregulation [6]. The PPG signal is measured from the vascular blood flow measurement sensor. The working principle of the PPG sensor is based on the emission of infrared light by an LED which penetrates the skin and blood vessels. This light is captured by the detector to measure the blood stream.

ECG and PPG sensors play a vital role in developing portable, reliable, non-intrusive and computationally efficient emotion detection systems. Understanding emotion using physiological signals is one of the active research field. The advantage of emotion detection using physiological signals is that these signals are unconscious responses of the human body and very difficult to conceal compared to audio-visual and gesture-based emotion detection. Two approaches are widely used for emotion assessment from bio-signals, discrete emotional models (DEM) and affective dimensional models (ADM) [7]. In DEM, the objective is to recognise and label standard emotional states such as joy, sadness, anger, or fear. In this approach, arbitrary affect state must be well defined and completely distinguishable from the rest. In ADM, the affective state is the combination of two parameters, namely, arousal and valence. Arousal is a measure of emotional stimulation and varies between high and low, whereas valence is a measure of pleasantness for the experienced emotion, ranging from very pleasant (positive) to very unpleasant (negative).

[1] Murugappan M., Murugappan S., Zheng BS. Frequency Band Analysis of Electrocardiogram (ECG) Signals for Human Emotional State Classification Using Discrete Wavelet Transform (DWT). J Phys Ther Sci. 2013;25(7):753–759.
[2] Kim, K.H., Bang, S.W. & Kim, S.R. Emotion recognition system using short-term monitoring of physiological signals. Med. Biol. Eng. Comput. (2004) 42: 419. 
[3] Haag A., Goronzy S., Schaich P., Williams J. (2004) Emotion Recognition Using Bio-sensors: First Steps towards an Automatic System. In: André E., Dybkjær L., Minker W., Heisterkamp P. (eds) Affective Dialogue Systems. ADS 2004. Lecture Notes in Computer Science, vol 3068. Springer, Berlin, Heidelberg. 

[4] Lee, M.S.; Lee, Y.K.; Pae, D.S.; Lim, M.T.; Kim, D.W.; Kang, T.K. Fast Emotion Recognition Based on Single Pulse PPG Signal with Convolutional Neural Network. Appl. Sci. 2019, 9, 3355. 

[5] Gacek A. (2012) An Introduction to ECG Signal Processing and Analysis. In: Gacek A., Pedrycz W. (eds) ECG Signal Processing, Classification and Interpretation. Springer, London. 

[6] Tamura, T.; Maeda, Y.; Sekine, M.; Yoshida, M. Wearable Photoplethysmographic Sensors—Past and Present. Electronics 2014, 3, 282-302.

[7] F. Agrafioti, D. Hatzinakos and A. K. Anderson, "ECG Pattern Analysis for Emotion Detection," in IEEE Transactions on Affective Computing, vol. 3, no. 1, pp. 102-115, Jan.-March 2012.