Discussing Solutions to the Data Imbalance Problem in Emotion Recognition

Junwei Chen

doi:10.54254/2755-2721/2025.PO24697

Applied and Computational EngineeringOpen access

Discussing Solutions to the Data Imbalance Problem in Emotion Recognition

Research Article

Open Access

Discussing Solutions to the Data Imbalance Problem in Emotion Recognition

Junwei Chen ^1*

¹ Maynooth International Engineering College, Fuzhou University, Fuzhou, Fujian, 350108, China

^*Corresponding author: 832203214@fzu.edu.cn

Published on 4 July 2025

ACE Vol.174

ISSN (Print): 2755-273X

ISSN (Online): 2755-2721

ISBN (Print): 978-1-80590-235-5

ISBN (Online): 978-1-80590-236-2

Download Cover

Abstract

Emotion recognition technology has been widely used in human-computer interaction, medical health and other fields. However, in practical applications, emotion datasets often have class imbalance problems, which lead to the model being seriously biased towards the majority class, significantly reducing the recognition accuracy and reliability of minority emotion classes. This paper focuses on comparing and analyzing methods such as ESC-GAN generative data augmentation technology, DER-GCN dialogue and event relationship perception graph model, and MultiEMO multimodal fusion framework to solve the problem of imbalanced emotion recognition categories, and explores the innovations and limitations in multiple scenarios. These methods compensate for minority emotions from different angles: for example, MultiEMO significantly improves the ability to classify minority emotions through cross-modal attention mechanism and weighted contrast loss, which can not only be applied to detect the psychological emotions of patients in the medical health field, but also help to provide support for fine-grained emotion classification in security scenarios. Experimental results show that these solutions significantly improve the accuracy and F1 value of emotion recognition, especially in extremely unbalanced categories. This paper provides a systematic reference for the selection of technology for high-value scenarios such as medical monitoring and intelligent security, promotes the interdisciplinary collaborative development in the field of emotional computing, and accelerates the application transformation of this technology in practice.

Keywords:

Emotion Recognition, Data Imbalance, Data Augmentation, Loss Optimization, Multimodal Fusion.

View PDF

References

[1]. J. Deng and F. Ren, "A Survey of Textual Emotion Recognition and Its Challenges, " in IEEE Transactions on Affective Computing, vol. 14, no. 1, pp. 49-67, 1 Jan.-March 2023, doi: 10.1109/TAFFC.2021.3053275.

[2]. W. Hamilton, Z. Ying and J. Leskovec, "Inductive representation learning on large graphs" in Proc. Adv. Neural Inf. Process. Syst., MIT Press, vol. 30, 2017.

[3]. B.-H. Su and C.-C. Lee, "Unsupervised cross-corpus speech emotion recognition using a multi-source cycle-GAN", IEEE Trans. Affect. Comput., vol. 14, no. 3, pp. 1991-2004, Jul./Sep. 2023, [online] Available: .

[4]. Y. Luo and B.-L. Lu, “Eeg data augmentation for emotion recognition using a conditional wasserstein gan, ” in 2018 40th annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, 2018, pp. 2535–2538.

[5]. B. Li, Y. Liu and X. Wang, "Gradient harmonized single-stage detector", Proc. AAAI Conf. Artif. Intell., vol. 33, no. 01, pp. 8577-8584, 2019.

[6]. S. Poria, D. Hazarika, N. Majumder, G. Naik, E. Cambria and R. Mihalcea, "MELD: A multimodal multi-party dataset for emotion recognition in conversations", Proc. 57th Annu. Meeting Assoc. Comput. Linguistics, pp. 527-536, 2019.

[7]. Zhang Y, Li Y, Liu X, et al. Leave no stone unturned: Mine extra knowledge for imbalanced facial expression recognition [J]. Advances in Neural Information Processing Systems, 2023, 36: 14414-14426.

[8]. Li Q, Huang P, Xu Y, et al. Generating and encouraging: An effective framework for solving class imbalance in multimodal emotion recognition conversation [J]. Engineering Applications of Artificial Intelligence, 2024, 133: 108523.

[9]. Singh K, Ahirwal M K, Pandey M. Subject wise data augmentation based on balancing factor for quaternary emotion recognition through hybrid deep learning model [J]. Biomedical Signal Processing and Control, 2023, 86: 105075.

[10]. Shi T, Huang S L. MultiEMO: An attention-based correlation-aware multimodal fusion framework for emotion recognition in conversations [C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023: 14752-14766.

[11]. C. Busso et al., "LEMOCAP: Interactive emotional dyadic motion capture database", Lang. Resour. Eval., vol. 42, no. 4, pp. 335-359, 2008, [online] Available: .

[12]. Ai W, Shou Y, Meng T, et al. Der-gcn: Dialog and event relation-aware graph convolutional neural network for multimodal dialog emotion recognition [J]. IEEE Transactions on Neural Networks and Learning Systems, 2024.

[13]. Meng T, Shou Y, Ai W, et al. Deep imbalanced learning for multimodal emotion recognition in conversations [J]. IEEE Transactions on Artificial Intelligence, 2024.

[14]. Zhang Z, Zhong S, Liu Y. Beyond mimicking under-represented emotions: deep data augmentation with emotional subspace constraints for EEG-based emotion recognition [C]//Proceedings of the AAAI conference on artificial intelligence. 2024, 38(9): 10252-10260.

[15]. S. Koelstra, C. Muhl, M. Soleymani, J.-S. Lee, A. Yazdani, T. Ebrahimi, T. Pun, A. Nijholt, and I. Patras, “Deap: A database for emotion analysis; using physiological signals, ” IEEE transactions on affective computing, vol. 3, no. 1, pp. 18–31, 2011.

[16]. Li A, Wu M, Ouyang R, et al. A Multimodal-Driven Fusion Data Augmentation Framework for Emotion Recognition [J]. IEEE Transactions on Artificial Intelligence, 2025.

[17]. P. Schmidt, A. Reiss, R. Duerichen, C. Marberger, and K. Van Laerhoven, “Introducing wesad, a multimodal dataset for wearable stress and affect detection, ” in Proceedings of the 20th ACM international conference on multimodal interaction, 2018, pp. 400–408.

References

[2]. W. Hamilton, Z. Ying and J. Leskovec, "Inductive representation learning on large graphs" in Proc. Adv. Neural Inf. Process. Syst., MIT Press, vol. 30, 2017.

[5]. B. Li, Y. Liu and X. Wang, "Gradient harmonized single-stage detector", Proc. AAAI Conf. Artif. Intell., vol. 33, no. 01, pp. 8577-8584, 2019.

[11]. C. Busso et al., "LEMOCAP: Interactive emotional dyadic motion capture database", Lang. Resour. Eval., vol. 42, no. 4, pp. 335-359, 2008, [online] Available: .

[13]. Meng T, Shou Y, Ai W, et al. Deep imbalanced learning for multimodal emotion recognition in conversations [J]. IEEE Transactions on Artificial Intelligence, 2024.

[16]. Li A, Wu M, Ouyang R, et al. A Multimodal-Driven Fusion Data Augmentation Framework for Emotion Recognition [J]. IEEE Transactions on Artificial Intelligence, 2025.

Cite this article

Chen,J. (2025). Discussing Solutions to the Data Imbalance Problem in Emotion Recognition. Applied and Computational Engineering,174,23-31.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

About volume

Volume title: Proceedings of CONF-CDS 2025 Symposium: Data Visualization Methods for Evaluatio

ISBN: 978-1-80590-235-5(Print) / 978-1-80590-236-2(Online)

Editor: Marwan Omar, Elisavet Andrikopoulou

Conference website: https://2025.confcds.org/portsmouth.html

Conference date: 30 July 2025

Series: Applied and Computational Engineering

Volume number: Vol.174

ISSN: 2755-2721(Print) / 2755-273X(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:

1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.

2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.

3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).