A Transformer-Based Approach with Data Augmentation for Multilabel Emotional Context Detection

Mohsin Hasan Hussein, Department of Computer Science, College of Computer Science and Information Technology, University of Kerbala, Karbala, Iraq
Marem H. Abdulabas, Department of Information Technology, College of Computer Science and Information Technology, University of Kerbala, Karbala, Iraq
Azha Talal Mohammed Ali, Department of Computer Science, College of Computer Science and Information Technology, University of Kerbala, Karbala, IraqFollow
Homam Aziz Ghazi, Department of Computer Science, College of Computer Science and Information Technology, University of Kerbala, Karbala, Iraq

Abstract

Emotion identification in texts is becoming increasingly difficult because of the wide variety of ways emotions are represented. This study uses a fine-tuned Robustly Optimized Bidirectional Encoder Representations from Transformers Approach

(RoBERTa) to offer a Transformer-based model for identifying multilabel emotional context in textual data. To balance emotion categories and enhance the model's capacity for generalization, data augmentation is applied on two different datasets: Semantic Evaluation and Cross-lingual Emotion Dataset (SemEval and XED) English corpus. This stage is considered one of the most important steps in preprocessing as it greatly helps to improve the results. The RoBERTa model was then used to extract and comprehend the deep context of emotional expressions. The proposed model shows robust performance on both SemEval-2018 and XED datasets with F1-macro/F1-micro of 0.924/0.93, 0.707/0.748, and 0.729/0.77 on single, double and triple emotions respectively. Further, it gives consistent results on XED with F1-macro/F1-micro of 0.846/0.846 on single emotions, 0.819/0.833 for double emotions and 0.825/0.834 for triple emotions.

Recommended Citation

Hussein, Mohsin Hasan; Abdulabas, Marem H.; Ali, Azha Talal Mohammed; and Ghazi, Homam Aziz (2026) "A Transformer-Based Approach with Data Augmentation for Multilabel Emotional Context Detection," Al-Bahir: Vol. 9: Iss. 2, Article 1.
Available at: https://doi.org/10.55810/2313-0083.1146

References

[1] Anwer S. Deep Neural Network and Transformer Models for Emotion Recognition. Bilad Alrafidain J Eng Sci Technol [Internet] 2025 Mar 15;4(1):100—12. https://doi.org/10.56990/ bajest/2025.040109. Available from:.

[2] Deng J, Ren F. A Survey of Textual Emotion Recognition and Its Challenges. IEEE Trans 2021;14(1):49—67. Available from: https://doi.org/10.1109/TAFFC.2021.3053275.

[3] Esteban Ortiz-Ospina. The rise of social media. Our World Data [Internet]; 2019. Available from: https://archive. ourworldindata.org/20260305-101825/rise-of-social-media.html.

[4] Tamilkodi R, Sujatha B, Leelavathy N. Emotion detection in text: advances in sentiment analysis. Int J Syst Assur Eng Manag [Internet] 2025;16:552—60. Available from: https:// doi.org/10.1007/s13198-024-02597-0.

[5] Chutia T, Baruah N. A review on emotion detection by using deep learning techniques. Artif Intell Rev [Internet]2024 Aug 1;57(203).https://doi.org/10.1007/s10462-024- 10831-1. Available from:.

[6] Nandwani P, Verma R. A review on sentiment analysis and emotion detection from text. Soc Netw Anal Min [Internet] 2021;11(1):1—19. Available from: https://doi.org/10.1007/ s13278-021-00776-6.

[7] Maruf A Al, Khanam F, Haque MM, Jiyad ZM, Mridha MF, Aung Z. Challenges and Opportunities of Text-Based Emotion Detection: A Survey. IEEE Access [Internet] 2024; 12:18416—50. Available from: https://doi.org/10.1109/ ACCESS.2024.3356357.

[8] Murthy AR, Anil Kumar KM. A Review of Different Approaches for Detecting Emotion from Text. IOP Conf Ser Mater Sci Eng [Internet] 2021;1110(1):012009. Available from: https://doi.org/10.1088/1757-899x/1110/1/012009.

[9] Acheampong FA, Nunoo-Mensah H, Chen W. Transformer Models for Text-based Emotion Detection: A Review of BERT-based Approaches. Artif Intell Rev [Internet] 2021; 54(8):5789—829. Available from: https://doi.org/10.1007/ s10462-021-09958-2.

[10] Devlin J, Chang MW, Lee K, Toutanova K. BERT : Pretraining of Deep Bidirectional Transformers for Language Understanding. In: NAACL-HLT 2019 [internet]; 2019. p. 4171—86. Available from: https://doi.org/10.18653/v1/N19- 1423.

[11] Kumar JA, Cambria E, Trueman TE. Transformer-Based Bidirectional Encoder Representations for Emotion Detection from Text. In: 2021 IEEE symposium series on computational intelligence, SSCI 2021 - proceedings [internet]. Institute of Electrical and Electronics Engineers Inc.; 2021. Available from: https://doi.org/10.1109/SSCI50451.2021.9660152.

[12] Steinbrenner T, Lalk C, Targan K, Schaffrath J, Eberhardt S, Haberkamp A, et al. Explaining anxiety prediction in psychotherapy transcripts: The role of patient linguistic features and theoretical constructs. Behav Res Ther [Internet] 2025;194(August 2024):104857. Available from: https:// doi.org/10.1016/j.brat.2025.104857.

[13] Ameer I, Bo€lücü N, Siddiqui MHF, Can B, Sidorov G, Gelbukh A. Multi-label emotion classification in texts using transfer learning. Expert Syst Appl [Internet] 2023;213. Available from: https://doi.org/10.1016/j.eswa. 2022.118534.

[14] Ju X, Zhang D, Li J, Zhou G. Transformer-based Label Set Generation for Multi-modal Multi-label Emotion Detection. In: MM 2020 - proceedings of the 28th ACM international conference on multimedia [internet]. Association for Computing Machinery, Inc; 2020. p. 512—20. Available from: https://dl.acm.org/doi/10.1145/3394171.3413577.

[15] Sallah A, Arbi Abdellaoui Alaoui E, Agoujil S, Ahmad Wani M, Hammad M, Maleh Y, et al. Fine-Tuned Understanding: Enhancing Social Bot Detection with Transformer-Based Classification. IEEE Access [Internet] 2024;12: 118250—69. Available from: https://doi.org/10.1109/ ACCESS.2024.3440657.

[16] Li J, Ji D, Li F, Zhang M, Liu Y. HiTrans: A TransformerBased Context- and Speaker-Sensitive Model for Emotion Detection in Conversations. In: the 28th international conference on computational linguistics [internet]. Online; 2020. p. 4190—4194200. Available from: https://doi.org/ 10.18653/v1/2020.coling-main.

[17] Mei Y, Abisado M. BERT-based two-channel neural network model text emotion analysis. Salud, Cienc y Tecnol [Internet] 2025;5. Available from: https://doi.org/10.56294/ saludcyt20251444.

[18] Mahima MA, Patel NC, Ravichandran S, Aishwarya N, Maradithaya S. A Text-Based Hybrid Approach for Multiple Emotion Detection Using Contextual and Semantic Analysis. In: Proceedings of the 2021 IEEE international conference on innovative computing, intelligent communication and smart electrical systems, ICSES 2021 [internet]. Institute of Electrical and Electronics Engineers Inc.; 2021. Available from: https://doi.org/10.1109/ICSES52305.2021.9633843.

[19] Kumar T, Mahrishi M, Sharma G. Emotion recognition in Hindi text using multilingual BERT transformer. Multimed Tool Appl 2023;82(27):42373—94. Available from: https:// doi.org/10.1007/s11042-023-15150-1.

[20] Das A, Hoque MM, Sharif O, Dewan MAA, Siddique N. TEmoX: Classification of Textual Emotion Using Ensemble of Transformers. IEEE Access [Internet] 2023;11: 109803—18. Available from: https://doi.org/10.1109/ ACCESS.2023.3319455.

[21] Sajikumar S. Emotion detection in text: a comprehensive analysis using classical, deep learning, and transformer-based models [internet]. National College of Ireland; 2024. Available from: https://norma.ncirl.ie/7963/1/sreelakshmisajikumar.pdf.

[22] Qasim A, Mehak G, Hussain N, Gelbukh A, Sidorov G. Detection of Depression Severity in Social Media Text Using Transformer-Based Models. Inf [Internet] 2025 Feb 1; 16(2). Available from: https://doi.org/10.3390/info16020114.

[23] Joshi D, Bharatbhai Joshi D, Ashokbhai Dhokai A, Pavankumar Agrawal N, Pankaj Agrawal A, Kumar A, et al. Emotion Detection using Transformer Model with Deep Learning. Int J Eng Res Technol [Internet] 2025;14(3). Available from: https://doi.org/10.17577/IJERTV14IS030116.

[24] Maazallahi A, Asadpour M, Bazmi P. Advancing emotion recognition in social media: A novel integration of heterogeneous neural networks with fine-tuned language models. Inf Process Manag [Internet] 2025;62(2):103974. Available from: https://doi.org/10.1016/j.ipm.2024.103974.

[25] Barcena Ruiz G, Gil Herrera RDJ. Textual emotion detection with complementary BERT transformers in a Condorcet’s Jury theorem assembly. Knowledge-Based Syst [Internet] 2025; 326(June):114070. Available from: https://doi.org/10.1016/ j.knosys.2025.114070.

[26] Mohammad SM, Bravo-Marquez F, Salameh M, Kiritchenko S. SemEval-2018 Task 1: Affect in Tweets. In: 12th international workshop on semantic evaluation (SemEval-2018) [internet]; 2018. p. 1—17. Available from: https://doi.org/10.18653/v1/s18-1001.

[27] Ohman € E, Pamies M, Kajava K, Tiedemann J. XED: A Multilingual Dataset for Sentiment Analysis and Emotion Detection. In: 28th international conference on computational linguistics [internet]; 2020. p. 6542—52. Available from: https://doi.org/10.18653/v1/2020.coling-main.575.

[28] Semary NA, Ahmed W, Amin K, Pławiak P, Hammad M. Improving sentiment classification using a RoBERTa-based hybrid model. Front Hum Neurosci 2023;17(December):1—10.

[29] Baziotis C, Athanasiou N, Chronopoulou A, Kolovou A, Paraskevopoulos G, Ellinas N, et al. NTUA-SLP at SemEval2018 Task 1: Predicting Affective Content in Tweets with Deep Attentive RNNs and Transfer Learning. In: 12th international workshop on semantic evaluation (SemEval2018) [internet]; 2018. p. 245—55. Available from: https:// doi.org/10.18653/v1/s18-1037.

[30] Acheampong FA, Henry NM, Chen W. Transformer Models for Text-based Emotion Detection: A Review of BERT-based Approaches. Artif Intell Rev [Internet] 2021;54(1):1—41. Available from: https://doi.org/10.1007/s10462-021-09958-2.

[31] Buechel S, Hahn U. Emotion Embeddings ― Learning Stable and Homogeneous Abstractions from Heterogeneous Affective Datasets. IEEE Trans Affect Comput [Internet] 2025:1—18. Available from: https://doi.org/10.1109/TAFFC.2025.3635686.

[32] Ameer I, Bolücü € N, Siddiqui MHF, Can B, Sidorov G, Gelbukh A. Multi-label emotion classification in texts using transfer learning. Expert Syst Appl [Internet] 2023;213. Available from: https://doi.org/10.1016/j.eswa.2022.118534.

[33] Hinojosa Lee MC, Braet J, Springael J. Performance Metrics for Multilabel Emotion Classification: Comparing Micro, Macro, and Weighted F1-Scores. Appl Sci [Internet] 2024; 14(21). Available from: https://doi.org/10.3390/app14219863.

[34] Belay TD, Azime IA, Ayele AA, Sidorov G, Klakow D, Slusallek P, et al. Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understanding. Proc - Int Conf Comput Linguist COLING 2025:3523—40.

Download