Multidisciplinary Deep Learning Approaches for Emotion Recognition in Human–Computer Interaction Systems: A Comprehensive Review

Abstract

Emotion recognition has emerged as a pivotal research frontier at the intersection of affective computing, signal processing, and human–computer interaction (HCI), driven by the growing demand for systems capable of perceiving, interpreting, and responding to human affective states in real time. This review synthesises the state of the art in deep learning-based emotion recognition, examining the multidisciplinary convergence of computer vision, speech processing, physiological signal analysis, and natural language understanding that underlies contemporary affect-sensing pipelines. The paper traces the evolution from handcrafted feature engineering to end-to-end representation learning, critically analysing convolutional neural networks, recurrent architectures, attention mechanisms, transformer-based models, and multimodal fusion strategies that have redefined benchmark performance on datasets such as IEMOCAP, CREMA-D, AffectNet, DEAP, and MELD. A structured taxonomy is proposed to classify existing approaches along the axes of modality (unimodal versus multimodal), learning paradigm (supervised, self-supervised, and few-shot), and application context (embodied agents, adaptive tutoring, mental health monitoring, and automotive safety). Architectural blueprints, mathematical formulations of loss functions and fusion operators, and comparative performance tables are presented to ground the discussion in empirical evidence. The review further examines real-world deployment through a case study of affect-aware conversational agents, surveys the software and hardware ecosystem supporting reproducible research, and evaluates recognition accuracy, latency, and robustness across cross-corpus and cross-cultural conditions. Persistent challenges—including data scarcity, annotation subjectivity, domain shift, and the fundamental ambiguity of emotional expression—are discussed alongside ethical considerations surrounding consent, bias, and affective privacy. The review concludes by outlining promising directions, including foundation-model-driven affective reasoning, neuro-symbolic integration, and personalised continual learning, offering a roadmap for researchers seeking to advance emotionally intelligent human–computer interaction systems.