From Unimodal to Multimodal: AI-Driven Mental Health Assessment Using Linguistic, Acoustic, and Visual Cues

Abstract

Mental health disorders, including depression, anxiety, bipolar disorder, and schizophrenia, represent a significant global health challenge due to their high prevalence and complex diagnostic requirements. Traditional assessment methods primarily rely on clinical interviews and self-reported questionnaires, which are often subjective, time-consuming, and unsuitable for continuous or large-scale monitoring. In recent years, advances in artificial intelligence (AI) have enabled the development of multimodal mental health assessment systems that integrate diverse behavioral and physiological signals such as speech, text, facial expressions, and biosignals. This paper presents a systematic review of eight influential studies that employ multimodal machine learning and deep learning techniques for mental health analysis. The reviewed works are critically analyzed based on the datasets utilized, model architectures, multimodal fusion strategies, and evaluation metrics. Comparative findings demonstrate that multimodal approaches consistently outperform unimodal systems, offering improved accuracy, robustness, and generalization across various mental health conditions. Despite these encouraging outcomes, several challenges remain, including limited availability of high-quality labeled data, lack of model interpretability, insufficient clinical validation, and ethical concerns related to privacy and real-world deployment. This review identifies key research gaps and highlights future directions aimed at developing reliable, explainable, and ethically responsible AI-driven systems to support mental healthcare assessment and intervention