多模态行为感知在高校学生心理健康监测与预警中的应用进展

余思慧¹ 卞晓谦^{2（通讯作者）}

1深圳职业技术大学，广东深圳，518055；

2澳门城市大学，中国澳门特别行政区，999078；

摘要：多模态行为感知为高校学生心理健康监测与预警提供了新的数据来源和分析路径。本文基于近年多模态机器学习、数字表型、技术介导心理评估和学校心理健康服务研究，梳理多模态行为感知由“数据融合”走向“预警证据建构”的应用进展。结果表明，相关技术主要沿四个方向演进：数据层面由单一或并列模态采集转向跨模态时空对齐、个体基线建模和低干扰连续感知；特征层面由人工设计的声学、视觉、生理浅层指标转向深度模型学习的语义化、动态化和个体化表征；融合层面由决策层后融合转向注意力机制、Transformer结构和跨模态交互建模；应用层面由静态症状识别扩展为动态监测、风险识别、预警提示和服务衔接。上述变化的实质，是研究者开始从“能否预测心理状态”转向“何种行为变化可以作为风险预警证据”。未来研究需进一步明确心理状态与模态线索的对应关系，发展可解释、可复核的多模态融合模型，并在真实校园场景中检验其生态效度、伦理边界和服务转化价值。

关键词：多模态学习分析；心理健康监测；风险预警；行为感知；数据融合；证据建构；数字表型；高校学生

参考文献

[1]Al Sahili, Z., Patras, I., & Purver, M. (2024). Multimodal machine learning in mental health: A survey of data, algorithms, and challenges. arXiv. https://doi.org/10.48550/arXiv.2407.16804

[2]Baltrušaitis, T., Ahuja, C., & Morency, L.-P. (2019). Multimodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2), 423–443. https://doi.org/10.1109/TPAMI.2018.2798607

[3]Chang, W. (2026). AI-based multimodal data fusion for psychological stress assessment and cardiovascular risk early warning among university students in the Greater Bay Area. European Heart Journal Supplements, 28(Supplement_2), suag002.024. https://doi.org/10.1093/eurheartjsupp/suag002.024

[4]Li, S., Liu, S., & Wu, M. (2026). Construction of mental health analysis model based on multi-modal feature learning and fusion network. International Journal of Information and Communication Technology, 27(10), 1–21. https://doi.org/10.1504/IJICT.2026.151685

[5]Liang, P. P., Zadeh, A., & Morency, L.-P. (2024). Foundations and trends in multimodal machine learning: Principles, challenges, and open questions. ACM Computing Surveys, 56(10), Article 264. https://doi.org/10.1145/3656580

[6]Oddleifson, C., Vengurlekar, I. N., Hendrix, C., Levis, T., Anderson, E., Eklund, K., & Kilgus, S. (2025). A scoping review of validity evidence for technology-mediated assessments of youth mental health. School Mental Health, 17, 316–335. https://doi.org/10.1007/s12310-025-09760-3

[7]Schmidt, P., Reiss, A., Duerichen, R., Marberger, C., & Van Laerhoven, K. (2018). Introducing WESAD, a multimodal dataset for wearable stress and affect detection. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (pp. 400–408). Association for Computing Machinery. https://doi.org/10.1145/3242969.3242985

[8]Tolosa, E., Ihianle, I. K., Machado, P., Yahaya, S. W., & Lotfi, A. (2026). From patterns to deviations: Detecting behavioural drift for mental health monitoring using smartphone and wearable data. Electronics, 15(4), 885. https://doi.org/10.3390/electronics15040885

[9]Torous, J., Kiang, M. V., Lorme, J., & Onnela, J.-P. (2016). New tools for new research in psychiatry: A scalable and customizable platform to empower data driven smartphone research. JMIR Mental Health, 3(2), e16. https://doi.org/10.2196/mental.5165

[10]Torous, J., Onnela, J.-P., & Keshavan, M. (2017). New dimensions and new tools to realize the potential of RDoC: Digital phenotyping via smartphones and connected devices. Translational Psychiatry, 7(3), e1053. https://doi.org/10.1038/tp.2017.25

[11]Wang, R., Chen, F., Chen, Z., Li, T., Harari, G., Tignor, S., Zhou, X., Ben-Zeev, D., & Campbell, A. T. (2014). StudentLife: Assessing mental health, academic performance and behavioral trends of college students using smartphones. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (pp. 3–14). Association for Computing Machinery. https://doi.org/10.1145/2632048.2632054

[12]Wang, Z., Wu, W., Zeng, C., & Shen, J. (2025). PhysioFormer: Integrating multimodal physiological signals and symbolic regression for explainable affective state prediction. PLOS ONE, 20(10), e0335221. https://doi.org/10.1371/journal.pone.0335221。