切换至 "中华医学电子期刊资源库"

中华肥胖与代谢病电子杂志 ›› 2026, Vol. 12 ›› Issue (02) : 125 -132. doi: 10.3877/cma.j.issn.2095-9605.2026.02.007

综述

多模态情绪识别技术在减重代谢手术患者心理评估与依从性管理中的研究进展
裴瑶, 高丽莲, 刘家晖, 杨华, 方欣怡, 艾星星, 王思雨, 张桂娟()   
  1. 510632 广州,暨南大学护理学院
  • 收稿日期:2026-01-07 出版日期:2026-05-31
  • 通信作者: 张桂娟

Multimodal emotion recognition in psychological evaluation and adherence management for bariatric and metabolic surgery patients: a review

Yao Pei, Lianlian Gao, Jiahui Liu, Hua Yang, Xinyi Fang, Xixing Ai, Siyu Wang, Guijuan Zhang()   

  1. Faculty of Nursing, Jinan University, Guangzhou 510630, China
  • Received:2026-01-07 Published:2026-05-31
  • Corresponding author: Guijuan Zhang
引用本文:

裴瑶, 高丽莲, 刘家晖, 杨华, 方欣怡, 艾星星, 王思雨, 张桂娟. 多模态情绪识别技术在减重代谢手术患者心理评估与依从性管理中的研究进展[J/OL]. 中华肥胖与代谢病电子杂志, 2026, 12(02): 125-132.

Yao Pei, Lianlian Gao, Jiahui Liu, Hua Yang, Xinyi Fang, Xixing Ai, Siyu Wang, Guijuan Zhang. Multimodal emotion recognition in psychological evaluation and adherence management for bariatric and metabolic surgery patients: a review[J/OL]. Chinese Journal of Obesity and Metabolic Diseases(Electronic Edition), 2026, 12(02): 125-132.

随着全球肥胖症发病率逐年递增,减重代谢手术已成为治疗重度肥胖及代谢综合征的重要手段。术后患者容易产生焦虑、抑郁、身体形象混乱以及服药依从性差等问题从而对长远的治疗效果造成不利影响。目前临床上心理测量方法主要是以主观问卷和谈话为主,不能及时、全面、动态地监测患者情绪状态和行为变化。MER为减重代谢手术患者心理状态量化评估和依从性管理搭建了客观化、连续化、智能化的新通道。未来仍要开展多中心临床验证研究,进而评估其应用效果,同时强化模型可解释性和隐私保护方面的设计,进而推进MER跟护理信息系统深度结合,以促进精准化连续性心理护理模式良好发展。

With the continuous increase in the global prevalence of obesity, bariatric and metabolic surgery (BMS) has gradually been introduced to address severe obesity and metabolic syndrome. Post-operative patients may be anxious or depressed, suffer from changes in body image, and lose motivation for treatment; thus, their recovery may be impaired. Most current clinical psychological evaluations are self-report questionnaires and interviews; thus, they fail to provide timely, objective and continuous data on alterations in patients' emotions and behaviour. MER is a new objective that can quantify the psychological status and adherence behaviour of patients with BMS more intelligently. Future work will include multiple centre clinical validation trials to examine the actual effects. In addition, improve the interpretability and privacy protection design of the model, promote deeper integration of MER with nursing information systems, and advance the development of precision and continuous psychological nursing care models.

图1 多模态情绪识别在减重手术患者心理评估与依从性管理中的应用流程图
[1]
GBD 2021 Adult BMI Collaborators. Global, regional, and national prevalence of adult overweight and obesity, 1990-2021, with forecasts to 2050: a forecasting study for the Global Burden of Disease Study 2021 [J]. Lancet, 2025, 405(10481): 813-838.
[2]
Kim JC, Kim MG, Park JK, et al. Outcomes and Adverse Events After Bariatric Surgery: An Updated Systematic Review and Meta-analysis, 2013-2023 [J]. J Metab Bariatr Surg, 2023, 12(2): 76-88.
[3]
Benítez T, Caixàs A, Rebasa P, et al. Psychopathological profile before and after bariatric surgery [J]. Sci Rep, 2023, 13(1): 16172.
[4]
Bruze G, Järvholm K, Norrbäck M, et al. Mental health from 5 years before to 10 years after bariatric surgery in adolescents with severe obesity: a Swedish nationwide cohort study with matched population controls [J]. Lancet Child Adolesc Health, 2024, 8(2): 135-146.
[5]
de Vries CEE, Terwee CB, Al Nawas M, et al. Outcomes of the first global multidisciplinary consensus meeting including persons living with obesity to standardize patient-reported outcome measurement in obesity treatment research [J]. Obes Rev, 2022, 23(8): e13452.
[6]
Fu J, Wang H, Na R, et al. Recent advancements in digital health management using multi-modal signal monitoring [J]. Math Biosci Eng, 2023, 20(3): 5194-5222.
[7]
Guo R, Guo H, Wang L, et al. Development and application of emotion recognition technology-a systematic literature review [J]. BMC Psychol, 2024, 12(1): 95.
[8]
Wu Y, Mi Q, Gao T, et al. A Comprehensive Review of Multimodal Emotion Recognition: Techniques, Challenges, and Future Directions. Biomimetics (Basel) [J]. 2025, 10(7): 418.
[9]
Lian H, Lu C, Li S, et al. A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face. Entropy (Basel) [J]. 2023, 25(10): 1440.
[10]
Knight SR, Ng N, Tsanas A, et al. Mobile devices and wearable technology for measuring patient outcomes after surgery: a systematic review [J]. NPJ Digit Med, 2021, 4(1): 157.
[11]
Lodewyk K, Wiebe M, Dennett L, et al. Wearables research for continuous monitoring of patient outcomes: A scoping review [J]. PLOS Digit Health, 2025, 4(5): e0000860.
[12]
Jacobs A, Monpellier VM, Torensma B, et al. Influence of mental and behavioral factors on weight loss after bariatric surgery: A systematic review and meta-analysis [J]. Obes Rev, 2024, 25(6): e13729.
[13]
Pyykkö JE, Zwartjes M, Nieuwdorp M, et al. Differences in Psychological Health and Weight Loss after Bariatric Metabolic Surgery between Patients with and without Pain Syndromes [J]. Obes Surg, 2024, 34(5): 1693-1703.
[14]
Chan JKY, Vartanian LR. Psychological predictors of adherence to lifestyle changes after bariatric surgery: A systematic review [J]. Obes Sci Pract, 2024, 10(1): e741.
[15]
Summerville S, Kirwan E, Sutin AR, et al. Personality trait associations with quality-of-life outcomes following bariatric surgery: a systematic review. Health Qual Life Outcomes [J]. 2023, 21(1): 32.
[16]
Pyykkö JE, van Olst N, Gerdes VEA, et al. Relations between trajectories of weight loss and changes in psychological health over a period of 2 years following bariatric metabolic surgery [J]. Qual Life Res, 2025, 34(5): 1345-1361.
[17]
Magurano MR, Napolitano D, Bozzetti M, et al. Psychological assessment and psychosocial outcomes in bariatric surgery candidates: a retrospective study [J]. Healthcare (Basel), 2025, 13(11): 1294. Published 2025 May 29.
[18]
Kops NL, Vivan MA, Fülber ER, et al. Preoperative binge eating and weight loss after bariatric surgery: a systematic review and meta-analysis [J]. Obes Surg, 2021, 31(3): 1239-1248.
[19]
Hany M, Yassin HAA, Hamdy A, et al. Prevalence and associated factors of eating disorders among metabolic bariatric surgery patients in egypt [J]. J Eat Disord, 2025, 13(1): 182.
[20]
Brunault P, Bourbao-Tournois C, Ballon N, et al. Psychiatric, psychological and addiction management in obes surg: early identification for better support [J]. J Visc Surg, 2023, 160(2S): S22-S29.
[21]
Block CKA R, Marek RJ. Presurgical psychological evaluation: risk factor identification and mitigation [J]. J Clin Psychol Med Settings, 2020, 27(2): 396-405.
[22]
Francois Z, Rizvi A. Psychiatric complications of bariatric surgery[M/OL]//StatPearls. Treasure Island (FL): StatPearls Publishing, 2024[2025-12-27]. [

URL    
[23]
Tolvanen L, Svensson Å, Hemmingsson E, et al. Perceived and preferred social support in patients experiencing weight regain after bariatric surgery-a qualitative study [J]. Obes Surg, 2021, 31(3): 1256-1264.
[24]
Bi Y, He L, Yan F, et al. Personal, external, and psychological factors influencing adherence to nutrition and diet in patients undergoing metabolic/bariatric surgery: a systematic synthesis of mixed methods research [J]. Acta Diabetol, 2024, 61(9): 1081-1095.
[25]
Dandgey S, Patten E. Psychological considerations for the holistic management of obesity [J]. Clin Med (Lond), 2023, 23(4): 318-322.
[26]
Magurano MR, Napolitano D, Bozzetti M, et al. Psychological Assessment and Psychosocial Outcomes in Bariatric Surgery Candidates: A Retrospective Study [J]. Healthcare, 2025, 13(11): 1294.
[27]
Butt M, Wagner A, Rigby A, et al. Associations of social desirability on psychological assessment outcomes for surgical weight loss patients [J]. J Clin Psychol Med Settings, 2021, 28(2): 384-393.
[28]
Woods R, Moga AM, Ribeiro PAB, et al. Evolution of depressive symptoms from before to 24 months after bariatric surgery: A systematic review and meta-analysis [J]. Obes Rev, 2023, 24(5): e13557.
[29]
Ames GE, Koball AM, Clark MM, et al. Behavioral interventions to attenuate driven overeating and weight regain after bariatric surgery [J]. Front Endocrinol (Lausanne), 2022, 13: 934680.
[30]
Bosc L, Mathias F, Monsaingeon M, et al. Long-term changes in body image after bariatric surgery: An observational cohort study [J]. PLoS One, 2022, 17(12): e0276167.
[31]
Rabah SM, Albrahim RA, Almajed EH, et al. Desire for body contouring surgery after bariatric surgery: a nationwide cross? sectional study in saudi arabia [J]. Plast Reconstr Surg Glob Open, 2023, 11(12): e5483.
[32]
Mauro MFFP, Papelbaum M, BrasilM AA, et al. Mental health and weight regain after bariatric surgery: associations between weight regain and psychiatric and eating-related comorbidities[J]. Arch Endocrinol Metab, 2024, 68: e230208
[33]
de Lourdes M, Cerqueira L, Pinto-Bastos A, et al. Understanding uncontrolled eating after bariatric surgery: the role of excessive skin and body image shame [J]. J Clin Med, 2021, 10(13): 2967.
[34]
Schrempft S, Jiménez-Sánchez C, Baysson H, et al. Pathways linking BMI trajectories and mental health in an adult population-based cohort: role of emotional eating and body dissatisfaction [J]. Int J Obes (Lond), 2025; 49(7): 1317-1326.
[35]
Goldschmidt AB, Smith KE, Thomas JG, et al. Real-time versus retrospective self-report assessment of loss-of-control eating in adults undergoing bariatric surgery [J]. Obesity (Silver Spring), 2023, 31(2): 363-373.
[36]
Ramaswamy MPA, Palaniswamy S. Multimodal emotion recognition: A comprehensive review, trends, and challenges [J]. Wiley Interdiscip Rev Data Min Knowl Discov, 2024, 14(6): e1563.
[37]
Lian H, Lu C, Li S, et al. A survey of deep learning-based multimodal emotion recognition: speech, text, and face [J]. Entropy, 2023, 25(10): 1440.
[38]
Kaur M, Kumar M. Facial emotion recognition: A comprehensive review [J]. Expert Syst, 2024, 41(10): e13670.
[39]
Ahmed N, Al Aghbari Z, Girija S, et al. A systematic survey on multimodal emotion recognition using learning algorithms [J]. Intell Syst Appl, 2023, 17, 200171.
[40]
Poria S, Cambria E, Bajpai R, et al. A review of affective computing: From unimodal analysis to multimodal fusion [J]. Information Fusion, 2017, 37: 98-125.
[41]
Shoumy NJ, Ang LM, Seng KP, et al. Multimodal big data affective analytics: A comprehensive survey using text, audio, visual and physiological signals [J]. J Netw Comput Appl, 2020, 149: 102447.
[42]
Abdullah SMSA, Ameen SYA, Sadeeq MA, et al. Multimodal emotion recognition using deep learning [J]. J Appl Sci Technol Trends, 2021, 2: 52-58.
[43]
Salas-Cáceres J, Lorenzo-Navarro J, Freire-Obregón D, et al. Multimodal emotion recognition based on a fusion of audiovisual information with temporal dynamics [J]. Multimed Tools Appl, 2025, 84(23): 27327-27343.
[44]
Manderino LM, et al. Cognitive dysfunction predicts poorer emotion recognition in bariatric surgery candidates [J]. Obes Surg, 2016, 26(9): 2302-2308.
[45]
Vaioli G, Scarpina F. Facial emotion recognition in obesity and in fibromyalgia: a systematic review [J]. NeuroSci, 2021, 2(2): 127-141.
[46]
Preece DA, Ditzer J, Gross JJ, et al. Emotion regulation [M]. Oxford Research Encyclopedia of Psychology, 2025.
[47]
Kumar AJR, Bhanu B. Micro-expression classification based on landmark relations with graph attention convolutional network [C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
[48]
Wan WJ, Wang SJ, Liu YJ, et al. CAS(ME) 3: a third generation facial spontaneous micro-expression database with depth information and high ecological validity [J]. IEEE Trans Pattern Anal Mach Intell, 2022, 45(3): 2782-2800.
[49]
Robinson A, Husband A, Slight R, et al. Designing digital health technology to support patients before and after bariatric surgery: qualitative study exploring patient desires, suggestions, and reflections to support lifestyle behavior change [J]. JMIR Hum Factors, 2022, 9(1): e29782.
[50]
Huang EY, Chung D, Hollandsworth HM, et al. Bite by byte: can fitness wearables help bariatric patients lose more weight after surgery? [J]. Surg Endosc, 2023, 37(8): 6526-6531.
[51]
Yang C, Kessler M, Taebi N, et al. Smartphone application-based follow-up care of patients after bariatric surgery: A mixed-method study of usability [J]. Digit Health, 2022, 8:20552076221129072.
[52]
Sysko R, Michaelides A, Costello K, et al. An initial test of the efficacy of a digital health intervention for bariatric surgery candidates [J]. Obes Surg, 2022, 32(11): 3641-3649.
[53]
Jiang DM, Chen JX, Zhang PW, et al. Accurate EEG-based emotion recognition on combined features using deep convolutional neural networks [J]. IEEE Access, 2019, 7: 44317-44328.
[54]
Jiang D, Chen J, Zhang P, et al. Emotion recognition from spatiotemporal EEG representations with hybrid convolutional recurrent neural networks via wearable multi-channel headset [J]. Comput Commun, 2020, 154: 58-65.
[55]
Jiang D, Yang L, Xia X, et al. Integrating deep and shallow models for multi-modal depression analysis - Hybrid architectures [J]. IEEE Trans Affect Comput, 2018, 12(1): 239-253.
[56]
Law S, Dong S, Zhou F, et al. Bariatric surgery and mental health outcomes: an umbrella review [J]. Front Endocrinol (Lausanne), 2023, 14: 1283621.
[57]
Guo Z, Yang M, Lin L, et al. E-MFNN: an emotion-multimodal fusion neural network framework for emotion recognition [J]. PeerJ Comput Sci. 2024, 10:e1977.
[58]
Sumithran P, Roberts L, Caterson ID, et al. Incidence of adverse mental health outcomes after sleeve gastrectomy compared with gastric bypass and restrictive bariatric procedures: a retrospective cohort study [J]. Obesity, 31(7): 1913-1923.
[59]
Nandini D, Yadav J, Singh V, et al. An ensemble deep learning framework for emotion recognition through wearable devices multi-modal physiological signals [J]. Sci Rep, 2025, 15(1): 17263.
[60]
Guo JY, Yang NL, Liang H, et al. Application effects of mindful eating intervention in patients after bariatric and metabolic surgery [J]. Chin J Bases Clin Gen Surg, 2025, 32(5): 596-602.
[61]
Rhee KE, Boutelle KN, Liang J, et al. Effect of attendance of the child on body weight, energy intake, and physical activity in childhood obesity treatment: a randomized clinical trial [J]. JAMA Pediatr, 2017, 171(7): 622-628.
[62]
Li J, Washington P. A comparison of personalized and generalized approaches to emotion recognition using consumer wearable devices: machine learning study [J]. JMIR AI, 2024, 3: e52171.
[63]
Konttinen H, Sjöholm K, Carlsson LMS, et al. Fifteen-year changes in health-related quality of life after bariatric surgery and non-surgical obesity treatment [J]. Int J Obes (Lond), 2024, 48(10): 1447-1456.
[64]
Cheung HC, Strodl E, Musial J, et al. Associations between diet composition, dietary pattern, and weight outcomes after bariatric surgery: a systematic review [J]. Int J Obes (Lond), 2023, 47(9): 764-790.
[65]
Sundgot-Borgen C, Bond DS, Sniehotta FF, et al. Associations of changes in physical activity and sedentary time with weight recurrence after bariatric surgery: a 5-year prospective study [J]. Int J Obes (Lond), 2023, 47(6): 463-470.
[66]
Nielsen MS, Christensen BJ, Schmidt JB, et al. Predictors of weight loss after bariatric surgery-a cross-disciplinary approach combining physiological, social, and psychological measures [J]. Int J Obes (Lond), 2020, 44(11): 2291-2302.
[1] 王小慧, 赵云茜, 赵德莉, 靳思雨, 罗旭芳. 集束化气道护理在吸入性损伤气管切开患者中的临床效果观察[J/OL]. 中华损伤与修复杂志(电子版), 2026, 21(04): 284-288.
[2] 冯颖, 祝红娟, 王倩, 高猛猛, 郭佳宁, 马妍. 负压伤口疗法联合生物活性敷料在糖尿病足溃疡中的应用研究进展[J/OL]. 中华损伤与修复杂志(电子版), 2026, 21(04): 309-314.
[3] 吕斐翠, 卢芳燕, 王璐. 肝移植受者体力活动的范围综述[J/OL]. 中华移植杂志(电子版), 2026, 20(02): 137-143.
[4] 汪甲甲, 黄敏. 肝病相关血小板减少症合并腹股沟疝术后应用疝气带及精细化护理的效果[J/OL]. 中华疝和腹壁外科杂志(电子版), 2026, 20(04): 445-447.
[5] 马建惠, 白洁, 赵叶, 张丽. 奥瑞姆自护理论干预模式在食管裂孔疝术后患者中的应用效果[J/OL]. 中华疝和腹壁外科杂志(电子版), 2026, 20(03): 289-295.
[6] 郭芸青, 周敏露, 郁烨. 基于生物反馈理论的术前心理干预在腹股沟疝修补术患者中的应用[J/OL]. 中华疝和腹壁外科杂志(电子版), 2026, 20(03): 346-350.
[7] 丁育, 林倩, 王志, 吴文睿, 张克林, 赵海璇. 肝移植术后早期感染风险简易评分模型构建及分级护理策略[J/OL]. 中华肝脏外科手术学电子杂志, 2026, 15(04): 619-626.
[8] 常炜, 谢剑锋. 心搏骤停后低温治疗:能够获益[J/OL]. 中华重症医学电子杂志, 2026, 12(02): 157-160.
[9] 孔琴, 岳慧玲, 侯卫华, 郑智, 何阿祥, 赵松, 陆群峰. 基于互联网技术赋能的医护康协作模式在前交叉韧带重建术后患者康复中的应用与评价[J/OL]. 中华临床医师杂志(电子版), 2026, 20(04): 265-274.
[10] 国际血管联盟中国分部护理专业委员会. 子宫肌瘤行动脉栓塞介入治疗护理规范专家共识(2026年)[J/OL]. 中华介入放射学电子杂志, 2026, 14(03): 311-322.
[11] 胡琴, 黄艳艳, 向华, 李兰, 李程伊. 水化治疗预防介入术后造影剂肾病发生的最佳证据总结[J/OL]. 中华介入放射学电子杂志, 2026, 14(02): 201-207.
[12] 张雪儿, 郝婧, 赵贝, 陈紫薇, 韩优, 阳柳柳, 陈倩倩. 复方聚乙二醇电解质散联合西甲硅油肠道准备的护理方案构建与应用效果[J/OL]. 中华胃肠内镜电子杂志, 2026, 13(02): 130-133.
[13] 尹星, 杜健, 王俊雅, 崔琴琴, 杜康, 王盈盈. 急性阑尾炎患者经肛超级微创术的护理经验分享[J/OL]. 中华胃肠内镜电子杂志, 2026, 13(02): 139-140.
[14] 杨妍, 俞素英, 张慧嫔, 何睿, 蒋永萍, 李小芳. 脑心健康管理师主导的互联网+延续性护理对脑卒中患者自我管理行为的影响[J/OL]. 中华脑血管病杂志(电子版), 2026, 20(03): 301-307.
[15] 胡佳敏, 廖佳怡, 邹菲燕, 刘相伶, 周振宇, 刘红华. 基于CiteSpace的脑卒中跌倒护理领域的研究热点和趋势的可视化分析[J/OL]. 中华脑血管病杂志(电子版), 2026, 20(03): 320-326.
阅读次数
全文


摘要


AI


AI小编
你好!我是《中华医学电子期刊资源库》AI小编,有什么可以帮您的吗?