Towards AI-based Mental Health Providers

Monitoring the mental well-being of people, especially those who have limited access to medical healthcare providers, is important for improving quality of life. Towards this goal, we have developed multiple systems and platforms to enable people to self-screen and monitor their mental health throughout their daily lives, outside of doctor visits.

aiMSE: Towards and AI-based Online Mental Status Examination

User-facing interface for taking the MSE.
User-facing interface for taking the MSE.

Mental status examination (MSE) is an important tool used by mental health providers for assessing mental health. Currently, MSEs are conducted by licensed professionals, which is a barrier for patients in low-income and remote areas. We propose an AI-based Personal Online Mental Status Examination (aiMSE), the first interactive mental status examination platform, where users can self-administer MSEs at home through a web browser, using only a camera and microphone. aiMSE uses multimodal image, speech and natural language processing algorithms to detect signs of abnormalities in mental functioning and recommend them for further examination by a mental health specialist.

Conversational AI Therapist for Homes

Deployments for the AI therapist system.
Deployments for the AI therapist system.

There is a growing number of smart devices, sensors, and actuators throughout our environments. In this project, we take advantage of these devices to continuously screen the day-to-day functioning and infer mental wellness of an occupant. Unlike existing “chatbot” works that identify the mental status of users through conversation, our AI therapist additionally leverages smart devices, sensors, and mobile robot systems (e.g., robot vacuum cleaners) throughout the home to infer mental well-being and assesses a user’s daily functioning.

Publications

AI Therapist for Daily Functioning Assessment and Intervention Using Smart Home Devices
AI Therapist for Daily Functioning Assessment and Intervention Using Smart Home Devices
Jingping Nie, Minghui Zhao, Stephen Xia, Xinghua Sun, Hanya Shao, Yuang Fan, Matthias Preindl, Xiaofan Jiang
Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems   ·   06 Nov 2022   ·   doi:10.1145/3560905.3568074
aiMSE: Toward an AI-Based Online Mental Status Examination
aiMSE: Toward an AI-Based Online Mental Status Examination
Yanchen Liu, Stephen Xia, Jingping Nie, Peter Wei, Zhan Shu, Jeffrey Andrew Chang, Xiaofan Jiang
IEEE Pervasive Computing   ·   01 Oct 2022   ·   doi:10.1109/MPRV.2022.3172419
Conversational AI Therapist for Daily Function Screening in Home Environments
Conversational AI Therapist for Daily Function Screening in Home Environments
Jingping Nie, Hanya Shao, Minghui Zhao, Stephen Xia, Matthias Preindl, Xiaofan Jiang
Proceedings of the 1st ACM International Workshop on Intelligent Acoustic Systems and Applications   ·   27 Jun 2022   ·   doi:10.1145/3539490.3539603