Wearables for Emotion Sensing
Monitoring the emotional well-being of a person is important in many domains, such as health monitoring, elderly care, depression treatment, entertainment, so on and so forth. We present SPIDERS which is a low-cost, wireless, glasses-based platform for continuous in-situ monitoring of user’s facial expressions and real emotions. SPIDERS costs less than $20 to assemble and can continuously run for up to 9 hours before recharging.
SPIDERS has a four-layer system architecture. Each layer in the architecture directly enables the applications or functionalities in the layer directly above. We present algorithms to provide four core functions (eye shape and eyebrow movements, pupillometry, zygomaticus muscle movements, and head movements). The algorithms use the bio-signals acquired from three non-contact sensors (infrared camera, proximity sensor, an inertial measuring unit). SPIDERS can be also extended to include functionalities provided by contact-based sensors, such as heart rate sensors and 8-channel EEG sensors.
Core Function Library
We present novel and robust vision-based techniques to perform pupillometry as well as eye & eyebrow shape detection using IR-band gray-scale images, from an IR camera positioned at a low angle from the eye as to not block the field of view of the user. A proximity sensor is used to detect the movements of zygomaticus muscle (smile muscle), which aims at adding another dimension of knowledge to the facial expression.
Advanced Functionalities
SPIDERS distinguishes between different classes of facial expression and real emotion states based on these four bio-signals. We prototype advanced functionalities, including facial expression detection and real emotion classification. Specifically, we deploy a novel facial expression detector based on landmarks and optical flow that leverages changes in a user’s eyebrows and eye shapes to achieve an accuracy that outperforming the state-of-the-art approach. We also implement a pupillometry-based real emotion classifier with higher accuracy than other low-cost wearable platforms that use sensors requiring skin contact.
Code and Media
Publications
2021
2020
