HomePal: Developing a Smart Speaker-Based System for In-Home Loneliness Assessment for Older Adults

Overview

Experiences of loneliness are prevalent among older adults. Loneliness is a painful and pernicious state occurring when there is a perceived discrepancy between one’s optimal levels of social interactions and actual social relationships. Lonely older adults are more likely to experience functional decline, including activities of daily living, mobility, and stair climbing.  Loneliness has been associated with negative health outcomes, such as increased morbidity and mortality, dementia risk, and cognitive impairment.  With longer life expectancy rates, the number of older adults at risk for loneliness will increase substantially, presenting challenges to the healthcare system. However, studies reported that loneliness is a reversible condition, which could be achieved through appropriate interventions, such as improving physical health and social relationships. Detection measures are imperative to identify both lonely and at-risk individuals early enough to intervene before adverse health outcomes occur. The assessment of loneliness is challenging due to the social stigma of being labeled as lonely, resulting in underestimation. Although multidimensional scales exist that do not explicitly use the word “lonely,” healthcare providers do not routinely assess loneliness using validated instruments, making it challenging to be aware of the extent of loneliness among their older adult patients.

To tackle the problem, we will use digital biomarker data extracted from smart speakers and passive sensing devices to automatically assess the level of loneliness (as correlates of self-report scores on the UCLA Loneliness Scale) among community-dwelling older adults. The increasing adoption of smart speakers (e.g., Amazon Echo) presents a unique opportunity for identifying lonely individuals in the community. Increasing evidence supports the extraction of key linguistic and acoustic features from speech data as digital biomarkers for detecting individuals with mental health conditions, including loneliness. Additionally, daily computer and phone use patterns are related to loneliness, therefore smart speakers have the potential to provide streams of behavioral data that represent user loneliness, such as use frequency, duration of interactions with the virtual agent (e.g., Alexa), and types of commands (e.g., music vs. conversations). In this project, we develop, deploy, and validate a smart speaker-based system for in-home assessment of loneliness in older adults (its proposed name is “HomePal”). The system will consist of two main novel components. First, we will equip smart speakers with the functionality required to gather both speech (acoustic and prosodic features) and behavioral data (e.g., smart speaker use patterns, in-home mobility, sleep) from older adults through IoT devices. Second, we will develop and validate novel multi-class machine learning (ML) models—semi-supervised type of Generative Adversarial Networks (SGANs) —utilizing the collected speech and behavioral data to estimate loneliness scores in older adults. Compared to the existing binary classification, the multi-class classification provides sufficient resolution to assess the individual's loneliness level in a fine-grained scale. This will allow us to identify not only “already” lonely individuals but also “at-risk” individuals. Moreover, it will allow us to monitor the slight changes in loneliness and provide more information on the effectiveness of future loneliness reduction intervention mechanisms.

People

Faculty

• Tamer Nadeem
• Jane Chung