Project Aiden
​
Advancing Autism Care Through Wearable Physiological Sensing
Overview
Project Aiden is my thesis project completed during my final semester as a Master of Design student at UC Berkeley. My dedication to this project is deeply personal, inspired by my younger brother, who has 'profound autism'. This project specifically addresses the unique communication challenges faced by individuals with profound autism, who have severe cognitive impairments and significant difficulties in standard communication.
​
​​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
Objectives:
​
-
Innovative Communication Approach: To explore non-verbal, physiological, and emotion-based communication methods, diverging from traditional AAC (Alternative and Augmentative Communication) devices, which often fall short for this specific demographic.
-
Utilization of Wearable Technology: The project hypothesizes the effective use of wearable physiological sensing and affect-based detection systems in communicating basic needs, like thirst, for those with profound autism.
Deliverables:
​
-
Development of 'Aiden': A user-friendly AAC platform named 'Aiden' was developed, which functions as a caregiver assistant app. It is customized and tailored for individual use, providing real-time assistance based on detected physiological cues.
-
Machine Learning Analysis: Advanced machine learning models were employed to analyze collected data for discerning patterns related to physiological states and emotional cues, with a focus on identifying indicators of thirst.
-
Real-Time Assistance and Chatbot Integration: 'Aiden' offers real-time visualization and alerts for caregivers based on physiological data, and includes a chatbot for easy input and access to critical information about the person under care.
Making Profound Autism Visible
The term 'Profound Autism' describes individuals on the autism spectrum with severe intellectual disabilities and limited communication abilities, often with an IQ below 50. They require 24-hour care and face complex challenges such as self-injury, aggression, and epilepsy.
Mainstream autism advocacy and media often focus on more independent individuals, leaving those with profound autism feeling forgotten. My commitment to this project stems from the desire to make a meaningful difference, not only for my brother but also for others like him.
Illustration inspired by my younger brother, Anshu: Capturing the everyday challenges he bravely faces at 21
Caregiver Perspectives on Autism: Call for Enhanced Support
Existing communication methods prove inadequate for individuals with profound autism, leaving them with limited means to express their needs and emotions. However, primary caregivers, with their intimate knowledge of these individuals, often excel at deciphering their non-traditional forms of communication. Despite their invaluable role, caregivers face a heightened risk of depression and anxiety due to the demands of round-the-clock care. Therefore, it becomes imperative to offer comprehensive support systems and respite opportunities to sustain the well-being of both individuals with profound autism and their dedicated caregivers.
​
.png)
.png)
.png)
A storyboard illustrating the everyday challenges faced by families navigating life with individuals with profound autism.
Enhancing Traditional Communication
-
Much research and technology have focused on augmenting traditional communication, even generating verbal speech. However, these solutions often require volitional training and fine motor control.
​​
​​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
-
For those with nascent or non-existent volitional speech-related muscle control, such solutions fall short, especially for minimally verbal or non-verbal individuals with autism.
​​
-
Augmentative and Alternative Communication (AAC) methods, including pictures, gestures, sign language, visual aids, and speech-output devices like computers, provide valuable alternatives. However, they may also require a minimum level of cognitive ability.
​​
Electromyography (EMG) Sensors used to generate silent speech
Symbol-based Alternative and Augmentative Device
User Research
For user research, I conducted interviews with four primary caregivers, including my own mother, all of whom provide care for children diagnosed with profound autism. The objective was to gain insights into the diverse range of needs within this demographic and the assistive technologies they rely on in their daily lives. These interviews provided valuable understanding of their specific requirements and preferences.
​​
.jpg)
Yael and Yonathan
User Research Insights
Roopa and Rohan
Akiko and Luke
Kiran and Anshu
-
Rohan, aged 18, non-verbal, has recently become comfortable using Proloquo after more than a decade of struggle with communication. Despite this progress, he still faces challenges.
-
Yonathan (18 years old) utilizes spell-to-speak technology, but his reliance on it varies. Sometimes, he prefers not to spell out his thoughts and expects his mother to understand him intuitively.
-
Luke, aged 24, under Akiko's care, relies on picture-based communication. While he can express basic needs, he struggles with conveying more complex emotions and desires.
-
Anshu, aged 21, with very low cognitive ability, does not currently utilize any communication technology.
Despite their access to assistive technologies, none of the users fully rely on them, indicating gaps in current solutions and the ongoing struggle for autonomy and normalcy in their daily lives. All the caregivers expressed interest in platform that could provide information about physical state (e.g., pain, emotions, hunger) without requiring active input from a user.
​​
Harnessing Physiology for Communication
-
An alternative approach involves exploring non-speech communication, including affect and physiological analyses.
-
These studies delve into understanding emotions and intent through physiological and behavioral cues.
-
This avenue acknowledges the unique challenges faced by individuals with minimal speech capabilities, offering a promising direction for addressing their communication needs.
​​
EmbracePlus: Advanced smartwatch for continuous health monitoring
Ideation and Process
-
Idea Generation: In the initial stages of ideation, I conceptualized a caregiver assistant app customized for individuals with profound autism, aiming to provide personalized real-time assistance and insights.
​
-
Exploration of Physiological Data: I considered leveraging physiological data, such as heart rate and Electrodermal Activity (EDA), to understand basic needs. The goal was to identify patterns in this data that could indicate when assistance or support was required.
​
-
Data Collection and Analysis: My plan involved collecting relevant physiological data and analyzing it to uncover patterns indicative of specific needs. This process included thorough examination and preprocessing of the data to ensure accuracy and reliability.
​
-
Machine Learning Model Development: I intended to develop machine learning models capable of detecting and alerting caregivers when specific needs were identified based on the analyzed physiological data. These models would be trained to recognize patterns associated with various needs.
​
-
Creation of User-Friendly Platform: Alongside model development, I aimed to design a user-friendly platform where caregivers could visualize the individual's needs using the collected physiological data. This platform would empower caregivers to respond promptly and effectively to ensure the well-being of the individual.
How might we optimize the capture of naturalistic non-verbal communication, enhance data quality, personalize machine learning methods, and design an engaging Augmentative and Alternative Communication (AAC) interface to improve communication for individuals with profound autism?
Process Diagram
Data Collection and Analysis
Gathering data directly from the profound autism community posed challenges for me due to highly individualized needs. Insights from my participation in an adult autism/DD conference at Stanford prompted a pivot towards customized solutions.
Data collection began with neurotypical individuals, emphasizing hydration needs initially. This pivot from profound autism to neurotypical data collection was due to time constraints and participant availability. The hypothesis that effective communication solutions for neurotypical individuals would extend to those with profound autism streamlined context-based data collection and analysis, as neurotypical individuals could offer insights into their communication needs.
​
-
Collection Method and Participants: Data meticulously collected from 10 neurotypical individuals, each for a duration of 30-40 minutes, using the Empatica E4 sensor to measure physiological responses related to different states, including dehydration.
​​
-
Protocol and Sensor Usage: Participants adhered to a protocol refraining from liquid intake after waking up, enabling capture of data in both dehydrated and hydrated states.
.jpeg)
.jpeg)
Data Collection using Empatica E4
Machine Learning and Predictive Analysis
Data preprocessing emerged as the most challenging aspect of the project due to the complexity of the collected raw data. To address this, the FLIRT (Feature Generation Toolkit for Wearable Data) was employed, allowing for efficient processing of the data.
Subsequently, various machine learning models were evaluated to distinguish between dehydration and hydration states. Leveraging the Random Forest algorithm for hydration state prediction yielded a remarkable accuracy score of 0.891, signifying robust model performance in accurately identifying changes in hydration levels.
Designing a Preliminary AAC Platform
-
Aimed at caregivers, visualizing basic needs and offering real-time alerts and notifications.
-
Empatica E4 sensor integration for continuous physiological data monitoring.
-
Intuitive UI catering to the specific needs of users with profound autism and their caregivers.
Design Explorations for UI
Feedback: "I don't believe using a graph is the optimal method to present this information on the UI. While graphs display predictions, you could represent the UI in a more user-friendly and intuitive manner."
Making the UI more intuitive and visual, while also considering the technical literacy of the caregivers.
Initial Setup and Personalization
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
Comprehensive data input phase for caregivers, including hydration, meal preferences, communication methods, allergies, and personal data.
Real-Time Monitoring
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
Alert System and Post-Alert Logging​
​
​
​
Interactive Chatbot
The platform employs advanced AI, including GPT-4 and RAG models, to power an intuitive chatbot that simplifies database queries and information management. The platform employs advanced AI, including GPT-4 and RAG models, to power an intuitive chatbot that simplifies database queries and information management.
Key Takeaways
-
Complexity of Real-time and Human Data: Acknowledged the challenges in working with real-time and human data, especially in the context of applying machine learning models. This experience underlined the complexity of integrating technology with human-centric data and the need for extensive research for successful implementation.
​​
-
Deep Dive into Autism and Physiology: Through this project, I gained profound insights into autism and human physiology. I learned about the diverse challenges faced by individuals with autism, reinforcing the need for more dedicated efforts in developing targeted interventions for this demographic.
​​
-
Significance of Personalization: This journey underscored the critical role of personalization in technology. I observed firsthand how tailored solutions could transform a good customer experience into a great one, especially in the context of AAC tools like 'Aiden'.
​​
-
Diversity within the Autism Spectrum: The project illuminated the vast diversity within the autism spectrum, highlighting that no single solution could meet everyone's needs. This realization emphasizes the importance of customizable and adaptable approaches in the development of support tools for individuals with autism.