From Robo Doctors to Mood Trackers, Emotion Sensing Is Coming
The proliferation of smartphones and wearable devices has enabled people to track all kinds of data about their physical health, from heart rate to steps walked in a day. Now, tech companies have their sights set on helping us—and perhaps others—monitor our mental health.
“We’ve gotten very good at quantifying the physical self,” said Affectiva chief marketing officer Gabi Zijderveld. “For the most part in consumer health, mental well-being is still missing from that equation. … Looking out to the future, what personally excites me is it’s almost an inevitable path that we’re going to get better at quantifying mood and mental health.”
Zijderveld’s startup is among those working on that. She spoke on a panel in Boston Thursday with representatives from other companies developing emotion-sensing technologies, including Microsoft researcher Mary Czerwinski and Sensoree founder Kristin Neidlinger. The discussion was part of the annual Connected Health Symposium organized by Partners HealthCare.
The conversation made it clear that emotion-sensing technology has a long way to go, particularly in applying it to healthcare. But the field is progressing quickly, and there are a lot of ideas for how the technology could make our lives better.
Waltham, MA-based Affectiva, which spun out of the MIT Media Lab in 2009, has built its business primarily by selling its emotion-recognition software and analysis tools to brands and businesses that use it to make marketing and advertising content more engaging for target customers, Zijderveld said. But in recent years, Affectiva has branched out into video games, online education, the automotive sector, robotics, and healthcare, she said. The company now licenses its technology to other companies that can embed the software into their products.
In healthcare, Affectiva envisions its technology being used in telemedicine, so doctors could track patients’ moods and communicate better. It could also help monitor the mental health of people suffering from depression and anxiety disorders, or serve as a check against biases of patients reporting how well an experimental treatment is working, Zijderveld said.
“Healthcare is a bit early days for us,” she said. “There’s a lot of research being done using our technology. To date, there’s not a lot of solutions in the market, but we expect that’ll change in the next couple of years.”
Meanwhile, Microsoft’s research arm has launched a number of emotion-tracking projects in recent years, experimenting with smartphone sensors, biometric sensors, Microsoft’s Kinect sensors, voice analysis, and more, Czerwinski said.
“We’ve tried all these different ways of sensing, and then what you try to do is just-in-time interventions,” Czerwinski said. For example, if a mobile app senses a person’s stress might be escalating, it could suggest an activity—such as a breathing exercise, or taking a walk to cool down—to counter those anxious feelings.
Or, perhaps a software program could log data on stress patterns over time. Using context, it could help a person understand what triggers stress, such as work meetings or hanging out with certain people. Those are “things you might not be aware of,” Czerwinski said. (Some other companies working on tracking mental health are Boston-based Neumitra and San Francisco-based Ginger.io.)
Sensoree, based in San Francisco, sits at the intersection of healthcare and fashion. It designs clothing equipped with a variety of biometric sensors that aim to track the wearer’s current mood and communicate it through visual or tactile displays on the garment.
During the panel discussion, Neidlinger brought out a data scientist, Rachel, wearing one of Sensoree’s products, a “mood sweater.” It features a bowl-shaped collar that lights up in different colors based on the sensors’ interpretation of how the wearer feels. It turned pink as Rachel walked out on stage, which Neidlinger said indicated she was “a little excited and ruffled.” (It later turned blue, indicating she had calmed down.)
Of course, some might not like the idea of literally wearing their emotions on their sleeve, collar, or other article of clothing. But Sensoree’s products could help caregivers get a better read on how patients feel, Neidlinger said. The clothing could also help people who have trouble identifying and understanding their emotions to know when their mood is “dipping down,” Neidlinger said—like a Pavlovian signal for sadness, she said.
“We help people who might have a hard time communicating how they’re feeling, or even knowing how they’re feeling,” Neidlinger said.
Microsoft’s Czerwinski said intelligent virtual assistants present a big opportunity for emotion-sensing tech. Think Siri (or in Microsoft’s case, Cortana), but with empathy and deep medical knowledge. Companies such as Wisconsin-based iDAvatars are already working on this.
“Conversational agents could be your teledoctor, your teletherapist—someone who just knows you really well … and can help take care of you,” Czerwinski said. The personal virtual assistant would relay important information to loved ones and healthcare providers. The key, she added, would be to build a strong level of trust and a rapport between the user and the software.
(12)
