"Voice biomarker" tech analyzes your voice for signs of depression

Illustration: Natalie Peeples/Axios

Software that analyzes snippets of your speech to identify mental health problems is rapidly making its way into call centers, medical clinics and telehealth platforms.

• The idea is to detect illnesses that might otherwise go untreated.

Why it matters: Proponents of "voice biomarker" technology say the underlying artificial intelligence is good enough to recognize depression, anxiety and a host of other maladies.

• But its growing ubiquity raises privacy concerns similar to those brought about by facial recognition.

Driving the news: Hospitals and insurance companies are installing voice biomarker software for inbound and outbound calls, so that — with patients' explicit permission — they can identify in real time if someone they're chatting with may be anxious or depressed, and refer them for help.

• The technology is also making inroads in doctors' offices, clinical trials and senior centers — to identify problems on the spot and monitor patients remotely.
• Employers are looking to use voice biomarkers on their employee help lines.
• Companies that sell the technology — such as Kintsugi, Winterlight Labs, Ellipsis Health and Sonde Health — are raising money, racking up clients and pointing to growing clinical evidence of its efficacy.
• Apps designed for personal use, such as Sonde Mental Fitness, encourage people to keep a daily voice journal so AI can detect stress and other changes.

Where it stands: A scant but growing number of scientific studies support claims that voice biomarkers can help screen for everything from depression to cardiovascular problems to respiratory ailments such as COVID-19, asthma and COPD.

• Depressed patients "take more pauses" and "stop more often," Maria Espinola, a psychologist and assistant professor at the University of Cincinnati College of Medicine, told the New York Times.
• "Their speech is generally more monotone, flatter and softer," she said. "They also have a reduced pitch range and lower volume."

How it works: Unlike Siri and Alexa, vocal biomarker systems analyze how you talk — prosody, intonation, pitch, etc. — but not what you say.

• Your voice sample is run through a machine-learning model that uses a capacious database of anonymized voices for comparison.

What they're saying: "From as little as 20 seconds of free-form speech, we're able to detect with 80% accuracy if somebody is struggling with depression or anxiety," Kintsugi CEO Grace Chang tells Axios.

• That figure comes from comparing the results of Kintsugi's tech to a mental health professional's clinical assessment.
• "When we're integrated into a call center where there is a nurse on the line, the nurse can ask additional questions" when there's a positive match, Chang said.
• Before it's activated, patients are asked if they consent to having their voice analyzed for health screening purposes. About 80% agree, says Chang.

Where it stands: Kinsugi is seeking Food and Drug Administration clearance for its product to become a recognized diagnostic tool.

• Sonde Health positions its system differently. "We are an early warning system — we are not a diagnostic device," Sonde CEO David Liu tells Axios.
• Sonde's technology — which screens for depression, anxiety and respiratory problems — is being integrated into hearing aids and tested by the Cognitive Behavior Institute.

• "From a few seconds of 'ahhh...' we're pretty able to tell if you have symptoms of respiratory disease," Liu tells Axios.

Yes, but: Skepticism and ethical questions surround voice biomarker tech, which observers describe as promising but not foolproof — and ripe for potential misuse.

• "There is still a long way to go before AI-powered vocal biomarkers can be endorsed by the clinical community," reads an editorial published by medical journal The Lancet. "Better ethical and technical standards are required for this research to fully realize the potential for vocal biomarkers in the early detection of disease."
• One risk is that the systems "may increase systemic biases towards people from specific regions, backgrounds, or with a specific accent," per a review in the journal Digital Biomarkers.
• “My research suggests that the ability of AI to identify vocal biomarkers is often oversold, or else glosses over the highly subjective processes involved in building these systems," writes Beth Semel, an assistant professor of anthropology at Princeton who has studied vocal biomarkers since 2015.

For example: Amazon's Halo wearable has drawn heat for its vocal tone tracking feature, which uses a form of voice biomarker technology.

• "People of color and women are traditionally more discriminated against by AI systems, and Amazon's own AI research has struggled with this very problem in the past," per tech news site Protocol.

The bottom line: The zeal of Big Pharma, insurance and health care companies will likely propel voice biomarker tech forward, and its early detection powers could prove helpful in diagnosing myriad conditions — if the related efficacy and privacy concerns are kept in check.