Conversational agents (CAs) like Amazon’s Alexa and Apple’s Siri are designed to answer questions, offer suggestions, and even display empathy. However, new research indicates that they fall short compared to humans in interpreting and exploring a user’s experience.
CAs are powered by large language models (LLMs) that ingest massive amounts of human-produced data, and thus can be prone to the same biases as the humans from which the information comes.
Researchers from Cornell University, Olin College, and Stanford University tested this theory by prompting CAs to display empathy while conversing with or about 65 distinct human identities.
Value Judgments and Harmful Ideologies
The team found that CAs make value judgments about certain identities – such as gay and Muslim – and can be encouraging of identities related to harmful ideologies, including Nazism.
“I think automated empathy could have tremendous impact and huge potential for positive things – for example, in education or the health care sector,” said lead author Andrea Cuadra, now a postdoctoral researcher at Stanford.
“It’s extremely unlikely that it (automated empathy) won’t happen,” she said, “so it’s important that as it’s happening, we have critical perspectives so that we can be more intentional about mitigating the potential harms.”
Cuadra will present “The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction” at CHI ’24, the Association of Computing Machinery conference on Human Factors in Computing Systems, May 11-18 in Honolulu. Research co-authors at Cornell University included Nicola Dell, associate professor, Deborah Estrin, professor of computer science, and Malte Jung, associate professor of information science.
Emotional Reactions vs. Interpretations
Researchers found that, in general, LLMs received high marks for emotional reactions, but scored low for interpretations and explorations. In other words, LLMs are able to respond to a query based on their training but are unable to dig deeper.
Dell, Estrin, and Jung said they were inspired to think about this work as Cuadra was studying the use of earlier-generation CAs by older adults.
“She witnessed intriguing uses of the technology for transactional purposes such as frailty health assessments, as well as for open-ended reminiscence experiences,” Estrin said. “Along the way, she observed clear instances of the tension between compelling and disturbing ‘empathy.’”
Reference: “The Illusion of Empathy? Notes on Displays of Emotion in Human-Computer Interaction” by Andrea Cuadra, Maria Wang, Lynn Andrea Stein, Malte F. Jung, Nicola Dell, Deborah Estrin and James A. Landay, 11 May 2024, CHI ’24.
DOI: 10.1145/3613904.3642336
News
Lingering Viral Fragments: The Hidden Cause of Long COVID
Long COVID, affecting 5-10% of COVID-19 patients, might be caused by the enduring presence of the virus in the body. Research suggests that viral fragments, possibly live, linger and lead to symptoms. Addressing this involves antiviral treatments, enhanced [...]
Hidden Scars: How COVID Lockdowns Altered Teen Brains Forever
Research from the University of Washington revealed that COVID-19 lockdowns led to accelerated cortical thinning in adolescents, impacting brain development significantly. This effect was more pronounced in females than males, raising concerns about long-term brain health. The study [...]
Simple Blood Test To Detect Dementia Before Symptoms Appear
UCLA researchers have identified placental growth factor (PlGF) as a potential blood biomarker for early detection of cognitive impairment and dementia. High PlGF levels correlate with increased vascular permeability, suggesting its role in the development [...]
Investing Goldman Sachs asks ‘Is curing patients a sustainable business model?’
Goldman Sachs analysts attempted to address a touchy subject for biotech companies, especially those involved in the pioneering “gene therapy” treatment: cures could be bad for business in the long run. “Is curing patients [...]
The risks of reversed chirality: Study highlights dangers of mirror organisms
A groundbreaking study evaluates the feasibility, risks, and ethical considerations of creating mirror bacteria with reversed chirality, highlighting potential threats to health and ecosystems. In a recent study published in Science, a team of researchers [...]
Alarming Mutation in H5N1 Virus Raises Pandemic Red Flags
NIH-funded study concludes that the risk of human infection remains low A recent study published in Science and funded by the National Institutes of Health (NIH) has found that a single alteration in a protein on the surface [...]
Scientists Discover Genetic Changes Linked to Autism, Schizophrenia
The Tbx1 gene influences brain volume and social behavior in autism and schizophrenia, with its deficiency linked to amygdala shrinkage and impaired social incentive evaluation. A study published in Molecular Psychiatry has linked changes in brain [...]
How much permafrost will melt this century, and where will its carbon go?
Among the many things global warming will be melting this century—sea ice, land glaciers and tourist businesses in seaside towns across the world—is permafrost. Lying underneath 15% of the northern hemisphere, permafrost consists of [...]
A Physics Discovery So Strange It’s Changing Quantum Theory
MIT physicists surprised to discover electrons in pentalayer graphene can exhibit fractional charge. New theoretical research from MIT physicists explains how it could work, suggesting that electron interactions in confined two-dimensional spaces lead to novel quantum states, [...]
Inside the Nano-Universe: New 3D X-Ray Imaging Transforms Material Science
A cutting-edge X-ray method reveals the 3D orientation of nanoscale material structures, offering fresh insights into their functionality. Researchers at the Swiss Light Source (SLS) have developed a groundbreaking technique called X-ray linear dichroic orientation tomography [...]
X-chromosome study reveals hidden genetic links to Alzheimer’s disease
Despite decades of research, the X-chromosome’s impact on Alzheimer’s was largely ignored until now. Explore how seven newly discovered genetic loci could revolutionize our understanding of the disease. Conventional investigations of the genetic contributors [...]
The Unresolved Puzzle of Long COVID: 30% of Young People Still Suffer After Two Years
A UCL study found that 70% of young people with long Covid recovered within 24 months, but recovery was less likely among older teenagers, females, and those from deprived backgrounds. Researchers emphasized the need [...]
Needle-Free: New Nano-Vaccine Effective Against All COVID-19 Variants
A new nano-vaccine developed by TAU and the University of Lisbon offers a needle-free, room-temperature-storable solution against COVID-19, targeting all key variants effectively. Professor Ronit Satchi-Fainaro’s lab at Tel Aviv University’s Faculty of Medical and [...]
Photoacoustic PDA-ICG Nanoprobe for Detecting Senescent Cells in Cancer
A study in Scientific Reports evaluated a photoacoustic polydopamine-indocyanine green (PDA-ICG) nanoprobe for detecting senescent cells. Senescent cells play a role in tumor progression and therapeutic resistance, with potential adverse effects such as inflammation and tissue [...]
How Dysregulated Cell Signaling Causes Disease
Cell signaling is crucial for cells to communicate and function correctly. Disruptions in these pathways, caused by genetic mutations or environmental factors, can lead to uncontrolled cell growth, improper immune responses, or errors in [...]
Scientists Develop Super-Strong, Eco-Friendly Plastic That Bacteria Can Eat
Researchers at the Weizmann Institute have developed a biodegradable composite material that could play a significant role in addressing the global plastic waste crisis. Billions of tons of plastic waste clutter our planet. Most [...]
