A computer-assisted needle misses its target, puncturing the spine. A diabetic patient goes rapidly downhill after a computer recommends an incorrect insulin dosage. An ultrasound fails to diagnose an obvious heart condition that is ultimately fatal.
These are just a few examples of incidents reported to the United States’ Food and Drug Administration involving health technology assisted by artificial intelligence (AI), and Australian researchers say they are an “early warning sign” of what could happen if regulators, hospitals and patients don’t take safety seriously in the rapidly evolving field.
“This is essentially showing us that when we’re putting in AI systems, we just need to be taking the safety of these systems really seriously,” Professor Farah Magrabi said.
Her team at the Australian Institute of Health Innovation at Macquarie University this month published a review of 266 safety events involving AI-assisted technology reported to the US watchdog. The article appeared in the Journal of the American Medical Informatics Association.
Only 16 per cent actually led to patients being harmed, but two-thirds were found to have the potential to cause harm, and 4 per cent were categorised as “near-miss events” in which users intervened.
Co-author Dr David Lyell said issues arose most commonly when users failed to enter the correct data, leading to an incorrect result, or misunderstood what the AI was actually telling them when it produced a result.
For example, one patient suffering a heart attack delayed medical care because an over-the-counter electrocardiogram device – which is not capable of detecting a heart attack – told them they had “normal sinus rhythm”.
“AI isn’t the answer; it’s part of a system that needs to support the provision of healthcare. And we do need to make sure that we have the systems in place that supports its effective use to promote healthcare for people,” Lyell said.
The researchers chose to analyse cases in the US, where the implementation of AI-enabled health devices is more advanced than in Australia. The US regulator has, to date, approved 521 artificial intelligence and machine learning-enabled medical devices, with 178 of those added in 2022.
In Australia, the Therapeutic Goods Administration (TGA) does not collect data about the number of approved devices in Australia that have AI or machine-learning components, but Magrabi said the regulator was taking the issue “very, very seriously”.
“AI isn’t the answer, it’s part of a system that needs to support the provision of health care.”
David Lyell, Australian Institute of Health Innovation, Macquarie University
AI has been used in healthcare devices for decades, but an explosion in data collection, computing power and advanced algorithms has opened up new frontiers, said David Hansen, chief executive of the CSIRO’s Australian E-Health Research Centre.
Sydney-based medical device start-up EMVision is one Australian company taking advantage of these advancements to develop a portable device for diagnosing stroke without the need for an MRI.
In the development stage, the company is using an advanced algorithm and high-powered computers to simulate stroke in numerous places in the brain, building up a database of synthetic images similar to MRIs which are then compared with real-life MRI and CT results from hospital clinical trials at Royal Melbourne, Liverpool and Princess Alexandra hospitals.
“We wouldn’t be able to do what we’re doing today if we didn’t have the [high-powered computer] infrastructure for the simulation,” said head of product development Forough Khandan.
Co-founder Scott Kirkland said the intention was not to completely replace CT and MRI scans, but to diagnose stroke in the first hour when treatment is most effective. The bedside device, set to be launched in 2025, will use a “traffic light” system based on a probability algorithm to help determine what type of stroke might have occurred.
“It’s better for an algorithm to give an ‘I don’t know’ than an incorrect answer, and have the wrong treatment and or triage process followed,” Kirkland said.
Radiology is at the forefront of the rapid adoption of AI in healthcare, especially in breast cancer screening and analysis of chest X-rays.
“A couple of years ago, almost no radiologist would say they use it, now a fair percentage would say that they use it in their daily work,” said clinical radiologist and AI safety researcher Dr Lauren Oakden Rayner.
Rayner, a member of the college, said the technology had many potential benefits, but Australian regulators and clinicians needed to better understand the risks of fully autonomous systems before putting them into hospitals, clinics and homes.
“Humans are legally and morally responsible for decision-making, and it’s taking some of that out of human hands,” she said. “There’s no reason autonomous AI systems can’t exist … but they obviously have to be tested very, very tightly.”
News
Magnetic nanoparticles that successfully navigate complex blood vessels may be ready for clinical trials
Every year, 12 million people worldwide suffer a stroke; many die or are permanently impaired. Currently, drugs are administered to dissolve the thrombus that blocks the blood vessel. These drugs spread throughout the entire [...]
Reviving Exhausted T Cells Sparks Powerful Cancer Tumor Elimination
Scientists have discovered how tumors secretly drain the energy from T cells—the immune system’s main cancer fighters—and how blocking that process can bring them back to life. The team found that cancer cells use [...]
Very low LDL-cholesterol correlates to fewer heart problems after stroke
Brigham and Women's Hospital's TIMI Study Group reports that in patients with prior ischemic stroke, very low achieved LDL-cholesterol correlated with fewer major adverse cardiovascular events and fewer recurrent strokes, without an apparent increase [...]
“Great Unified Microscope” Reveals Hidden Micro and Nano Worlds Inside Living Cells
University of Tokyo researchers have created a powerful new microscope that captures both forward- and back-scattered light at once, letting scientists see everything from large cell structures to tiny nanoscale particles in a single shot. Researchers [...]
Breakthrough Alzheimer’s Drug Has a Hidden Problem
Researchers in Japan found that although the Alzheimer’s drug lecanemab successfully removes amyloid plaques from the brain, it does not restore the brain’s waste-clearing system within the first few months of treatment. The study suggests that [...]
Concerning New Research Reveals Colon Cancer Is Skyrocketing in Adults Under 50
Colorectal cancer is striking younger adults at alarming rates, driven by lifestyle and genetic factors. Colorectal cancer (CRC) develops when abnormal cells grow uncontrollably in the colon or rectum, forming tumors that can eventually [...]
Scientists Discover a Natural, Non-Addictive Way To Block Pain That Could Replace Opioids
Scientists have discovered that the body can naturally dull pain through its own localized “benzodiazepine-like” peptides. A groundbreaking study led by a University of Leeds scientist has unveiled new insights into how the body manages pain, [...]
GLP-1 Drugs Like Ozempic Work, but New Research Reveals a Major Catch
Three new Cochrane reviews find evidence that GLP-1 drugs lead to clinically meaningful weight loss, though industry-funded studies raise concerns. Three new reviews from Cochrane have found that GLP-1 medications can lead to significant [...]
How a Palm-Sized Laser Could Change Medicine and Manufacturing
Researchers have developed an innovative and versatile system designed for a new generation of short-pulse lasers. Lasers that produce extremely short bursts of light are known for their remarkable precision, making them indispensable tools [...]
New nanoparticles stimulate the immune system to attack ovarian tumors
Cancer immunotherapy, which uses drugs that stimulate the body’s immune cells to attack tumors, is a promising approach to treating many types of cancer. However, it doesn’t work well for some tumors, including ovarian [...]
New Drug Kills Cancer 20,000x More Effectively With No Detectable Side Effects
By restructuring a common chemotherapy drug, scientists increased its potency by 20,000 times. In a significant step forward for cancer therapy, researchers at Northwestern University have redesigned the molecular structure of a well-known chemotherapy drug, greatly [...]
Lipid nanoparticles discovered that can deliver mRNA directly into heart muscle cells
Cardiovascular disease continues to be the leading cause of death worldwide. But advances in heart-failure therapeutics have stalled, largely due to the difficulty of delivering treatments at the cellular level. Now, a UC Berkeley-led [...]
The basic mechanisms of visual attention emerged over 500 million years ago, study suggests
The brain does not need its sophisticated cortex to interpret the visual world. A new study published in PLOS Biology demonstrates that a much older structure, the superior colliculus, contains the necessary circuitry to perform the [...]
AI Is Overheating. This New Technology Could Be the Fix
Engineers have developed a passive evaporative cooling membrane that dramatically improves heat removal for electronics and data centers Engineers at the University of California San Diego have created an innovative cooling system designed to greatly enhance [...]
New nanomedicine wipes out leukemia in animal study
In a promising advance for cancer treatment, Northwestern University scientists have re-engineered the molecular structure of a common chemotherapy drug, making it dramatically more soluble and effective and less toxic. In the new study, [...]
Mystery Solved: Scientists Find Cause for Unexplained, Deadly Diseases
A study reveals that a protein called RPA is essential for maintaining chromosome stability by stimulating telomerase. New findings from the University of Wisconsin-Madison suggest that problems with a key protein that helps preserve chromosome stability [...]
