The brain, despite its comparatively shallow structure with limited layers, operates efficiently, whereas modern AI systems are characterized by deep architectures with numerous layers. This raises the question: Can brain-inspired shallow architectures rival the performance of deep architectures, and if so, what are the fundamental mechanisms that enable this?
Neural network learning methods are inspired by the brain's functioning, yet there are fundamental differences between how the brain learns and how deep learning operates. A key distinction lies in the number of layers each employs.
Deep learning systems often have many layers, sometimes extending into the hundreds, which allows them to effectively learn complex classification tasks. In contrast, the human brain has a much simpler structure with far fewer layers. Despite its relatively shallow architecture and the slower, noisier nature of its processes, the brain is remarkably adept at handling complex classification tasks efficiently.
Research on Shallow Learning Mechanisms in the Brain
The key question driving new research is the possible mechanism underlying the brain's efficient shallow learning — one that enables it to perform classification tasks with the same accuracy as deep learning. In an article published in Physica A, researchers from Bar-Ilan University in Israel show how such shallow learning mechanisms can compete with deep learning.
Credit: Prof. Ido Kanter, Bar-Ilan University
"Instead of a deep architecture, like a skyscraper, the brain consists of a wide shallow architecture, more like a very wide building with only very few floors," said Prof. Ido Kanter, of Bar-Ilan's Department of Physics and Gonda (Goldschmied) Multidisciplinary Brain Research Center, who led the research.
"The capability to correctly classify objects increases where the architecture becomes deeper, with more layers. In contrast, the brain's shallow mechanism indicates that a wider network better classifies objects," said Ronit Gross, an undergraduate student and one of the key contributors to this work.
"Wider and higher architectures represent two complementary mechanisms," she added. Nevertheless, the realization of very wide shallow architectures, imitating the brain's dynamics, requires a shift in the properties of advanced GPU technology, which is capable of accelerating deep architecture, but fails in the implementation of wide shallow ones.
Reference: "Efficient shallow learning mechanism as an alternative to deep learning" by Ofek Tevet, Ronit D. Gross, Shiri Hodassman, Tal Rogachevsky, Yarden Tzach, Yuval Meir and Ido Kanter, 11 January 2024, Physica A: Statistical Mechanics and its Applications.
DOI: 10.1016/j.physa.2024.129513
News
Magnetic nanoparticles that successfully navigate complex blood vessels may be ready for clinical trials
Every year, 12 million people worldwide suffer a stroke; many die or are permanently impaired. Currently, drugs are administered to dissolve the thrombus that blocks the blood vessel. These drugs spread throughout the entire [...]
Reviving Exhausted T Cells Sparks Powerful Cancer Tumor Elimination
Scientists have discovered how tumors secretly drain the energy from T cells—the immune system’s main cancer fighters—and how blocking that process can bring them back to life. The team found that cancer cells use [...]
Very low LDL-cholesterol correlates to fewer heart problems after stroke
Brigham and Women's Hospital's TIMI Study Group reports that in patients with prior ischemic stroke, very low achieved LDL-cholesterol correlated with fewer major adverse cardiovascular events and fewer recurrent strokes, without an apparent increase [...]
“Great Unified Microscope” Reveals Hidden Micro and Nano Worlds Inside Living Cells
University of Tokyo researchers have created a powerful new microscope that captures both forward- and back-scattered light at once, letting scientists see everything from large cell structures to tiny nanoscale particles in a single shot. Researchers [...]
Breakthrough Alzheimer’s Drug Has a Hidden Problem
Researchers in Japan found that although the Alzheimer’s drug lecanemab successfully removes amyloid plaques from the brain, it does not restore the brain’s waste-clearing system within the first few months of treatment. The study suggests that [...]
Concerning New Research Reveals Colon Cancer Is Skyrocketing in Adults Under 50
Colorectal cancer is striking younger adults at alarming rates, driven by lifestyle and genetic factors. Colorectal cancer (CRC) develops when abnormal cells grow uncontrollably in the colon or rectum, forming tumors that can eventually [...]
Scientists Discover a Natural, Non-Addictive Way To Block Pain That Could Replace Opioids
Scientists have discovered that the body can naturally dull pain through its own localized “benzodiazepine-like” peptides. A groundbreaking study led by a University of Leeds scientist has unveiled new insights into how the body manages pain, [...]
GLP-1 Drugs Like Ozempic Work, but New Research Reveals a Major Catch
Three new Cochrane reviews find evidence that GLP-1 drugs lead to clinically meaningful weight loss, though industry-funded studies raise concerns. Three new reviews from Cochrane have found that GLP-1 medications can lead to significant [...]
How a Palm-Sized Laser Could Change Medicine and Manufacturing
Researchers have developed an innovative and versatile system designed for a new generation of short-pulse lasers. Lasers that produce extremely short bursts of light are known for their remarkable precision, making them indispensable tools [...]
New nanoparticles stimulate the immune system to attack ovarian tumors
Cancer immunotherapy, which uses drugs that stimulate the body’s immune cells to attack tumors, is a promising approach to treating many types of cancer. However, it doesn’t work well for some tumors, including ovarian [...]
New Drug Kills Cancer 20,000x More Effectively With No Detectable Side Effects
By restructuring a common chemotherapy drug, scientists increased its potency by 20,000 times. In a significant step forward for cancer therapy, researchers at Northwestern University have redesigned the molecular structure of a well-known chemotherapy drug, greatly [...]
Lipid nanoparticles discovered that can deliver mRNA directly into heart muscle cells
Cardiovascular disease continues to be the leading cause of death worldwide. But advances in heart-failure therapeutics have stalled, largely due to the difficulty of delivering treatments at the cellular level. Now, a UC Berkeley-led [...]
The basic mechanisms of visual attention emerged over 500 million years ago, study suggests
The brain does not need its sophisticated cortex to interpret the visual world. A new study published in PLOS Biology demonstrates that a much older structure, the superior colliculus, contains the necessary circuitry to perform the [...]
AI Is Overheating. This New Technology Could Be the Fix
Engineers have developed a passive evaporative cooling membrane that dramatically improves heat removal for electronics and data centers Engineers at the University of California San Diego have created an innovative cooling system designed to greatly enhance [...]
New nanomedicine wipes out leukemia in animal study
In a promising advance for cancer treatment, Northwestern University scientists have re-engineered the molecular structure of a common chemotherapy drug, making it dramatically more soluble and effective and less toxic. In the new study, [...]
Mystery Solved: Scientists Find Cause for Unexplained, Deadly Diseases
A study reveals that a protein called RPA is essential for maintaining chromosome stability by stimulating telomerase. New findings from the University of Wisconsin-Madison suggest that problems with a key protein that helps preserve chromosome stability [...]
