The brain, despite its comparatively shallow structure with limited layers, operates efficiently, whereas modern AI systems are characterized by deep architectures with numerous layers. This raises the question: Can brain-inspired shallow architectures rival the performance of deep architectures, and if so, what are the fundamental mechanisms that enable this?
Neural network learning methods are inspired by the brain's functioning, yet there are fundamental differences between how the brain learns and how deep learning operates. A key distinction lies in the number of layers each employs.
Deep learning systems often have many layers, sometimes extending into the hundreds, which allows them to effectively learn complex classification tasks. In contrast, the human brain has a much simpler structure with far fewer layers. Despite its relatively shallow architecture and the slower, noisier nature of its processes, the brain is remarkably adept at handling complex classification tasks efficiently.
Research on Shallow Learning Mechanisms in the Brain
The key question driving new research is the possible mechanism underlying the brain's efficient shallow learning — one that enables it to perform classification tasks with the same accuracy as deep learning. In an article published in Physica A, researchers from Bar-Ilan University in Israel show how such shallow learning mechanisms can compete with deep learning.
Credit: Prof. Ido Kanter, Bar-Ilan University
"Instead of a deep architecture, like a skyscraper, the brain consists of a wide shallow architecture, more like a very wide building with only very few floors," said Prof. Ido Kanter, of Bar-Ilan's Department of Physics and Gonda (Goldschmied) Multidisciplinary Brain Research Center, who led the research.
"The capability to correctly classify objects increases where the architecture becomes deeper, with more layers. In contrast, the brain's shallow mechanism indicates that a wider network better classifies objects," said Ronit Gross, an undergraduate student and one of the key contributors to this work.
"Wider and higher architectures represent two complementary mechanisms," she added. Nevertheless, the realization of very wide shallow architectures, imitating the brain's dynamics, requires a shift in the properties of advanced GPU technology, which is capable of accelerating deep architecture, but fails in the implementation of wide shallow ones.
Reference: "Efficient shallow learning mechanism as an alternative to deep learning" by Ofek Tevet, Ronit D. Gross, Shiri Hodassman, Tal Rogachevsky, Yarden Tzach, Yuval Meir and Ido Kanter, 11 January 2024, Physica A: Statistical Mechanics and its Applications.
DOI: 10.1016/j.physa.2024.129513
News
Miller School Researchers Pioneer Nanovanilloid-Based Brain Cooling for Traumatic Injury
A multidisciplinary team at the University of Miami Miller School of Medicine has developed a breakthrough nanodrug platform that may prove beneficial for rapid, targeted therapeutic hypothermia after traumatic brain injury (TBI). Their work, published in ACS [...]
COVID-19 still claims more than 100,000 US lives each year
Centers for Disease Control and Prevention researchers report national estimates of 43.6 million COVID-19-associated illnesses and 101,300 deaths in the US during October 2022 to September 2023, plus 33.0 million illnesses and 100,800 deaths [...]
Nanomedicine in 2026: Experts Predict the Year Ahead
Progress in nanomedicine is almost as fast as the science is small. Over the last year, we've seen an abundance of headlines covering medical R&D at the nanoscale: polymer-coated nanoparticles targeting ovarian cancer, Albumin recruiting nanoparticles for [...]
Lipid nanoparticles could unlock access for millions of autoimmune patients
Capstan Therapeutics scientists demonstrate that lipid nanoparticles can engineer CAR T cells within the body without laboratory cell manufacturing and ex vivo expansion. The method using targeted lipid nanoparticles (tLNPs) is designed to deliver [...]
The Brain’s Strange Way of Computing Could Explain Consciousness
Consciousness may emerge not from code, but from the way living brains physically compute. Discussions about consciousness often stall between two deeply rooted viewpoints. One is computational functionalism, which holds that cognition can be [...]
First breathing ‘lung-on-chip’ developed using genetically identical cells
Researchers at the Francis Crick Institute and AlveoliX have developed the first human lung-on-chip model using stem cells taken from only one person. These chips simulate breathing motions and lung disease in an individual, [...]
Cell Membranes May Act Like Tiny Power Generators
Living cells may generate electricity through the natural motion of their membranes. These fast electrical signals could play a role in how cells communicate and sense their surroundings. Scientists have proposed a new theoretical [...]
This Viral RNA Structure Could Lead to a Universal Antiviral Drug
Researchers identify a shared RNA-protein interaction that could lead to broad-spectrum antiviral treatments for enteroviruses. A new study from the University of Maryland, Baltimore County (UMBC), published in Nature Communications, explains how enteroviruses begin reproducing [...]
New study suggests a way to rejuvenate the immune system
Stimulating the liver to produce some of the signals of the thymus can reverse age-related declines in T-cell populations and enhance response to vaccination. As people age, their immune system function declines. T cell [...]
Nerve Damage Can Disrupt Immunity Across the Entire Body
A single nerve injury can quietly reshape the immune system across the entire body. Preclinical research from McGill University suggests that nerve injuries may lead to long-lasting changes in the immune system, and these [...]
Fake Science Is Growing Faster Than Legitimate Research, New Study Warns
New research reveals organized networks linking paper mills, intermediaries, and compromised academic journals Organized scientific fraud is becoming increasingly common, ranging from fabricated research to the buying and selling of authorship and citations, according [...]
Scientists Unlock a New Way to Hear the Brain’s Hidden Language
Scientists can finally hear the brain’s quietest messages—unlocking the hidden code behind how neurons think, decide, and remember. Scientists have created a new protein that can capture the incoming chemical signals received by brain [...]
Does being infected or vaccinated first influence COVID-19 immunity?
A new study analyzing the immune response to COVID-19 in a Catalan cohort of health workers sheds light on an important question: does it matter whether a person was first infected or first vaccinated? [...]
We May Never Know if AI Is Conscious, Says Cambridge Philosopher
As claims about conscious AI grow louder, a Cambridge philosopher argues that we lack the evidence to know whether machines can truly be conscious, let alone morally significant. A philosopher at the University of [...]
AI Helped Scientists Stop a Virus With One Tiny Change
Using AI, researchers identified one tiny molecular interaction that viruses need to infect cells. Disrupting it stopped the virus before infection could begin. Washington State University scientists have uncovered a method to interfere with a key [...]
Deadly Hospital Fungus May Finally Have a Weakness
A deadly, drug-resistant hospital fungus may finally have a weakness—and scientists think they’ve found it. Researchers have identified a genetic process that could open the door to new treatments for a dangerous fungal infection [...]
