Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

  • Photo from THE NEURO-SYMBOLIC CONCEPT LEARNER: INTERPRETING SCENES, WORDS, AND SENTENCES FROM NATURAL SUPERVISION; Edited by MIT Technology Review
  • Intelligent Machines

    Two rival AI approaches combine to let machines learn about the world like a child

    Together, deep learning and symbolic reasoning create a program that learns in a remarkably humanlike way.

    Over the decades since the inception of artificial intelligence, research in the field has fallen into two main camps. The “symbolists” have sought to build intelligent machines by coding in logical rules and representations of the world. The “connectionists” have sought to construct artificial neural networks, inspired by biology, to learn about the world. The two groups have historically not gotten along.

    But a new paper from MIT, IBM, and DeepMind shows the power of combining the two approaches, perhaps pointing a way forward for the field. The team, led by Josh Tenenbaum, a professor at MIT’s Center for Brains, Minds, and Machines, created a computer program called a neuro-symbolic concept learner (NS-CL) that learns about the world (albeit a simplified version) just as a child might—by looking around and talking.

    The system consists of several pieces. One neural network is trained on a series of scenes made up of a small number of objects. Another neural network is trained on a series of text-based question-answer pairs about the scene, such as “Q: What’s the color of the sphere?” “A: Red.” This network learns to map the natural language questions to a simple program that can be run on a scene to produce an answer. 

    Sign up for The Algorithm
    Artificial intelligence, demystified

    The NS-CL system is also programed to understand symbolic concepts in text such as “objects,” “object attributes,” and “spatial relationship.” That knowledge helps NS-CL answer new questions about a different scene—a type of feat that is far more challenging using a connectionist approach alone. The system thus recognizes concepts in new questions and can relate them visually to the scene before it.

    "This is an exciting approach," says Brenden Lake, an assistant professor at NYU. "Neural pattern recognition allows the system to see, while symbolic programs allow the system to reason. Together, the approach goes beyond what current deep learning systems can do."

    In other words, the hybrid system addresses key limitations of both earlier approaches by combining them. It overcomes the scalability problems of symbolism, which has historically struggled to encode the complexity of human knowledge in an efficient way. But it also tackles one of the most common problems with neural networks: the fact that they need huge amounts of data.

    It is possible to train just a neural network to answer questions about a scene by feeding in millions of examples as training data. But a human child doesn’t require such a vast amount of data in order to grasp what a new object is or how it relates to other objects. Also, a network trained that way has no real understanding of the concepts involved—it’s just a vast pattern-matching exercise. So such a system would be prone to making very silly mistakes when faced with new scenarios. This is a common problem with today’s neural networks and underpins shortcomings that are easily exposed (see “AI’s language problem”).

    Connectionism purists may object to the fact that the system requires some knowledge to be hard-coded in. But the work is important because it nudges us closer to engineering a form of intelligence that seems more like our own. Cognitive scientists believe that the human mind goes through some similar steps, and that this underpins the flexibility of human learning.

    More practically, it could also unlock new applications of AI because the new technology requires far less training data. Robot systems, for example, could finally learn on the fly, rather than spend significant time training for each unique environment they’re in.

    “This is really exciting because it’s going to get us past this dependency on huge amounts of labeled data,” says David Cox, the scientist who leads the MIT-IBM Watson AI lab.

    The researchers behind the study are now developing a version that works on photographs of real scenes. This could prove valuable for many practical applications of computer vision.

    Learn from the humans leading the way in machine learning at EmTech Next. Register Today!
    June 11-12, 2019
    Cambridge, MA

    Register now
    More from Intelligent Machines

    Artificial intelligence and robots are transforming how we work and live.

    Want more award-winning journalism? Subscribe to Print + All Access Digital.
    • Print + All Access Digital {! insider.prices.print_digital !}*

      {! insider.display.menuOptionsLabel !}

      The best of MIT Technology Review in print and online, plus unlimited access to our online archive, an ad-free web experience, discounts to MIT Technology Review events, and The Download delivered to your email in-box each weekday.

      See details+

      12-month subscription

      Unlimited access to all our daily online news and feature stories

      6 bi-monthly issues of print + digital magazine

      10% discount to MIT Technology Review events

      Access to entire PDF magazine archive dating back to 1899

      Ad-free website experience

      The Download: newsletter delivery each weekday to your inbox

      The MIT Technology Review App

    /3
    You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.