Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning

by   Weili Nie, et al.

Humans have an inherent ability to learn novel concepts from only a few samples and generalize these concepts to different situations. Even though today's machine learning models excel with a plethora of training data on standard recognition tasks, a considerable gap exists between machine-level pattern recognition and human-level concept learning. To narrow this gap, the Bongard Problems (BPs) were introduced as an inspirational challenge for visual cognition in intelligent systems. Albeit new advances in representation learning and learning to learn, BPs remain a daunting challenge for modern AI. Inspired by the original one hundred BPs, we propose a new benchmark Bongard-LOGO for human-level concept learning and reasoning. We develop a program-guided generation technique to produce a large set of human-interpretable visual cognition problems in action-oriented LOGO language. Our benchmark captures three core properties of human cognition: 1) context-dependent perception, in which the same object may have disparate interpretations given different contexts; 2) analogy-making perception, in which some meaningful concepts are traded off for other meaningful concepts; and 3) perception with a few samples but infinite vocabulary. In experiments, we show that the state-of-the-art deep learning methods perform substantially worse than human subjects, implying that they fail to capture core human cognition properties. Finally, we discuss research directions towards a general architecture for visual reasoning to tackle this benchmark.


page 1

page 2

page 3

page 4


Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions

A significant gap remains between today's visual pattern recognition mod...

Bridging Cognitive Programs and Machine Learning

While great advances are made in pattern recognition and machine learnin...

KANDINSKYPatterns – An experimental exploration environment for Pattern Analysis and Machine Intelligence

Machine intelligence is very successful at standard recognition tasks wh...

How intelligent are convolutional neural networks?

Motivated by the Gestalt pattern theory, and the Winograd Challenge for ...

Solving Bongard Problems with a Visual Language and Pragmatic Reasoning

More than 50 years ago Bongard introduced 100 visual concept learning pr...

The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domain

The abilities to form and abstract concepts is key to human intelligence...

Tunneling Neural Perception and Logic Reasoning through Abductive Learning

Perception and reasoning are basic human abilities that are seamlessly c...

Please sign up or login with your details

Forgot password? Click here to reset