The paradigm for intelligence was logical reasoning, and the idea of what an internal representation would look like was it would be some kind of symbolic structure. That has completely changed with these big neural nets.

The quote "The paradigm for intelligence was logical reasoning, and the idea of what an internal representation would look like was it would be some kind of symbolic structure. That has completely changed with these big neural nets." by Geoffrey Hinton reflects a significant shift in the understanding of artificial intelligence (AI) and cognitive science. Hinton, a leading researcher in the field of machine learning and neural networks, is commenting on the evolution of how intelligence is conceptualized and modeled in machines. Historically, logical reasoning was the central framework for understanding intelligence, and it was believed that internal representations of knowledge were structured as symbols or abstract concepts, much like human language or formal logic.

Hinton highlights how this traditional paradigm of intelligence has been dramatically altered with the advent of neural networks—specifically, deep learning models. Neural networks, which are computational models inspired by the structure of the human brain, do not rely on symbolic reasoning. Instead, they process data through interconnected layers of nodes (neurons), allowing the system to learn patterns and representations from large amounts of information without explicit, human-defined rules. This shift in thinking has opened up new ways of creating intelligent systems that can solve complex problems more effectively than traditional symbolic AI.

The symbolic structure that Hinton refers to was once considered the ideal way to model knowledge in AI systems, where concepts and logic were represented in an understandable, structured form. However, neural networks have shown that intelligence can emerge from systems that learn patterns through data and experience, rather than from pre-programmed symbols or rules. This change represents a move toward more flexible and scalable AI models that can adapt and improve over time, much like the way humans learn through experience.

In essence, Hinton’s quote marks a paradigm shift in how AI is developed and understood. While traditional models focused on logical reasoning and symbolic representations, the rise of neural networks emphasizes learning from vast datasets, which has led to more advanced and powerful forms of artificial intelligence. This transformation is at the heart of much of the recent progress in AI, including advances in computer vision, natural language processing, and autonomous systems.