Danyal Akarca
iai.tv
Originally posted 30 Jan 24
Here is an excerpt:
So, evolution shapes systems that are capable of solving competing problems that are both internal (e.g., how to expend energy) and external (e.g., how to act to survive), but in a way that can be highly efficient, in many cases elegant, and often surprising. But how does this evolutionary story of biological intelligence contrast with the current paradigm of AI?
In some ways, quite directly. Since the 50s, neural networks were developed as models that were inspired directly from neurons in the brain and the strength of their connections, in addition to many successful architectures of the past being directly motivated by neuroscience experimentation and theory. Yet, AI research in the modern era has occurred with a significant absence of thought of intelligent systems in nature and their guiding principles. Why is this? There are many reasons. But one is that the exponential growth of computing capabilities, enabled by increases of transistors on integrated circuits (observed since the 1950s, known as Moore’s Law), has permitted AI researchers to leverage significant improvements in performance without necessarily requiring extraordinarily elegant solutions. This is not to say that modern AI algorithms are not widely impressive – they are. It is just that the majority of the heavy lifting has come from advances in computing power rather than their engineered design. Consequently, there has been relatively little recent need or interest from AI experts to look to the brain for inspiration.
But the tide is turning. From a hardware perspective, Moore’s law will not continue ad infinitum (at 7 nanometers, transistor channel lengths are now nearing fundamental limits of atomic spacing). We will therefore not be able to leverage ever improving performance delivered by increasingly compact microprocessors. It is likely therefore that we will require entirely new computing paradigms, some of which may be inspired by the types of computations we observe in the brain (the most notable being neuromorphic computing). From a software and AI perspective, it is becoming increasingly clear that – in part due to the reliance on increases to computational power – the AI research field will need to refresh its conceptions as to what makes systems intelligent at all. For example, this will require much more sophisticated benchmarks of what it means to perform at human or super-human performance. In sum, the field will need to form a much richer view of the possible space of intelligent systems, and how artificial models can occupy different places in that space.
Key Points:
- Evolutionary pressures: Efficient, resource-saving brains are advantageous for survival, leading to optimized solutions for learning, memory, and decision-making.
- AI's reliance on brute force: Modern AI often achieves performance through raw computing power, neglecting principles like energy efficiency.
- Shifting AI paradigm: Moore's Law's end and limitations in conventional AI call for exploration of new paradigms, potentially inspired by the brain.
- Neurobiology's potential: Brain principles like network structure, local learning, and energy trade-offs can inform AI design for efficiency and novel functionality.
- Embodied AI with constraints: Recent research incorporates space and communication limitations into AI models, leading to features resembling real brains and potentially more efficient information processing.
