[Consiousness][Choe] Consciousness prior - Bengio group

Mar 3, 2021

Interesting papers from the Bengio group.

Note: When Bengio says “conscious”, he means “conscious, as in conscious and deliberate effort”, not “I feel like this or that kind of conscious”. See our publications on consciousness: /pub_topic/#Predictive.

quote

Yoshua Bengio

Two new papers from my group & collaborators on system 2 architectures, led by Anirudh Goyal, to exploit attention, sparsity and bottlenecks of communication to favour modularity, systematic generalization and out-of-distribution generalization by exploiting inductive biases inspired by human conscious processing (you can check this earlier paper https://arxiv.org/abs/2011.15091 for an overview of these inductive biases).

#1: Coordination Among Neural Modules Through a Shared Global Workspace, https://arxiv.org/abs/2103.01197

#2: Neural Production Systems, https://arxiv.org/abs/2103.01937

These all derive from the early inspiration and vision of my ‘consciousness prior’ paper from 2017: https://arxiv.org/abs/1709.08568. We have learned a lot since then, of course, but I believe this is just the beginning of a significant turn in the style of deep learning to achieve the flexibility and generalization power which conscious processing provides to humans. Self-attention and transformers were just the beginning of a great adventure. Congrats Anirudh Goyal for the amazing progress.

← Back to all articles         Quick Navigation:    Next:[ j ] – Prev:[ k ] – List:[ l ]