[LLM][Hinton] Will digital intelligence replace biological intelligence : Geoff Hinton

Oct 30, 2024

Here’s an interesting essay (Nature magazine): How close is AI to human-level intelligence?

The article mentions these as limitations/needs/observations:

LLMs can't do long multi-step planning that require abstract reasoning and generalization
LLMs can come up with embedding with low Kolmogorov complexity (algorithmic complexity), but it cannot get better than now due to lack of more data.
Predicting the next token is good but not good enough to reach AGI. It needs to generate large chunks all at once.
LLMs need to build good world models. Analysis of LLM's internal representations show a hint of this, but it's a mess.
LLMs lack internal feedback that requires recurrent connections. All connections are feedforward, and feedback only through error back propagation.
Need too much data (unlike humans?)

← Back to all articles Quick Navigation: Next:[ j ] – Prev:[ k ] – List:[ l ]