Eric James Beyer
Interesting Engineering
Originally posted 18 DEC 21
Here is an excerpt:
In the wake of these successes, Martin began to wonder whether or not the same principle could be applied to higher-level cognitive functions like language processing.
“I said, let’s just look at neural networks that are successful and see if they’re anything like the brain. My bet was that it would work, at least to some extent.”
To find out, Martin and colleagues compared data from 43 artificial neural network language models against fMRI and ECoG neural recordings taken while subjects listened to or read words as part of a text. The AI models the group surveyed covered all the major classes of available neural network approaches for language-based tasks. Some of them were more basic embedding models like GloVe, which clusters semantically similar words together in groups. Others, like the models known as GPT and BERT, were far more complex. These models are trained to predict the next word in a sequence or predict a missing word within a certain context, respectively.
“The setup itself becomes quite simple,” Martin explains. “You just show the same stimuli to the models that you show to the subjects [...]. At the end of the day, you’re left with two matrices, and you test if those matrices are similar.”
And the results?
“I think there are three-and-a-half major findings here,” Schrimpf says with a laugh. “I say ‘and a half’ because the last one we still don’t fully understand.”
Machine learning that mirrors the brain
The finding that sticks out to Martin most immediately is that some of the models predict neural data extremely well. In other words, regardless of how good a model was at performing a task, some of them appear to resemble the brain’s cognitive mechanics for language processing. Intriguingly, the team at MIT identified the GPT model variants as the most brain-like out of the group they looked at.
