When you purchase through links on our articles, Future and its syndication partners may earn a commission.

The latest generative AI models are capable of producing amazing and magical, human-like results. But do they really understand anything? This will be a big no according to the latest study from MIT (via Technical spot).

More specifically, the key question is whether LLMs or the broad linguistic models at the heart of the most powerful chatbots are able to construct accurate internal models of the world. And the answer that MIT researchers have largely arrived at is no, they can’t.

To find out, the MIT team developed new metrics for testing AI that go beyond simple response accuracy measures and rely on what are called deterministic finite automations, or DFA.

A DFA is a problem composed of a sequence of interrelated steps that rely on a set of rules. Among other tasks, navigating the streets of New York was chosen for the research.

The MIT team found that some generative AI models are capable of providing highly accurate turn-by-turn driving instructions in New York, but only under ideal circumstances. When the researchers closed some streets and added detours, performance dropped. In fact, the internal maps implicitly generated by the LLMs through their training processes were full of non-existent streets and other inconsistencies.

“I was surprised by how quickly performance deteriorated as soon as we added a detour. If we close just 1 percent of possible streets, the accuracy immediately drops from almost 100 percent to just 67 percent,” explains lead author of the research paper, Keyon Vafa.

The main lesson here is that the remarkable accuracy of LLMs in some contexts can be misleading. “Often we see these models doing impressive things and think that they must have understood something about the world. I hope that we can convince people that this is an issue that needs to be thought about very carefully and that we “We shouldn’t rely on our own intuitions to answer them,” says Ashesh Rambachan, lead author of the paper.

More broadly, this research is a reminder of what is really happening with the latest LLMs. All they really do is predict which word to put next in a sequence based on gargantuan amounts of text scraped, indexed and correlated. Reasoning and understanding are not an integral part of this process.

Your next upgrade

Best processor for gaming: The best chips from Intel and AMD.
Best gaming motherboard: The good boards.
Best graphics card: Your perfect pixel pusher is waiting for you.
Best SSD for Gaming: Enter the game before the others.

What this new MIT research has shown is that LLMs can do remarkably well without actually understanding the rules. At the same time, this accuracy can quickly collapse in the face of real-world variables.

Of course, this won’t be entirely new to anyone familiar with using chatbots. We’ve all seen how quickly a convincing interaction with a chatbot can degrade into hallucination or simply gibberish following some type of interrogative prompt.

But this MIT study is helpful in crystallizing this anecdotal experience into a more formal explanation. We all knew that chatbots simply predict words. But sometimes the incredible accuracy of some answers can start to convince you that something magical might just be happening.

This latest study reminds us that this is certainly not the case. Well, unless incredibly accurate but ultimately insane word prediction is your idea of ​​magic.