We Are Reinventing Intelligence Without Understanding It. Like Birds with Flight

Birds learned to fly without understanding how insects did it, and we humans are developing forms of artificial intelligence without having fully understood how natural intelligence works. "As far as we know, children learn by creating models of reality, which they continually test through observation," said Riccardo Zecchina, Vodafone Chair in Machine Learning and Data Science, at the recent Donor Event, organized by Bocconi University to thank those who support it. "Artificial intelligence, through machine learning techniques, learns by processing an amount of data that is unmanageable for a human being."

Among the most amazing developments in artificial intelligence in recent years, Professor Zecchina cites powerful language generators such as ChatGPT, the ability to "predict" the structure of new proteins, and therefore contribute to the development of new medicines, and the formulation of new mathematical theorems.

Prof. Zecchina described the language generators as "autoregressive models," which generate one word after another based on the dataset they were trained on. To train this kind of model, researchers take a sentence and hide part of it, making the system predict how it should be completed, and continue until the prediction is accurate. ChatGPT has been trained on a corpus equivalent to 100 million books.

Systems like ChatGPT are constantly evolving and the improvement is impressive. Among the differences between the version of ChatGPT freely available from November 2022 and the newer GPT-4, Prof. Zecchina highlighted the different responses to the prompt, "Here we have a book, 9 eggs, a laptop computer, a bottle, and a nail. Please tell me how to stack them on each other in a stable way." By turning the two responses into drawings, the results can be seen below.

The improvement is, to say the least, enormous. Recommending placing the eggs between the book and the computer, moreover, GPT-4 specifies, "Arrange the 9 eggs in a 3-by-3 square on top of the book, leaving some space between them. The eggs will form a second layer and distribute the weight evenly," suggesting that the system has created a model for itself to distinguish what is stable and what is not.

"Within the system," Prof. Zecchina said, "complex interactions develop. It was observed, for example, that after learning some programming languages, it also improved its natural language performance."

Although calls for a moratorium in the development of artificial intelligence are multiplying, and there are even fears of extinction, Zecchina clarified that, for now, the realistic risks are the tendency to reproduce the biases implicit in the training dataset, the possible replacement of some workers and the concentration of power in the hands of a few. "These systems still lack the ability to plan," said the Vodafone Chair in Machine Learning and Data Science, "the ability to develop hypotheses, counterclaims and alternative hypotheses. If they were ever to acquire this ability, they could be driven by an indication to achieve certain goals, and the situation would become more serious."

We live, in short, in a time when some positive prospects, such as a strong acceleration in the pace of scientific discovery, are certain, but our ability to progress experimentally has outpaced our theoretical understanding. The scientific community must remain vigilant and take every opportunity to observe, verify, and experiment to look behind the curtain that some companies active in artificial intelligence tend to erect for competitive reasons.