Physics Nobel Prize 2021, AI And Machine Learning
The Physics Nobel Prize for 2021 went to three physicists for their contributions to understand complex systems. One-fourth of the prize went to the Japanese Syukuro Manabe and to the German Klaus Hasselman. One-half of the prize went to the Italian theoretical physicist Giorgio Parisi.
Syukuro Manabe and Klauss Hasselman were awarded the prize:
“for the physical modelling of Earth’s climate, quantifying variability and reliably predicting global warming”
On the other hand, Giorgio Parisi got the prize
“for the discovery of the interplay of disorder and fluctuations in physical systems from atomic to planetary scales”
Since I am a theoretical physicist myself, I must say that was very happy that Giorgio Parisi got the prize this year. I had the internal feeling that he would get the prize this year since he also got the Wolf Prize early this year, which, is a good predictor of future Nobel prize winners.
Parisi’s contribution to Physics
If you are a physicist and especially a theoretical physicist, you will have to study or read at some point Parisi’s articles during your research career. His major contributions in Physics have been in quantum field theory, statistical physics and complex systems.
In quantum field theory, Parisi is widely known for his derivation with Guido Altarelli, of the equations that govern the Parton densities in quantum chromodynamics. He also derived exact solutions of the so-called Sherington-Kirkpatrick equations, that is an important equation related to the theory of spin glasses. In the field of spin glasses, Parisi has made many important contributions that have revolutionised the field.
Parisi’s contribution to Neural Networks and AI
Giorgio Parisi has made numerous contributions to the field of complex systems, the reason why he got the prize, and in the field of Neural Networks and Artificial Intelligence (AI).
A complex system is a system that is composed of large interacting units that usually interact with each other in a disordered way. These units could be for example atoms, molecules, genes, neurons, proteins, living organisms, the human brain, etc. The reason why these systems are called “complex” is because very difficult to model their interaction and predict their future evolution.
Many ideas of how to formulate and predict complex systems interaction and evolution come from the field of statistical physics, a field in which Giorgio Parisi is a leading expert.
One important point to understand is the phenomena of spin glasses. This phenomenon occurs in the field of condensed matter where some atom spins are completely misaligned from the rest of the atoms in a material. The state of the atom spins in these materials is mostly dominated by randomness and only cooperative behaviour of atom spins occurs at some low temperature called the freezing temperature.
Very often in statistical physics, one has to calculate the so-called partition function, denoted with Z, which is, in general, a function of many variables of the physical system. The partition function is a very important quantity because with it one can calculate the so-called free energy function which is one of the most important quantities in statistical physics.
However, there is a serious problem in calculating the free energy F because it is proportional to the logarithm of the partition function ln(Z) which is a difficult quantity to calculate analytically for a large number of interacting particles. In connection with the theory of spin glasses, Giorgio Parisi was faced with calculating ln(Z) and to bay-pass its calculation complexity, he used a very simple and powerful mathematical trick which is usually called the “replica trick” in statistical physics. The trick consists in expressing
So, you can see that instead of calculating ln(Z) on the left, one can calculate its equivalent expression on the r.h.s which is much easier to compute for complex systems. You can see that on the r.h.s appears Z to power n. One usually has to average the equation above in the system variables, and this is done by creating n copies of the system or n replicas each with its partition function Z.
What has all this to do with AI and machine learning?
It is a well-known fact that statistical physics has many applications in the theory of neural networks and statistical learning theory. For example, one can model the interaction of neural networks in AI as those of complex systems and use the theory of statistical physics to calculate important quantities.
Very often one is faced with optimisation problems in machine learning when is necessary to minimise the training and test data cost function. For example, in supervised learning the cost function C can be a function of a set of neural networks x over a dataset of points D. In my previous article, I showed you how to calculate the linear test data error by using statistical theory. The final expression for linear regression was quite simple to calculate but in the case of neural networks, it can be very difficult to calculate the expectation value.
In such cases, one reduces the expected value to a Gibbs sampling functional which is a function of ln(Z) where the latter is expressed as an exponential integral of the cost function C. After one uses the replica trick to calculate ln(Z). All this procedure is complicated to show here and the reader can find more information on the internet about this topic.
So by concluding, one can see how the work of Giorgio Parisi made it possible for the further expansion the field of neural networks and AI in general. Indeed, he made many other contributions in these fields that are too long to describe here.
