Just learned about an article about the hot AI topic titled “Learning-at-Criticality in Large Language Models for Quantum Field Theory and Beyond“. Most interestingly, they showed that their Learning-at-Criticality (LaC) concept can be closely connected to the 2nd order phase transition or spontaneous symmetry breaking mechanism in physics, especially in quantum field theory. It is not surprising that AI, as an emergent phenomenon, is behaving like critical phenomena in complex systems that are well studied in physics. But how to develop and take advantage of this critical ability of AI is still a challenging job. This article points out a way that we may apply well developed techniques on phase transitions in physics to AI studies. And it also claims that LaC with its peak generalization capacity will thrive in situations where information is scarce.
We may think more ambitiously and consider the bigger picture. As well as LaC, we also need Thinking-at-Criticality (TaC). Scientific research requires a full spectrum of ideas and talents. In particular, we need two distinct types of researchers who occupy opposite ends of this spectrum (in Freeman Dyson’s terms): “birds,” the visionaries who see the big picture, and “frogs,” the technical experts who are detail-oriented and master the complexities of their specific niche (see my CORE article on further discussions). These “birds” are those who think at criticality. Current AI models are doing better and better as “frogs”, which may be achieved through sheer information overload and over-training. The next breakthrough may lie in producing these AI “birds”, which could be what AGI really means in the end. Even better, what if we combine the capabilities of both “bird” and “frog” into one single model, a super-intelligence?
How can we produce AI “birds”? This will depend on our ability to find those mysterious critical points within the vast parameter space of AI models. Power-law scalings observed in the path lengths of various AI activities may be a good starting point. More advanced math and physics tools will certainly help us to identify all possible critical points, some of which will undoubtedly lead to AGI and/or SI. Of course, there may be many different types of “birds” or “birds” in different fields (e.g., science birds, physics birds, QFT birds, etc.). The biggest question now is, what the role of human beings will be (besides understanding and applying it) afterwards?
