Enlightenment 2.0
The AI intelligent model class has ushered in new powerhouses.At the 2021 Beijing Zhiyuan Conference held a few days ago, the Zhiyuan Artificial Intelligence Research Institute officially releasedEnlightenment 2.0 Intelligent model. Its parameter scale has reached 1.75 trillion, which is 10 times that of OpenAI GPT-3, the representative of the field a year ago. Compared with Google’s Switch Transformers super language model, the parameter scale of Wudao 2.0 is also 150 billion more. It is worth mentioning that less than three months have passed since Enlightenment 1.0 debuted. In this process, the training direction of the model has also changed from mainly Chinese text to a collection of text and vision. Therefore, Enlightenment 2.0 can be used for more different tasks, and its versatility has been further improved.
The FastMoE technology newly developed by Zhiyuan is the key to Enlightenment 2.0 becoming a mega-level model. The MoE (Mixture of Experts) scheme currently adopted by Google, due to its distributed training framework and customized hardware requirements, makes it impossible for most people to get the opportunity to use and research. Enlightenment’s FastMoE is the first MoE system that supports the PyTorch framework. It has the characteristics of “easy to use, flexibility, and high performance”, and supports massively parallel training and complex balancing strategies such as Switch and Gshard. In contrast, it can provide a lower threshold and more flexibility.
According to Zhiyuan’s introduction, Enlightenment 2.0 “has been close to breaking through the Turing Test in terms of poetry creation, couplets, text summaries, anthropomorphic questions and answers, and painting.” At the conference, the official also showed the virtual student “Hua Zhibing” jointly developed with Xiaoice Company (from Microsoft). Its development direction is to surpass humans in a number of cognitive intelligences, and to have creative ability on the basis of recognition. , To help AI “move from perceptual intelligence to the era of cognitive intelligence.”