The Life Cycle of Knowledge in Big Language Models:A Survey

作     者：Boxi Cao Hongyu Lin Xianpei Han Le Sun Boxi Cao;Hongyu Lin;Xianpei Han;Le Sun

作者机构：Chinese Information Processing LaboratoryBeijing 100190China State Key Laboratory of Computer ScienceInstitute of SoftwareChinese Academy of SciencesBeijing 100190China University of Chinese Academy of SciencesBeijing 101408China 

出 版 物：《Machine Intelligence Research》 (机器智能研究（英文版）)

年 卷 期：2024年第21卷第2期

页      面：217-238页

核心收录：

学科分类：12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 081104[工学-模式识别与智能系统] 08[工学] 0835[工学-软件工程] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：supported by the National Natural Science Foundation of China(No.62122077) CAS Project for Young Scientists in Basic Research,China(No.YSBR-040) 

主　　题：Pre-trained language model knowledge acquisition knowledge representation knowledge probing knowledge editing knowledge application 

摘      要：Knowledge plays a critical role in artificial ***,the extensive success of pre-trained language models(PLMs)has raised significant attention about how knowledge can be acquired,maintained,updated and used by language *** the enormous amount of related studies,there is still a lack of a unified view of how knowledge circulates within language models throughout the learning,tuning,and application processes,which may prevent us from further understanding the connections between current progress or realizing existing *** this survey,we revisit PLMs as knowledge-based systems by dividing the life circle of knowledge in PLMs into five critical periods,and investigating how knowledge circulates when it is built,maintained and *** this end,we systematically review existing studies of each period of the knowledge life cycle,summarize the main challenges and current limitations,and discuss future directions.