The Life Cycle of Knowledge in Big Language Models:A Survey
作者机构:Chinese Information Processing LaboratoryBeijing 100190China State Key Laboratory of Computer ScienceInstitute of SoftwareChinese Academy of SciencesBeijing 100190China University of Chinese Academy of SciencesBeijing 101408China
出 版 物:《Machine Intelligence Research》 (机器智能研究(英文版))
年 卷 期:2024年第21卷第2期
页 面:217-238页
核心收录:
学科分类:12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 081104[工学-模式识别与智能系统] 08[工学] 0835[工学-软件工程] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术(可授工学、理学学位)]
基 金:supported by the National Natural Science Foundation of China(No.62122077) CAS Project for Young Scientists in Basic Research,China(No.YSBR-040)
主 题:Pre-trained language model knowledge acquisition knowledge representation knowledge probing knowledge editing knowledge application
摘 要:Knowledge plays a critical role in artificial ***,the extensive success of pre-trained language models(PLMs)has raised significant attention about how knowledge can be acquired,maintained,updated and used by language *** the enormous amount of related studies,there is still a lack of a unified view of how knowledge circulates within language models throughout the learning,tuning,and application processes,which may prevent us from further understanding the connections between current progress or realizing existing *** this survey,we revisit PLMs as knowledge-based systems by dividing the life circle of knowledge in PLMs into five critical periods,and investigating how knowledge circulates when it is built,maintained and *** this end,we systematically review existing studies of each period of the knowledge life cycle,summarize the main challenges and current limitations,and discuss future directions.