检索结果-南通市图书馆

Improving Extraction of Chinese Open Relations Using pre-trained language model and Knowledge Enhancement

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Data Intelligence 2023年第4期5卷 962-989页

作者： Chaojie Wen Xudong Jia Tao Chen Faculty of Intelligent Manufacturing Wuyi UniversityJiangmenGuangdongChina

Open Relation Extraction(ORE)is a task of extracting semantic relations from a text *** ORE systems have significantly improved their efficiency in obtaining Chinese relations,when compared with conventional systems which heavily depend on feature engineering or syntactic ***,the ORE systems do not use robust neural networks such as pre-trained language models to take advantage of large-scale unstructured data *** respons to this issue,a new system entitled Chinese Open Relation Extraction with Knowledge Enhancement(CORE-KE)is presented in this *** CORE-KE system employs a pre-trained language model(with the support of a Bidirectional Long Short-Term Memory(BiLSTM)layer and a Masked Conditional Random Field(Masked CRF)layer)on unstructured data in order to improve Chinese open relation *** descriptions in Wikidata and additional knowledge(in terms of triple facts)extracted from Chinese ORE datasets are used to fine-tune the pre-trained language *** addition,syntactic features are further adopted in the training stage of the CORE-KE system for knowledge *** results of the CORE-KE system on two large-scale datasets of open Chinese entities and relations demonstrate that the CORE-KE system is superior to other ORE *** F1-scores of the CORE-KE system on the two datasets have given a relative improvement of 20.1%and 1.3%,when compared with benchmark ORE systems,*** source code is available at https:/***/cjwen15/CORE-KE.

关键词： Chinese open relation extraction pre-trained language model Knowledge enhancement

Adapter Based on pre-trained language models for Classification of Medical Text

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Electronic Research and Application 2024年第3期8卷 129-134页

作者： Quan Li University of Science and Technology of China Hefei 230026Anhui ProvinceChina

We present an approach to classify medical text at a sentence level *** the inherent complexity of medical text classification,we employ adapters based on pre-trained language models to extract information from medical text,facilitating more accurate classification while minimizing the number of trainable *** experiments conducted on various datasets demonstrate the effectiveness of our approach.

关键词： Classification of medical text Adapter pre-trained language model

The Life Cycle of Knowledge in Big language models:A Survey

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Machine Intelligence Research 2024年第2期21卷 217-238页

作者： Boxi Cao Hongyu Lin Xianpei Han Le Sun Chinese Information Processing Laboratory Beijing 100190China State Key Laboratory of Computer Science Institute of SoftwareChinese Academy of SciencesBeijing 100190China University of Chinese Academy of Sciences Beijing 101408China

Knowledge plays a critical role in artificial ***,the extensive success of pre-trained language models(PLMs)has raised significant attention about how knowledge can be acquired,maintained,updated and used by language *** the enormous amount of related studies,there is still a lack of a unified view of how knowledge circulates within language models throughout the learning,tuning,and application processes,which may prevent us from further understanding the connections between current progress or realizing existing *** this survey,we revisit PLMs as knowledge-based systems by dividing the life circle of knowledge in PLMs into five critical periods,and investigating how knowledge circulates when it is built,maintained and *** this end,we systematically review existing studies of each period of the knowledge life cycle,summarize the main challenges and current limitations,and discuss future directions.

关键词： pre-trained language model knowledge acquisition knowledge representation knowledge probing knowledge editing knowledge application

Personality Trait Detection via Transfer Learning

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Computers, Materials & Continua 2024年第2期78卷 1933-1956页

作者： Bashar Alshouha Jesus Serrano-Guerrero Francisco Chiclana Francisco P.Romero Jose A.Olivas Department of Information Technologies and Systems University of Castilla-La ManchaCiudad Real13071Spain Institute of Artificial Intelligence School of Computer Science and InformaticsDe Montfort UniversityThe GatewayLeicesterLE19BHUK

Personality recognition plays a pivotal role when developing user-centric solutions such as recommender systems or decision support systems across various domains,including education,e-commerce,or human ***-ditional machine learning techniques have been broadly employed for personality trait identification;nevertheless,the development of new technologies based on deep learning has led to new opportunities to improve their *** study focuses on the capabilities of pre-trained language models such as BERT,RoBERTa,ALBERT,ELECTRA,ERNIE,or XLNet,to deal with the task of personality *** models are able to capture structural features from textual content and comprehend a multitude of language facets and complex features such as hierarchical relationships or long-term *** makes them suitable to classify multi-label personality traits from reviews while mitigating computational *** focus of this approach centers on developing an architecture based on different layers able to capture the semantic context and structural features from ***,it is able to fine-tune the previous models using the MyPersonality dataset,which comprises 9,917 status updates contributed by 250 Facebook *** status updates are categorized according to the well-known Big Five personality model,setting the stage for a comprehensive exploration of personality *** test the proposal,a set of experiments have been performed using different metrics such as the exact match ratio,hamming loss,zero-one-loss,precision,recall,F1-score,and weighted *** results reveal ERNIE is the top-performing model,achieving an exact match ratio of 72.32%,an accuracy rate of 87.17%,and 84.41%of *** findings demonstrate that the tested models substantially outperform other state-of-the-art studies,enhancing the accuracy by at least 3%and confirming them as powerful tools for personality *** findings represent substantial a

关键词： Personality trait detection pre-trained language model big five model transfer learning

Vulnerability Detection of Ethereum Smart Contract Based on SolBERT-BiGRU-Attention Hybrid Neural model

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Computer modeling in Engineering & Sciences 2023年第10期137卷 903-922页

作者： Guangxia Xu Lei Liu Jingnan Dong Cyberspace Institute of Advanced Technology Guangzhou UniversityGuangzhou510006China School of Software Engineering Chongqing University of Posts and TelecommunicationsChongqing400065China School of Computer Science and Technology Chongqing University of Posts and TelecommunicationsChongqing400065China

In recent years,with the great success of pre-trained language models,the pre-trained BERT model has been gradually applied to the field of source code ***,the time cost of training a language model from zero is very high,and how to transfer the pre-trained language model to the field of smart contract vulnerability detection is a hot research direction at *** this paper,we propose a hybrid model to detect common vulnerabilities in smart contracts based on a lightweight pre-trained languagemodel BERT and connected to a bidirectional gate recurrent *** downstream neural network adopts the bidirectional gate recurrent unit neural network model with a hierarchical attention mechanism to mine more semantic features contained in the source code of smart contracts by using their *** experiments show that our proposed hybrid neural network model SolBERT-BiGRU-Attention is fitted by a large number of data samples with smart contract vulnerabilities,and it is found that compared with the existing methods,the accuracy of our model can reach 93.85%,and the Micro-F1 Score is 94.02%.

关键词： Smart contract pre-trained language model deep learning recurrent neural network blockchain security

Medical Named Entity Recognition from Un-labelled Medical Records based on pre-trained language models and Domain Dictionary

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Data Intelligence 2021年第3期3卷 402-417页

作者： Chaojie Wen Tao Chen Xudong Jia Jiang Zhu Faculty of Intelligent Manufacturing Wuyi UniversityJiangmen 529020China

Medical named entity recognition(NER)is an area in which medical named entities are recognized from medical texts,such as diseases,drugs,surgery reports,anatomical parts,and examination *** medical NER methods do not make full use of un-labelled medical texts embedded in medical *** address this issue,we proposed a medical NER approach based on pre-trained language models and a domain ***,we constructed a medical entity dictionary by extracting medical entities from labelled medical texts and collecting medical entities from other resources,such as the YiduN4 K data ***,we employed this dictionary to train domain-specific pre-trained language models using un-labelled medical ***,we employed a pseudo labelling mechanism in un-labelled medical texts to automatically annotate texts and create pseudo ***,the BiLSTM-CRF sequence tagging model was used to fine-tune the pre-trained language *** experiments on the un-labelled medical texts,which were extracted from Chinese electronic medical records,show that the proposed NER approach enables the strict and relaxed F1 scores to be 88.7%and 95.3%,respectively.

关键词： Medical named entity recognition pre-trained language model Domain dictionary Pseudo labelling Un-labelled medical data

DynamicRetriever:A pre-trained model-based IR System Without an Explicit Index

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Machine Intelligence Research 2023年第2期20卷 276-288页

作者： Yu-Jia Zhou Jing Yao Zhi-Cheng Dou Ledell Wu Ji-Rong Wen Gaoling School of Artificial Intelligence Renmin University of ChinaBeijing 100872China Beijing Academy of Artificial Intelligence Beijing 100084China

Web search provides a promising way for people to obtain information and has been extensively *** the surge of deep learning and large-scale pre-training techniques,various neural information retrieval models are proposed,and they have demonstrated the power for improving search(especially,the ranking)*** these existing search methods follow a common paradigm,i.e.,index-retrieve-rerank,where they first build an index of all documents based on document terms(i.e.,sparse inverted index)or representation vectors(i.e.,dense vector index),then retrieve and rerank retrieved documents based on the similarity between the query and documents via ranking *** this paper,we explore a new paradigm of information retrieval without an explicit index but only with a pre-trained ***,all of the knowledge of the documents is encoded into model parameters,which can be regarded as a differentiable indexer and optimized in an end-to-end ***,we propose a pre-trained model-based information retrieval(IR)system called DynamicRetriever,which directly returns document identifiers for a given *** such a framework,we implement two variants to explore how to train the model from scratch and how to combine the advantages of dense retrieval *** with existing search methods,the model-based IR system parameterizes the traditional static index with a pre-training model,which converts the document semantic mapping into a dynamic and updatable *** experiments conducted on the public search benchmark Microsoft machine reading comprehension(MS MARCO)verify the effectiveness and potential of our proposed new paradigm for information retrieval.

关键词： Information retrieval(IR) document retrieval model-based IR pre-trained language model differentiable search index

EPT:Data Augmentation with Embedded Prompt Tuning for LowResource Named Entity Recognition

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

武汉大学学报:自然科学英文版 2023年第4期28卷 299-308页

作者： YU Hongfei NI Kunyu XU Rongkang YU Wenjun HUANG Yu College of Informatics Huazhong Agricultural UniversityWuhan 430070HubeiChina

Data augmentation methods are often used to address data scarcity in natural language processing(NLP).However,token-label misalignment,which refers to situations where tokens are matched with incorrect entity labels in the augmented sentences,hinders the data augmentation methods from achieving high scores in token-level tasks like named entity recognition(NER).In this paper,we propose embedded prompt tuning(EPT)as a novel data augmentation approach to low-resource *** address the problem of token-label misalignment,we implicitly embed NER labels as prompt into the hidden layer of pre-trained language model,and therefore entity tokens masked can be predicted by the finetuned ***,EPT can generate high-quality and high-diverse data with various entities,which improves performance of *** datasets of cross-domain NER are available,we also explore NER domain adaption with *** experimental results show that EPT achieves substantial improvement over the baseline methods on low-resource NER tasks.

关键词： data augmentation token-label misalignment named entity recognition pre-trained language model prompt

一种基于金融文本情感分析的股票指数预测新方法

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论