检索结果-南通市图书馆

Improving BERT fine-tuning via Self-Ensemble and Self-Distillation

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Computer Science & Technology 2023年第4期38卷 853-866页

作者：许一格邱锡鹏周浬皋黄萱菁 School of Computer Science Fudan UniversityShanghai 200433China Shanghai Key Laboratory of Intelligent Information Processing Fudan UniversityShanghai 200433China Huawei Technologies Co. Ltd.Hangzhou 310052China

fine-tuning pre-trained language models like BERT have become an effective way in natural language processing(NLP)and yield state-of-the-art results on many downstream *** studies on adapting BERT to new tasks mainly focus on modifying the model structure,re-designing the pre-training tasks,and leveraging external data and *** fine-tuning strategy itself has yet to be fully *** this paper,we improve the fine-tuning of BERT with two effective mechanisms:self-ensemble and *** self-ensemble mechanism utilizes the checkpoints from an experience pool to integrate the teacher *** order to transfer knowledge from the teacher model to the student model efficiently,we further use knowledge distillation,which is called self-distillation because the distillation comes from the model itself through the time *** on the GLUE benchmark and the Text Classification benchmark show that our proposed approach can significantly improve the adaption of BERT without any external data or *** conduct exhaustive experiments to investigate the efficiency of the self-ensemble and self-distillation mechanisms,and our proposed approach achieves a new state-of-the-art result on the SNLI dataset.

关键词： BERT deep learning fine-tuning natural language processing(NLP) pre-training model

Comparing fine-tuning, Zero and Few-Shot Strategies with Large Language Models in Hate Speech Detection in English

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Computer Modeling in Engineering & Sciences 2024年第9期140卷 2849-2868页

作者： Ronghao Pan JoséAntonio García-Díaz Rafael Valencia-García Departamento de Informática y Sistemas Universidad de MurciaCampus de EspinardoMurcia30100Spain

Large Language Models(LLMs)are increasingly demonstrating their ability to understand natural language and solve complex tasks,especially through text *** of the relevant capabilities is contextual learning,which involves the ability to receive instructions in natural language or task demonstrations to generate expected outputs for test instances without the need for additional training or gradient *** recent years,the popularity of social networking has provided a medium through which some users can engage in offensive and harmful online *** this study,we investigate the ability of different LLMs,ranging from zero-shot and few-shot learning to *** experiments show that LLMs can identify sexist and hateful online texts using zero-shot and few-shot approaches through information ***,it is found that the encoder-decoder model called Zephyr achieves the best results with the fine-tuning approach,scoring 86.811%on the Explainable Detection of Online Sexism(EDOS)test-set and 57.453%on the Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter(HatEval)***,it is confirmed that the evaluated models perform well in hate text detection,as they beat the best result in the HatEval task *** error analysis shows that contextual learning had difficulty distinguishing between types of hate speech and figurative ***,the fine-tuned approach tends to produce many false positives.

关键词： Hate speech detection zero-shot few-shot fine-tuning natural language processing

Mining and fine-tuning sugar uptake system for titer improvement of milbemycins in Streptomyces bingchenggensis

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Synthetic and Systems Biotechnology 2020年第3期5卷 214-221页

作者： Pinjiao Jin Shanshan Li Yanyan Zhang Liyang Chu Hairong He Zhuoxu Dong Wensheng Xiang School of Life Science Northeast Agricultural UniversityNo.59 Mucai StreetXiangfang DistrictHarbin150030China State Key Laboratory for Biology of Plant Diseases and Insect Pests Institute of Plant ProtectionChinese Academy of Agricultural SciencesBeijing100193China College of Pharmacy Henan University of Chinese MedicineZhengzhou 450046HenanChina

Dramatic decrease of sugar uptake is a general phenomenon in Streptomyces at stationary phase,when antibiotics are extensively *** produced by Streptomyces bingchenggensis are a group of valuable macrolide biopesticides,while the low yield and titer impede their broad applications in agricultural *** that inadequate sugar uptake generally hinders titer improvement of desired products,we mined the underlying sugar uptake systems and fine-tuned their expression in this ***,we screened the candidates at both genomic and transcriptomic level in ***,two ATP-binding cassette transporters named TP2 and TP5 were characterized to improve milbemycin titer and yield ***,the appropriate native temporal promoters were selected and used to tune the expression of TP2 and TP5,resulting in a maximal milbemycin A3/A4 titer increase by 36.9%to 3321 mg/***,TP2 and TP5 were broadly finetuned in another two macrolide biopesticide producers Streptomyces avermitilis and Streptomyces cyaneogriseus,leading to a maximal titer improvement of 34.1%and 52.6%for avermectin B1a and nemadectin,*** work provides useful transporter tools and corresponding engineering strategy for Streptomyces.

关键词： Streptomyces Sugar uptake system fine-tuning Titer improvement Milbemycins Macrolide biopesticides

Optimizing Enterprise Conversational AI: Accelerating Response Accuracy with Custom Dataset fine-tuning

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Intelligent Information Management 2024年第2期16卷 65-76页

作者： Yash Kishore Independent Researcher Austin USA

As the realm of enterprise-level conversational AI continues to evolve, it becomes evident that while generalized Large Language Models (LLMs) like GPT-3.5 bring remarkable capabilities, they also bring forth formidable challenges. These models, honed on vast and diverse datasets, have undoubtedly pushed the boundaries of natural language understanding and generation. However, they often stumble when faced with the intricate demands of nuanced enterprise applications. This research advocates for a strategic paradigm shift, urging enterprises to embrace a fine-tuning approach as a means to optimize conversational AI. While generalized LLMs are linguistic marvels, their inability to cater to the specific needs of businesses across various industries poses a critical challenge. This strategic shift involves empowering enterprises to seamlessly integrate their own datasets into LLMs, a process that extends beyond linguistic enhancement. The core concept of this approach centers on customization, enabling businesses to fine-tune the AI’s functionality to fit precisely within their unique business landscapes. By immersing the LLM in industry-specific documents, customer interaction records, internal reports, and regulatory guidelines, the AI transcends its generic capabilities to become a sophisticated conversational partner aligned with the intricacies of the enterprise’s domain. The transformative potential of this fine-tuning approach cannot be overstated. It enables a transition from a universal AI solution to a highly customizable tool. The AI evolves from being a linguistic powerhouse to a contextually aware, industry-savvy assistant. As a result, it not only responds with linguistic accuracy but also with depth, relevance, and resonance, significantly elevating user experiences and operational efficiency. In the subsequent sections, this paper delves into the intricacies of fine-tuning, exploring the multifaceted challenges and abundant opportunities it presents. I

关键词： fine-tuning Dataset AI Conversational Enterprise LLM

Classification of Conversational Sentences Using an Ensemble Pre-Trained Language Model with the fine-Tuned Parameter

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Computers, Materials & Continua 2024年第2期78卷 1669-1686页

作者： R.Sujatha K.Nimala Department of Networking and Communications SRM Institute of Science and TechnologyKattankulathurChengalpattuTamilnadu603203India

Sentence classification is the process of categorizing a sentence based on the context of the *** categorization requires more semantic highlights than other tasks,such as dependence parsing,which requires more syntactic *** existing strategies focus on the general semantics of a conversation without involving the context of the sentence,recognizing the progress and comparing *** ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation *** conversational sentences are classified into four categories:information,question,directive,and *** classification label sequences are for analyzing the conversation progress and predicting the pecking order of the *** of Bidirectional Encoder for Representation of Transformer(BERT),Robustly Optimized BERT pretraining Approach(RoBERTa),Generative Pre-Trained Transformer(GPT),DistilBERT and Generalized Autoregressive Pretraining for Language Understanding(XLNet)models are trained on conversation corpus with *** tuning approach is carried out for better performance on sentence *** Ensemble of Pre-trained Language Models with a Hyperparameter tuning(EPLM-HT)system is trained on an annotated conversation *** proposed approach outperformed compared to the base BERT,GPT,DistilBERT and XLNet transformer *** proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.

关键词： Bidirectional encoder for representation of transformer conversation ensemble model fine-tuning generalized autoregressive pretraining for language understanding generative pre-trained transformer hyperparameter tuning natural language processing robustly optimized BERT pretraining approach sentence classification transformer models

Regulation of insect behavior by non-coding RNAs

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Science China(Life Sciences) 2024年第6期67卷 1106-1118页

作者： Jing He Le Kang State Key Laboratory of Integrated Management of Pest Insects and Rodents Institute of ZoologyChinese Academy of SciencesBeijing 100101China Beijing Institutes of Life Sciences Chinese Academy of SciencesBeijing 100101China College of Life Science Hebei UniversityBaoding 071002China

The adaptation of insects to environments relies on a sophisticated set of behaviors controlled by molecular and physiological *** the past several decades,accumulating studies have unveiled the roles of non-coding RNAs(nc RNAs)in regulating insect *** RNAs assume particularly pivotal roles in the behavioral plasticity of insects by rapidly responding to environmental *** RNAs also contribute to the maintenance of homeostasis of insects by fine-tuning the expression of target ***,a comprehensive review of nc RNAs'roles in regulating insect behaviors has yet to be ***,we present the recent progress in our understanding of how nc RNAs regulate various insect behaviors,including flight and movement,social behavior,reproduction,learning and memory,and *** refine the intricate mechanisms by which nc RNAs modulate the function of neural,motor,reproductive,and other physiological systems,as well as gene expression in insects like fruit flies,social insects,locusts,and ***,we discuss potential avenues for future studies in nc RNA-mediated insect behaviors.

关键词： ncRNAs insect behavior plasticity fine-tuning environmental adaptation

A multiple-pass ring oscillator based dual-loop phase-locked loop

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Journal of Semiconductors 2009年第10期30卷 132-136页

作者：陈丹凤任俊彦邓晶晶李巍李宁 State Key Laboratory of ASIC & System Fudan University

A dual-loop phase-locked loop（PLL）for wideband operation is *** dual-loop architecture combines a coarse-tuning loop with a fine-tuning one,enabling a wide tuning range and low voltage-controlled oscillator（VCO）gain without poisoning phase noise and reference spur suppression *** analysis of the phase noise and reference spur of the dual-loop PLL is emphasized.A novel multiple-pass ring VCO is designed for the dual-loop *** utilizes both voltage-control and current-control simultaneously in the delay cell. The PLL is fabricated in Jazz 0.18-μm RF CMOS *** measured tuning range is from 4.2 to 5.9 *** achieves a low phase noise of–99 dBc/Hz@1 MHz offset from a 5.5 GHz carrier.

关键词： coarse-tuning dual-loop fine-tuning phase-locked loop phase noise

LLM4CP:Adapting Large Language Models for Channel Prediction

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Journal of Communications and Information Networks 2024年第2期9卷 113-125页

作者： Boxun Liu Xuanyu Liu Shijian Gao Xiang Cheng Liuqing Yang State Key Laboratory of Advanced Optical Communication Systems and Networks School of ElectronicsPeking UniversityBeijing 100871China Internet of Things Thrust The Hong Kong University of Science and Technology(Guangzhou)Guangzhou 511400China Internet of Things Thrust and Intelligent Transportation Thrust The Hong Kong University of Science and Technology(Guangzhou)Guangzhou 511400China Department of Electronic and Computer Engineering and Department of Civil and Environmental Engineering The Hong Kong University of Science and TechnologyHong KongChina

Channel prediction is an effective approach for reducing the feedback or estimation overhead in massive multi-input multi-output (m-MIMO) systems. However, existing channel prediction methods lack precision due to model mismatch errors or network generalization issues. Large language models (LLMs) have demonstrated powerful modeling and generalization abilities, and have been successfully applied to cross-modal tasks, including the time series analysis. Leveraging the expressive power of LLMs, we propose a pre-trained LLM-empowered channel prediction(LLM4CP)method to predict the future downlink channel state information (CSI) sequence based on the historical uplink CSI sequence. We fine-tune the network while freezing most of the parameters of the pre-trained LLM for better cross-modality knowledge transfer. To bridge the gap between the channel data and the feature space of the LLM,preprocessor, embedding, and output modules are specifically tailored by taking into account unique channel characteristics. Simulations validate that the proposed method achieves state-of-the-art (SOTA) prediction performance on full-sample, few-shot, and generalization tests with low training and inference costs.

关键词： channel prediction massive multi-input multi-output(m-MIMO) large language models(LLMs) fine-tuning time-series

Enhancing Fire Detection Performance Based on fine-Tuned YOLOv10

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Computers, Materials & Continua 2024年第11期81卷 2281-2298页

作者： Trong Thua Huynh Hoang Thanh Nguyen Du Thang Phu Faculty of Information Technology 2 Posts and Telecommunications Institute of Technology(PTIT)Ho Chi Minh City700000Vietnam

In recent years,early detection and warning of fires have posed a significant challenge to environmental protection and human *** learning models such as Faster R-CNN(Faster Region based Convolutional Neural Network),YOLO(You Only Look Once),and their variants have demonstrated superiority in quickly detecting objects from images and videos,creating new opportunities to enhance automatic and efficient fire *** YOLO model,especially newer versions like YOLOv10,stands out for its fast processing capability,making it suitable for low-latency ***,when applied to real-world datasets,the accuracy of fire prediction is still not *** study improves the accuracy of YOLOv10 for real-time applications through model fine-tuning techniques and data *** core work of the research involves creating a diverse fire image dataset specifically suited for fire detection applications in buildings and factories,freezing the initial layers of the model to retain general features learned from the dataset by applying the Squeeze and Excitation attention mechanism and employing the Stochastic Gradient Descent(SGD)with a momentum optimization algorithm to enhance accuracy while ensuring real-time fire *** results demonstrate the effectiveness of the proposed fire prediction approach,where the YOLOv10 small model exhibits the best balance compared to other YOLO family models such as nano,medium,and ***,the study provides an experimental evaluation to highlight the effectiveness of model fine-tuning compared to the YOLOv10 baseline,YOLOv8 and Faster R-CNN based on two criteria:accuracy and prediction time.

关键词： Fire detection accuracy prediction time fine-tuning real-time YOLOv10 Faster R-CNN