检索结果-南通市图书馆

Optimizing Fine-Tuning in Quantized Language Models:An In-Depth Analysis of Key Variables

Computers, Materials & Continua 2025年第1期82卷 307-325页

作者： Ao Shen Zhiquan Lai Dongsheng Li Xiaoyu Hu National Key laboratory of parallel and distributed Computing National University of Defense TechnologyChangsha410073China Strategic Assessments and Consultation Institute Academy of Military ScienceBeijing100091China

Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language processing(NLP),driven by the pre-training and fine-tuning *** this approach allows models to specialize in specific tasks with reduced training costs,the substantial memory requirements during fine-tuning present a barrier to broader ***-Efficient Fine-Tuning(PEFT)techniques,such as Low-Rank Adaptation(LoRA),and parameter quantization methods have emerged as solutions to address these challenges by optimizing memory usage and computational *** these,QLoRA,which combines PEFT and quantization,has demonstrated notable success in reducing memory footprints during fine-tuning,prompting the development of various QLoRA *** these advancements,the quantitative impact of key variables on the fine-tuning performance of quantized LLMs remains *** study presents a comprehensive analysis of these key variables,focusing on their influence across different layer types and depths within LLM *** investigation uncovers several critical findings:(1)Larger layers,such as MLP layers,can maintain performance despite reductions in adapter rank,while smaller layers,like self-attention layers,aremore sensitive to such changes;(2)The effectiveness of balancing factors depends more on specific values rather than layer type or depth;(3)In quantization-aware fine-tuning,larger layers can effectively utilize smaller adapters,whereas smaller layers struggle to do *** insights suggest that layer type is a more significant determinant of fine-tuning success than layer depth when optimizing quantized ***,for the same discount of trainable parameters,reducing the trainable parameters in a larger layer is more effective in preserving fine-tuning accuracy than in a smaller *** study provides valuable guidance for more efficient fine-tuning strategies and opens avenues for further research into optimizing LLM

关键词： Large-scale Language Model Parameter-Efficient Fine-Tuning parameter quantization key variable trainable parameters experimental analysis

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

A Unified Co-Processor Architecture for Matrix Decomposition

引用

Journal of Computer science & technology 2010年第4期25卷 874-885页

作者：窦勇周杰邬贵明姜晶菲雷元武倪时策 National laboratory for parallel & distributed processing National University of Defense Technology National laboratory for parallel & distributed processing.National University of Defense technology

QR and LU decompositions are the most important matrix decomposition algorithms. Many studies work on accelerating these algorithms by FPGA or ASIC in a case by case style. In this paper, we propose a unified framework for the matrix decomposition algorithms, combining three QR decomposition algorithms and LU algorithm with pivoting into a unified linear array structure. The QR and LU decomposition algorithms exhibit the same two-level loop structure and the same data dependency. Utilizing the similarities in loop structure and data dependency of matrix decomposition, we unify a fine-grained algorithm for all four matrix decomposition algorithms. Furthermore, we present a unified co-processor structure with a scalable linear array of processing elements （PEs）, in which four types of PEs are same in the structure of memory channels and PE connections, but the only difference exists in the internal structure of data path. Our unified co-processor, which is IEEE 32-bit floating-point precision, is implemented and mapped onto a Xilinx Virtex5 FPGA chip. Experimental results show that our co-processors can achieve speedup of 2.3 to 14.9 factors compared to a Pentium Dual CPU with double SSE threads.

关键词： co-processor matrix decomposition fine-grained parallel FPGA

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Flip-flops soft error rate evaluation approach considering internal single-event transient

引用

science China(Information sciences) 2015年第6期58卷 159-170页

作者： SONG RuiQiang CHEN ShuMing HE YiBai DU YanKang College of Computer science National University of Defense Technology science and technology on parallel and distributed processing laboratory

The internal single-event transient(SET) induced upset in flip-flops is becoming significant with the increase of the operating frequency. However, the conventional soft error rate(SER) evaluation approach could only produce an approximate upset prediction result caused by the internal SET. In this paper, we propose an improved SER evaluation approach based on Monte Carlo simulation. A novel SET-based upset model is implemented in the proposed evaluation approach to accurately predict upsets caused by the internal SET. A test chip was fabricated in a commercial 65 nm bulk process to validate the accuracy of the improved SER evaluation approach. The predicted single-event upset cross-sections are consistent with the experimental data.

关键词： soft error rate Monte Carlo internal SET single-event upset flip-flops

来源：

同方期刊数据库评论

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

DKP-SLAM:A Visual SLAM for Dynamic Indoor Scenes Based on Object Detection and Region Probability

引用

Computers, Materials & Continua 2025年第1期82卷 1329-1347页

作者： Menglin Yin Yong Qin Jiansheng Peng College of Automation Guangxi University of Science and TechnologyLiuzhou545000China Department of Artificial Intelligence and Manufacturing Hechi UniversityHechi546300China Key laboratory of AI and Information processing Education Department of Guangxi Zhuang Autonomous RegionHechi546300China Guangxi Key laboratory of Sericulture Ecology and Applied Intelligent technology School of Chemistry and BioengineeringHechi UniversityHechi546300China

In dynamic scenarios,visual simultaneous localization and mapping(SLAM)algorithms often incorrectly incorporate dynamic points during camera pose computation,leading to reduced accuracy and *** paper presents a dynamic SLAM algorithm that leverages object detection and regional dynamic ***,a parallel thread employs the YOLOX object detectionmodel to gather 2D semantic information and compensate for missed ***,an improved K-means++clustering algorithm clusters bounding box regions,adaptively determining the threshold for extracting dynamic object contours as dynamic points *** process divides the image into low dynamic,suspicious dynamic,and high dynamic *** the tracking thread,the dynamic point removal module assigns dynamic probability weights to the feature points in these *** with geometric methods,it detects and removes the dynamic *** final evaluation on the public TUM RGB-D dataset shows that the proposed dynamic SLAM algorithm surpasses most existing SLAM algorithms,providing better pose estimation accuracy and robustness in dynamic environments.

关键词： Visual SLAM dynamic scene YOLOX K-means++clustering dynamic probability

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Demand-Driven Memory Leak Detection Based on Flow-and Context-Sensitive Pointer Analysis

引用

Journal of Computer science & technology 2009年第2期24卷 347-356页

作者：王戟马晓东董威徐厚峰刘万伟 National laboratory for parallel and distributed processing National University of Defense Technology

We present a demand-driven approach to memory leak detection algorithm based on flow- and context-sensitive pointer analysis. The detection algorithm firstly assumes the presence of a memory leak at some program point and then runs a backward analysis to see if this assumption can be disproved. Our algorithm computes the memory abstraction of programs based on points-to graph resulting from flow- and context-sensitive pointer analysis. We have implemented the algorithm in the SUIF2 compiler infrastructure and used the implementation to analyze a set of C benchmark programs. The experimental results show that the approach has better precision with satisfied scalability as expected.

关键词： flow-sensitive memory leak detection demand-driven static analysis

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Representation learning on textual network with personalized Page Rank

引用

science China(Information sciences) 2021年第11期64卷 95-104页

作者： Teng LI Yong DOU National laboratory for parallel and distributed processing National University of Defense Technology

Representation learning on textual network or textual network embedding, which leverages rich textual information associated with the network structure to learn low-dimensional embedding of vertices, has been useful in a variety of tasks. However, most approaches learn textual network embedding by using direct neighbors. In this paper, we employ a powerful and spatially localized operation: personalized Page Rank(PPR) to eliminate the restriction of using only the direct connection relationship. Also, we analyze the relationship between PPR and spectral-domain theory, which provides insight into the empirical performance boost. From the experiment, we discovered that the proposed method provides a great improvement in linkprediction tasks, when compared to existing methods, achieving a new state-of-the-art on several real-world benchmark datasets.

关键词： representation learning network embedding PageRank textual network personalized PageRank

来源：

同方期刊数据库评论

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Scalability of 3D deterministic particle transport on the Intel MIC architecture

引用

Nuclear science and Techniques 2015年第5期26卷 88-97页

作者：王庆林刘杰龚春叶邢座程 science and technology on parallel and distributed processing laboratory National University of Defense Technology science and technology on Space Physics laboratory

The key to large-scale parallel solutions of deterministic particle transport problem is single-node computation performance. Hence, single-node computation is often parallelized on multi-core or many-core computer architectures. However, the number of on-chip cores grows quickly with the scale-down of feature size in semiconductor technology. In this paper, we present a scalability investigation of one energy group time-independent deterministic discrete ordinates neutron transport in 3D Cartesian geometry(Sweep3D) on Intel's Many Integrated Core(MIC) architecture, which can provide up to 62 cores with four hardware threads per core now and will own up to 72 in the future. The parallel programming model, Open MP, and vector intrinsic functions are used to exploit thread parallelism and vector parallelism for the discrete ordinates method, respectively. The results on a 57-core MIC coprocessor show that the implementation of Sweep3 D on MIC has good scalability in performance. In addition, the application of the Roofline model to assess the implementation and performance comparison between MIC and Tesla K20 C Graphics processing Unit(GPU) are also reported.

关键词：计算机体系结构可扩展性粒子输运三维几何英特尔麦克风离散坐标法计算性能

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Deep reinforcement learning:a survey

引用

Frontiers of Information technology & Electronic Engineering 2020年第12期21卷 1726-1744页

作者： Hao-nan WANG Ning LIU Yi-yun ZHANG Da-wei FENG Feng HUANG Dong-sheng LI Yi-ming ZHANG science and technology on parallel and distributed processing laboratory National University of Defense TechnologyChangsha 41OOOOChina

Deep reinforcement learning(RL)has become one of the most popular topics in artificial intelligence *** has been widely used in various fields,such as end-to-end control,robotic control,recommendation systems,and natural language dialogue *** this survey,we systematically categorize the deep RL algorithms and applications,and provide a detailed review over existing deep RL algorithms by dividing them into modelbased methods,model-free methods,and advanced RL *** thoroughly analyze the advances including exploration,inverse RL,and transfer ***,we outline the current representative applications,and analyze four open problems for future research.

关键词： Reinforcement learning Deep reinforcement learning Reinforcement learning applications

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Surveying concurrency bug detectors based on types of detected bugs

引用

science China(Information sciences) 2017年第3期60卷 5-31页

作者： Zhendong WU Kai LU Xiaoping WANG science and technology on parallel and distributed processing laboratory National University of Defense Technology College of Computer National University of Defense Technology

Concurrency bugs widely exist in concurrent programs and have caused severe failures in the real world. Researchers have made significant progress in detecting concurrency bugs, which improves software reliability. In this paper, we survey the most up-to-date and well-known concurrency bug detectors. We categorize the existing detectors based on the types of concurrency bugs. Consequently, we analyze data race detectors, atomicity violation detectors, order violation detectors, and deadlock detectors, respectively. We also discuss some other techniques which are mostly related to concurrency bug detection, including schedule bounding techniques, interleaving optimizing techniques, path expanding techniques, and deterministic replay techniques. Additionally, we statistically analyze the reviewed detectors and get some interesting findings, for instance, nearly 86% of previous detectors focus on data races and atomicity violations, and dynamic approaches are popular(74%). We also discuss the limitations of previous detectors, finding that 91% of previous detectors suffer from false negatives and 64% of previous detectors suffer from runtime overhead. Based on the reviewed detectors and statistical analysis, we conclude some future research directions, including accuracy, performance,applicability, and integrality.

关键词： concurrency bug detection data race atomicity violation order violation deadlock

来源：

同方期刊数据库评论

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Managing Data-Objects in Dynamically Reconfigurable Caches

引用

Journal of Computer science & technology 2010年第2期25卷 232-245页

作者：杨学军吴俊杰曾坤唐玉华 National laboratory for parallel and distributed processing School of ComputerNational University of Defense Technology

The widening gap between processor and memory speeds makes cache an important issue in the computer system design. Compared with work set of programs, cache resource is often rare. Therefore, it is very important for a computer system to use cache efficiently. Toward a dynamically reconfigurable cache proposed recently, DOOC （Data- Object Oriented Cache）, this paper proposes a quantitative framework for analyzing the cache requirement of data-objects, which includes cache capacity, block size, associativity and coherence protocol. And a kind of graph coloring algorithm dealing with the competition between data-objects in the DOOC is proposed as well. Finally, we apply our approaches to the compiler management of DOOC. We test our approaches on both a single-core platform and a four-core platform. Compared with the traditional caches, the DOOC in both platforms achieves an average reduction of 44.98% and 49.69% in miss rate respectively. And its performance is very close to the ideal optimal cache.

关键词： cache data-object data reuse data-object oriented cache (DOOC) graph coloring

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

欢迎您,

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

请选择保存的检索档案：

请选择收藏分类：

通借通还

欢迎您,

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：