检索结果-南通市图书馆

Tsinghua Science and Technology 2024年第6期29卷 1851-1871页

作者： Maojie Sun Fang Dong Zhaowu Huang Junzhou Luo School of Computer Science and Engineering Southeast UniversityNanjing 211189China Najing Iron and Steel Co. Nanjing 210035China

The steel plate is one of the main products in steel industries,and its surface quality directly affects the final product *** to detect surface defects of steel plates in real time during the production process is a challenging *** single or fixed model compression method cannot be directly applied to the detection of steel surface defects,because it is difficult to consider the diversity of production tasks,the uncertainty caused by environmental factors,such as communication networks,and the influence of process and working conditions in steel plate *** this paper,we propose an adaptive model compression method for steel surface defect online detection based on expert knowledge and working ***,we establish an expert system to give lightweight model parameters based on the correlation between defect types and manufacturing ***,lightweight model parameters are adaptively adjusted according to working conditions,which improves detection accuracy while ensuring real-time *** experimental results show that compared with the detection method of constant lightweight parameter model,the proposed method makes the total detection time cut down by 23.1%,and the deadline satisfaction ratio increased by 36.5%,while upgrading the accuracy by 4.2%and reducing the false detection rate by 4.3%.

关键词： steel surface defect detection inference acceleration model compression expert knowledge pruning quantization

A Novel Quantization and model compression Approach for Hardware Accelerators in Edge Computing

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Computers, Materials & Continua 2024年第8期80卷 3021-3045页

作者： Fangzhou He Ke Ding DingjiangYan Jie Li Jiajun Wang Mingzhe Chen State Key Laboratory of Intelligent Vehicle Safety Technology Chongqing401133China Foresight Technology Institute Chongqing Changan Automobile Co.Ltd.Chongqing400023China School of Computer Science and Engineering Chongqing University of Science and TechnologyChongqing401331China

Massive computational complexity and memory requirement of artificial intelligence models impede their deploy-ability on edge computing devices of the Internet of Things(IoT).While Power-of-Two(PoT)quantization is pro-posed to improve the efficiency for edge inference of Deep Neural Networks(DNNs),existing PoT schemes require a huge amount of bit-wise manipulation and have large memory overhead,and their efficiency is bounded by the bottleneck of computation latency and memory *** tackle this challenge,we present an efficient inference approach on the basis of PoT quantization and model *** integer-only scalar PoT quantization(IOS-PoT)is designed jointly with a distribution loss regularizer,wherein the regularizer minimizes quantization errors and training ***,two-stage model compression is developed to effectively reduce memory requirement,and alleviate bandwidth usage in communications of networked heterogenous learning *** product look-up table(P-LUT)inference scheme is leveraged to replace bit-shifting with only indexing and addition operations for achieving low-latency computation and implementing efficient edge ***,comprehensive experiments on Residual Networks(ResNets)and efficient architectures with Canadian Institute for Advanced Research(CIFAR),ImageNet,and Real-world Affective Faces Database(RAF-DB)datasets,indicate that our approach achieves 2×∼10×improvement in the reduction of both weight size and computation cost in comparison to state-of-the-art methods.A P-LUT accelerator prototype is implemented on the Xilinx KV260 Field Programmable Gate Array(FPGA)platform for accelerating convolution operations,with performance results showing that P-LUT reduces memory footprint by 1.45×,achieves more than 3×power efficiency and 2×resource efficiency,compared to the conventional bit-shifting scheme.

关键词： Edge computing model compression hardware accelerator power-of-two quantization

Effective model compression via Stage-wise Pruning

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Machine Intelligence Research 2023年第6期20卷 937-951页

作者： Ming-Yang Zhang Xin-Yi Yu Lin-Lin Ou School of Information Engineering Zhejiang University of TechnologyHangzhou310000China

Automated machine learning(AutoML)pruning methods aim at searching for a pruning strategy automatically to reduce the computational complexity of deep convolutional neural networks(deep CNNs).However,some previous work found that the results of many Auto-ML pruning methods cannot even surpass the results of the uniformly pruning *** this paper,the ineffectiveness of Auto-ML pruning,which is caused by unfull and unfair training of the supernet,is shown.A deep supernet suffers from unfull training because it contains too many *** overcome the unfull training,a stage-wise pruning(SWP)method is proposed,which splits a deep supernet into several stage-wise supernets to reduce the candidate number and utilize inplace distillation to supervise the stage ***,a wide supernet is hit by unfair training since the sampling probability of each channel is ***,the fullnet and the tinynet are sampled in each training iteration to ensure that each channel can be ***,the proxy performance of the subnets trained with SWP is closer to the actual performance than that of most of the previous AutoML pruning ***,experiments show that SWP achieves the state-of-the-art in both CIFAR-10 and ImageNet under the mobile setting.

关键词： Automated machine learning(AutoML) channel pruning model compression distillation convolutional neural networks(CNN)

A pruning-then-quantization model compression framework for facial emotion recognition

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Intelligent and Converged Networks 2023年第3期4卷 225-236页

作者： Han Sun Wei Shao Tao Li Jiayu Zhao Weitao Xu Linqi Song the Department of Computer Science City University of Hong KongHong Kong 999077China the Department of Computer Science City University of Hong KongHong Kong 999077Chinaand also with City University of Hong Kong Shenzhen Research InstituteShenzhen 518057China the Shenzhen Konka Electronic Technology Co. Ltd.Shenzhen 518053China

Facial emotion recognition achieves great success with the help of large neural models but also fails to be applied in practical situations due to the large model size of neural *** bridge this gap,in this paper,we combine two mainstream model compression methods(pruning and quantization)together,and propose a pruningthen-quantization framework to compress the neural models for facial emotion recognition *** on three datasets show that our model could achieve a high model compression ratio and maintain the model’s high performance ***,We analyze the layer-wise compression performance of our proposed framework to explore its effect and adaptability in fine-grained modules.

关键词： model compression facial emotion recognition Resnet

Acoustic model compression with Knowledge Transfer

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Acoustic Model Compression with Knowledge Transfer

Channel pruning on frequency response

第十四届全国人机语音通讯学术会议

作者： Jiangyan Yi Jianhua Tao Zhengqi Wen Ya Li Hao Ni National Laboratory of Pattern Recognition Institute of AutomationChinese Academy of Sciences School of Computer and Control Engineering University of Chinese Academy of Sciences CAS Center for Excellence in Brain Science and Intelligence Technology Institute of AutomationChinese Academy of Sciences

Mobile devices have limited computing power and limited ***,large deep neural network（DNN） based acoustic models are not well suited for application on mobile *** order to alleviate this problem,this paper proposes to compress acoustic models by using knowledge *** approach forces a large teacher model to transfer generalized knowledge to a small student *** student model is trained with a linear interpolation of hard probabilities and soft probabilities to learn generalized knowledge from the teacher *** hard probabilities are generated from a Gaussian mixture model hidden Markov model（GMM-HMM） *** soft probabilities are computed from a teacher model（DNN or RNN）.Experiments on AMI corpus show that a small student model obtains 2.4%relative WER improvement over a large teacher model with almost 7.6 times compression ratio.

关键词： model compression knowledge transfer deep neural networks automatic speech recognition

来源： cnki会议评论

在线全文

cnki会议

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2025年第1期68卷 159-170页

作者： Hang LIN Yifan PENG Lin BIE Chenggang YAN Xibin ZHAO Yue GAO School of Software Tsinghua University School of Automation Hangzhou Dianzi University

Network pruning has a significant role in reducing network parameters and accelerating the inference time of the network. Some existing methods prune the network based on the frequency of the data, and finally obtain a sub-network with high accuracy. However, according to our experimental analysis, different frequencies of information in the data contribute differently to the accuracy of the model, and using this information directly for pruning without making a selection will lead to incorrect results. We believe that pruning should retain the convolutional kernels in the network that process important information, while those kernels that process unimportant information should be removed. In this paper, we first investigate the meaning of each frequency band information in the spectrum and their contribution to the prediction accuracy of the network,and according to these results, we propose a new pruning method based on frequency response(PFR). Our PFR finds and removes the convolutional kernels in the network that specialize in processing unimportant information, resulting in a compact neural network model. PFR obtains significant experimental results on different datasets, for example, a 56.0% raduction of float points operations(FLOPs) on Res Net-50 and only 0.37% of Top-1 accuracy degradation on the Image Net dataset.

关键词： deep learning model compression filter pruning channel pruning frequency response

同方期刊数据库

A revisit to MacKay algorithm and its application to deep network compression

在线全文

学校读者我要写书评

暂无评论

Frontiers of Computer Science 2020年第4期14卷 39-54页

作者： Chune LI Yongyi MAO Richong ZHANG Jinpeng HUAI School of Computer Science and Engineering Beihang UniversityBeijing100191China School of Electrical Engineering and Computer Science University of OttawaOttawaK1N6N5Canada

An iterative procedure introduced in MacKay’s evidence framework is often used for estimating the hyperparameter in empirical *** with the use of a particular form of prior,the estimation of the hyperparameter reduces to an automatic relevance determination model,which provides a soft way of pruning model *** the effectiveness of this estimation procedure,it has stayed primarily as a heuristic to date and its application to deep neural network has not yet been *** paper formally investigates the mathematical nature of this procedure and justifies it as a well-principled algorithm framework,which we call the MacKay *** an application,we demonstrate its use in deep neural networks,which have typically complicated structure with millions of parameters and can be pruned to reduce the memory requirement and boost computational *** experiments,we adopt MacKay algorithm to prune the parameters of both simple networks such as LeNet,deep convolution VGG-like networks,and residual netowrks for large image classification *** results show that the algorithm can compress neural networks to a high level of sparsity with little loss of prediction accuracy,which is comparable with the state-of-the-art.

关键词： deep learning MacKay algorithm model compression neural network

Enhancing the robustness of object detection via 6G vehicular edge computing

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Digital Communications and Networks 2022年第6期8卷 923-931页

作者： Chen Chen Guorun Yao Chenyu Wang Sotirios Goudos Shaohua Wan School of Telecommunication Engineering Xidian UniversityXi'an710071China Department of Physics ELEDIA@AUTHAristotle University of Thessaloniki54124ThessalonikiGreece Shenzhen Institute for Advanced Study University of Electronic Science and Technology of ChinaShenzhen518110China

Academic and industrial communities have been paying significant attention to the 6th Generation (6G) wireless communication systems after the commercial deployment of 5G cellular communications. Among the emerging technologies, Vehicular Edge Computing (VEC) can provide essential assurance for the robustness of Artificial Intelligence (AI) algorithms to be used in the 6G systems. Therefore, in this paper, a strategy for enhancing the robustness of AI model deployment using 6G-VEC is proposed, taking the object detection task as an example. This strategy includes two stages: model stabilization and model adaptation. In the former, the state-of-the-art methods are appended to the model to improve its robustness. In the latter, two targeted compression methods are implemented, namely model parameter pruning and knowledge distillation, which result in a trade-off between model performance and runtime resources. Numerical results indicate that the proposed strategy can be smoothly deployed in the onboard edge terminals, where the introduced trade-off outperforms the other strategies available.

关键词： 6G Vehicular edge computing Object detection Feature fusion model compression model deployment

维普期刊数据库

Communication-efficient federated continual learning for distributed learning system with Non-IID data

在线全文

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2023年第2期66卷 94-113页

作者： Zhao ZHANG Yong ZHANG Da GUO Shuang ZHAO Xiaolin ZHU School of Electronic Engineering Beijing University of Posts and Telecommunications Beijing Key Laboratory of Work Safety Intelligent Monitoring Beijing University of Posts and Telecommunication

Due to the privacy preserving capabilities and the low communication costs, federated learning has emerged as an efficient technique for distributed deep learning/machine learning training. However, given the typical heterogeneous data distributions in the realistic scenario, federated learning faces the challenge of performance degradation on non-independent identically distributed(Non-IID) data across clients. Therefore, we propose federated continual learning to improve the performance on Non-IID data by introducing the knowledge of the other local models. Specifically, we propose a novel federated continual learning method called Fed SI, adapting the synaptic intelligence method to the federated learning scenario. Furthermore,in order to reduce the communication overheads, we propose the bidirectional compression and error compensation(BCEC) algorithm to produce the communication-efficient federated continual learning method,called CFedSI. Specifically, the proposed BCEC algorithm compresses both the uplink and the downlink transmission data and utilizes the error compensation locally to ensure training divergence. Experiments show that CFedSI improves the accuracy on Non-IID data by up to 46% with KDDCUP’99 dataset, 23%with CICIDS2017 dataset, 22% with MNIST dataset, and 8% with FashionMNIST dataset, along with the reduced communication overheads.

关键词： distributed learning system federated learning continual learning model compression error compensation

PokerNet:Expanding Features Cheaply via Depthwise Convolutions

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

International Journal of Automation and computing 2021年第3期18卷 432-442页

作者： Wei Tang Yan Huang Liang Wang National Laboratory of Pattern Recognition(NLPR) Institute of AutomationChinese Academy of SciencesBeijing 100190China Center for Research on Intelligent Perception and Computing(CRIPAC) Institute of AutomationChinese Academy of SciencesBeijing 100190China

Pointwise convolution is usually utilized to expand or squeeze features in modern lightweight deep ***,it takes up most of the overall computational cost(usually more than 90%).This paper proposes a novel Poker module to expand features by taking advantage of cheap depthwise *** a result,the Poker module can greatly reduce the computational cost,and meanwhile generate a large number of effective features to guarantee the *** proposed module is standardized and can be employed wherever the feature expansion is *** varying the stride and the number of channels,different kinds of bottlenecks are designed to plug the proposed Poker module into the ***,a lightweight model can be easily *** conducted on benchmarks reveal the effectiveness of our proposed Poker *** our Poker Net models can reduce the computational cost by 7.1%-15.6%.Poker Net models achieve comparable or even higher recognition accuracy than previous state-of-the-art(SOTA)models on the Image Net ILSVRC2012 classification *** is available at https://***/diaomin/pokernet.

关键词： Deep learning depthwise convolution lightweight deep model model compression model acceleration

维普期刊数据库