检索结果-南通市图书馆

Intelligent and Converged Networks 2020年第3期1卷 234-242页

作者： Junyu Dong Wenjun Wu Yang Gao Xiaoxi Wang Pengbo Si Faculty of Information Technology Beijing University of TechnologyBeijing 100022China

Nowadays,Edge Information System(EIS)has received a lot of *** EIS,distributed machine learning(DML),which requires fewer computing resources,can implement many artificial intelligent applications ***,due to the dynamical network topology and the fluctuating transmission quality at the edge,work node selection affects the performance of DML a *** this paper,we focus on the Internet of Vehicles(IoV),one of the typical scenarios of EIS,and consider the DML-based High Definition(HD)mapping and intelligent driving decision model as the *** worker selection problem is modeled as a Markov Decision Process(MDP),maximizing the DML model aggregate performance related to the timeliness of the local model,the transmission quality of model parameters uploading,and the effective sensing area of the worker.A Deep Reinforcement learning(DRL)based solution is proposed,called the Worker Selection based on Policy Gradient(PG-WS)*** policy mapping from the system state to the worker selection action is represented by a deep neural *** episodic simulations are built and the REINFORCE algorithm with baseline is used to train the policy *** show that the proposed PG-WS algorithm outperforms other comparation methods.

关键词： edge information system internet of vehicles distributed machine learning deep reinforcement learning worker selection

Adaptive Load Balancing for Parameter Servers in distributed machine learning over Heterogeneous Networks

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

ZTE Communications 2023年第1期21卷 72-80页

作者： CAI Weibo YANG Shulin SUN Gang ZHANG Qiming YU Hongfang University of Electronic Science and Technology of China Chengdu 611731China ZTE Corporation Shenzhen 518057China

In distributed machine learning(DML)based on the parameter server(PS)architecture,unbalanced communication load distribution of PSs will lead to a significant slowdown of model synchronization in heterogeneous networks due to low utilization of *** address this problem,a network-aware adaptive PS load distribution scheme is proposed,which accelerates model synchronization by proactively adjusting the communication load on PSs according to network *** evaluate the proposed scheme on MXNet,known as a realworld distributed training platform,and results show that our scheme achieves up to 2.68 times speed-up of model training in the dynamic and heterogeneous network environment.

关键词： distributed machine learning network awareness parameter server load distribution heterogeneous network

distributed Deep Reinforcement learning:A Survey and a Multi-player Multi-agent learning Toolbox

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

machine Intelligence Research 2024年第3期21卷 411-430页

作者： Qiyue Yin Tongtong Yu Shengqi Shen Jun Yang Meijing Zhao Wancheng Ni Kaiqi Huang Bin Liang Liang Wang Institute of Automation Chinese Academy of SciencesBeijing 100190China School of Artificial Intelligence University of Chinese Academy of SciencesBeijing 100049China Department of Automation Tsinghua UniversityBeijing 100084China Center for Excellence in Brain Science and Intelligence Technology Chinese Academy of SciencesBeijing 100190China

With the breakthrough of AlphaGo,deep reinforcement learning has become a recognized technique for solving sequential decision-making *** its reputation,data inefficiency caused by its trial and error learning mechanism makes deep reinforcement learning difficult to apply in a wide range of *** methods have been developed for sample efficient deep reinforcement learning,such as environment modelling,experience transfer,and distributed modifications,among which distributed deep reinforcement learning has shown its potential in various applications,such as human-computer gaming and intelligent *** this paper,we conclude the state of this exciting field,by comparing the classical distributed deep reinforcement learning methods and studying important components to achieve efficient distributed learning,covering single player single agent distributed deep reinforcement learning to the most complex multiple players multiple agents distributed deep reinforcement ***,we review recently released toolboxes that help to realize distributed deep reinforcement learning without many modifications of their non-distributed *** analysing their strengths and weaknesses,a multi-player multi-agent distributed deep reinforcement learning toolbox is developed and released,which is further validated on Wargame,a complex environment,showing the usability of the proposed toolbox for multiple players and multiple agents distributed deep reinforcement learning under complex ***,we try to point out challenges and future trends,hoping that this brief review can provide a guide or a spark for researchers who are interested in distributed deep reinforcement learning.

关键词： Deep reinforcement learning distributed machine learning self-play population-play toolbox

A Tutorial on Federated learning from Theory to Practice:Foundations,Software Frameworks,Exemplary Use Cases,and Selected Trends

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

IEEE/CAA Journal of Automatica Sinica 2024年第4期11卷 824-850页

作者： M.Victoria Luzón Nuria Rodríguez-Barroso Alberto Argente-Garrido Daniel Jiménez-López Jose M.Moyano Javier Del Ser Weiping Ding Francisco Herrera Department of Software Engineering Andalusian Research Institute in Data Science and Computational Intelligence(DaSCI)University of GranadaGranada 18071Spain Department of Computer Science and Artificial Intelligence Andalusian Research Institute in Data Science and Computational Intelligence(DaSCI)University of GranadaGranada 18071Spain Department of Communications Engineering University of the Basque Country(UPV/EHU)and also with TECNALIABasque Research&Technology Alliance(BRTA)Spain School of Information Science and Technology Nantong UniversityNantong 226019China

When data privacy is imposed as a necessity,Federated learning(FL)emerges as a relevant artificial intelligence field for developing machine learning(ML)models in a distributed and decentralized *** allows ML models to be trained on local devices without any need for centralized data transfer,thereby reducing both the exposure of sensitive data and the possibility of data interception by malicious third *** paradigm has gained momentum in the last few years,spurred by the plethora of real-world applications that have leveraged its ability to improve the efficiency of distributed learning and to accommodate numerous participants with their data *** virtue of FL,models can be learned from all such distributed data sources while preserving data *** aim of this paper is to provide a practical tutorial on FL,including a short methodology and a systematic analysis of existing software ***,our tutorial provides exemplary cases of study from three complementary perspectives:i)Foundations of FL,describing the main components of FL,from key elements to FL categories;ii)Implementation guidelines and exemplary cases of study,by systematically examining the functionalities provided by existing software frameworks for FL deployment,devising a methodology to design a FL scenario,and providing exemplary cases of study with source code for different ML approaches;and iii)Trends,shortly reviewing a non-exhaustive list of research directions that are under active investigation in the current FL *** ultimate purpose of this work is to establish itself as a referential work for researchers,developers,and data scientists willing to explore the capabilities of FL in practical applications.

关键词： Data privacy distributed machine learning federated learning software frameworks

Elastic Optimization for Stragglers in Edge Federated learning

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Big Data Mining and Analytics 2023年第4期6卷 404-420页

作者： Khadija Sultana Khandakar Ahmed Bruce Gu Hua Wang Institute for Sustainable Industries and Liveable Cities(ISILC) Victoria UniversityMelbourne 3011Australia Intelligent Technology Innovation Lab(ITIL) Institute for Sustainable Industries and Liveable Cities(ISILC)Victoria UniversityMelbourne 3011Australia. Shandong Computer Science Center(National Supercomputer Center) Jinan 250101China

To fully exploit enormous data generated by intelligent devices in edge computing,edge federated learning(EFL)is envisioned as a promising *** distributed collaborative training in EFL deals with delay and privacy issues compared to traditional centralized model ***,the existence of straggling devices,responding slow to servers,degrades model *** consider data heterogeneity from two aspects:high dimensional data generated at edge devices where the number of features is greater than that of observations and the heterogeneity caused by partial device *** large number of features,computation overhead on the devices increases,causing edge devices to become *** incorporation of partial training results causes gradients to be diverged which further exaggerates when more training is performed to reach local *** this paper,we introduce elastic optimization methods for stragglers due to data heterogeneity in edge federated ***,we define the problem of stragglers in ***,we formulate an optimization problem to be solved at edge *** customize a benchmark algorithm,FedAvg,to obtain a new elastic optimization algorithm(FedEN)which is applied in local training of edge *** mitigates stragglers by having a balance between lasso and ridge penalization thereby generating sparse model updates and enforcing parameters as close as to local *** have evaluated the proposed model on MNIST and CIFAR-10 *** experiments demonstrate that our approach improves run time training performance by achieving average accuracy with less communication *** results confirm the improved performance of our approach over benchmark algorithms.

关键词： edge computing federated learning distributed machine learning regularization stragglers

FedTC:A Personalized Federated learningMethod with Two Classifiers

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Computers, Materials & Continua 2023年第9期76卷 3013-3027页

作者： Yang Liu Jiabo Wang Qinbo Liu Mehdi Gheisari Wanyin Xu Zoe L.Jiang Jiajia Zhang School of Computer Science and Technology Harbin Institute of TechnologyShenzhen518055China Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies Shenzhen518055China Research Center for Cyberspace Security Peng Cheng LaboratoryShenzhen518055China

Centralized training of deep learning models poses privacy risks that hinder their *** learning(FL)has emerged as a solution to address these risks,allowing multiple clients to train deep learning models collaborativelywithout sharing ***,FL is vulnerable to the impact of heterogeneous distributed data,which weakens convergence stability and suboptimal performance of the trained model on local *** is due to the discarding of the old local model at each round of training,which results in the loss of personalized information in the model critical for maintaining model accuracy and ensuring *** this paper,we propose FedTC,a personalized federated learning method with two classifiers that can retain personalized information in the local model and improve the model’s performance on local *** divides the model into two parts,namely,the extractor and the classifier,where the classifier is the last layer of the model,and the extractor consists of other *** classifier in the local model is always retained to ensure that the personalized information is not *** receiving the global model,the local extractor is overwritten by the globalmodel’s extractor,and the classifier of the globalmodel serves as anadditional classifier of the localmodel toguide local *** FedTCintroduces a two-classifier training strategy to coordinate the two classifiers for local model *** results on Cifar10 and Cifar100 datasets demonstrate that FedTC performs better on heterogeneous data than current studies,such as FedAvg,FedPer,and local training,achieving a maximum improvement of 27.95%in model classification test accuracy compared to FedAvg.

关键词： distributed machine learning federated learning data hetero-geneity non-independent identically distributed

Collaborative deep learning across multiple data centers

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2020年第8期63卷 169-179页

作者： Haibo MI Kele XU Dawei FENG Huaimin WANG Yiming ZHANG Zibin ZHENG Chuan CHEN Xu LAN College of Computer National University of Defense Technology School of Data and Computer Science Sun Yat-sen University School of Electronic Engineering and Computer Science Queen Mary University of London

Valuable training data is often owned by independent organizations and located in multiple data centers. Most deep learning approaches require to centralize the multi-datacenter data for performance purpose. In practice, however, it is often infeasible to transfer all data of different organizations to a centralized data center owing to the constraints of privacy regulations. It is very challenging to conduct the geo-distributed deep learning among data centers without the privacy leaks. Model averaging is a conventional choice for data parallelized training and can reduce the risk of privacy leaks, but its ineffectiveness is claimed by previous studies as deep neural networks are often non-convex. In this paper, we argue that model averaging can be effective in the decentralized environment by using two strategies, namely, the cyclical learning rate(CLR) and the increased number of epochs for local model training. With the two strategies,we show that model averaging can provide competitive performance in the decentralized mode compared to the data-centralized one. In a practical environment with multiple data centers, we conduct extensive experiments using state-of-the-art deep network architectures on different types of data. Results demonstrate the effectiveness and robustness of the proposed method.

关键词： collaborative learning multiple datacenters distributed machine learning

Overhead-free Noise-tolerant Federated learning: A New Baseline

同方期刊数据库评论

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

machine Intelligence Research 2024年第3期21卷 526-537页

作者： Shiyi Lin Deming Zhai Feilong Zhang Junjun Jiang Xianming Liu Xiangyang Ji Department of Computer Science and Technology Harbin Institute of TechnologyHarbin150000China Department of Automation Tsinghua UniversityBeijing100084China

Federated learning (FL) is a promising decentralized machine learning approach that enables multiple distributed clients to train a model jointly while keeping their data private. However, in real-world scenarios, the supervised training data stored in local clients inevitably suffer from imperfect annotations, resulting in subjective, inconsistent and biased labels. These noisy labels can harm the collaborative aggregation process of FL by inducing inconsistent decision boundaries. Unfortunately, few attempts have been made towards noise-tolerant federated learning, with most of them relying on the strategy of transmitting overhead messages to assist noisy labels detection and correction, which increases the communication burden as well as privacy risks. In this paper, we propose a simple yet effective method for noise-tolerant FL based on the well-established co-training framework. Our method leverages the inherent discrepancy in the learning ability of the local and global models in FL, which can be regarded as two complementary views. By iteratively exchanging samples with their high confident predictions, the two models “teach each other” to suppress the influence of noisy labels. The proposed scheme enjoys the benefit of overhead cost-free and can serve as a robust and efficient baseline for noise-tolerant federated learning. Experimental results demonstrate that our method outperforms existing approaches, highlighting the superiority of our method.

关键词： Federated learning noise-label learning privacy-preserving machine learning edge intelligence distributed machine learning