检索结果-南通市图书馆

Robust cooperative multi-agent reinforcement learning via multi-view message certification

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2024年第4期67卷 131-145页

作者： Lei YUAN Tao JIANG Lihe LI Feng CHEN Zongzhang ZHANG Yang YU national key laboratory for novel software technology Nanjing University Polixir Technologies

Many multi-agent scenarios require message sharing among agents to promote coordination,hastening the robustness of multi-agent communication when policies are deployed in a message perturbation environment. Major relevant studies tackle this issue under specific assumptions, like a limited number of message channels would sustain perturbations, limiting the efficiency in complex scenarios. In this paper,we take a further step in addressing this issue by learning a robust cooperative multi-agent reinforcement learning via multi-view message certification, dubbed CroMAC. Agents trained under CroMAC can obtain guaranteed lower bounds on state-action values to identify and choose the optimal action under a worst-case deviation when the received messages are perturbed. Concretely, we first model multi-agent communication as a multi-view problem, where every message stands for a view of the state. Then we extract a certificated joint message representation by a multi-view variational autoencoder(MVAE) that uses a product-of-experts inference network. For the optimization phase, we do perturbations in the latent space of the state for a certificate guarantee. Then the learned joint message representation is used to approximate the certificated state representation during training. Extensive experiments in several cooperative multi-agent benchmarks validate the effectiveness of the proposed CroMAC.

关键词： multi-agent reinforcement learning robust communication adversarial training multi-view learning message certification

LAMDA-SSL:a comprehensive semi-supervised learning toolkit

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2024年第1期67卷 306-307页

作者： Lin-Han JIA Lan-Zhe GUO Zhi ZHOU Yu-Feng LI national key laboratory for novel software technology Nanjing University

Machine learning, particularly deep learning, has achieved remarkable success across a wide range of tasks. However,most of these tasks demand a substantial amount of labeled training data, which can be challenging to obtain in many real-world applications due to difficulties, costs, or time constraints associated with labeling. In this context,semi-supervised learning (SSL) has emerged as a promising paradigm to alleviate the dependency on large labeled datasets by harnessing the power of unlabeled data [1–3].

关键词：

Learning in games:a systematic review

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2024年第7期67卷 133-159页

作者： Rong-Jun QIN Yang YU national key laboratory for novel software technology Nanjing University Polixir Technologies

Game theory studies the mathematical models for self-interested individuals. Nash equilibrium is arguably the most central solution in game theory. While finding the Nash equilibrium in general is known as polynomial parity arguments on directed graphs(PPAD)-complete, learning in games provides an alternative to approximate Nash equilibrium, which iteratively updates the player's strategy through interactions with other players. Rules and models have been developed for learning in games, such as fictitious play and no-regret learning. Particularly, with recent advances in online learning and deep reinforcement learning,techniques from these fields greatly boost the breakthroughs in learning in games from theory to *** a result, we have witnessed many superhuman game AI systems. The techniques used in these systems evolve from conventional search and learning to purely reinforcement learning(RL)-style learning methods,gradually getting rid of the domain knowledge. In this article, we systematically review the above techniques,discuss the trend of basic learning rules towards a unified framework, and recap applications in large ***, we discuss some future directions and make the prospect of future game AI systems. We hope this article will give some insights into designing novel approaches.

关键词： non-cooperative games learning in games no-regret learning reinforcement learning superhuman AI

Random Subspace Sampling for Classification with Missing Data

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Journal of Computer Science & technology 2024年第2期39卷 472-486页

作者：曹云浩吴建鑫 State key laboratory for novel software technology Nanjing UniversityNanjing 210093China

Many real-world datasets suffer from the unavoidable issue of missing values,and therefore classification with missing data has to be carefully handled since inadequate treatment of missing values will cause large *** this paper,we propose a random subspace sampling method,RSS,by sampling missing items from the corresponding feature histogram distributions in random subspaces,which is effective and efficient at different levels of missing *** most established approaches,RSS does not train on fixed imputed ***,we design a dynamic training strategy where the filled values change dynamically by resampling during ***,thanks to the sampling strategy,we design an ensemble testing strategy where we combine the results of multiple runs of a single model,which is more efficient and resource-saving than previous ensemble ***,we combine these two strategies with the random subspace method,which makes our estimations more robust and *** effectiveness of the proposed RSS method is well validated by experimental studies.

关键词： missing data random subspace neural network ensemble learning

Improving actionable warning identification via the refined warning-inducing context representation

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2024年第5期67卷 321-322页

作者： Xiuting GE Chunrong FANG Xuanye LI Quanjun ZHANG Jia LIU Zhihong ZHAO Zhenyu CHEN State key laboratory for novel software technology Nanjing University

Static analysis tools(SATs) have shown potential ability in software quality assurance [1]. However, SATs report an overwhelming number of warnings, where most warnings are unactionable ones(i.e., warnings that are not acted on or fixed by developers). It is not a trivial task for developers to find actionable warnings(i.e., warnings that are acted on and fixed by developers) in all reported warnings. Hence,the excessive unactionable warnings seriously impede the usability of SATs [2].

关键词：

Model gradient: unified model and policy learning in model-based reinforcement learning

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Frontiers of Computer Science 2024年第4期18卷 117-128页

作者： Chengxing JIA Fuxiang ZHANG Tian XU Jing-Cheng PANG Zongzhang ZHANG Yang YU national key laboratory for novel software technology Nanjing UniversityNanjing 210023China Polixir Technologies Nanjing 210000China

Model-based reinforcement learning is a promising direction to improve the sample efficiency of reinforcement learning with learning a model of the *** model learning methods aim at fitting the transition data,and commonly employ a supervised learning approach to minimize the distance between the predicted state and the real *** supervised model learning methods,however,diverge from the ultimate goal of model learning,i.e.,optimizing the learned-in-the-model *** this work,we investigate how model learning and policy learning can share the same objective of maximizing the expected return in the real *** find model learning towards this objective can result in a target of enhancing the similarity between the gradient on generated data and the gradient on the real *** thus derive the gradient of the model from this target and propose the Model Gradient algorithm(MG)to integrate this novel model learning approach with policy-gradient-based policy *** conduct experiments on multiple locomotion control tasks and find that MG can not only achieve high sample efficiency but also lead to better convergence performance compared to traditional model-based reinforcement learning approaches.

关键词： reinforcement learning model-based reinforcement learning Markov decision process

When debugging encounters artificial intelligence: state of the art and open challenges

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2024年第4期67卷 42-84页

作者： Yi SONG Xiaoyuan XIE Baowen XU School of Computer Science Wuhan University State key laboratory of novel software technology Nanjing University

Both software debugging and artificial intelligence techniques are hot topics in the current field of software engineering. Debugging techniques, which comprise fault localization and program repair, are an important part of the software development lifecycle for ensuring the quality of software systems. As the scale and complexity of software systems grow, developers intend to improve the effectiveness and efficiency of software debugging via artificial intelligence(artificial intelligence for software debugging, AI4SD). On the other hand, many artificial intelligence models are being integrated into safety-critical areas such as autonomous driving, image recognition, and audio processing, where software debugging is highly necessary and urgent(software debugging for artificial intelligence, SD4AI). An AI-enhanced debugging technique could assist in debugging AI systems more effectively, and a more robust and reliable AI approach could further guarantee and support debugging techniques. Therefore, it is important to take AI4SD and SD4AI into consideration comprehensively. In this paper, we want to show readers the path, the trend, and the potential that these two directions interact with each other. We select and review a total of 165 papers in AI4SD and SD4AI for answering three research questions, and further analyze opportunities and challenges as well as suggest future directions of this cross-cutting area.

关键词： software debugging fault localization program repair artificial intelligence machine learning

A survey on model-based reinforcement learning

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2024年第2期67卷 59-84页

作者： Fan-Ming LUO Tian XU Hang LAI Xiong-Hui CHEN Weinan ZHANG Yang YU national key laboratory for novel software technology Nanjing University Polixir. ai Department of Computer Science and Engineering Shanghai Jiao Tong University

Reinforcement learning(RL) interacts with the environment to solve sequential decision-making problems via a trial-and-error approach. Errors are always undesirable in real-world applications, even though RL excels at playing complex video games that permit several trial-and-error attempts. To improve sample efficiency and thus reduce errors, model-based reinforcement learning(MBRL) is believed to be a promising direction, as it constructs environment models in which trial-and-errors can occur without incurring actual costs. In this survey, we investigate MBRL with a particular focus on the recent advancements in deep RL. There is a generalization error between the learned model of a non-tabular environment and the actual environment. Consequently, it is crucial to analyze the disparity between policy training in the environment model and that in the actual environment, guiding algorithm design for improved model learning, model utilization, and policy training. In addition, we discuss the recent developments of model-based techniques in other forms of RL, such as offline RL, goal-conditioned RL, multi-agent RL, and meta-RL. Furthermore,we discuss the applicability and benefits of MBRL for real-world tasks. Finally, this survey concludes with a discussion of the promising future development prospects for MBRL. We believe that MBRL has great unrealized potential and benefits in real-world applications, and we hope this survey will encourage additional research on MBRL.

关键词： reinforcement learning model-based reinforcement learning planning model learning model learning with reduced error model usage

Understanding and Detecting Inefficient Image Displaying Issues in Android Apps

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Journal of Computer Science & technology 2024年第2期39卷 434-459页

作者：李文杰马骏蒋炎岩许畅马晓星 State key laboratory of novel software technology Nanjing UniversityNanjing 210023China Department of Computer Science and technology Nanjing UniversityNanjing 210023China

Mobile applications(apps for short)often need to display ***,inefficient image displaying(IID)issues are pervasive in mobile apps,and can severely impact app performance and user *** paper first establishes a descriptive framework for the image displaying procedures of IID *** on the descriptive framework,we conduct an empirical study of 216 real-world IID issues collected from 243 popular open-source Android apps to validate the presence and severity of IID issues,and then shed light on these issues’characteristics to support research on effective issue *** the findings of this study,we propose a static IID issue detection tool TAPIR and evaluate it with 243 real-world Android ***,49 and 64 previously-unknown IID issues in two different versions of 16 apps reported by TAPIR are manually confirmed as true positives,respectively,and 16 previously-unknown IID issues reported by TAPIR have been confirmed by developers and 13 have been ***,we further evaluate the performance impact of these detected IID issues and the performance improvement if they are *** results demonstrate that the IID issues detected by TAPIR indeed cause significant performance degradation,which further show the effectiveness and efficiency of TAPIR.

关键词： Android application(app) inefficient image displaying(IID) performance empirical study static analysis