检索结果-南通市图书馆

SpikingMiniLM: energy-efficient spiking transformer for natural language understanding

Science China(Information Sciences) 2024年

作者： Jiayu ZHANG Jiangrong SHEN Zeke WANG Qinghai GUO Rui YAN Gang PAN Huajin TANG College of Computer Science and Technology Zhejiang University The State key lab of Brain-machine intelligence Zhejiang University Collaborative Innovation Center of Artificial intelligence Zhejiang University advanced computing and Storage laboratory Huawei Technologies Co. Ltd. College of Computer Science and Technology Zhejiang University of Technology MOE Frontier Science Center for Brain Science and Brain-machine Integration Zhejiang University

In the era of large-scale pretrained models, artificial neural networks (ANNs) have excelled in natural language understanding (NLU) tasks. However, their success often necessitates substantial computational resources and energy consumption. To address this, we explore the potential of spiking neural networks (SNNs) in NLU—a promising avenue with demonstrated advantages, including reduced power consumption and improved efficiency due to their event-driven characteristics. We propose the SpikingMiniLM,a novel spiking Transformer model tailored for natural language understanding. We first introduce a multistep encoding method to convert text embeddings into spike trains. Subsequently, we redesign the attention mechanism and residual connections to make our model operate on the pure spike-based paradigm without any normalization technique. To facilitate stable and fast convergence, we propose a general parameter initialization method grounded in the stable firing rate principle. Furthermore, we apply an ANN-to-SNN knowledge distillation to overcome the challenges of pretraining SNNs. Our approach achieves a macroaverage score of 75.5 on the dev sets of the GLUE benchmark, retaining 98%of the performance exhibited by the teacher model MiniLMv2. Our smaller model also achieves similar performance to BERTMINIwith fewer parameters and much lower energy consumption, underscoring its competitiveness and resource efficiency in NLU tasks.

关键词：

来源：

同方期刊数据库评论

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Domain-specific feature elimination:multi-source domain adaptation for image classification

引用

Frontiers of Computer Science 2023年第4期17卷 171-179页

作者： Kunhong WU Fan JIA Yahong HAN College of intelligence and computing Tianjin UniversityTianjin 300350China Tianjin key lab of machine Learning Tianjin UniversityTianjin 300350China

Multi-source domain adaptation utilizes multiple source domains to learn the knowledge and transfers it to an unlabeled target *** address the problem,most of the existing methods aim to minimize the domain shift by auxiliary distribution alignment objectives,which reduces the effect of domain-specific ***,without explicitly modeling the domain-specific features,it is not easy to guarantee that the domain-invariant representation extracted from input domains contains domain-specific information as few as *** this work,we present a different perspective on MSDA,which employs the idea of feature elimination to reduce the influence of domain-specific *** design two different ways to extract domain-specific features and total features and construct the domain-invariant representations by eliminating the domain-specific features from total *** experimental results on different domain adaptation datasets demonstrate the effectiveness of our method and the generalization ability of our model.

关键词： multi-source domain adaptation generalization feature elimination

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Dynamic parameterized learning for unsupervised domain adaptation

引用

Frontiers of Information Technology & Electronic Engineering 2023年第11期24卷 1616-1632页

作者： Runhua JIANG Yahong HAN College of intelligence and computing Tianjin UniversityTianjin 300350China Tianjin key lab of machine Learning Tianjin UniversityTianjin 300350China

Unsupervised domain adaptation enables neural networks to transfer from a labeled source domain to an unlabeled target domain by learning domain-invariant *** approaches achieve this by directly matching the marginal distributions of these two *** of them,however,ignore exploration of the dynamic trade-off between domain alignment and semantic discrimination learning,thus rendering them susceptible to the problems of negative transfer and outlier *** address these issues,we introduce the dynamic parameterized learning ***,by exploring domain-level semantic knowledge,the dynamic alignment parameter is proposed,to adaptively adjust the optimization steps of domain alignment and semantic discrimination ***,for obtaining semantic-discriminative and domain-invariant representations,we propose to align training trajectories on both source and target *** experiments are conducted to validate the effectiveness of the proposed methods,and extensive comparisons are conducted on seven datasets of three visual tasks to demonstrate their practicability.

关键词： Unsupervised domain adaptation Optimization steps Domain alignment Semantic discrimination

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Instance-sequence reasoning for video question answering

引用

Frontiers of Computer Science 2022年第6期16卷 93-101页

作者： Rui LIU Yahong HAN College of intelligence and computing Tianjin UniversityTianjin300350China Tianjin key lab of machine Learning Tianjin UniversityTianjin300350China

Video question answering(Video QA)involves a thorough understanding of video content and question language,as well as the grounding of the textual semantic to the visual content of ***,to answer the questions more accurately,not only the semantic entity should be associated with certain visual instance in video frames,but also the action or event in the question should be localized to a corresponding temporal *** turns out to be a more challenging task that requires the ability of conducting reasoning with correlations between instances along temporal *** this paper,we propose an instance-sequence reasoning network for video question answering with instance grounding and temporal *** our model,both visual instances and textual representations are firstly embedded into graph nodes,which benefits the integration of intra-and ***,we propose graph causal convolution(GCC)on graph-structured sequence with a large receptive field to capture more causal connections,which is vital for visual grounding and instance-sequence ***,we evaluate our model on TVQA+dataset,which contains the groundtruth of instance grounding and temporal localization,three other Video QA datasets and three multimodal language processing *** experiments demonstrate the effectiveness and generalization of the proposed ***,our method outperforms the state-of-the-art methods on these benchmarks.

关键词： video question answering instance grounding graph causal convolution

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Dual collaboration for decentralized multi-source domain adaptation

引用

Frontiers of Information Technology & Electronic Engineering 2022年第12期23卷 1780-1794页

作者： Yikang WEI Yahong HAN College of intelligence and computing Tianjin UniversityTianjin 300350China Tianjin key lab of machine Learning Tianjin UniversityTianjin 300350China

The goal of decentralized multi-source domain adaptation is to conduct unsupervised multi-source domain adaptation in a data decentralization scenario. The challenge of data decentralization is that the source domains and target domain lack cross-domain collaboration during training. On the unlabeled target domain, the target model needs to transfer supervision knowledge with the collaboration of source models, while the domain gap will lead to limited adaptation performance from source models. On the labeled source domain, the source model tends to overfit its domain data in the data decentralization scenario, which leads to the negative transfer problem. For these challenges, we propose dual collaboration for decentralized multi-source domain adaptation by training and aggregating the local source models and local target model in collaboration with each other. On the target domain, we train the local target model by distilling supervision knowledge and fully using the unlabeled target domain data to alleviate the domain shift problem with the collaboration of local source models. On the source domain, we regularize the local source models in collaboration with the local target model to overcome the negative transfer problem. This forms a dual collaboration between the decentralized source domains and target domain, which improves the domain adaptation performance under the data decentralization scenario. Extensive experiments indicate that our method outperforms the state-of-the-art methods by a large margin on standard multi-source domain adaptation datasets.

关键词： Multi-source domain adaptation Data decentralization Domain shift Negative transfer

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Weakly supervised action anticipation without object annotations

引用

Frontiers of Computer Science 2023年第2期17卷 101-110页

作者： Yi ZHONG Jia-Hui PAN Haoxin LI Wei-Shi ZHENG School of Computer Science and Engineering Sun Yat-sen UniversityGuangzhou 510006China key laboratory of machine intelligence and advanced computing Ministry of EducationGuangzhou 510006China

Anticipating future actions without observing any partial videos of future actions plays an important role in action prediction and is also a challenging *** obtain abundant information for action anticipation,some methods integrate multimodal contexts,including scene object ***,extensively labelling each frame in video datasets requires considerable *** this paper,we develop a weakly supervised method that integrates global motion and local finegrained features from current action videos to predict next action label without the need for specific scene context ***,we extract diverse types of local features with weakly supervised learning,including object appearance and human pose representations without ground ***,we construct a graph convolutional network for exploiting the inherent relationships of humans and objects under present *** evaluate the proposed model on two datasets,the MPII-Cooking dataset and the EPIC-Kitchens dataset,and we demonstrate the generalizability and effectiveness of our approach for action anticipation.

关键词： action anticipation weakly supervised learning relation modelling graph convolutional network

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Rescue to the Curse of universality

引用

Science China(Information Sciences) 2023年第9期66卷 134-150页

作者： Yongwei ZHAO Zidong DU Qi GUO Zhiwei XU Yunji CHEN State key lab of Processors Institute of Computing Technology Chinese Academy of Sciences Beijing Academy of Artificial intelligence University of Chinese Academy of Sciences

From the very beginning of computers, universality has been the core focus in the building of computing machines, such as the universal Turing machine, von Neumann architecture, random-access machines, and universal circuits. Academia has taken universality as the primary principle ever since. However,the Curse of universality, implied from ***’s Universal Circuit, states that computers based on logic circuits cannot be both universal and efficient, as the cost of universality is Ω(n log2n). Though the Curse has been hidden by the rapid advancement of semiconductor technologies, it has been wielding its effects noticeably in recent years. Due to the ending of Dennard scaling and Moore’s law, general-purpose processors leave less room for improvement. Therefore, domain-specific architectures(DSAs), such as deep learning processors, have been exploding, leading to the new golden age of computer architectures. For DSAs, universality is traded off for optimal efficiency. However, we predict that universality will once again be a major concern for post-golden-age computers. In this paper, we discuss how much universality could an efficient computer keep. As a rescue to the Curse, we define and discuss quasi-universal architectures. Quasi-universal architectures can solve any computable problem and are efficient for a wide range of problems. The proposed Recursive-Encapsulated(RENC) architecture achieves maximal universality while keeping optimal efficiency as found in specialized architectures. The discovery of RENCsuggests that current golden-age architectures are not Pareto optimal.

关键词： universality general-purpose architecture specialized architecture deep learning processor universal circuit

来源：

同方期刊数据库评论

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Clause-level Relationship-aware Math Word Problems Solver

引用

machine intelligence Research 2022年第5期19卷 425-438页

作者： Chang-Yang Wu Xin Lin Zhen-Ya Huang Yu Yin Jia-Yu Liu Qi Liu Gang Zhou anhui Province key laboratory of Big Data Analysis and Application School of Data ScienceUniversity of Science and Technology of ChinaHefei 230026China Institute of Artificial intelligence Hefei Comprehensive National Science CenterHefei 230088China laboratory of Mathematical Engineering and advanced computing Information Engineering UniversityZhengzhou 450001China

Automatically solving math word problems,which involves comprehension,cognition,and reasoning,is a crucial issue in artificial intelligence *** math word problem solvers mainly work on word-level relationship extraction and the generation of expression solutions while lacking consideration of the clause-level *** this end,inspired by the theory of two levels of process in comprehension,we propose a novel clause-level relationship-aware math solver(CLRSolver)to mimic the process of human comprehension from lower level to higher ***,in the lower-level processes,we split problems into clauses according to their natural division and learn their *** the higher-level processes,following human′s multi-view understanding of clause-level relationships,we first apply a CNN-based module to learn the dependency relationships between clauses from word relevance in a local ***,we propose two novel relationship-aware mechanisms to learn dependency relationships from the clause semantics in a global ***,we enhance the representation of clauses based on the learned clause-level dependency *** expression generation,we develop a tree-based decoder to generate the mathematical *** conduct extensive experiments on two datasets,where the results demonstrate the superiority of our framework.

关键词： Artificial intelligence(AI) artificial neural network(ANN) computational mathematics machine intelligence machine learning

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Current Situation and Development of intelligence Robots

引用

ZTE Communications 2016年第B12期14卷 25-34页

作者： REN Fuji SUN Xiao anhui key lab of affective computing and advanced intelligence machine Hefei University of Technology Hefei 230009 China Tokushima University Tokushima 770800 Japan

Industrial intelligent robots are treated as a measure of na- tional scientific level and technology innovation, and also the important symbol of high-level manufacturing, while service intelligent robots can directly affect people＇ s daily lives. The development of artificial robots in different areas is at- tracting much attention around the world. This article re- views the current situation and development of Chinese and international intelligent robot markets including industrial ro- bots and service robots. The intelligent robot technology and the classification of robots are also discussed. Finally, appli- cations of intelligent robots in various fields are concluded and the development trends and outlook of intelligent robots are explored.

关键词： intelligent robot artificial intelligence development trends

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Chip design with machine learning: a survey from algorithm perspective

引用

Science China(Information Sciences) 2023年第11期66卷 69-99页

作者： Wenkai HE Xiaqing LI Xinkai SONG Yifan HAO Rui ZHANG Zidong DU Yunji CHEN State key lab of Processor Institute of Computing Technology Chinese Academy of Sciences University of Chinese Academy of Sciences Cambricon Technologies

Chip design with machine learning(ML) has been widely explored to achieve better designs,lower runtime costs, and no human-in-the-loop process. However, with tons of work, there is a lack of clear links between the ML algorithms and the target problems, causing a huge gap in understanding the potential and possibility of ML in future chip design. This paper comprehensively surveys existing studies in chip design with ML from an algorithm perspective. To achieve this goal, we first propose a novel and systematical taxonomy that divides target problems in chip design into three categories. Then, to solve the target problems with ML algorithms, we formulate the three categories as three ML problems *** on the taxonomy, we conduct a comprehensive survey in terms of target problems based on different ML algorithms. Finally, we conclude three key challenges for existing studies and highlight several insights for the future development of chip design with machine learning. By constructing a clear link between chip design problems and ML solutions, we hope the survey can shed light on the road to chip design intelligence from previous chip design automation.

关键词： chip design machine learning chip design automation design result estimation design optimization and correction design construction

来源：

同方期刊数据库评论

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

欢迎您,

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

请选择保存的检索档案：

请选择收藏分类：

通借通还

欢迎您,

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：