检索结果-南通市图书馆

Why over-parameterization of deep neural networks does not overfit?

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2021年第1期64卷 242-244页

作者： zhi-hua zhou National Key Laboratory for Novel Software Technology Nanjing University

Deep neural networks often come with a huge number of parameters,even larger than the number of training examples,but it seems that these over-parameterized models have not suffered from *** is quite strange and why over-parameterization does not overfit?poses a fundamental question concerning the mysteries behind the success of deep neural networks.

关键词： Information Systems and Communication Service

Abductive learning: towards bridging machine learning and logical reasoning

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2019年第7期62卷 220-222页

作者： zhi-hua zhou National Key Laboratory for Novel Software Technology Nanjing University

In the history of artificial intelligence research,machine learning and logical reasoning have almost been separately developed. It is often argued that advanced intelligent technologies would emerge when machine lear... 详细信息

关键词： pro towards bridging machine learning and logical reasoning Abductive learning

Learnware： on the future of machine learning

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Frontiers of Computer Science 2016年第4期10卷 589-590页

作者： zhi-hua zhou National Key Laboratory for Novel Software Technology Nanjing University Nanjing 210023 China

Current machine learning techniques have achieved great success; however, there are many deficiencies. First, to train a strong model, a large amount of training examples are required, whereas collecting the data, particularly data with labels, is expensive or even difficult in many real tasks. Second, once a model has been trained, if environment changes, which often happens in real tasks, the model can hardly perform well or even become useless. Third,

关键词： Learnware future machine learning

维普期刊数据库

Learnware:small models do big

在线全文

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2024年第1期67卷 39-50页

作者： zhi-hua zhou zhi-Hao TAN National Key Laboratory for Novel Software Technology Nanjing University

There are complaints about current machine learning techniques such as the requirement of a huge amount of training data and proficient training skills,the difficulty of continual learning,the risk of catastrophic forgetting,and the leaking of data privacy/*** research efforts have been focusing on one of those concerned issues separately,paying less attention to the fact that most issues are entangled in *** prevailing big model paradigm,which has achieved impressive results in natural language processing and computer vision applications,has not yet addressed those issues,whereas becoming a serious source of carbon *** article offers an overview of the learnware paradigm,which attempts to enable users not to need to build machine learning models from scratch,with the hope of reusing small models to do things even beyond their original purposes,where the key ingredient is the specification which enables a trained model to be adequately identified to reuse according to the requirement of future users who know nothing about the model in advance.

关键词： artificial intelligence machine learning learnware

Rehearsal:learning from prediction to decision

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Frontiers of Computer Science 2022年第4期16卷 1-3页

作者： zhi-hua zhou National Key Laboratory for Novel Software Technology Nanjing UniversityNanjing210023China

Machine learning[1]studies focus mostly on prediction,where a model is built from a set of observational data formaking correct predictions on unseen *** has beenaddressed very well by modem techniques such as deeplea... 详细信息

关键词： such has see

维普期刊数据库

A brief introduction to weakly supervised learning

在线全文

学校读者我要写书评

暂无评论

National Science Review 2018年第1期5卷 44-53页

作者： zhi-hua zhou National Key Laboratory for Novel Software Technology Nanjing University

Supervised learning techniques construct predictive models by learning from a large number of training examples, where each training example has a label indicating its ground-truth output. Though current techniques have achieved great success; it is noteworthy that in many tasks it is difficult to get strong supervision information like fully ground-truth labels due to the high cost of the data-labeling process. Thus,it is desirable for machine-learning techniques to work with weak supervision. This article reviews some research progress of weakly supervised learning, focusing on three typical types of weak supervision:incomplete supervision, where only a subset of training data is given with labels; inexact supervision, where the training data are given with only coarse-grained labels; and inaccurate supervision, where the given labels are not always ground-truth.

关键词： machine learning weakly supervised learning supervised learning

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Deep forest

National Science Review 2019年第1期6卷 74-86页

作者： zhi-hua zhou Ji Feng National Key Laboratory for Novel Software Technology Nanjing University

Current deep-learning models are mostly built upon neural networks, i.e. multiple layers of parameterized differentiable non-linear modules that can be trained by backpropagation. In this paper, we explore the possibility of building deep models based on non-differentiable modules such as decision trees. After a discussion about the mystery behind deep neural networks, particularly by contrasting them with shallow neural networks and traditional machine-learning techniques such as decision trees and boosting machines,we conjecture that the success of deep neural networks owes much to three characteristics, ***-by-layer processing, in-model feature transformation and sufficient model complexity. On one hand,our conjecture may offer inspiration for theoretical understanding of deep learning; on the other hand, to verify the conjecture, we propose an approach that generates deep forest holding these characteristics. This is a decision-tree ensemble approach, with fewer hyper-parameters than deep neural networks, and its model complexity can be automatically determined in a data-dependent way. Experiments show that its performance is quite robust to hyper-parameter settings, such that in most cases, even across different data from different domains, it is able to achieve excellent performance by using the same default setting. This study opens the door to deep learning based on non-differentiable modules without gradient-based adjustment, and exhibits the possibility of constructing deep models without backpropagation.

关键词： deep forest deep learning machine learning ensemble methods decision trees

Dropout Rademacher complexity of deep neural networks

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2016年第7期59卷 173-184页

作者： Wei GAO zhi-hua zhou National Key Laboratory for Novel Software Technology Nanjing University Collaborative Innovation Center of Novel Software Technology and Industrialization Nanjing University

Great successes of deep neural networks have been witnessed in various real applications. Many algorithmic and implementation techniques have been developed; however, theoretical understanding of many aspects of deep neural networks is far from clear. A particular interesting issue is the usefulness of dropout,which was motivated from the intuition of preventing complex co-adaptation of feature detectors. In this paper,we study the Rademacher complexity of different types of dropouts, and our theoretical results disclose that for shallow neural networks(with one or none hidden layer) dropout is able to reduce the Rademacher complexity in polynomial, whereas for deep neural networks it can amazingly lead to an exponential reduction.

关键词： artificial intelligence machine learning deep learning dropout Rademacher complexity

Structural diversity for decision tree ensemble learning

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Frontiers of Computer Science 2018年第3期12卷 560-570页

作者： Tao SUN zhi-hua zhou National Key Laboratory for Novel Software Technology Nanjing University Nanjing 210023 China

Decision trees are a kind of off-the-shelf predictive models, and they have been successfully used as the base learners in ensemble learning. To construct a strong classi- fier ensemble, the individual classifiers should be accurate and diverse. However, diversity measure remains a mystery although there were many attempts. We conjecture that a deficiency of previous diversity measures lies in the fact that they consider only behavioral diversity, i.e., how the classifiers behave when making predictions, neglecting the fact that classifiers may be potentially different even when they make the same predictions. Based on this recognition, in this paper, we advocate to consider structural diversity in addition to behavioral diversity, and propose the TMD （tree matching diversity） measure for decision trees. To investigate the usefulness of TMD, we empirically evaluate performances of selective ensemble approaches with decision forests by incorporating different diversity measures. Our results validate that by considering structural and behavioral diversities together, stronger ensembles can be constructed. This may raise a new direction to design better diversity measures and ensemble methods.

关键词： ensemble learning structural diversity decisiontree