检索结果-南通市图书馆

A Method to Build a Super Small but Practically Accurate language model for Handheld Devices

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Computer Science & Technology 2003年第6期18卷 747-755页

作者：吴根清郑方 CenterofSpeechTechnology StateKeyLaboratoryofIntelligentTechnologyandSystemsDepartmentofComputerScienceandTechnologyTsinghuaUniversityBeijing100084P.R.China TheauthorcurrentlyisalsowithBeijingd-EarTechnologiesCo. Ltd.

In this paper, an important question, whether a small language model can be practically accurate enough, is raised. Afterwards, the purpose of a language model, the problems that a language model faces, and the factors that affect the performance of a language model,are analyzed. Finally, a novel method for language model compression is proposed, which makes the large language model usable for applications in handheld devices, such as mobiles, smart phones, personal digital assistants (PDAs), and handheld personal computers (HPCs). In the proposed language model compression method, three aspects are included. First, the language model parameters are analyzed and a criterion based on the importance measure of n-grams is used to determine which n-grams should be kept and which removed. Second, a piecewise linear warping method is proposed to be used to compress the uni-gram count values in the full language model. And third, a rank-based quantization method is adopted to quantize the bi-gram probability values. Experiments show that by using this compression method the language model can be reduced dramatically to only about 1M bytes while the performance almost does not decrease. This provides good evidence that a language model compressed by means of a well-designed compression technique is practically accurate enough, and it makes the language model usable in handheld devices.

关键词： language model language model compression piecewise linear warping tank-based quantization

Improved world-based language model

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Harbin Institute of Technology(New Series) 2004年第1期11卷 114-120页

作者：陈勇 CHAN Kwok-ping Dept. of Computer Science and Information Systems University of Hong Kong Hong KongChina

In order to construct a good language model used in the postprocessing phase of a recognition system. A smoothing technique must be used to solve the data sparseness problem. In the past, many smoothing techniques have been proposed. Among them, Katz’s smoothing technique is well known. However, we found that a weakness with the Katz’s smoothing technique. We improved this approach by incorporating one kind of special Chinese language information and Chinese word class information into the language model. We tested the new smoothing technique with a Chinese character recognition system. The experimental result showed that a better performance can be achieved.

关键词： language model computational language pattern recognition

A Bit Progress on Word-Based language model

在线全文

学校读者我要写书评

暂无评论

Journal of Shanghai University(English Edition) 2003年第2期7卷 148-155页

作者：陈勇陈国评 DepartmentofComputerScienceandInformationSystem

A good language model is essential to a postprocessing algorithm for recognition systems. In the past, researchers have presented various language models, such as character based language models, word based language model, syntactical rules language model, hybrid models, etc . The word N gram model is by far an effective and efficient model, but one has to address the problem of data sparseness in establishing the model. Katz and Kneser et al. respectively presented effective remedies to solve this challenging problem. In this study, we proposed an improvement to their methods by incorporating Chinese language specific information or Chinese word class information into the system.

关键词： language model pattern recognition Chinese character recognition.

Summary-based model of Information Retrieval in language model Framework

在线全文

学校读者我要写书评

暂无评论

Summary-based Model of Information Retrieval in Language mod...

Investigation of Long Short-Term Memory for Tibetan language model

第四届全国信息检索与内容安全学术会议

作者： Li Wei-jiang~(1,+) Zhao Tie-Jun~2 ~1 Computer Application Key Laboratory of Yunnan Province,Kunmmg University of Science and Technology,Kunming 650051,China ~2 School of Computer Science and Technology,Harbin Institute of Technology,Harbin 150001,China

Standard IR treats the whole document as relevant,the implication of this being that using terms from non-relevant sections of these documents for IR may also cause query *** paper investigates a novel retrieval model that aims to concentrate more user’s attention on the parts of the text that possess a high density of relevant information,while,at the same time,to provide enough information to support their retrieval *** is proposed that an automatically generated summary of each document,biased to a user’s query,can provide such a *** on the experimental results on TREC collections,it is showed the retrieval model based on summary is more effective than standard one based on document.

关键词： Information Retrieval Summary language model smoothing methods

来源： cnki会议评论

在线全文

cnki会议

学校读者我要写书评

暂无评论

Investigation of Long Short-Term Memory for Tibetan Language...

CPT: a pre-trained unbalanced transformer for both Chinese language understanding and generation

第十四届全国人机语音通讯学术会议

作者： Tongtong Shen Longbiao Wang Xie Chen Kuntharrgyal Khysru Jianwu Dang Tianjin Key Lab.of Cognitive Computing and Application Tianjin University University of Cambridge Japan Advanced Institute of Science and Technology

Nowadays,recurrent neural network language model（RNNLM） has achieved excellent performance in many speech and natural language processing（NLP） ***,it is hard to train due to the gradient vanishing *** problem can be relieved by long short-term memory language model（LSTMLM）.But training of LSTMLM poses great difficulties for low-resource language such as Tibetan since it requires a large quantities of data to guarantee its good *** paper mainly explores the performance of LSTMLM for Tibetan by utilizing its *** addition,domain adaptation is applied to solve the low resource issue to obtain a better ***,an interpolated language model is proposed to get further improvements,which better exploits the complementary attributes between N-gram LM and *** are conducted with two data sets with different sizes,small Tibetan data set（STD） and large Tibetan data set（LTD） *** results show that the proposed LMs show better *** to Knerser-Ney 3-gram trained with STD,our domain adapted LM,interpolated LM and interpolated LM with domain adaptation yield consistent gains,28.8%,18.0%and 34.2%in relative perplexity reduction respectively.

关键词： Long short-term memory language model Tibetan perplexity low resource

来源： cnki会议评论

在线全文

cnki会议

学校读者我要写书评

暂无评论

Science China(Information Sciences) 2024年第5期67卷 43-55页

作者： Yunfan SHAO Zhichao GENG Yitao LIU Junqi DAI Hang YAN Fei YANG Zhe LI Hujun BAO Xipeng QIU School of Computer Science Fudan University Shanghai Key Laboratory of Intelligent Information Processing Fudan University Zhejiang Lab

In this paper, we take the advantage of previous pre-trained models(PTMs) and propose a novel Chinese pre-trained unbalanced transformer(CPT). Different from previous Chinese PTMs, CPT is designed to utilize the shared knowledge between natural language understanding(NLU) and natural language generation(NLG) to boost the performance. CPT consists of three parts: a shared encoder, an understanding decoder, and a generation decoder. Two specific decoders with a shared encoder are pretrained with masked language modeling(MLM) and denoising auto-encoding(DAE) tasks, *** the partially shared architecture and multi-task pre-training, CPT can(1) learn specific knowledge of both NLU or NLG tasks with two decoders and(2) be fine-tuned flexibly that fully exploits the potential of the model. Moreover, the unbalanced transformer saves the computational and storage cost, which makes CPT competitive and greatly accelerates the inference of text generation. Experimental results on a wide range of Chinese NLU and NLG tasks show the effectiveness of CPT.

关键词： pre-trained model transformer language model generation unified model

Incorporating Linguistic Structure into Maximum Entropy language models

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Journal of Computer Science & Technology 2003年第1期18卷 131-136页

作者：方高林高文王兆其 Department of Computer Science and Engineering Harbin Institute of Technology Harbin 150001 P.R. China Department of Computer Science and Engineering Harbin Institute of Technology Harbin 150001 P.R. ChinaInstitute of Computing Technology The Chinese Academy of Sciences Beijing 100080 P.R. China Institute of Computing Technology The Chinese Academy of Sciences Beijing 100080 P.R. China

In statistical language models, how to integrate diverse linguistic knowledge in a general framework for long-distance dependencies is a challenging issue. In this paper, an improved language model incorporating linguistic structure into maximum entropy framework is *** proposed model combines trigram with the structure knowledge of base phrase in which trigram is used to capture the local relation between words, while the structure knowledge of base phrase is considered to represent the long-distance relations between syntactical structures. The knowledge of syntax, semantics and vocabulary is integrated into the maximum entropy *** results show that the proposed model improves by 24% for language model perplexity and increases about 3% for sign language recognition rate compared with the trigram model.

关键词： maximum entropy language model base phrase identification sign language recognition

Constructing Maximum Entropy language models for Movie Review Subjectivity Analysis

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Computer Science & Technology 2008年第2期23卷 231-239页

作者：陈博何慧郭军 Pattern Recognition and Intelligent System Laboratory School of Information Engineering Beijing University of Posts and TelecommunicationsBeijing 100876China

Document subjectivity analysis has become an important aspect of web text content mining. This problem is similar to traditional text categorization, thus many related classification techniques can be adapted here. However, there is one significant difference that more language or semantic information is required for better estimating the subjectivity of a document. Therefore, in this paper, our focuses are mainly on two aspects. One is how to extract useful and meaningful language features, and the other is how to construct appropriate language models efficiently for this special task. For the first issue, we conduct a Global-Filtering and Local-Weighting strategy to select and evaluate language features in a series of n-grams with different orders and within various distance-windows. For the second issue, we adopt Maximum Entropy （MaxEnt） modeling methods to construct our language model framework. Besides the classical MaxEnt models, we have also constructed two kinds of improved models with Gaussian and exponential priors respectively. Detailed experiments given in this paper show that with well selected and weighted language features, MaxEnt models with exponential priors are significantly more suitable for the text subjectivity analysis task.

关键词： exponential prior language model maximum entropy n-gram subjectivity analysis

Two language models Using Chinese Semantic Parsing

在线全文

学校读者我要写书评

暂无评论

Tsinghua Science and Technology 2006年第5期11卷 582-588页

作者：李明琴王侠王作英 Department of Electronic Engineering Tsinghua University

This paper presents two language models that utilize a Chinese semantic dependency parsing technique for speech recognition. The models are based on a representation of the Chinese semantic structure with dependency relations, A semantic dependency parser was described to automatically tag the semantic class for each word with 90.9% accuracy and parse the sentence semantic dependency structure with 75.8% accuracy. The Chinese semantic parsing technique was applied to structure language models to develop two language models, the semantic dependency model （SDM） and the headword trigram model （HTM）. These language models were evaluated using Chinese speech recognition. The experiments show that both models outperform the word trigram model in terms of the Chinese character recognition error rate.

关键词： language model semantic parsing dependency grammar speech recognition