检索结果-南通市图书馆

Optimizing Big Data Retrieval and job scheduling Using Deep Learning Approaches

在线全文

学校读者我要写书评

暂无评论

Computer Modeling in Engineering & Sciences 2023年第2期134卷 783-815页

作者： Bao Rong Chang Hsiu-Fen Tsai Yu-Chieh Lin Department of Computer Science and Information Engineering National University of KaohsiungKaohsiungTaiwan Department of Fragrance and Cosmetic Science Kaohsiung Medical UniversityKaohsiungTaiwan

Big data analytics in business intelligence do not provide effective data retrieval methods and job scheduling that will cause execution inefficiency and low system *** paper aims to enhance the capability of data retrieval and job scheduling to speed up the operation of big data analytics to overcome inefficiency and low throughput ***,integrating stacked sparse autoencoder and Elasticsearch indexing explored fast data searching and distributed indexing,which reduces the search scope of the database and dramatically speeds up data ***,exploiting a deep neural network to predict the approximate execution time of a job gives prioritized job scheduling based on the shortest job first,which reduces the average waiting time of job *** a result,the proposed data retrieval approach outperforms the previous method using a deep autoencoder and Solr indexing,significantly improving the speed of data retrieval up to 53%and increasing system throughput by 53%.On the other hand,the proposed job scheduling algorithmdefeats both first-in-first-out andmemory-sensitive heterogeneous early finish time scheduling algorithms,effectively shortening the average waiting time up to 5%and average weighted turnaround time by 19%,respectively.

关键词： Stacked sparse autoencoder Elasticsearch distributed indexing data retrieval deep neural network job scheduling

On/Off-Line Prediction Applied to job scheduling on Non-Dedicated NOWs

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Computer Science & Technology 2011年第1期26卷 99-116页

作者： Mauricio Hanzich Porfidio Hernandez Francesc Gine Francesc Solsona Josep L.Lerida Department of Computer Application in Science and Engineering Barcelona Supercomputing CenterBarcelonaSpain Department of Computer Architecture & Operating Systems Autonomous University of BarcelonaBarcelonaSpain Department of Computer Science University of LleidaLleidaSpain

This paper proposes a prediction engine designed for non-dedicated clusters, which is able to estimate the turnaround time for parallel applications, even in the presence of serial workload of the workstation owner. The prediction engine can be configured to work with three different estimation kernels： a Historical kernel, a Simulation kernel based on analytical models and an integration of both, named Hybrid kernel. These estimation proposals were integrated into a scheduling system, named CISNE, which can be executed in an on-line or off-line mode. The accuracy of the proposed estimation methods was evaluated in relation to different job scheduling policies in a real and a simulated cluster environment. In both environments, we observed that the Hybrid system gives the best results because it combines the ability of a simulation engine to capture the dynamism of a non-dedicated environment together with the accuracy of the historical methods to estimate the application runtime considering the state of the resources.

关键词： prediction method non-dedicated cluster cluster computing job scheduling simulation

Network and data location aware approach for simultaneous job scheduling and data replication in large-scale data grid environments

在线全文

学校读者我要写书评

暂无评论

Frontiers of Computer Science 2014年第3期8卷 391-408页

作者： Najme MANSOURI Department of Computer Science Shahid Bahonar University of Kerman Kerman 97175-569 Iran

Data Grid integrates graphically distributed resources for solving data intensive scientific applications. Effective scheduling in Grid can reduce the amount of data transferred among nodes by submitting a job to a node, where most of the requested data files are available. scheduling is a traditional problem in parallel and distributed system. However, due to special issues and goals of Grid, traditional approach is not effective in this environment any more. Therefore, it is necessary to propose methods specialized for this kind of parallel and distributed system. Another solution is to use a data replication strategy to create multiple copies of files and store them in convenient locations to shorten file access times. To utilize the above two concepts, in this paper we develop a job scheduling policy, called hierarchical job scheduling strategy （HJSS）, and a dynamic data replication strategy, called advanced dynamic hierarchical replication strategy （ADHRS）, to improve the data access efficiencies in a hierarchical Data Grid. HJSS uses hierarchical scheduling to reduce the search time for an appropriate computing node. It considers network characteristics, number of jobs waiting in queue, file locations, and disk read speed of storage drive at data sources. Moreover, due to the limited storage capacity, a good replica replacement algorithm is needed. We present a novel replacement strategy which deletes files in two steps when free space is not enough for the new replica： first, it deletes those files with minimum time for transferring. Second, if space is still insufficient then it considers the last time the replica was requested, number of access, size of replica and file transfer time. The simulation results show that our proposed algorithm has better performance in comparison with other algorithms in terms of job execution time, number of intercommunications, number of replications, hit ratio, computing resource usage and storage usage.

关键词： data replication data grid optorSim job scheduling simulation

A Heuristic for the job scheduling Problem with a Common Due Window on Parallel and Non-Identical Machines

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Systems Engineering and Electronics 2001年第2期12卷 6-11页

作者： Huang Decai College of information Engineering, Zhejiang University of Technology,Hangzhou 310014, P. R. China College of Information Engineering Zhejiang University of Technology Hangzhou 310014 P. R. China

In this paper, we give a mathematical model for earliness-tardiness job scheduling problem with a common due window on parallel and non-identical machines. Because the job scheduling problem discussed in the paper contains a problem of minimizing make-span, which is NP-complete on parallel and uniform machines, a heuristic algorithm is presented to find an approximate solution for the scheduling problem after proving an important theorem. Two numerical examples illustrate that the heuristic algorithm is very useful and effective in obtaining the near-optimal solution.

关键词： Common due window job scheduling Earliness-tardiness JIT.

Fuzzy Logic-Based Secure and Fault Tolerant job scheduling in Grid

在线全文

学校读者我要写书评

暂无评论

Tsinghua Science and Technology 2007年第S1期12卷 45-50页

作者：王乘蒋从锋刘小虎 Engineering Computing and Simulation Institute Huazhong University of Science and Technology

The uncertainties of grid sites security are main hurdle to make the job scheduling secure, reliable and fault-tolerant. Most existing scheduling algorithms use fixed-number job replications to provide fault tolerant ability and high scheduling success rate, which consume excessive resources or can not provide sufficient fault tolerant functions when grid security conditions change. In this paper a fuzzy-logic-based self-adaptive replication scheduling (FSARS) algorithm is proposed to handle the fuzziness or uncertainties of job replication number which is highly related to trust factors behind grid sites and user jobs. Remote sensing-based soil moisture extraction (RSBSME) workload experiments in real grid environment are performed to evaluate the proposed approach and the results show that high scheduling success rate of up to 95% and less grid resource utilization can be achieved through FSARS. Extensive experiments show that FSARS scales well when user jobs and grid sites increase.

关键词： fault tolerance grid security fuzzy logic job scheduling self-adaptive replication

QoS-aware simulation job scheduling algorithm in virtualized cloud environment

在线全文

学校读者我要写书评

暂无评论

International Journal of Modeling, Simulation, and Scientific Computing 2020年第5期11卷 208-225页

作者： Zhen Li Bin Chen Xiaocheng Liu Dandan Ning Xiaogang Qiu College of Information and Communication National University of Defense Technology Jiefang Park Road Wuhan 430019P.R.China College of System Engineering National University of Defense Technology Deya Road Changsha 410073P.R.China

Cloud computing is attracting an increasing number of simulation applications running in the virtualized cloud data *** applications are submitted to the cloud in the form of simulation ***,the management and scheduling of simulation jobs are playing an essential role to offer efficient and high productivity computational *** this paper,we design a management and scheduling service framework for simulation jobs in two-tier virtualization-based private cloud data center,named simulation execution as a service(SimEaaS).It aims at releasing users from complex simulation running settings,while guaranteeing the QoS requirements ***,a novel job scheduling algorithm named adaptive deadline-aware job size adjustment(ADaSA)algorithm is designed to realize high job responsiveness under QoS requirement for *** tries to make full use of the idle fragmentation resources by tuning the number of requested processes of submitted jobs in the queue adaptively,while guaranteeing that jobs’deadline requirements are not *** experiments with trace-driven simulation are conducted to evaluate the performance of our *** results show that ADaSA outperforms both cloud-based job scheduling algorithm KCEASY and traditional EASY in terms of response time(up to 90%)and bounded slow down(up to 95%),while obtains approximately equivalent deadline-missed *** also outperforms two representative moldable scheduling algorithms in terms of deadline-missed rate(up to 60%).

关键词： job scheduling simulation execution as a service virtualized cloud

A job scheduling Model and Heuristic Algorithm for Minimizing the Range of Lateness and Makespan on Parallel Machines

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Systems Science and Systems Engineering 1998年第1期8卷 51-56页

作者： HUANG Decai Zhejiang University of technology,Hangzhou 310014 JING Ling Beijing University of aeronaut & astronaut, 100083

In this paper ,based on JIT management technique, a multi objective job scheduling model is presented. After a new algorithm is presented for minimizing the range of lateness on a single machine , a heuristic algorithm (named M-R algorithm)is given to be applied to the parallel machines’ systems . Finally , an illustrative example is given.

关键词： job scheduling production management FMS parallel machines

A COMPARATIVE REVIEW OF job scheduling FOR MAPREDUCE

在线全文

学校读者我要写书评

暂无评论

A COMPARATIVE REVIEW OF JOB SCHEDULING FOR MAPREDUCE

SMART: Speedup job Completion Time by scheduling Reduce Tasks

2011 IEEE International Conference on Cloud Computing and Intelligence Systems(CCIS2011)

作者： Dongjin Yoo Kwang Mong Sim Multi-Agent and Cloud Computing Systems Laboratory School of Information and CommunicationGwangju Institute of Science and Technology（GIST）

MapReduce is an emerging paradigm for data intensive processing with support of cloud computing *** provides convenient programming interfaces to distribute data intensive works in a cluster *** strengths of MapReduce are fault tolerance,an easy programming structure and high scalability.A variety of applications have adopted MapReduce including scientific analysis,web data processing and high performance *** Intensive computing systems,such as Hadoop and Dryad,should provide an efficient scheduling mechanism for enhanced utilization in a shared cluster *** problems of scheduling mapreduce jobs are mostly caused by locality and synchronization ***,there is a need to schedule multiple jobs in a shared cluster with fairness *** introducing the scheduling problems with regards to locality,synchronization and fairness constraints,this paper reviews a collection of scheduling methods for handling these issues in *** addition,this paper compares different scheduling methods evaluating their features,strengths and *** resolving synchronization overhead,two categories of studies;asynchronous processing and speculative execution are *** fairness constraints with locality improvement,delay scheduling in Hadoop and Quincy scheduler in Dryad are discussed.

关键词： MapReduce job scheduling Locality Fairness

来源： cnki会议评论

在线全文

cnki会议

学校读者我要写书评

暂无评论

Journal of Computer Science & Technology 2022年第4期37卷 763-778页

作者：董加卿何泽昊龚媛媛于沛文田臣窦万春陈贵海夏耐管浩然 State Key Laboratory of Media Convergence and Communication Communication University of China Beijing 100024China State Key Laboratory for Novel Software Technology Nanjing UniversityNanjing 210023China School of Computer Science The University of SydneySydney NSW 2006Australia

Distributed computing systems have been widely used as the amount of data grows exponentially in the era of information explosion. job completion time (JCT) is a major metric for assessing their effectiveness. How to reduce the JCT for these systems through reasonable scheduling has become a hot issue in both industry and academia. Data skew is a common phenomenon that can compromise the performance of such distributed computing systems. This paper proposes SMART, which can effectively reduce the JCT through handling the data skew during the reducing phase. SMART predicts the size of reduce tasks based on part of the completed map tasks and then enforces largest-first scheduling in the reducing phase according to the predicted reduce task size. SMART makes minimal modifications to the original Hadoop with only 20 additional lines of code and is readily deployable. The robustness and the effectiveness of SMART have been evaluated with a real-world cluster against a large number of datasets. Experiments show that SMART reduces JCT by up to 6.47%, 9.26%, and 13.66% for Terasort, WordCount and InvertedIndex respectively with the Purdue MapReduce benchmarks suite (PUMA) dataset.

关键词： job scheduling job completion time MapReduce Hadoop