• Complex
  • Title
  • Author
  • Keyword
  • Abstract
  • Scholars
Search

Author:

Zhao, Xu (Zhao, Xu.) | Dong, Xiaoshe (Dong, Xiaoshe.) | Cao, Haijun (Cao, Haijun.) | Fan, Yuanquan (Fan, Yuanquan.) | Zhu, Huo (Zhu, Huo.)

Indexed by:

Abstract:

In MapReduce model, the job execution time was prolonged by the straggler tasks in heterogeneity environments . The LATE scheduler has introduced the longest remaining time strategy , but it also has some drawbacks such as inaccurate estimated time and the wasting of system resources. In order to solve these problem, we propose two main algorithm : The parameter dynamic-tuning algorithm based history estimates progress of a task accurately since it dynamically tunes the weight of each phase of a map task and a reduce task according to the historical values of the weights; The evaluation-scheduling algorithm reduce the wasting of system resources by evaluating the free slot before launching a straggler task on this node. The two main algorithm are implemented in hadoop 0.20.1. The environment results are satisfaction to our expects and significantly reduce the wasting of system resources. © 2012 IEEE.

Keyword:

Dynamic tuning Hadoop Heterogeneous environments Map-reduce MapReduce models Parameter dynamics Straggler tasks System resources

Author Community:

  • [ 1 ] [Zhao, Xu;Dong, Xiaoshe;Cao, Haijun;Fan, Yuanquan;Zhu, Huo]Department of Computer Science and Technology, Xi'an Jiaotong University, China

Reprint Author's Address:

Show more details

Related Keywords:

Related Article:

Source :

Proceedings - 7th ChinaGrid Annual Conference, ChinaGrid 2012

ISSN: 9780769548166

Year: 2012

Publish Date: 2012

Page: 49-56

Language: English

Cited Count:

WoS CC Cited Count: 0

SCOPUS Cited Count: 3

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 7

FAQ| About| Online/Total:408/168627504
Address:XI'AN JIAOTONG UNIVERSITY LIBRARY(No.28, Xianning West Road, Xi'an, Shaanxi Post Code:710049) Contact Us:029-82667865
Copyright:XI'AN JIAOTONG UNIVERSITY LIBRARY Technical Support:Beijing Aegean Software Co., Ltd.