Indexed by:
Abstract:
In MapReduce model, the job execution time was prolonged by the straggler tasks in heterogeneity environments . The LATE scheduler has introduced the longest remaining time strategy , but it also has some drawbacks such as inaccurate estimated time and the wasting of system resources. In order to solve these problem, we propose two main algorithm : The parameter dynamic-tuning algorithm based history estimates progress of a task accurately since it dynamically tunes the weight of each phase of a map task and a reduce task according to the historical values of the weights; The evaluation-scheduling algorithm reduce the wasting of system resources by evaluating the free slot before launching a straggler task on this node. The two main algorithm are implemented in hadoop 0.20.1. The environment results are satisfaction to our expects and significantly reduce the wasting of system resources. © 2012 IEEE.
Keyword:
Reprint Author's Address:
Email:
Source :
Proceedings - 7th ChinaGrid Annual Conference, ChinaGrid 2012
ISSN: 9780769548166
Year: 2012
Publish Date: 2012
Page: 49-56
Language: English
Cited Count:
WoS CC Cited Count: 0
SCOPUS Cited Count: 3
ESI Highly Cited Papers on the List: 0 Unfold All
WanFang Cited Count:
Chinese Cited Count:
30 Days PV: 7
Affiliated Colleges: