Earlier stage for straggler detection and handling using combined CPU test and LATE methodology

Anwar H. Katrawi, Rosni Abdullah, Mohammed Anbar, Ammar Kamal Abasi

Abstract


Using Map Reduce in Hadoop helps in lowering the execution time and power consumption for large scale data. However, there can be a delay in job processing in circumstances where tasks are assigned to bad or congested machines called "straggler tasks"; which increases the time, power consumptions and therefore increasing the costs and leading to a poor performance of computing systems. This research proposes a hybrid Map Reduce framework referred to as the Combinatory Late-Machine (CLM) framework. Implementation of this framework will facilitate early and timely detection and identification of stragglers thereby facilitating prompt appropriate and effective actions.

Keywords


Big data; Combinatory late-machine; Hadoop; Mapreduce; Straggler



DOI: http://doi.org/10.11591/ijece.v10i5.pp%25p
Total views : 18 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.