A Study of Heuristically-Based Parametric Performance Improvement/Optimization Algorithms for BigData Computing

Jongyeop Kim, Noh Jin Park, Nohpill Park

Research output: Contribution to book or proceedingConference articlepeer-review

3 Scopus citations

Abstract

Performance optimization for mapreduce computing in Hadoop platform is a tedious yet challenging problem due to the complexity of system organization with an extensive list of configuration parameters to be considered. In order to address and resolve this problem, various parameter optimization algorithms are proposed in this research from a naive exhaustive method to a random and a couple of heuristically-based greedy methods to vie with the exponentially nature of the search process for the possible best parameter setting. Extensive benchmark-based experiments have been conducted to validate the performance viability of the mapreduce computations by the benchmark programs such as TestDFSIO, TeraSort, to mention a couple. The experimental results demonstrate the proposed heuristically-based algorithms in greedy manner provide a promising answer to the problem of the research how to optimize the systems configuration parameter set at a computationally viable and feasible cost.

Original languageEnglish
Title of host publicationProceedings - 4th International Conference on Applied Computing and Information Technology, 3rd International Conference on Computational Science/Intelligence and Applied Informatics, 1st International Conference on Big Data, Cloud Computing, Data Science and Engineering, ACIT-CSII-BCD 2016
EditorsWeimin Li, Simon Xu, Nam Nguyen, Takaaki Goto
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages413-418
Number of pages6
ISBN (Electronic)9781509048717
DOIs
StatePublished - May 1 2017
Event4th International Conference on Applied Computing and Information Technology, 3rd International Conference on Computational Science/Intelligence and Applied Informatics and 1st International Conference on Big Data, Cloud Computing, Data Science and Engineering, ACIT-CSII-BCD 2016 - Las Vegas, United States
Duration: Dec 12 2016Dec 14 2016

Publication series

NameProceedings - 4th International Conference on Applied Computing and Information Technology, 3rd International Conference on Computational Science/Intelligence and Applied Informatics, 1st International Conference on Big Data, Cloud Computing, Data Science and Engineering, ACIT-CSII-BCD 2016

Conference

Conference4th International Conference on Applied Computing and Information Technology, 3rd International Conference on Computational Science/Intelligence and Applied Informatics and 1st International Conference on Big Data, Cloud Computing, Data Science and Engineering, ACIT-CSII-BCD 2016
Country/TerritoryUnited States
CityLas Vegas
Period12/12/1612/14/16

Keywords

  • Big Data
  • Configuration
  • Hadoop
  • Performance tuning

Fingerprint

Dive into the research topics of 'A Study of Heuristically-Based Parametric Performance Improvement/Optimization Algorithms for BigData Computing'. Together they form a unique fingerprint.

Cite this