Optimized Common Parameter Set Extraction by Benchmarking Applications on a Big Data Platform

Jongyeop Kim; Abhilash Kancharla; Jongho Seol; Noh Jin Park; Nohpill Park

doi:10.1109/SNPD.2018.8441059

Optimized Common Parameter Set Extraction by Benchmarking Applications on a Big Data Platform

Jongyeop Kim, Abhilash Kancharla, Jongho Seol, Noh Jin Park, Nohpill Park

Information Technology

Research output: Contribution to book or proceeding › Chapter

Abstract

This research proposes the methodology to extract common configuration parameter set by applying multiple benchmark applications including TeraSort., TestDFSIO, and MrBench on the Hadoop Distributed File System. In the process of determining parameter set for each stage, one parameter and its associated values selected which is reduced system performance in terms of overall execution time difference are measured by multiple applications on a Hadoop cluster. The experimental results demonstrate the proposed extended greedy manner provide a feasible benchmark model for the multiple tasks. In this way, we have found several parameter value sets that can reduce the execution time by 27% of the values provided by Hadoop default.

Original language	American English
Title of host publication	Proceedings of 2018 19th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD)
DOIs	https://doi.org/10.1109/SNPD.2018.8441059
State	Published - Aug 23 2018

DC Disciplines

Engineering
Computer Engineering

Access to Document

10.1109/SNPD.2018.8441059License: Unspecified

Cite this

@inbook{ed5c5c8119134b938e762283258663ba,

title = "Optimized Common Parameter Set Extraction by Benchmarking Applications on a Big Data Platform",

abstract = " This research proposes the methodology to extract common configuration parameter set by applying multiple benchmark applications including TeraSort., TestDFSIO, and MrBench on the Hadoop Distributed File System. In the process of determining parameter set for each stage, one parameter and its associated values selected which is reduced system performance in terms of overall execution time difference are measured by multiple applications on a Hadoop cluster. The experimental results demonstrate the proposed extended greedy manner provide a feasible benchmark model for the multiple tasks. In this way, we have found several parameter value sets that can reduce the execution time by 27% of the values provided by Hadoop default. ",

author = "Jongyeop Kim and Abhilash Kancharla and Jongho Seol and Park, {Noh Jin} and Nohpill Park",

note = "This research proposes the methodology to extract common configuration parameter set by applying multiple benchmark applications including TeraSort., TestDFSIO, and MrBench on the Hadoop Distributed File System. In the process of determining parameter set for each stage, one parameter and its associated values selected which is reduced system performance in terms of overall execution time difference are measured by multiple applications on a Hadoop cluster.",

year = "2018",

month = aug,

day = "23",

doi = "10.1109/SNPD.2018.8441059",

language = "American English",

booktitle = "Proceedings of 2018 19th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD)",

}

Optimized Common Parameter Set Extraction by Benchmarking Applications on a Big Data Platform. / Kim, Jongyeop; Kancharla, Abhilash; Seol, Jongho et al.
Proceedings of 2018 19th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD). 2018.

Research output: Contribution to book or proceeding › Chapter

TY - CHAP

T1 - Optimized Common Parameter Set Extraction by Benchmarking Applications on a Big Data Platform

AU - Kim, Jongyeop

AU - Kancharla, Abhilash

AU - Seol, Jongho

AU - Park, Noh Jin

AU - Park, Nohpill

N1 - This research proposes the methodology to extract common configuration parameter set by applying multiple benchmark applications including TeraSort., TestDFSIO, and MrBench on the Hadoop Distributed File System. In the process of determining parameter set for each stage, one parameter and its associated values selected which is reduced system performance in terms of overall execution time difference are measured by multiple applications on a Hadoop cluster.

PY - 2018/8/23

Y1 - 2018/8/23

N2 - This research proposes the methodology to extract common configuration parameter set by applying multiple benchmark applications including TeraSort., TestDFSIO, and MrBench on the Hadoop Distributed File System. In the process of determining parameter set for each stage, one parameter and its associated values selected which is reduced system performance in terms of overall execution time difference are measured by multiple applications on a Hadoop cluster. The experimental results demonstrate the proposed extended greedy manner provide a feasible benchmark model for the multiple tasks. In this way, we have found several parameter value sets that can reduce the execution time by 27% of the values provided by Hadoop default.

AB - This research proposes the methodology to extract common configuration parameter set by applying multiple benchmark applications including TeraSort., TestDFSIO, and MrBench on the Hadoop Distributed File System. In the process of determining parameter set for each stage, one parameter and its associated values selected which is reduced system performance in terms of overall execution time difference are measured by multiple applications on a Hadoop cluster. The experimental results demonstrate the proposed extended greedy manner provide a feasible benchmark model for the multiple tasks. In this way, we have found several parameter value sets that can reduce the execution time by 27% of the values provided by Hadoop default.

UR - https://doi.org/10.1109/SNPD.2018.8441059

U2 - 10.1109/SNPD.2018.8441059

DO - 10.1109/SNPD.2018.8441059

M3 - Chapter

BT - Proceedings of 2018 19th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD)

ER -

Optimized Common Parameter Set Extraction by Benchmarking Applications on a Big Data Platform

Abstract

DC Disciplines

Access to Document

Other files and links

Fingerprint

Cite this