TY - GEN
T1 - Lcio
T2 - 2nd International Symposium on Benchmarking, Measuring, and Optimization, Bench 2019
AU - Bachstein, Matthew
AU - Wang, Feiyi
AU - Oral, Sarp
N1 - Publisher Copyright:
© Springer Nature Switzerland AG 2020.
PY - 2020
Y1 - 2020
N2 - Performance of file systems shift during their life cycles. Evaluating this performance change over time is not trivial. Complexity arises in the interplay between external (i.e. application I/O workloads) and internal (i.e. the filesystem state) factors. Many benchmarks can test how a filesystem performs at the current snapshot state, but to observe the change over time necessitates that the filesystem state mutate (age) between benchmark runs. For a large-scale HPC parallel filesystem, the sheer scale and amount of interacting components during I/O operations magnify these challenges. There have been several approaches that address different aspects of filesystem aging, from creating statistically realistic filesystem images to file age distributions. The common drawbacks are the scale to be evaluated and the time needed to converge; none of the methods in literature targeted network or parallel file systems. Also, none were evaluated with a filesystem image over 300 GiB, most under 50 GiB, yet almost all took between a half hour to 7 h to converge. For a large-scale parallel file system, these methods are impractical as far as time and resources needed (a typical large PFS is in the PB range). Additionally, HPC filesystem I/O workloads are drastically different from local system workloads used in earlier studies. This paper presents the design, implementation and evaluation of LCIO synthetic filesystem aging benchmark, which aims to address the question of “how will the filesystem perform at different stages of its life cycle?”. As such, being able to answer that question as realistically as feasible in a reasonable time is where LCIO contributes.
AB - Performance of file systems shift during their life cycles. Evaluating this performance change over time is not trivial. Complexity arises in the interplay between external (i.e. application I/O workloads) and internal (i.e. the filesystem state) factors. Many benchmarks can test how a filesystem performs at the current snapshot state, but to observe the change over time necessitates that the filesystem state mutate (age) between benchmark runs. For a large-scale HPC parallel filesystem, the sheer scale and amount of interacting components during I/O operations magnify these challenges. There have been several approaches that address different aspects of filesystem aging, from creating statistically realistic filesystem images to file age distributions. The common drawbacks are the scale to be evaluated and the time needed to converge; none of the methods in literature targeted network or parallel file systems. Also, none were evaluated with a filesystem image over 300 GiB, most under 50 GiB, yet almost all took between a half hour to 7 h to converge. For a large-scale parallel file system, these methods are impractical as far as time and resources needed (a typical large PFS is in the PB range). Additionally, HPC filesystem I/O workloads are drastically different from local system workloads used in earlier studies. This paper presents the design, implementation and evaluation of LCIO synthetic filesystem aging benchmark, which aims to address the question of “how will the filesystem perform at different stages of its life cycle?”. As such, being able to answer that question as realistically as feasible in a reasonable time is where LCIO contributes.
UR - http://www.scopus.com/inward/record.url?scp=85087003590&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-49556-5_24
DO - 10.1007/978-3-030-49556-5_24
M3 - Conference contribution
AN - SCOPUS:85087003590
SN - 9783030495558
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 251
EP - 261
BT - Benchmarking, Measuring, and Optimizing - 2nd BenchCouncil International Symposium, Bench 2019, Revised Selected Papers
A2 - Gao, Wanling
A2 - Zhan, Jianfeng
A2 - Fox, Geoffrey
A2 - Lu, Xiaoyi
A2 - Stanzione, Dan
PB - Springer
Y2 - 14 November 2019 through 16 November 2019
ER -