TY - GEN
T1 - Performance and scalability evaluation of the Ceph parallel file system
AU - Wang, Feiyi
AU - Nelson, Mark
AU - Oral, Sarp
AU - Atchley, Scott
AU - Weil, Sage
AU - Settlemyer, Bradley W.
AU - Caldwell, Blake
AU - Hill, Jason
PY - 2013
Y1 - 2013
N2 - Ceph is an emerging open-source parallel distributed file and storage system. By design, Ceph leverages unreliable commodity storage and network hardware, and provides reliability and fault-tolerance via controlled object placement and data replication. This paper presents our file and block I/O performance and scalability evaluation of Ceph for scientific high-performance computing (HPC) environments. Our work makes two unique contributions. First, our evaluation is performed under a realistic setup for a large-scale capability HPC environment using a commercial high-end storage system. Second, our path of investigation, tuning efforts, and findings made direct contributions to Ceph's development and improved code quality, scalability, and performance. These changes should benefit both Ceph and the HPC community at large.
AB - Ceph is an emerging open-source parallel distributed file and storage system. By design, Ceph leverages unreliable commodity storage and network hardware, and provides reliability and fault-tolerance via controlled object placement and data replication. This paper presents our file and block I/O performance and scalability evaluation of Ceph for scientific high-performance computing (HPC) environments. Our work makes two unique contributions. First, our evaluation is performed under a realistic setup for a large-scale capability HPC environment using a commercial high-end storage system. Second, our path of investigation, tuning efforts, and findings made direct contributions to Ceph's development and improved code quality, scalability, and performance. These changes should benefit both Ceph and the HPC community at large.
UR - http://www.scopus.com/inward/record.url?scp=84891722452&partnerID=8YFLogxK
U2 - 10.1145/2538542.2538562
DO - 10.1145/2538542.2538562
M3 - Conference contribution
AN - SCOPUS:84891722452
SN - 9781450325059
T3 - Proceedings of the 8th Parallel Data Storage Workshop, PDSW 2013 - Held in Conjunction with SC 2013: The International Conference for High Performance Computing, Networking, Storage and Analysis
SP - 14
EP - 19
BT - Proceedings of the 8th Parallel Data Storage Workshop, PDSW 2013 - Held in Conjunction with SC 2013
PB - Association for Computing Machinery
T2 - 8th Parallel Data Storage Workshop, PDSW 2013 - Held in Conjunction with the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2013
Y2 - 17 November 2013 through 22 November 2013
ER -