TY - GEN
T1 - Design, implementation, and evaluation of transparent pNFS on lustre
AU - Yu, Weikuan
AU - Drokin, Oleg
AU - Vetter, Jeffrey S.
PY - 2009
Y1 - 2009
N2 - Parallel NFS (pNFS) is an emergent open standard for parallelizing data transfer over a variety of I/O protocols. Prototypes of pNFS are actively being developed by industry and academia to examine its viability and possible enhancements. In this paper, we present the design, implementation, and evaluation of lpNFS, a Lustre-based parallel NFS. We achieve our primary objective in designing lpNFS as an enabling technology for transparent pNFS accesses to an opaque Lustre file system. We optimize the data flow paths in lpNFS by using two techniques: (a) fast memory coping for small messages, and (b) page sharing for zero-copy bulk data transfer. Our initial performance evaluation shows that the performance of lpNFS is comparable to that of original Lustre. Given these results, we assert that lpNFS is a promising approach to combining the benefits of pNFS and Lustre, and it exposes the underlying capabilities of Lustre file systems while transparently supporting pNFS clients.
AB - Parallel NFS (pNFS) is an emergent open standard for parallelizing data transfer over a variety of I/O protocols. Prototypes of pNFS are actively being developed by industry and academia to examine its viability and possible enhancements. In this paper, we present the design, implementation, and evaluation of lpNFS, a Lustre-based parallel NFS. We achieve our primary objective in designing lpNFS as an enabling technology for transparent pNFS accesses to an opaque Lustre file system. We optimize the data flow paths in lpNFS by using two techniques: (a) fast memory coping for small messages, and (b) page sharing for zero-copy bulk data transfer. Our initial performance evaluation shows that the performance of lpNFS is comparable to that of original Lustre. Given these results, we assert that lpNFS is a promising approach to combining the benefits of pNFS and Lustre, and it exposes the underlying capabilities of Lustre file systems while transparently supporting pNFS clients.
KW - Keywords
KW - Lustre
KW - NFSv4
KW - PNFS
UR - http://www.scopus.com/inward/record.url?scp=70450091447&partnerID=8YFLogxK
U2 - 10.1109/IPDPS.2009.5160993
DO - 10.1109/IPDPS.2009.5160993
M3 - Conference contribution
AN - SCOPUS:70450091447
SN - 9781424437504
T3 - IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium
BT - IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium
T2 - 23rd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2009
Y2 - 23 May 2009 through 29 May 2009
ER -