TY - GEN
T1 - Multi-scale CAFE framework for simulating fracture in heterogeneous materials implemented in fortran co-arrays and MPI
AU - Shterenlikht, Anton
AU - Margetts, Lee
AU - Arregui-Mena, Jose D.
AU - Cebamanos, Luis
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2017/1/30
Y1 - 2017/1/30
N2 - Fortran coarrays have been used as an extension to the standard for over 20 years, mostly on Cray systems. Their appeal to users increased substantially when they were standardised in 2010. In this work we show that coarrays offer simple and intuitive data structures for 3D cellular automata (CA) modelling of material microstructures. We show how coarrays can be used together with an MPI finite element (FE) library to create a two-way concurrent hierarchical and scalable multi-scale CAFE deformation and fracture framework. Design of a coarray cellular automata microstructure evolution library CGPACK is described. A highly portable MPI FE library ParaFEM was used in this work. We show that independently CGPACK and ParaFEM programs can scale up well into tens of thousands of cores. Strong scaling of a hybrid ParaFEM/CGPACK MPI/coarray multi-scale framework was measured on an important solid mechanics practical example of a fracture of a steel round bar under tension. That program did not scale beyond 7 thousand cores. Excessive synchronisation might be one contributing factor to relatively poor scaling. Therefore we conclude with a comparative analysis of synchronisation requirements in MPI and coarray programs. Specific challenges of synchronising a coarray library are discussed.
AB - Fortran coarrays have been used as an extension to the standard for over 20 years, mostly on Cray systems. Their appeal to users increased substantially when they were standardised in 2010. In this work we show that coarrays offer simple and intuitive data structures for 3D cellular automata (CA) modelling of material microstructures. We show how coarrays can be used together with an MPI finite element (FE) library to create a two-way concurrent hierarchical and scalable multi-scale CAFE deformation and fracture framework. Design of a coarray cellular automata microstructure evolution library CGPACK is described. A highly portable MPI FE library ParaFEM was used in this work. We show that independently CGPACK and ParaFEM programs can scale up well into tens of thousands of cores. Strong scaling of a hybrid ParaFEM/CGPACK MPI/coarray multi-scale framework was measured on an important solid mechanics practical example of a fracture of a steel round bar under tension. That program did not scale beyond 7 thousand cores. Excessive synchronisation might be one contributing factor to relatively poor scaling. Therefore we conclude with a comparative analysis of synchronisation requirements in MPI and coarray programs. Specific challenges of synchronising a coarray library are discussed.
KW - Computer aided engineering
KW - Mechanical engineering
KW - Microstructure
KW - Parallel algorithms
KW - Parallel programming
KW - Scientific computing
KW - Supercomputers
UR - http://www.scopus.com/inward/record.url?scp=85015206785&partnerID=8YFLogxK
U2 - 10.1109/PAW.2016.006
DO - 10.1109/PAW.2016.006
M3 - Conference contribution
AN - SCOPUS:85015206785
T3 - Proceedings of PAW 2016: 1st PGAS Applications Workshop - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis
SP - 1
EP - 8
BT - Proceedings of PAW 2016
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 1st PGAS Applications Workshop, PAW 2016
Y2 - 14 November 2016
ER -