Nexus: A modular workflow management system for quantum simulation codes

The management of simulation workflows represents a significant task for the individual computational researcher. Automation of the required tasks involved in simulation work can decrease the overall time to solution and reduce sources of human error. A new simulation workflow management system, Nexus, is presented to address these issues. Nexus is capable of automated job management on workstations and resources at several major supercomputing centers. Its modular design allows many quantum simulation codes to be supported within the same framework. Current support includes quantum Monte Carlo calculations with QMCPACK, density functional theory calculations with Quantum Espresso or VASP, and quantum chemical calculations with GAMESS. Users can compose workflows through a transparent, text-based interface, resembling the input file of a typical simulation code. A usage example is provided to illustrate the process. Program summary Program title: Nexus Catalogue identifier: AEXN-v1-0 Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEXN-v1-0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: University of Illinois/NCSA Open Source License (UIUC/NCSA) No. of lines in distributed program, including test data, etc.: 92059 No. of bytes in distributed program, including test data, etc.: 1414101 Distribution format: tar.gz Programming language: Python. Computer: Workstations, clusters, and supercomputing resources at OLCF, ALCF, NERSC, NCSA, TACC, and RPI. Operating system: Linux. RAM: 100 megabytes Classification: 6.5, 7.3. External routines: NumPy. Optionally SciPy, Matplotlib, and H5Py. Nature of problem: Quantum simulation workflows with multiple dependencies between various electronic structure codes. Job management of simulations on arbitrary workstation and supercomputing environments. Aggregation and analysis of output data. Solution method: Modular and extensible class-based framework to represent atomic structures, simulation instances, input files, output data, job requests, and host machines. Automatic identification of time ordered simulation networks (workflows) and simultaneous management of independent workflows including writing of input files, submission of jobs on available resources, job monitoring, and collection and post processing of output data. High-level and minimal user interface to generate input files, compose workflows, and perform data analysis. Running time: Total run time of the Nexus program is close to the overall wall time of the managed simulations. For the enclosed user example, the running time is approximately 30 seconds on a 16 core workstation.