Workflows in a secure environment

Petascale simulations on the largest supercomputers in the US require advanced data management techniques in order to optimize the application scientist time, and to optimize the time spent on the supercomputers. Researchers in such problems are starting to require workflow automation during their simulations in order to monitor the simulations, and in order to automate many of the complex analysis which must take place from the data that is generated from these simulations. Scientific workflows are being used to monitor simulations running on these supercomputers by applying a series of complex analysis, and finally producing images and movies from the variables produced in the simulation, or from the derived quantities produced by the analysis. The typical scenario is where the large calculation runs on the supercomputer, and the auxiliary diagnostics/monitors are run on resources, which are either on the local area network of the supercomputer, or over the wide area network. The supercomputers at one of the largest centers are highly secure, and the only method to log into the center is interactive authentication by using One Time Passwords (OTP) that are generated by a security device and expire in half a minute. Therefore, grid certificates are not a current option on these machines in the Department of Energy at Oak Ridge National Laboratory. In this paper we describe how we have extended the Kepler scientific workflow management system to be able to run operations on these supercomputers, how workflows themselves can be executed as batch jobs, and finally, how external data-transfer operations can be utilized when they need to perform authentication for their own as well.