Abstract
We explore how to optimally deploy several different types of machine-learned surrogate models used in rotorcraft aerodynamics on HPC. We first developed three different rotorcraft models at three different orders of magnitude (2M, 44M, and 212M trainable parameters) to use as test models. Then we developed a benchmark, which we call 'smiBench', that uses synthetic data to test a wide range of alternative configurations to study optimal deployment scenarios. We discovered several different types of optimal deployment scenarios depending on the model size and inference frequency. For most cases, it makes sense to use multiple inference servers, each bound to a GPU with a load balancer distributing the requests across multiple GPUs. We tested three different types of inference server deployments: (1) a custom Flask-based HTTP inference server, (2) TensorFlow Serving with gRPC protocol, and (3) RedisAI server with SmartRedis clients using the RESP protocol. We also tested three different types of load balancing techniques for multi-GPU inferencing: (1) Python concurrent.futures thread pool, (2) HAProxy, and (3) mpi4py. We investigated deployments on both DoD HPCMP's SCOUT and DoE OLCF's Summit POWER9 supercomputers, demonstrated the ability to inference a million samples per second using 192 GPUs, and studied multiple scenarios on both Nvidia T4 and V100 GPUs. Moreover, we studied a range of concurrency levels, both on the client-side and the server-side, and provide optimal configuration advice based on the type of deployment. Finally, we provide a simple Python-based framework for benchmarking machine-learned surrogate models using the various inference servers.
Original language | English |
---|---|
Title of host publication | Proceedings of MLHPC 2021 |
Subtitle of host publication | Workshop on Machine Learning in High Performance Computing Environments, Held in conjunction with SC 2021: The International Conference for High Performance Computing, Networking, Storage and Analysis |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 21-32 |
Number of pages | 12 |
ISBN (Electronic) | 9781665411240 |
DOIs | |
State | Published - 2021 |
Externally published | Yes |
Event | 7th IEEE/ACM Workshop on Machine Learning in High Performance Computing Environments, MLHPC 2021 - St. Louis, United States Duration: Nov 15 2021 → … |
Publication series
Name | Proceedings of MLHPC 2021: Workshop on Machine Learning in High Performance Computing Environments, Held in conjunction with SC 2021: The International Conference for High Performance Computing, Networking, Storage and Analysis |
---|
Conference
Conference | 7th IEEE/ACM Workshop on Machine Learning in High Performance Computing Environments, MLHPC 2021 |
---|---|
Country/Territory | United States |
City | St. Louis |
Period | 11/15/21 → … |
Funding
Distribution Statement A: Approved for Public Release. This material is based upon work supported by, or in part by, the Department of Defense High Performance Computing Modernization Program (HPCMP) under User Productivity, Enhanced Technology Transfer, and Training (PET) contracts #GS04T09DBC0017 and #47QFSA18K0111. Any opinions, finding and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the DoD HPCMP. This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725.
Keywords
- HPC
- inference
- production
- surrogate