Streaming Detection and Classification Performance of a POWER9 Edge Supercomputer

We present training and inference benchmarks on a POWER9 edge supercomputer called SCOUT using two production-level artificial intelligence systems: The Video Processing Exploitation Framework (VPEF) and the Ordnance Threat Target Automated Recognition (OTTAR) system. For training benchmarks, we use Horovod to train ResNet-50 on synthetic data for a baseline benchmark, and then train Faster R-CNN on an available WASABI dataset using SCOUT's six-way V100 GPU nodes. For inference benchmarks, we use GStreamer to stream both synthetic and real Motion Imagery (MI) and then use Single-Shot Detector trained on MobileNetV2 data with real and synthetic MI at four different resolutions (720p, 1080p, 1280p, and 2160p) while measuring average inference time. We also test reduced-precision inference performance using TensorRT and distributed inference performance using the Inference Benchmark "iBench"framework. We compare our results with equivalent work on x86_64 systems and provide suggestions on tuning for optimal performance. We find that V100s work well for training and offline inferencing in batches, while T4 GPUs outperform V100 GPUs only in very specific usage scenarios, such as streaming detection at reduced precision.