Scalable Interactive Autonomous Navigation Simulations on HPC

Wesley Brewer; Joel Bretheim; John Kaniarz; Peilin Song; Burhman Gates

doi:10.1109/HPEC55821.2022.9926384

Scalable Interactive Autonomous Navigation Simulations on HPC

Wesley Brewer, Joel Bretheim, John Kaniarz, Peilin Song, Burhman Gates

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We present our work of enabling HPC in an interactive real-time autonomy loop. The workflow consists of many different software components deployed within Singu-larity containers and communicating using both the Robotic Operating System's (ROS) publish-subscribe system and the Message Passing Interface (MPI). We use Singularity's container networking interface (CNI) to enable virtual networking within the containers, so that multiple containers can run the various components using different IP addresses on the same compute node. The Virtual Autonomous Navigation Environment Environmental Sensor Engine (VANE: ESE) is used for physically-realistic simulation of LIDAR along with the Autonomous Navigation Virtual Environment Laboratory (ANVEL) for vehicle simulation. VANE: ESE sends Velodyne UDP LIDAR packets directly to the Robotic Technology Kernel (RTK) and is distributed across multiple compute nodes via MPI along with OpenMP for shared memory parallelism within each compute node. The user interfaces with the navigation environment using an XFCE desk-top with virtual workspaces running over a VNC containerized deployment through a double-hop ssh tunnel, which uses noVNC (a JavaScript-based VNC client) to provide a browser-based client interface. We automate the complete launch process using a custom iLauncher plugin. We benchmark scalable performance with multiple vehicle simulations on four different HPC systems and discuss our findings.

Original language	English
Title of host publication	2022 IEEE High Performance Extreme Computing Conference, HPEC 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781665497862
DOIs	https://doi.org/10.1109/HPEC55821.2022.9926384
State	Published - 2022
Externally published	Yes
Event	2022 IEEE High Performance Extreme Computing Conference, HPEC 2022 - Virtual, Online, United States Duration: Sep 19 2022 → Sep 23 2022

Publication series

Name	2022 IEEE High Performance Extreme Computing Conference, HPEC 2022

Conference

Conference	2022 IEEE High Performance Extreme Computing Conference, HPEC 2022
Country/Territory	United States
City	Virtual, Online
Period	09/19/22 → 09/23/22

Funding

DISTRIBUTION A. Approved for public release; distribution unlimited. This material is based upon work supported by, or in part by, the Department of Defense High Performance Computing Modernization Program (HPCMP) under User Productivity, Enhanced Technology Transfer, and Training (PET) contracts #GS04T09DBC0017 and #47QFSA18K0111. Any opinions, finding and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the DoD HPCMP.

Funders	Funder number
Department of Defense High Performance Computing Modernization Program
HPCMP	04T09DBC0017, 47QFSA18K0111

Keywords

autonomous
navigation
real-time
scalable
vehicle
virtual desktop

Access to Document

10.1109/HPEC55821.2022.9926384

Cite this

Brewer, W., Bretheim, J., Kaniarz, J., Song, P., & Gates, B. (2022). Scalable Interactive Autonomous Navigation Simulations on HPC. In 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022 (2022 IEEE High Performance Extreme Computing Conference, HPEC 2022). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/HPEC55821.2022.9926384

@inproceedings{83f4524ff3344b58aaad87c31ad350a2,

title = "Scalable Interactive Autonomous Navigation Simulations on HPC",

abstract = "We present our work of enabling HPC in an interactive real-time autonomy loop. The workflow consists of many different software components deployed within Singu-larity containers and communicating using both the Robotic Operating System's (ROS) publish-subscribe system and the Message Passing Interface (MPI). We use Singularity's container networking interface (CNI) to enable virtual networking within the containers, so that multiple containers can run the various components using different IP addresses on the same compute node. The Virtual Autonomous Navigation Environment Environmental Sensor Engine (VANE: ESE) is used for physically-realistic simulation of LIDAR along with the Autonomous Navigation Virtual Environment Laboratory (ANVEL) for vehicle simulation. VANE: ESE sends Velodyne UDP LIDAR packets directly to the Robotic Technology Kernel (RTK) and is distributed across multiple compute nodes via MPI along with OpenMP for shared memory parallelism within each compute node. The user interfaces with the navigation environment using an XFCE desk-top with virtual workspaces running over a VNC containerized deployment through a double-hop ssh tunnel, which uses noVNC (a JavaScript-based VNC client) to provide a browser-based client interface. We automate the complete launch process using a custom iLauncher plugin. We benchmark scalable performance with multiple vehicle simulations on four different HPC systems and discuss our findings.",

keywords = "autonomous, navigation, real-time, scalable, vehicle, virtual desktop",

author = "Wesley Brewer and Joel Bretheim and John Kaniarz and Peilin Song and Burhman Gates",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022 ; Conference date: 19-09-2022 Through 23-09-2022",

year = "2022",

doi = "10.1109/HPEC55821.2022.9926384",

language = "English",

series = "2022 IEEE High Performance Extreme Computing Conference, HPEC 2022",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2022 IEEE High Performance Extreme Computing Conference, HPEC 2022",

}

Brewer, W, Bretheim, J, Kaniarz, J, Song, P & Gates, B 2022, Scalable Interactive Autonomous Navigation Simulations on HPC. in 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022. 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022, Institute of Electrical and Electronics Engineers Inc., 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022, Virtual, Online, United States, 09/19/22. https://doi.org/10.1109/HPEC55821.2022.9926384

Scalable Interactive Autonomous Navigation Simulations on HPC. / Brewer, Wesley; Bretheim, Joel; Kaniarz, John et al.
2022 IEEE High Performance Extreme Computing Conference, HPEC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. (2022 IEEE High Performance Extreme Computing Conference, HPEC 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Scalable Interactive Autonomous Navigation Simulations on HPC

AU - Brewer, Wesley

AU - Bretheim, Joel

AU - Kaniarz, John

AU - Song, Peilin

AU - Gates, Burhman

PY - 2022

Y1 - 2022

N2 - We present our work of enabling HPC in an interactive real-time autonomy loop. The workflow consists of many different software components deployed within Singu-larity containers and communicating using both the Robotic Operating System's (ROS) publish-subscribe system and the Message Passing Interface (MPI). We use Singularity's container networking interface (CNI) to enable virtual networking within the containers, so that multiple containers can run the various components using different IP addresses on the same compute node. The Virtual Autonomous Navigation Environment Environmental Sensor Engine (VANE: ESE) is used for physically-realistic simulation of LIDAR along with the Autonomous Navigation Virtual Environment Laboratory (ANVEL) for vehicle simulation. VANE: ESE sends Velodyne UDP LIDAR packets directly to the Robotic Technology Kernel (RTK) and is distributed across multiple compute nodes via MPI along with OpenMP for shared memory parallelism within each compute node. The user interfaces with the navigation environment using an XFCE desk-top with virtual workspaces running over a VNC containerized deployment through a double-hop ssh tunnel, which uses noVNC (a JavaScript-based VNC client) to provide a browser-based client interface. We automate the complete launch process using a custom iLauncher plugin. We benchmark scalable performance with multiple vehicle simulations on four different HPC systems and discuss our findings.

AB - We present our work of enabling HPC in an interactive real-time autonomy loop. The workflow consists of many different software components deployed within Singu-larity containers and communicating using both the Robotic Operating System's (ROS) publish-subscribe system and the Message Passing Interface (MPI). We use Singularity's container networking interface (CNI) to enable virtual networking within the containers, so that multiple containers can run the various components using different IP addresses on the same compute node. The Virtual Autonomous Navigation Environment Environmental Sensor Engine (VANE: ESE) is used for physically-realistic simulation of LIDAR along with the Autonomous Navigation Virtual Environment Laboratory (ANVEL) for vehicle simulation. VANE: ESE sends Velodyne UDP LIDAR packets directly to the Robotic Technology Kernel (RTK) and is distributed across multiple compute nodes via MPI along with OpenMP for shared memory parallelism within each compute node. The user interfaces with the navigation environment using an XFCE desk-top with virtual workspaces running over a VNC containerized deployment through a double-hop ssh tunnel, which uses noVNC (a JavaScript-based VNC client) to provide a browser-based client interface. We automate the complete launch process using a custom iLauncher plugin. We benchmark scalable performance with multiple vehicle simulations on four different HPC systems and discuss our findings.

KW - autonomous

KW - navigation

KW - real-time

KW - scalable

KW - vehicle

KW - virtual desktop

UR - http://www.scopus.com/inward/record.url?scp=85142217321&partnerID=8YFLogxK

U2 - 10.1109/HPEC55821.2022.9926384

DO - 10.1109/HPEC55821.2022.9926384

M3 - Conference contribution

AN - SCOPUS:85142217321

T3 - 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022

BT - 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022

Y2 - 19 September 2022 through 23 September 2022

ER -

Scalable Interactive Autonomous Navigation Simulations on HPC

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this