Diagnosing autonomous vehicle driving criteria with an adversarial evolutionary algorithm

Mark A. Coletti; Shang Gao; Spencer Paulissen; Nicholas Quentin Haas; Robert Patton

doi:10.1145/3449726.3459573

Diagnosing autonomous vehicle driving criteria with an adversarial evolutionary algorithm

Mark A. Coletti, Shang Gao, Spencer Paulissen, Nicholas Quentin Haas, Robert Patton

Learning Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Scopus citations

Abstract

We repurposed an adversarial evolutionary algorithm, Gremlin, from finding driving scenarios where a model of an autonomous vehicle drove poorly to troubleshooting driving quality evaluation criteria. We evaluated the driving performance of a "perfect driver"robot in a virtual town environment using the same fitness criteria intended for a deep learner (DL) trained driver. We found that the fitness evaluation criteria poorly handled turns, and used Gremlin to iteratively improve that criteria. We were confident that the same criteria could then be applied to the DL-based models as originally intended, and that this approach could be used as a general means of troubleshooting autonomous vehicle driving criteria.

Original language	English
Title of host publication	GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion
Publisher	Association for Computing Machinery, Inc
Pages	301-302
Number of pages	2
ISBN (Electronic)	9781450383516
DOIs	https://doi.org/10.1145/3449726.3459573
State	Published - Jul 7 2021
Event	2021 Genetic and Evolutionary Computation Conference, GECCO 2021 - Virtual, Online, France Duration: Jul 10 2021 → Jul 14 2021

Publication series

Name	GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion

Conference

Conference	2021 Genetic and Evolutionary Computation Conference, GECCO 2021
Country/Territory	France
City	Virtual, Online
Period	07/10/21 → 07/14/21

Funding

This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy (DOE). Funding provided by the U.S. Department of Energy Office of Energy Efficiency and Renewable Energy Vehicle Technologies Office. The views expressed in the article do not necessarily represent the views of the DOE or the U.S. Government. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Funders	Funder number
U.S. Department of Energy Office of Energy Efficiency and Renewable Energy Vehicle Technologies Office
U.S. Department of Energy
Office of Science	DE-AC05-00OR22725

Keywords

adversarial algorithms
autonomous vehicles
evolutionary algorithms

Access to Document

10.1145/3449726.3459573

Cite this

Coletti, M. A., Gao, S., Paulissen, S., Haas, N. Q., & Patton, R. (2021). Diagnosing autonomous vehicle driving criteria with an adversarial evolutionary algorithm. In GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion (pp. 301-302). (GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion). Association for Computing Machinery, Inc. https://doi.org/10.1145/3449726.3459573

Coletti, Mark A. ; Gao, Shang ; Paulissen, Spencer et al. / Diagnosing autonomous vehicle driving criteria with an adversarial evolutionary algorithm. GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery, Inc, 2021. pp. 301-302 (GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion).

@inproceedings{5942a08ca7ab406e9d5a49a4828030bf,

title = "Diagnosing autonomous vehicle driving criteria with an adversarial evolutionary algorithm",

abstract = "We repurposed an adversarial evolutionary algorithm, Gremlin, from finding driving scenarios where a model of an autonomous vehicle drove poorly to troubleshooting driving quality evaluation criteria. We evaluated the driving performance of a {"}perfect driver{"}robot in a virtual town environment using the same fitness criteria intended for a deep learner (DL) trained driver. We found that the fitness evaluation criteria poorly handled turns, and used Gremlin to iteratively improve that criteria. We were confident that the same criteria could then be applied to the DL-based models as originally intended, and that this approach could be used as a general means of troubleshooting autonomous vehicle driving criteria.",

keywords = "adversarial algorithms, autonomous vehicles, evolutionary algorithms",

author = "Coletti, {Mark A.} and Shang Gao and Spencer Paulissen and Haas, {Nicholas Quentin} and Robert Patton",

note = "Publisher Copyright: {\textcopyright} 2021 Owner/Author.; 2021 Genetic and Evolutionary Computation Conference, GECCO 2021 ; Conference date: 10-07-2021 Through 14-07-2021",

year = "2021",

month = jul,

day = "7",

doi = "10.1145/3449726.3459573",

language = "English",

series = "GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion",

publisher = "Association for Computing Machinery, Inc",

pages = "301--302",

booktitle = "GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion",

}

Coletti, MA, Gao, S, Paulissen, S, Haas, NQ & Patton, R 2021, Diagnosing autonomous vehicle driving criteria with an adversarial evolutionary algorithm. in GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion. GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion, Association for Computing Machinery, Inc, pp. 301-302, 2021 Genetic and Evolutionary Computation Conference, GECCO 2021, Virtual, Online, France, 07/10/21. https://doi.org/10.1145/3449726.3459573

Diagnosing autonomous vehicle driving criteria with an adversarial evolutionary algorithm. / Coletti, Mark A.; Gao, Shang; Paulissen, Spencer et al.
GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery, Inc, 2021. p. 301-302 (GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Diagnosing autonomous vehicle driving criteria with an adversarial evolutionary algorithm

AU - Coletti, Mark A.

AU - Gao, Shang

AU - Paulissen, Spencer

AU - Haas, Nicholas Quentin

AU - Patton, Robert

PY - 2021/7/7

Y1 - 2021/7/7

N2 - We repurposed an adversarial evolutionary algorithm, Gremlin, from finding driving scenarios where a model of an autonomous vehicle drove poorly to troubleshooting driving quality evaluation criteria. We evaluated the driving performance of a "perfect driver"robot in a virtual town environment using the same fitness criteria intended for a deep learner (DL) trained driver. We found that the fitness evaluation criteria poorly handled turns, and used Gremlin to iteratively improve that criteria. We were confident that the same criteria could then be applied to the DL-based models as originally intended, and that this approach could be used as a general means of troubleshooting autonomous vehicle driving criteria.

AB - We repurposed an adversarial evolutionary algorithm, Gremlin, from finding driving scenarios where a model of an autonomous vehicle drove poorly to troubleshooting driving quality evaluation criteria. We evaluated the driving performance of a "perfect driver"robot in a virtual town environment using the same fitness criteria intended for a deep learner (DL) trained driver. We found that the fitness evaluation criteria poorly handled turns, and used Gremlin to iteratively improve that criteria. We were confident that the same criteria could then be applied to the DL-based models as originally intended, and that this approach could be used as a general means of troubleshooting autonomous vehicle driving criteria.

KW - adversarial algorithms

KW - autonomous vehicles

KW - evolutionary algorithms

UR - http://www.scopus.com/inward/record.url?scp=85111029439&partnerID=8YFLogxK

U2 - 10.1145/3449726.3459573

DO - 10.1145/3449726.3459573

M3 - Conference contribution

AN - SCOPUS:85111029439

T3 - GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion

SP - 301

EP - 302

BT - GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion

PB - Association for Computing Machinery, Inc

T2 - 2021 Genetic and Evolutionary Computation Conference, GECCO 2021

Y2 - 10 July 2021 through 14 July 2021

ER -

Coletti MA, Gao S, Paulissen S, Haas NQ , Patton R. Diagnosing autonomous vehicle driving criteria with an adversarial evolutionary algorithm. In GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery, Inc. 2021. p. 301-302. (GECCO 2021 Companion - Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion). doi: 10.1145/3449726.3459573

Diagnosing autonomous vehicle driving criteria with an adversarial evolutionary algorithm

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this