Weaknesses in machine learning technology can have serious consequences, such as improperly trained facial recognition artificial intelligence yielding inaccurate identification. To improve machine learning, ORNL researchers developed Gremlin, a learning system designed to identify and address the worst-performing neural network feature sets.
Gremlin identifies problems within a machine learning system, often through inverting a model’s training metrics. For example, a model may be trained to drive a virtual autonomous car, so a simple training metric for that model might be maximizing the length of time before crashing; Gremlin would invert that metric to discover scenarios where the model crashes the soonest.
The system can then be used to update the model training data with more examples of those poor performing scenarios, and the model is retrained using that updated data.
Gremlin decreases time needed to address machine learning model weaknesses and can be scaled for application from laptop computers to machines like ORNL’s Summit supercomputer.
A flexible framework improving upon comparable systems, the technology can be used on machine learning models designed for most any application.
Funding for this project was provided by the Office of Energy Efficiency and Renewable Energy’s Vehicle Technologies Office and the DOE Office of Science’s Advanced Scientific Computing Research.
ORNL’s Mark Coletti led the development. ORNL’s Robert Patton and Quentin Haas also contributed to the development.