Enabling Long-range Exploration in Minimization of Multimodal Functions

Research output: Contribution to conferencePaperpeer-review

3 Scopus citations

Abstract

We consider the problem of minimizing multimodal loss functions with a large number of local optima. Since the local gradient points to the direction of the steepest slope in an infinitesimal neighborhood, an optimizer guided by the local gradient is often trapped in a local minimum. To address this issue, we develop a novel nonlocal gradient to skip small local minima by capturing major structures of the loss's landscape in blackbox optimization. The nonlocal gradient is defined by a directional Gaussian smoothing (DGS) approach. The key idea of DGS is to conducts 1D long-range exploration with a large smoothing radius along d orthogonal directions in Rd, each of which defines a nonlocal directional derivative as a 1D integral. Such long-range exploration enables the nonlocal gradient to skip small local minima. The d directional derivatives are then assembled to form the nonlocal gradient. We use the Gauss-Hermite quadrature rule to approximate the d 1D integrals to obtain an accurate estimator. The superior performance of our method is demonstrated in three sets of examples, including benchmark functions for global optimization, and two real-world scientific problems.

Original languageEnglish
Pages1639-1649
Number of pages11
StatePublished - 2021
Event37th Conference on Uncertainty in Artificial Intelligence, UAI 2021 - Virtual, Online
Duration: Jul 27 2021Jul 30 2021

Conference

Conference37th Conference on Uncertainty in Artificial Intelligence, UAI 2021
CityVirtual, Online
Period07/27/2107/30/21

Funding

This work was supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Applied Mathematics program under the contract ERJK359; and by the Artificial Intelligence Initiative at the Oak Ridge National Laboratory (ORNL). This research used resources of the Compute and Data Environment for Science at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

FundersFunder number
U.S. Department of Energy
Office of Science
Advanced Scientific Computing ResearchERJK359
Oak Ridge National LaboratoryDE-AC05-00OR22725

    Fingerprint

    Dive into the research topics of 'Enabling Long-range Exploration in Minimization of Multimodal Functions'. Together they form a unique fingerprint.

    Cite this