Abstract
We present a nonlocal variational image completion technique which admits simultaneous inpainting of multiple structures and textures in a unified framework. The recovery of geometric structures is achieved by using general convolution operators as a measure of behavior within an image. These are combined with a nonlocal exemplar-based approach to exploit the self-similarity of an image in the selected feature domains and to ensure the inpainting of textures. We also introduce an anisotropic patch distance metric to allow for better control of the feature selection within an image and present a nonlocal energy functional based on this metric. Finally, we derive an optimization algorithm for the proposed variational model and examine its validity experimentally with various test images.
| Original language | English |
|---|---|
| Pages (from-to) | 2140-2168 |
| Number of pages | 29 |
| Journal | SIAM Journal on Imaging Sciences |
| Volume | 13 |
| Issue number | 4 |
| DOIs | |
| State | Published - 2020 |
Funding
The work of the authors was partially supported by the US Department of Energy, Office of Science, Early Career Research Program grant ERKJ314, the US Department of Energy, Office of Advanced Scientific Computing Research grants ERKJ331 and ERKJ345, the National Science Foundation, Division of Mathematical Sciences, Computational Mathematics Program grant DMS-1620280, and the Behavioral Reinforcement Learning Lab at Lirio LLC. This manuscript has been co-authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan). Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International, Inc., for the US DOE’s National Nuclear Security Administration under contract DE-NA0003525. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the US DOE or the United States Government. \ast Received by the editors February 12, 2020; accepted for publication (in revised form) September 23, 2020; published electronically December 1, 2020. https://doi.org/10.1137/20M1317864 Funding: The work of the authors was partially supported by the US Department of Energy, Office of Science, Early Career Research Program grant ERKJ314, the US Department of Energy, Office of Advanced Scientific Computing Research grants ERKJ331 and ERKJ345, the National Science Foundation, Division of Mathematical Sciences, Computational Mathematics Program grant DMS-1620280, and the Behavioral Reinforcement Learning Lab at Lirio LLC. This manuscript has been co-authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan). Sandia National Laboratories is a multimission laboratory managed and operated by National Technology \& Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International, Inc., for the US DOE's National Nuclear Security Administration under contract DE-NA0003525. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the US DOE or the United States Government.
| Funders | Funder number |
|---|---|
| Behavioral Reinforcement Learning Lab at Lirio LLC | DE-AC05-00OR22725 |
| DOE Public Access Plan | |
| National Science Foundation | |
| U.S. Department of Energy | |
| Division of Mathematical Sciences | DMS-1620280 |
| Office of Science | ERKJ314 |
| National Nuclear Security Administration | DE-NA0003525 |
| Advanced Scientific Computing Research | ERKJ345, ERKJ331 |
Keywords
- Image inpainting
- Nonlocal
- Variational