Neural architecture search for deep image prior

 Kary Ho[1], Hailin Jin [2],  Andrew Gilbert [1], John Collomosse [1,2]
[1] University of Surrey,  [2] Adobe Research
In Journal of Computers and Graphics

Abstract

We present a neural architecture search (NAS) technique to enhance image denoising, inpainting, and super-resolution tasks under the recently proposed Deep Image Prior (DIP). We show that evolutionary search can automatically optimize the encoder-decoder (E-D) structure and meta-parameters of the DIP network, which serves as a content-specific prior to regularize these single image restoration tasks. Our binary representation encodes the design space for an asymmetric E-D network that typically converges to yield a content-specific DIP within 10–20 generations using a population size of 500. The optimized architectures consistently improve upon the visual quality of classical DIP for a diverse range of photo- graphic and artistic content.​
Architecture search space of NAS-DIP(-T). The Encoder-Decoder (E-D) network G (right) is formed of several E-D Units ( U n ) each with an Encoder E n ) and Decoder D n paired stage (zoomed, left) represented each by 7 bits plus an additional 4 Nbits R n to encode gated skip connections from E n to other decoder blocks in the network. Optionally the training epoch count T is encoded ( Section 3.1 ). Under DIP images are reconstructed from constant noise field Nby optimizing to find weights θthus overfitting the network to input image x under reconstruction loss e.g. here for denoising ( Eq. (3) ).

Paper

Neural architecture search for deep image prior, Kary Ho, Andrew Gilbert, Halin Jin, John Collomosse, In Proc Computer and Graphics, 2021

Video

​Citation

@inproceedings{Ho:CAG:2021,
AUTHOR = Ho, Kary and Gilbert, Andrew and Jin, Hailin and Collomosse, John",
TITLE = "Neural architecture search for deep image prior,",
BOOKTITLE = "In Proc Computer and Graphics",
YEAR = "2021",
​}