Style Image Prior

Style Generator Inversion for Image Enhancement and Animation
Aviv Gabbay and Yedid Hoshen

Inpainting

Corrupted	Ours	GT

Super-Resolution (128x128 to 1024x1024)

Bicubic	Ours	GT

Re-animation: Animating Obama from a video of Trump

Usage

Dependencies

• python >= 3.6
• numpy >= 1.15.4
• tensorflow-gpu >= 1.12.0
• keras >= 2.2.4
• opencv >= 3.4.4
• tqdm >= 4.28.1

Getting started

1. Clone the official StyleGAN repository.

2. Add the local StyleGAN project to PYTHONPATH.

   For bash users:

export PYTHONPATH=<path-to-stylegan-project>

Style Image Prior for Inpainting

Recovering missing parts of given images along with the respective latent codes can be done as follows:

inpainting.py --imgs-dir <input-imgs-dir> --masks-dir <output-masks-dir>
    --corruptions-dir <output-corruptions-dir> --restorations-dir <output-restorations-dir>
    --latents-dir <output-latents-dir>
    [--input-img-size INPUT_IMG_HEIGHT INPUT_IMG_WIDTH]
    [--perceptual-img-size EFFECTIVE_IMG_HEIGHT EFFECTIVE_IMG_WIDTH]
    [--mask-size MASK_HEIGHT MASK_WIDTH]
    [--learning-rate LEARNING_RATE]
    [--total-iterations TOTAL_ITERATIONS]

Style Image Prior for Super-Resolution

Performing super-resolution on given images can be done as follows:

super_resolution.py --lr-imgs-dir <input-imgs-dir> --hr-imgs-dir <output-imgs-dir>
    --latents-dir <output-latents-dir>
    [--lr-img-size LR_IMG_HEIGHT LR_IMG_WIDTH]
    [--hr-img-size HR_IMG_HEIGHT HR_IMG_WIDTH]
    [--learning-rate LEARNING_RATE]
    [--total-iterations TOTAL_ITERATIONS]

Note: StyleGAN inversion is very sensitive to the face alignment. The target face should be aligned exactly as done in the pipeline which CelebA-HQ was created by. You may use the alignment method implemented here: https://github.com/Puzer/stylegan-encoder/blob/master/align_images.py before applying any of the proposed image restoration methods.

Citing

If you find this project useful for your research, please cite

@article{gabbay2019styleimageprior,
  author    = {Aviv Gabbay and Yedid Hoshen},
  title     = {Style Generator Inversion for Image Enhancement and Animation},
  journal   = {arXiv preprint arXiv:1906.11880},
  year      = {2019}
}