Measuring the Biases and Effectiveness of Content-Style Disentanglement

This repository contains the official implementation of the Distance Correlation (DC) and Information Over Bias (IOB) metrics proposed in https://arxiv.org/abs/2008.12378 (to be presented in BMVC 2021). The two metrics can be used to assess the level of disentanglement between spatial content and vector style representations. Both metrics are ready to use with PyTorch implementation.

The repository is created by Xiao Liu*, Spyridon Thermos*, Gabriele Valvano*, Agisilaos Chartsias, Alison O'Neil, and Sotirios A. Tsaftaris, as a result of the collaboration between The University of Edinburgh and Canon Medical Systems Europe, supported by the Royal Academy of Engineering.

System Requirements

• Pytorch 1.5.1 or higher with GPU support
• Python 3.7.2 or higher
• SciPy 1.5.2 or higher
• CUDA toolkit 10 or newer

Note: you can also use PyTorch without GPU support to run the metrics, while it will take much more time.

Metric 1: Distance Correlation (DC) - Independence

To compute the DC between extracted:

• content tensors (C) and style vectors (S) - DC(C,S)
• content tensors (C) and input image tensors (I) - DC(I,C)
• style vectors (S) and input image tensors (I) - DC(I,S)

run the following command:

python compute_DC.py --root <path/to/extracted/tensors/and/vectors> --save </name/of/result/file>

Metric 2: Information Over Bias (IOB) - Informativeness

To compute the IOB between extracted:

• content tensors (C) and input image tensors (I) - IOB(I,C)
• style vectors (S) and input image tensors (I) - IOB(I,S)

run the following command:

python compute_IOB.py --root <path/to/extracted/tensors/and/vectors> --gpu <gpu_id> --save </name/of/result/file>

Applications

The two metrics have been tested on three popular models that exploit content-style disentanglement in the context of image-to-image translation, medical image segmentation, and pose estimation.

• MUNIT - official github implementation, paper
• SDNet - official github implementation, paper
• PANet - official github implementation, paper

To test a pre-trained model with our metrics, the image and content tensors as well as the style vectors need to prepared. We provide an example script to show how to extract the tensors and vectors and further save them with required format in usage/example_extract.py.

Citation

If you find our metrics useful please cite the following paper:

@inproceedings{liu2020metrics,
  author       = "Xiao Liu and Spyridon Thermos and Gabriele Valvano and Agisilaos Chartsias and Alison O'Neil and Sotirios A. Tsaftaris",
  title        = "Measuring the Biases and Effectiveness of Content-Style Disentanglement",
  booktitle    = "Proc. of the British Machine Vision Conference (BMVC)",
  year         = "2021"
}

License

All scripts are released under the MIT License.