CNN Visualization and Explanation

This project aims to visualize filters, feature maps, guided backpropagation from any convolutional layers of all pre-trained models on ImageNet available in tf.keras.applications (TF 2.3). This will help you observe how filters and feature maps change through each convolution layer from input to output.

With any image uploaded, you can also make the classification with any of the above models and generate GradCAM, Guided-GradCAM to see the important features based on which the model makes its decision.

If "art" is in your blood, you can use any model to generate hallucination-like visuals from your original images. For this feature, personally, I highly recommend trying with "InceptionV3" model as the deep-dream images generated from this model are appealing.

With the current version, there are 26 pre-trained models.

How to use

Run with your resource

• Clone this repo:

git clone https://github.com/nguyenhoa93/cnn-visualization-keras-tf2
cd cnn-visualization-keras-tf2

• Create virtualev:

conda create -n cnn-vis python=3.6
conda activate cnn-vs
bash requirements.txt

• Run demo with the file visualization.ipynb

Run on Google Colab

• Go to this link: https://colab.research.google.com/github/nguyenhoa93/cnn-visualization-keras-tf2/blob/master/visualization.ipynb
• Change your runtime type to Python3
• Choose GPU as your hardware accelerator.
• Run the code.

Voila! You got it.

Briefs

Method	Brief
Filter visualization	Simply plot the learned filters. * Step 1: Find a convolutional layer. * Step 2: Get weights at a convolution layer, they are filters at this layer. * Step 3: Plot filter with the values from step 2. This method does not require an input image. VGG-16, block1_conv1
Feature map visualization	Plot the feature maps obtained when fitting an image to the network. * Step 1: Find a convolutional layer. * Step 2: Build a feature model from the input up to that convolutional layer. * Step 3: Fit the image to the feature model to get feature maps. * Step 4: Plot the feature map. VGG-16, block1_conv1 VGG-16, block5_conv3
Guided Backpropagation	Backpropagate from a particular convolution layer to input image with modificaton of the gradient of ReLU. VGG-16: block1_conv1 & block5_conv3
GradCAM	* Step 1: Determine the last convolutional layer * Step 2: Perform gradient from `pre-softmax` layer to last convolutional layer and the apply global average pooling to obtain weights for neurons' importance. * Step 3: Linearly combinate feature map of last convolutional layer and weights, then apply ReLu on that linear combination. InceptionV3, explanation for "lakeside" class
Guided-GradCAM	* Step 1: Calculate guided backpropagation from last convolutional layer to input. * Step 2: Upsample GradCAM to the size of input * Step 3: Apply element-wise multiplication of guided backpropagation and GradCAM InceptionV3, explanation for "boathouse" class
Deep Dream	See more in this excellent tutorial from François Chollet: https://keras.io/examples/generative/deep_dream/ InceptionV3
Original image

References

1. How to Visualize Filters and Feature Maps in Convolutional Neural Networks by Machine Learning Mastery
2. Pytorch CNN visualzaton by utkuozbulak: https://github.com/utkuozbulak
3. CNN visualization with TF 1.3 by conan7882: https://github.com/conan7882/CNN-Visualization
4. Deep Dream Tutorial from François Chollet: https://keras.io/examples/generative/deep_dream/