PytorchRevelio is a collection of classes and functions that allow you to investigate MLP and convolutional networks written in Pytorch. These classes and functions enable you to visualize features that neurons and filters have learned or illustrate a saliency map for a given image. Even a Deep-Dream function is provided to have more fun. You can calculate visualization of learned features and saliency maps with several different methods. Although Some of these methods are better than others, we included weaker methods for educational purposes. For detail about different methods you can see this blog post Reveling what neural networks see and learn: PytorchRevelio
Keywords: Pytorch, MLP Neural Networks, Convolutional Neural Networks, Deep Learning, Visualization, Saliency Map, Guided Gradient
With PytorchRevelio you can investigate MLP and Convolutional neural networks that are written in Pytorch. There is no matter that you have writtern the network or it is a Pytorch build-in neural network such as VGG-19.
Download These Files and put them in your code directory:
- PytorchRevelio.py
- utilities_PytorchRevelio.py
- imagenet_labels Folder
In the following, we introduce provided methods and show their use cases with different examples.
Visualizing learned feature maps and illustrating saliency maps are highly sensitive to hyper-parameters. Therefore, make sure to choose a good set of hyper-parameters to obtain good results.
| Methods of PytorchRevelio | Reference (Paper, Webpage, etc) |
|---|---|
| activation_maximization(...) | Visualizing higher-layer features of a deep network |
| activation_maximization_with_gaussian_blurring(...) | Visualizing GoogLeNet Classes |
| activation_maximization_with_bilateral_blurring(...) | Class visualization with bilateral filters |
| saliency_map(...) | Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps |
| saliency_map_guided(...) | Striving for Simplicity: The All Convolutional Net |
| deep_dream(...) | Inceptionism: Going Deeper into Neural Networks |
| grad_cam(...) | Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization |
For detail about different methods you can see this blog post Reveling what neural networks see and learn: PytorchRevelio
- The higher methods in the list perform better for feature visualization:
- activation_maximization_with_bilateral_blurring
- activation_maximization_with_gaussian_blurring
- activation_maximization
- The higher methods in the list perform better for saliency map:
- grad_cam
- saliency_map_guided
- saliency_map
You can use the below text to cite PytorchRevelio.
Farhad Dalirani. (2021, May 23). farhad-dalirani/PytorchRevelio: PytorchRevelio-V1.1.0 (Version V1.1.0). Zenodo. http://doi.org/10.5281/zenodo.4782563
- Visualizing features of Alexnet with activation_maximization:
import matplotlib.pyplot as plt
import numpy as np
import torch.nn as nn
import torch
import torchvision
from torchvision import transforms
from PytorchRevelio import PytorchRevelio
from utilities_PytorchRevelio import imagenet_labels
if __name__ == '__main__':
# load pretrained Alexnet
alexnet_net = torchvision.models.alexnet(pretrained=True)
# choose GPU if it is available
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
print('Device: {}'.format(device))
# put network on device
alexnet_net.to(device)
# print name of modules
for key, value in PytorchRevelio.layers_name_type(alexnet_net):
print('+' * 10)
print(key)
print('-' * 10)
print(value)
# network transformer for input image
img_transformer = transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
# for different convolutional filter and neuron in fully connected layer
# show representation
first_layer_name = 'features.0'
last_layer_name = 'classifier.6'
for layer_name in alexnet_net.named_modules():
layer_name = layer_name[0]
# select convolutional and fully connected layers for visualization
layer = PytorchRevelio.return_module_by_name(network=alexnet_net, module_name=layer_name)
if isinstance(layer, nn.Conv2d):
filter_neuron_num = layer.out_channels
layer_type = 'Conv2d'
num_iter = 150
lr = 1
elif isinstance(layer, nn.Linear):
filter_neuron_num = layer.out_features
layer_type = 'Linear'
num_iter = 500
lr = 1
else:
continue
# from each layer select 8 filter our neurons
filters_neuron_indexs = np.random.choice([i for i in range(filter_neuron_num)], size=8)
# for each selected filter or neuron, calculate representation
plt.figure()
for i, filter_neuron_index in enumerate(filters_neuron_indexs):
img = PytorchRevelio.activation_maximization(network=alexnet_net, img_transformer=img_transformer,
in_img_size=(224, 224, 3),
first_layer_name=first_layer_name, layer_name=layer_name,
filter_or_neuron_index=filter_neuron_index, num_iter=num_iter,
lr=lr, device=device)
# to cpu and normalize for illustration purpose
img = PytorchRevelio.tensor_outputs_to_image(img)
# Illustrate
ax = plt.subplot(2, 4, i+1)
plt.imshow(img)
if layer_name != last_layer_name:
ax.set_title("{}".format(filter_neuron_index))
else:
ax.set_title("{}, {}".format(filter_neuron_index, imagenet_labels(class_number=filter_neuron_index)))
plt.suptitle('Layer Name: {}, Type: {}'.format(layer_name, layer_type))
ax.axis('off')
print('Processing of layer {}, filter/neuron {} is done.'.format(layer_name, filter_neuron_index))
plt.show()Some of the outputs:
- Visualizing features of VGG-11 with activation_maximization_with_gaussian_blurring:
import matplotlib.pyplot as plt
import numpy as np
import torch.nn as nn
import torch
import torchvision
from torchvision import transforms
from PytorchRevelio import PytorchRevelio
from utilities_PytorchRevelio import imagenet_labels
if __name__ == '__main__':
# load pretrained VGG11
vgg11_net = torchvision.models.vgg11(pretrained=True)
# choose GPU if it is available
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
print('Device: {}'.format(device))
# put network on device
vgg11_net.to(device)
# print name of modules
for key, value in vgg11_net.named_modules():
print('+' * 10)
print(key)
print('-' * 10)
print(value)
# network transformer for input image
img_transformer = transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
# for different convolutional filter and neuron in fully connected layer
# show representation
first_layer_name = 'features.0'
last_layer_name = 'classifier.6'
for layer_name in vgg11_net.named_modules():
layer_name = layer_name[0]
# select convolutional and fully connected layers for visualization
layer = PytorchRevelio.return_module_by_name(network=vgg11_net, module_name=layer_name)
if isinstance(layer, nn.Conv2d):
filter_neuron_num = layer.out_channels
layer_type = 'Conv2d'
num_iter = 450
lr = 0.09
start_sigma = 2.5
end_sigma = 0.5
elif isinstance(layer, nn.Linear):
filter_neuron_num = layer.out_features
layer_type = 'Linear'
num_iter = 450
lr = 0.09
start_sigma = 7.5,
end_sigma = 2.5,
else:
continue
# from each layer select 8 filter our neurons
filters_neuron_indexs = np.random.choice([i for i in range(filter_neuron_num)], size=8)
# for each selected filter or neuron, calculate representation
plt.figure()
for i, filter_neuron_index in enumerate(filters_neuron_indexs):
img = PytorchRevelio.activation_maximization_with_gaussian_blurring(
network=vgg11_net, img_transformer=img_transformer,
in_img_size=(224, 224, 3),
first_layer_name=first_layer_name,
layer_name=layer_name,
filter_or_neuron_index=filter_neuron_index,
num_iter=num_iter,
start_sigma=start_sigma,
end_sigma=end_sigma,
lr=lr,
device=device)
# to cpu and normalize for illustration purpose
img = PytorchRevelio.tensor_outputs_to_image(img)
# Illustrate
ax = plt.subplot(2, 4, i+1)
plt.imshow(img)
if layer_name != last_layer_name:
ax.set_title("{}".format(filter_neuron_index))
else:
ax.set_title("{}, {}".format(filter_neuron_index, imagenet_labels(class_number=filter_neuron_index)))
plt.suptitle('Layer Name: {}, Type: {}'.format(layer_name, layer_type))
ax.axis('off')
print('Processing of layer {}, filter/neuron {} is done.'.format(layer_name, filter_neuron_index))
plt.show()Some of the outputs:
- Visualizing features of VGG-11 with activation_maximization_with_bilateral_blurring:
import matplotlib.pyplot as plt
import numpy as np
import torch.nn as nn
import torch
import torchvision
from torchvision import transforms
from PytorchRevelio import PytorchRevelio
from utilities_PytorchRevelio import imagenet_labels
from PIL import Image
if __name__ == '__main__':
# load pretrained VGG11
vgg11_net = torchvision.models.vgg11(pretrained=True)
# choose GPU if it is available
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
print('Device: {}'.format(device))
# put network on device
vgg11_net.to(device)
# print name of modules
for key, value in vgg11_net.named_modules():
print('+' * 10)
print(key)
print('-' * 10)
print(value)
# network transformer for input image
img_size = (224, 224, 3)
img_transformer = transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
# for different convolutional filter and neuron in fully connected layer
# show representation
first_layer_name = 'features.0'
last_layer_name = 'classifier.6'
for layer_name in vgg11_net.named_modules():
layer_name = layer_name[0]
# select convolutional and fully connected layers for visualization
layer = PytorchRevelio.return_module_by_name(network=vgg11_net, module_name=layer_name)
if isinstance(layer, nn.Conv2d):
filter_neuron_num = layer.out_channels
layer_type = 'Conv2d'
num_iter = 300
lr = 0.09
start_sigma_color = 25
end_sigma_color = 110
start_sigma_space = 25
end_sigma_space = 110
kernel_size = 3
elif isinstance(layer, nn.Linear):
filter_neuron_num = layer.out_features
layer_type = 'Linear'
num_iter = 300
lr = 0.09
start_sigma_color = 25
end_sigma_color = 110
start_sigma_space = 25
end_sigma_space = 110
kernel_size = 3
else:
continue
# from each layer select 8 filter our neurons
filters_neuron_indexs = np.random.choice([i for i in range(filter_neuron_num)], size=8)
# for each selected filter or neuron, calculate representation
plt.figure()
for i, filter_neuron_index in enumerate(filters_neuron_indexs):
img = PytorchRevelio.activation_maximization_with_bilateral_blurring(
network=vgg11_net,
img_transformer=img_transformer,
in_img_size=img_size,
first_layer_name=first_layer_name,
layer_name=layer_name,
filter_or_neuron_index=filter_neuron_index,
num_iter=num_iter,
start_sigma_color=start_sigma_color,
end_sigma_color=end_sigma_color,
start_sigma_space=start_sigma_space,
end_sigma_space=end_sigma_space,
kernel_size=kernel_size,
lr=lr,
device=device)
# to cpu and normalize for illustration purpose
img = PytorchRevelio.tensor_outputs_to_image(img)
# Illustrate
ax = plt.subplot(2, 4, i+1)
plt.imshow(img)
if layer_name != last_layer_name:
ax.set_title("{}".format(filter_neuron_index))
else:
ax.set_title("{}, {}".format(filter_neuron_index, imagenet_labels(class_number=filter_neuron_index)))
plt.suptitle('Layer Name: {}, Type: {}'.format(layer_name, layer_type))
ax.axis('off')
print('Processing of layer {}, filter/neuron {} is done.'.format(layer_name, filter_neuron_index))
plt.show()Some of the outputs:
- Saliency map of ResNet-18 with saliency_map:
import matplotlib.pyplot as plt
import torch
import torchvision
from torchvision import transforms
from PIL import Image
from PytorchRevelio import PytorchRevelio
if __name__ == '__main__':
# load pretrained resnet18
resnet18_net = torchvision.models.resnet18(pretrained=True)
# choose GPU if it is available
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
print('Device: {}'.format(device))
# put network on device
resnet18_net.to(device)
# print name of modules
for key, value in resnet18_net.named_modules():
print('+' * 10)
print(key)
print('-' * 10)
print(value)
# network transformer for input image
img_transformer = transforms.Compose([
transforms.Resize((224, 224)),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
# for different convolutional filter and neuron in fully connected layer
# show representation
first_layer_name = 'conv1'
last_layer_name = 'fc'
for input_image_name, class_number in [("test_images/kit_fox_278_imagenet.jpg", 278),
("test_images/bald_eagle_22_imagenet.jpg", 22),
("test_images/peacock_imagenet_84.jpg", 84),
("test_images/tiger_imagenet_292.jpg", 292),
("test_images/toucan_imagenet_96.jpg", 96),
("test_images/cello_imagenet_486.jpg", 486)]:
# read input image
input_image = Image.open(input_image_name).convert('RGB')
network_input_shape = (input_image.size[0], input_image.size[1], 3)
gradients = PytorchRevelio.saliency_map(network=resnet18_net,
input_image=input_image,
class_number=class_number,
img_transformer=img_transformer,
first_layer_name=first_layer_name,
last_layer_name=last_layer_name,
device=device)
gradients = PytorchRevelio.tensor_outputs_to_image(gradients)
plt.figure()
plt.subplot(1, 2, 1)
plt.imshow(gradients)
plt.subplot(1, 2, 2)
plt.imshow(input_image.resize(size=(224, 224)))
plt.show()Some of the outputs:
- Saliency map of ResNet-50 with saliency_map_guided:
import matplotlib.pyplot as plt
import torch
import torchvision
from torchvision import transforms
from PIL import Image
from PytorchRevelio import PytorchRevelio
if __name__ == '__main__':
# load pretrained resnet50
resnet18_net = torchvision.models.resnet50(pretrained=True)
# choose GPU if it is available
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
device = 'cpu'
print('Device: {}'.format(device))
# put network on device
resnet18_net.to(device)
# print name of modules
for key, value in resnet18_net.named_modules():
print('+' * 10)
print(key)
print('-' * 10)
print(value)
# network transformer for input image
img_transformer = transforms.Compose([
transforms.Resize((224, 224)),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
network_input_shape = (224, 224, 3)
# for different convolutional filter and neuron in fully connected layer
# show representation
first_layer_name = 'conv1'
last_layer_name = 'fc'
for input_image_name, class_number in [("test_images/kit_fox_278_imagenet.jpg", 278),
("test_images/bald_eagle_22_imagenet.jpg", 22),
("test_images/peacock_imagenet_84.jpg", 84),
("test_images/tiger_imagenet_292.jpg", 292),
("test_images/toucan_imagenet_96.jpg", 96),
("test_images/cello_imagenet_486.jpg", 486)]:
# read input image
input_image = Image.open(input_image_name).convert('RGB')
gradients = PytorchRevelio.saliency_map_guided(network=resnet18_net,
input_image=input_image,
class_number=class_number,
img_transformer=img_transformer,
first_layer_name=first_layer_name,
device=device)
gradients = PytorchRevelio.tensor_outputs_to_image(gradients)
plt.figure()
plt.subplot(1, 2, 1)
plt.imshow(gradients)
plt.subplot(1, 2, 2)
plt.imshow(input_image.resize(size=(224, 224)))
plt.show()Some of the outputs:
- Saliency map of ResNet-50 with grad_cam:
import matplotlib.pyplot as plt
import torch
import torchvision
from torchvision import transforms
from PIL import Image
from PytorchRevelio import PytorchRevelio
if __name__ == '__main__':
# load pretrained resnet50
vgg11_net = torchvision.models.resnet50(pretrained=True)
# choose GPU if it is available
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
device = 'cpu'
print('Device: {}'.format(device))
# put network on device
vgg11_net.to(device)
# print name of modules
for key, value in vgg11_net.named_modules():
print('+' * 10)
print(key)
print('-' * 10)
print(value)
# network transformer for input image
img_transformer = transforms.Compose([
# transforms.Resize((224, 224)),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
# for different convolutional filter and neuron in fully connected layer
# show representation
first_layer_name = 'conv1'
last_layer_name = 'fc'
for input_image_name, class_number in [("test_images/cheta-zebra-293-340.jpg", 293),
("test_images/cheta-zebra-293-340.jpg", 340),
("test_images/vulture-lion-23-291.jpg", 23),
("test_images/vulture-lion-23-291.jpg", 291),
("test_images/Gorilla-Lion-Tiger-366-291-292.jpg", 366),
("test_images/Gorilla-Lion-Tiger-366-291-292.jpg", 291),
("test_images/Gorilla-Lion-Tiger-366-291-292.jpg", 292),
("test_images/bull_mastiff_tabbycat_243_281.png", 243),
("test_images/bull_mastiff_tabbycat_243_281.png", 281),
("test_images/hen_imagenet_8.jpg", 8),
]:
# read input image
input_image = Image.open(input_image_name).convert('RGB')
input_image_size = input_image.size
guided_saliency, sum_last_conv_features_map, grad_cam_gradients = PytorchRevelio.grad_cam(
network=vgg11_net,
input_image=input_image,
input_image_size=input_image_size,
class_number=class_number,
img_transformer=img_transformer,
first_layer_name=first_layer_name,
selected_conv_layer_name="layer4.2.conv3",
device=device)
#
grad_cam_gradients = PytorchRevelio.tensor_outputs_to_image(grad_cam_gradients)
#
sum_last_conv_features_map = PytorchRevelio.tensor_outputs_to_image(sum_last_conv_features_map)
#
guided_saliency = PytorchRevelio.tensor_outputs_to_image(guided_saliency)
plt.figure()
ax = plt.subplot(2, 2, 1)
plt.imshow(input_image)
ax.set_title("Input Image")
ax = plt.subplot(2, 2, 2)
plt.imshow(guided_saliency)
ax.set_title("Guided Saliency")
ax = plt.subplot(2, 2, 3)
plt.imshow(sum_last_conv_features_map)
ax.set_title("Grad Cam")
ax = plt.subplot(2, 2, 4)
plt.imshow(grad_cam_gradients)
ax.set_title("Guided Grad Cam")
plt.show()Some of the outputs:
There are a lot of ways to undestand what a neural network sees in images. Therefore, we will add more methods to this repositories in future.
There are a lot of different methods for opening the black box of deep neural networks. Please add new methods to the repository or fix the existing mistakes.