Subhaditya Mukherjee SubhadityaMukherjee

dataset = dset.ImageFolder(root=dataroot,
                           transform=transforms.Compose([
                               transforms.Resize(image_size),
                               transforms.CenterCrop(image_size),
                               transforms.ToTensor(),
                               transforms.Normalize((.5, .5, .5),
                                                    (.5, .5, .5)),
                           ]))
dataloader = torch.utils.data.DataLoader(dataset,

dataroot = "art/"
workers = 10
batch_size = 128
image_size = 64
nc = 3
nz = 100
ngf = 64
ndf = 64

pip3 install matplotlib torch torchvision numpy pillow

import os
import random
import matplotlib.animation as animation
import matplotlib.pyplot as plt
import numpy as np
import torch
import torch.backends.cudnn as cudnn
import torch.nn as nn
import torch.nn.parallel

run_deep_dream_simple

We have to first pre process the image to be able to work with the pre processed network. (Basically converts it into the format inception was trained)
We then convert it into a tensor and split the execution into parts
We simply use the deepdream function to calculate the losses
We use the deprocess function to convert it back into an image from a tensor
clear_output is just a function to make sure the notebook doesnt get flooded with outputs and only the latest output is kept

def run_deep_dream_simple(img, steps=100, step_size=0.01):

    img = tf.keras.applications.inception_v3.preprocess_input(img)

main class simple deepdream

the @tf.function allows the function to be precompiled. Since it is compiled, it runs faster
tensorspec basically allows us to pre define the shapes of specific arrays as we are pre compiling it

call

here we are trying to find the gradients of the image
this method is called gradient ascent. This adds the gradients found in every layer to the image and thus increases the activations at that point as well which is what we want
GradientTape allows us to keep a sort of history of all the gradients and allows us to use it to calculate loss directly from the history
After we get the gradients, we normalize them
img = img + gradients * step_size is the main ascent function which maximizes the loss

main class simple deepdream

the @tf.function allows the function to be precompiled. Since it is compiled, it runs faster
tensorspec basically allows us to pre define the shapes of specific arrays as we are pre compiling it

call

here we are trying to find the gradients of the image
this method is called gradient ascent. This adds the gradients found in every layer to the image and thus increases the activations at that point as well which is what we want
GradientTape allows us to keep a sort of history of all the gradients and allows us to use it to calculate loss directly from the history
After we get the gradients, we normalize them
img = img + gradients * step_size is the main ascent function which maximizes the loss

calc_loss

We take the image and the model as inputs
expand dims basically adds an extra dimension to our input along the x axis to make it work with inception
for every activation in our layers, we calculate the loss and append it to a list
reduce_mean() and reduce_sum() are approximately the mean and sum equivalent for tensors instead of just plain arrays
Thus the sum is the total loss we get

def calc_loss(img, model):
 img_batch = tf.expand_dims(img, axis=0)

Pre trained

We use Inception Net v3 which is a pretrained network that already has some idea of the world.
We use imagenet weights which basically allows us to use transfer learning on the network
Instead of training from scratch we can just cherry pick layers and use our neural network on it

base_model = tf.keras.applications.InceptionV3(include_top=False,
                                               weights='imagenet')

Subhaditya Mukherjee SubhadityaMukherjee

run_deep_dream_simple

main class simple deepdream

call

main class simple deepdream

call

calc_loss

Pre trained

Model