[neural net jargons] #neural_net

conv_net.txt

depth = number of channel
path / kernel = [depth, convolutional sliding window weight]

sliding window weight = [height, width, depth], where depth equals to previous image channels

feature map_i = output[i, :, :] the ith layer of the output hyper image after convolution
stride = number of pixels when shifting the kernel window
valid_padding = stop slide at the edge (without padding)
same_padding = pad with zeros to make output image the same as input

bias in convolution: each kernel is associated with a bias (number of bias = number of output channels)

conv_net_dimension_computation.txt

Dimensionality
From what we've learned so far, how can we calculate the number of neurons of each layer in our CNN?

Given:

our input layer has a width of W and a height of H
our convolutional layer has a filter size F
we have a stride of S
a padding of P
and the number of filters K,
the following formula gives us the width of the next layer: W_out = (W−F+2P)/S+1.

The output height would be H_out = (H-F+2P)/S + 1.

And the output depth would be equal to the number of filters D_out = K.

The output volume would be W_out * H_out * D_out.

Knowing the dimensionality of each additional layer helps us understand how large our model is and how our decisions around filter size and stride affect the size of our network.

conv_tricks.txt

one by one convolution is equivalent to matrix pointwise multiplication

the functionality: cheap way to add depth and nonlinearities in conv operation

inception module: use simutanously 3x3->1x1, 5x5->1x1, 1x1 + pooling

depth concatinate all the outputs in the end

network_specs.txt

lennet:

conv1: num_filters = 6, kernel_size = 5, stride = 1

relu activation

pool_1: size = 2, stride = 2

conv2: num_filters = 16, kernel_size = 5, stride = 1

relu activate

pool_2: size=2, stride=2

flatten to features

fc_1: output_features = 120

relu activate

fc_2: output_features = 84

relu activate

fc_3 outpu_features = 10

softmax - xent

tensorflow_conv_net_parameter.txt

strides = [1, 2, 2, 1] means stride in [batch, input_height, input_width, input_channels]

conv kernel dim = [height, width, in_channel, out_channel]

pooling kernel dim: [batch, height, width, depth]  batch is alway 1, depth = 1 (as always), the output depth is unchanged