Skip to content

Instantly share code, notes, and snippets.

View sabyasachis's full-sized avatar
🐢
Overfitting to unseen data

Sabyasachi Sahoo sabyasachis

🐢
Overfitting to unseen data
View GitHub Profile
@sabyasachis
sabyasachis / small_large_filter_compare.csv
Last active January 25, 2020 07:41
Here are few things to consider while choosing the optimal size
Smaller Filter Sizes Larger Filter Sizes
It has a smaller receptive field as it looks at very few pixels at once. Larger receptive field per layer.
Highly local features extracted without much image overview. Quite generic features extracted spread across the image.
Therefore captures smaller, complex features in the image. Therefore captures the basic components in the image.
Amount of information extracted will be vast, maybe useful in later layers. Amount of information extracted are considerably lesser.
Slow reduction in the image dimension can make the network deep Fast reduction in the image dimension makes the network shallow
Better weight sharing Poorer weight sharing
In an extreme scenario, using a 1x1 convolution is like treating each pixel as a useful feature. Using a image sized filter is equivalent to a fully connected layer.
@sabyasachis
sabyasachis / small_large_filter_size_example_compare.csv
Created November 15, 2018 17:46
convolution example with small (3x3) and large filter sizes (5x5)
Smaller Filter Sizes Larger Filter Sizes
Two 3x3 kernels result in an image size reduction by 4 one 5x5 kernel results in same reduction.
We have used (3x3 + 3x3) = 18 weights. We used (5x5) = 25 weights.
So, we get lower no. of weights but more layers. Higher number of weights but lesser layers.
Therefore, computationally efficient. And, this is computationally expensive.
With more layers, it learns complex, more non-linear features. With less layers, it learns simpler non linear features.
With more layers, it necessitates the need for larger memory. And, it will use less memory for backpropogation.