Note on caps net

text.txt

1) Input MNIST: [28 x 28 x 1] ( 1 color channel)
2) Kernel = [9x9]
3) Apply kernel to input 256 times:
(28 - 9 ) + 1 = 20
Output is 256 x 20 x 20 
4) Relu: 256 x 20 x 20
5) Apply kernel again with stride = 2:
((20 - 9) + 1) = 6
Output is 256 x 6 x 6
6) We’re going to cut the stack up into 32 decks with 8 cards each deck.
We can call this deck a "capsule layer"
Each capsule layer has 36 "capsules"
7) (32 x 6 x 6) is the total number of calsules, each capsule has the dimension of 8
==> (32 x 6 x 6) x 8
8) W_ij is the weight between a capsule i in the first layer and capsule j in the second layer, and each pair of ij shares the same weight
(1 x 8) x (8 x 16) = (1 x 16) is the dimension of each capsule j
We have (32 x 6 x 6) = 1152 capsules, means that we have 1152 x 16 capsules j
In capsule j, we have 10 classes ==> 1152 x 10 x 16
9) b_ij = (1152 x 10 x 1)
10) c_ij = softmax(b_ij , dim = 1)
11) s_i = c_ij x u_i = (1152 x 10 x 1) x (1152 x 10 x 16) = 1152 x 10 x 16