lec

a.md

prob

• https://www.youtube.com/playlist?list=PLUl4u3cNGP60A3XMwZ5sep719_nh95qOe

• https://www.youtube.com/playlist?list=PLEEF5322B331C1B98

DL

• https://www.youtube.com/watch?v=vT1JzLTH4G4&list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv

• https://www.youtube.com/watch?v=OQQ-W_63UgQ&list=PL3FW7Lu3i5Jsnh1rnUwq_TcylNr7EkRe6

dl.md

lec 5 CNN kernel

• the kernel is a certain pattern we are looking for slide

loss functions

• ReLu

• Does not saturate (in +region)
• Very computationally efficient
• Converges much faster than sigmoid/tanh in practice (e.g. 6x)
• Actually more biologically plausible than sigmoid
• Not zero-centered output
• An annoyance: hint: what is the gradient when x < 0? undefined at x = 0

• Leaky ReLU Does not saturate

• Computationally efficient
• Converges much faster than sigmoid/tanh in practice! (e.g. 6x)
• will not “die”

• Exponential Linear Units (ELU)

• All benefits of ReLU
• Closer to zero mean outputs
• Negative saturation regime compared with Leaky ReLU adds some robustness to noise

lec 3 loss function

** slide lec 3

• multi-class SVM loss sum(max(0, Sj - Syi + 1)), where Sj = prediction score, Syi = true score slide 14

NLP.md

lect 3 GLoVe

1/2 sum f(Pij)(ui * vj - log Pij)

• mangnitude of ui vj: eventually capture log count
• skip gram: capture co-ocurances one window at a time
• glove: capture the cont of the overall statistics of how often these words appear [45:00]

lec 4 Softmax, intro NN

d/df -log softmax(f_y) = [y - t] = δ slide 25

how to ignore certain files https://www.atlassian.com/git/tutorials/saving-changes/gitignore