deeplearning

Applications
- Google Image search
- Google speech recognition
- Google Street View number recognition

https://en.wikipedia.org/wiki/Deep_learning

ANNs: A brief history
=====================

Why is it important to draw a line to classify
- Separate male female

Perceptron - Frank Rosenblatt (1958)
Logical diagram - http://i.stack.imgur.com/1c2Mb.png
Hardware perceptron (wires and motors) - http://www.scaruffi.com/mind/ai/perceptr.jpg

Explain how the weights are tuned by saying adjust them using error from correct answer


XOR problem: The case of Linear (in)Separability - Marvin Minsky & Seymour Papert (1969)
http://ecee.colorado.edu/~ecen4831/lectures/xor2.gif
Use truth table.

Multi-layer & Backpropagation - (Too many. One of them is Geoff Hinton)
Multi-layer describe as just stacking perceptrons
http://matlabgeeks.com/wp-content/uploads/2011/06/Multi-layer-perceptron.png
Error is propagated down into hidden layers. Mention that it's the chain-rule from differentiation

- More complicated problem -> more computation -> more layers
- Still (practical) had problems when more than 2 layers (Vanishing gradients, exploding gradients)
- Funding to AI startups with shitty returns. AI winter.


ReNNaisance 2007-ish
Layer-wise pretraining
http://www.dmi.usherb.ca/~larocheh/images/deep_learning.jpg
Learns (using unlabelled data) what the data looks like in general.

Pretty much caught up.

The situation now...

Problem with explaining. "It's like the brain!"
===============================================
https://www.youtube.com/watch?v=LzE1fF0_3q0
https://www.youtube.com/watch?v=1OYrh86dV9s

2012
Andrew Ng & Jeff Dean
http://www.nytimes.com/2012/06/26/technology/in-a-big-network-of-computers-evidence-of-machine-learning.html?_r=0


Big Companies buying the deep learning guys.
============================================

https://lh3.googleusercontent.com/-KzORn341UDw/VIxtoa5RhXI/AAAAAAAAfLc/uIqb2i7uwA8/w2048-h1536/photo.JPG
Four Heavenly Kings

2013
Geoffrey Hinton
http://techcrunch.com/2013/06/12/how-googles-acquisition-of-dnnresearch-allowed-it-to-build-its-impressive-google-photo-search-in-6-months/

Yann LeCun
http://www.businessinsider.com/zuckerberg-hires-deep-learning-professor-2013-12?IR=T&

2014
http://techcrunch.com/2014/01/26/google-deepmind/

Andrew Ng
http://www.technologyreview.com/news/527301/chinese-search-giant-baidu-hires-man-behind-the-google-brain/

http://www.technologyreview.com/news/524026/is-google-cornering-the-market-on-deep-learning/

FUD FUD FUD
===========
http://rt.com/usa/222015-scientists-dangers-artificial-intelligence/
http://www.wired.com/2015/01/elon-musk-ai-safety/

Why this is bad? AI Winter.


1. Do PhD in Machine Learning (Deep Learning)
2. Set up a company
3. ???
4. Acqhire!