deep-learning.md

#Deep Learning Model

Review

• Zachary C. Lipton, John Berkowitz and Charles Elkan, A Critical Review of Recurrent Neural Networks for Sequence Learning

Deep Architecture

• Rupesh Kumar Srivastava, Klaus Greff and Jurgen Schmidhuber, Highway Networks, 2015.
• Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun, Deep Residual Learning for Image Recognition, 2015
• Wojciech Zaremba, Ilya Sutskever and Oriol Vinyals, Recurrent Neural Network Regularization, 2014

Theoretical Analysis

• Guido Montufar, Razvan Pascanu, Kyunghyun Cho and Yoshua Bengio, On the Number of Linear Regions of Deep Neural Networks, NIPS 2014

Convolutional Neuarl Net For Graphs

• Joan Bruna, Wojciech Zaremba, Arthur Szlam and Yann LeCun, Spectral Networks and Deep Locally Connected Networks on Graphs, ICPR 2014
• Mathias Niepert, Mohamed Ahmed and Konstantin Kutzkov, Learning Convolutional Neural Networks for Graphs, Arxiv, 2016. pdf, slide

Attention & Memory for Recurrent Neural Net

Neural Turing Machine (External Memory)

• A. Graves, G. Wayne and I. Danihelka, Neural Turing Machines, CoRR 2014.
• L. Kaiser and I. Sutskever, Neural GPUs Learn Algorithms, CoRR 2015.
• W. Zaremba and I. Sutskever, Reinforcement Learning Neural Turing Machines, CoRR 2015.
• K. Kurach, M. Andrychowicz and I. Sutskever, Neural Random-Access Machines, CoRR 2015.
• E. Grefenstette, K.M. Hermann, M. Suleyman and P. Blunsom, Learning to Transduce with Unbounded Memory, NIPS 2015.
• A. Joulin and T. Mikolov, Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets, NIPS 2015.
• J. Weston, S. Chopra and A. Bordes, Memory Networks, CoRR 2014.
• A. Kumar, O. Irsoy, J. Su, J. Bradbury, R. English, B. Pierce, P. Ondruska, I. Gulrajani and R. Socher, Ask Me Anything: Dynamic Memory Networks for Natural Language Processing, CoRR 2015.
• A. Graves, G. Wayne, M. Reynolds, T. Harley, I. Danihelka, A. Grabska-Barwińska, S.G. Colmenarejo, E. Grefenstette, T. Ramalho, J. Agapiou, A.P. Badia, K.M. Hermann, Y. Zwols, G. Ostrovski, A. Cain, H. King, C. Summerfield, P. Blunsom, K. Kavukcuoglu and D. Hassabis, Hybrid computing using a neural network with dynamic external memory, Nature 2016.

Attentional Mechanism

• Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio, Neural Machine Translation by Jointly Learning to Align and Translate, CoRR 2014.
• Minh-Thang Luong Hieu Pham Christopher D. Manning, Effective Approaches to Attention-based Neural Machine Translation, EMNLP 2015.
• Li, Jiwei, Minh-Thang Luong, and Dan Jurafsky. "A hierarchical neural autoencoder for paragraphs and documents." arXiv preprint arXiv:1506.01057 (2015).
• Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiský, Phil Blunsom, Reasoning about Entailment with Neural Attention, CoRR 2016.

Deep Alternatives

• P. Kontschieder, M. Fiterau, A. Criminisi, and S. Rota Bulo'. Deep Neural Decision Forests. ICCV 2015.
• Zhi-hua Zhou, and Ji Feng, Deep Forest: Towards An Alternative to Deep Neural Networks, CoRR, 2017.