shagunsodhani

Introduction

The paper presents a framework and a set of synthetic toy tasks (classified into skill sets) for analyzing the performance of different machine learning algorithms.

Tasks

• Single/Two/Three Supporting Facts: Questions where a single(or multiple) supporting facts provide the answer. More is the number of supporting facts, tougher is the task.
• Two/Three Supporting Facts: Requires differentiation between objects and subjects.
• Yes/No Questions: True/False questions.
• Counting/List/Set Questions: Requires ability to count or list objects having a certain property.

shagunsodhani

Question Answering with Subgraph Embeddings

Introduction

• Open-domain Question Answering (Open QA) - efficiently querying large-scale knowledge base(KB) using natural language.
• Two main approaches:
  • Information Retrieval
    • Transform question (in natural language) into a valid query(in terms of KB) to get a broad set of candidate answers.
    • Perform fine-grained detection on candidate answers.
• Semantic Parsing

shagunsodhani

The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training

Introduction

• The paper explores the challenges involved in training deep networks, the effect of unsupervised pre-training on training process and visualizes the error function landscape for deep architectures.
• Link to the paper

Experiments

• Datasets used - Shapeset and MNIST.

shagunsodhani

Sequence to Sequence Learning with Neural Networks

Introduction

• The paper proposes a general and end-to-end approach for sequence learning that uses two deep LSTMs, one to map input sequence to vector space and another to map vector to the output sequence.
• For sequence learning, Deep Neural Networks (DNNs) requires the dimensionality of input and output sequences be known and fixed. This limitation is overcome by using the two LSTMs.
• Link to the paper

Model

shagunsodhani

Query Regression Networks for Machine Comprehension

Introduction

• Machine Comprehension (MC) - given a natural language sentence, answer a natural language question.
• End-To-End MC - can not use language resources like dependency parsers. The only supervision during training is the correct answer.
• Query Regression Network (QRN) - Variant of Recurrent Neural Network (RNN).
• Link to the paper

Related Work

shagunsodhani

Advances In Optimizing Recurrent Networks

Introduction

• Recurrent Neural Networks (RNNs) are very powerful at modelling sequences but they are not good at learning long-term dependencies.
• The paper discusses the reasons behind this difficulty and some suggestions to mitigate it.
• Link to the paper.

Optimization Difficulty

shagunsodhani

Key-Value Memory Networks for Directly Reading Documents

Introduction

• Knowledge Bases (KBs) are effective tools for Question Answering (QA) but are often too restrictive (due to fixed schema) and too sparse (due to limitations of Information Extraction (IE) systems).
• The paper proposes Key-Value Memory Networks, a neural network architecture based on Memory Networks that can leverage both KBs and raw data for QA.
• The paper also introduces MOVIEQA, a new QA dataset that can be answered by a perfect KB, by Wikipedia pages and by an imperfect KB obtained using IE techniques thereby allowing a comparison between systems using any of the three sources.
• Link to the paper.

Related Work

shagunsodhani

A Neural Conversational Model

Introduction

• The paper presents a domain agnostic approach for conversational modelling based on Sequence to Sequence Learning Framework.
• Link to the paper

Model

• Neural Conversational Model (NCM)

shagunsodhani

Deep Math: Deep Sequence Models for Premise Selection

Introduction

• Automated Theorem Proving (ATP) - Attempting to prove mathematical theorems automatically.
• Bottlenecks in ATP:
  • Autoformalization - Semantic or formal parsing of informal proofs.
  • Automated Reasoning - Reasoning about already formalised proofs.
• Paper evaluates the effectiveness of neural sequence models for premise selection (related to automated reasoning) without using hand engineered features.
• Link to the paper

shagunsodhani

Recurrent Neural Network Regularization

Introduction

• The paper explains how to apply dropout to LSTMs and how it could reduce overfitting in tasks like language modelling, speech recognition, image caption generation and machine translation.
• Link to the paper

Dropout

• Regularisation method that drops out (or temporarily removes) units in a neural network.