Siddha Ganju sidgan

#A Collection of NLP notes

##N-grams

###Calculating unigram probabilities:

P( w_i ) = count ( w_i ) ) / count ( total number of words )

In english..

Key-Value Memory Networks for Directly Reading Documents

Introduction

Knowledge Bases (KBs) are effective tools for Question Answering (QA) but are often too restrictive (due to fixed schema) and too sparse (due to limitations of Information Extraction (IE) systems).
The paper proposes Key-Value Memory Networks, a neural network architecture based on Memory Networks that can leverage both KBs and raw data for QA.
The paper also introduces MOVIEQA, a new QA dataset that can be answered by a perfect KB, by Wikipedia pages and by an imperfect KB obtained using IE techniques thereby allowing a comparison between systems using any of the three sources.
Link to the paper.

Related Work

tmux cheatsheet

As configured in my dotfiles.

start new:

tmux

start new with session name:

Mathematics: What do grad students in math do all day?

by Yasha Berchenko-Kogan

A lot of math grad school is reading books and papers and trying to understand what's going on. The difficulty is that reading math is not like reading a mystery thriller, and it's not even like reading a history book or a New York Times article.

The main issue is that, by the time you get to the frontiers of math, the words to describe the concepts don't really exist yet. Communicating these ideas is a bit like trying to explain a vacuum cleaner to someone who has never seen one, except you're only allowed to use words that are four letters long or shorter.

What can you say?

old rushed dashed squashed crushed smothered suffocated trapped rescued saved

	My problems with the paper:
	- There is no comparison of resulting video quality. The amount of encode time (and power
	expended) to produce a H.264 bit stream dramatically depends on the desired quality level;
	e.g. for x264 (state of the art SW encoder, already in 2010 when the paper was written), the
	difference between the fastest and best quality settings is close to 2 orders of magnitude
	in both speed and power use. This is not negligible!
	[NOTE: This is excluding quality-presets like "placebo", which are more demanding still.
	Even just comparing between different settings usable for real-time encoding, we still have
	at least an order of magnitude difference.]
	- They have their encoder, which is apparently based on JM 8.6 (not a good encoder!), for