eamartin

eamartin

Consider a binary classification problem with target t and output probability p.
Further, consider the batched version of this problem, with targets t_i and outputs p_i.
Assume independence between all elements in the batch.

The likelihood is
prod_i p_i^t_i * (1-p_i)^(1-t_i) = prod_i (p_i if t_i == 1 else (1-p_i))

Rather than maximizing the likelihood, we assume we can maximize a monotonic function of the likelihood, 
such as the log likelihood. If the problem was convex or we knew optimization would end at a global optima,
the maximizer for log likelihood would be the same as the maximizer for likelihood.

eamartin

I recently read https://arxiv.org/abs/1308.3432 , and want to make sure I'm understanding the 
straight-through gradient estimator correctly. In general, I'm interested in conditional computation 
and propagating gradients back through non-smooth functions (or discrete distributions).

My understanding:
Let HT(x) = int(x >= 0) be the hard threshold function. For forwards propagation, use the hard threshold 
function. For backwards propagation, replace all instances of HT(x) for some G(x) that has non-zero 
gradient in some set of measure > 0 and that approximates HT over the domain of x's. For instance, G 
can be identity function if x in [0, 1], or otherwise can be the sigmoid function.

eamartin

[emartin@eric-Thinkpad:~] virtualenv --no-site-packages qtile_bug
New python executable in qtile_bug/bin/python
Installing setuptools............done.
Installing pip...............done.

[emartin@eric-Thinkpad:~] source qtile_bug/bin/activate

(qtile_bug)[emartin@eric-Thinkpad:~] pip install xcffib
Downloading/unpacking xcffib
  Downloading xcffib-0.3.2.tar.gz (65kB): 65kB downloaded

eamartin

{
 "metadata": {
  "language": "Julia",
  "name": "",
  "signature": "sha256:52758898c2403685db8d857c76083099a23d7889718b8185d62e48903a88e488"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {

eamartin

def predict_expectation(aaf, covariates):
    sfs = aaf.predict_survival_function(covariates)
    xvals = sfs.index.values
    return simps(sfs.values, xvals, axis=0)

eamartin

FAIL tests/filesys/extended/dir-vine
pass tests/filesys/extended/grow-create
pass tests/filesys/extended/grow-dir-lg
pass tests/filesys/extended/grow-file-size
pass tests/filesys/extended/grow-root-lg
pass tests/filesys/extended/grow-root-sm
FAIL tests/filesys/extended/grow-seq-lg
pass tests/filesys/extended/grow-seq-sm
FAIL tests/filesys/extended/grow-sparse
pass tests/filesys/extended/grow-tell

eamartin

Looming motion
looming_1: http://www.hypnobeast.com/spirals/Double_Spiral_1280x1024.jpg
looming_2: http://static3.wikia.nocookie.net/__cb20130317010810/fallout/images/9/95/-pictures.4ever.eu-_optical_illusion,_spiral_158747.jpg

Binary disparity
bindis_1: http://www.indiana.edu/~p1013447/images/juleszrd.gif
bindis_2: http://www.cns.nyu.edu/~msl/3d.gif

Texture gradient
tg_1: http://www.cmap.polytechnique.fr/~maureen/vasarely3.jpg

eamartin

target remote localhost:1234
set architecture i8086
break *0x7c00
layout asm
layout reg
display/7i $pc - 4

eamartin

function householder!(x)
    x[1] = x[1] + sign(x[1]) .* norm(x)
    x ./= norm(x);
end

function tridiag_qr(T)
    Q = eye(size(T)...)
    R = copy(T)

    for i in 1:(size(R, 1) - 1)