Kevin Kwok antimatter15

Experiments with Reverse Mode Auto-Differentiation

Auto Differentiation is a technique used to calculate gradients of arbitrary computer programs. As opposed to symbolic differentiation, which occasionally results in an exponential blow-up in the size of the programs, and numerical differentiation, which estimates the gradient by running the target program dozens or hundreds of times, auto differentiation allows you to get out the gradient of a program after a single pass.

Reverse Mode Auto-Differentiation, especially in its imperative form has recently gained popularity due to projects like TF Eager, PyTorch, and HIPS Autograd. Existing auto differentiation libraries exploit operator overloading capabilities found in many languages to create data structures that incrementally track gradients.

Javascript lacks operator overloads, so defining special data structures loses much of its natural appeal. Rather than thinking about data structures, we can think about functions and how they compose, and how th

Keybase proof

I hereby claim:

I am antimatter15 on github.
I am kkwok (https://keybase.io/kkwok) on keybase.
I have a public key whose fingerprint is 1E98 6A49 07AC 5035 F965 9291 E1AC 1506 8D60 9685

To claim this, I am signing this object:

	thing
	= (list 1 2 3 4)
	= (1 . (2 . (3 . (4 . nil))))

	(car thing)
	= 1

	(cdr thing)
	= (2 . (3 . (4 . nil)))
	= cdr_thing

	// awaitable queue
	class AwaitableQueue {
	constructor(){
	this.queue = []
	this.resolvers = []
	}
	pop(){
	if(this.queue.length > 0) return Promise.resolve(this.queue.shift());
	return new Promise((resolve, reject) => this.resolvers.push(resolve) )
	}

	// set up a mock HTTP client/server API
	// should be pretty easy to replace this with
	// the real ones

	var endpoints = {}
	async function fetch(path, opts){
	let data = await endpoints[path](JSON.parse(opts.body))
	return { async json(){ return data } }
	}
	async function serve(path, fn){

	import java.util.Scanner;

	public class Add implements Comparable<Add>{
	private String name;
	private String lastName;
	private int number;


	public Add(String name, String lastName, int number) {
	this.name = name;

	import java.util.Scanner;

	public class Add implements Comparable<Add>{
	private String name;
	private String lastName;
	private int number;


	public Add(String name, String lastName, int number) {
	this.name = name;

	# Larger LSTM Network to Generate Text for Alice in Wonderland
	# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/include/

	import numpy
	from keras.models import Sequential
	from keras.layers import Dense
	from keras.layers import Dropout
	from keras.layers import LSTM, Activation
	from keras.callbacks import ModelCheckpoint, LambdaCallback, TensorBoard
	from keras.utils import np_utils

	test

	what

	merp