Deep Narain Singh deepnarainsingh

TPU VM Cheetsheat

This TPU VM cheatsheet uses and was tested with the following library versions:

Library	Version
JAX	0.3.25
FLAX	0.6.4
Datasets	2.10.1
Transformers	4.27.1

What I Wish I'd Known About Equity Before Joining A Unicorn

Disclaimer: This piece is written anonymously. The names of a few particular companies are mentioned, but as common examples only.

This is a short write-up on things that I wish I'd known and considered before joining a private company (aka startup, aka unicorn in some cases). I'm not trying to make the case that you should never join a private company, but the power imbalance between founder and employee is extreme, and that potential candidates would

Speed Improvements and Compression for Deep Learning

Notes

Quantized Convolutional Neural Networks for Mobile Devices CVPR 2016
Deep SimNets CVPR 2016
Quantized Convolutional Neural Networks for Mobile Devices (2016)
- Quantization is applied to both convolutional and fully connected layers. This method has the advantage of accelerating the convolutional layers runtime, which is very important given that those are the most computationally expensive layers in CNNs. The disadvantage of this method is that it leads to a small loss in accuracy.
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size (2016)
Use 1x1 convolutions to reduce the number of convolution maps. I.e. if we have 10 traditional convolution filters this will generate 10 feature maps. If then we have N convolutions (where N < 10) the res

Comprehensions in Python the Jedi way

by Bjørn Friese

Beautiful is better than ugly. Explicit is better than implicit.

-- The Zen of Python

I frequently deal with collections of things in the programs I write. Collections of droids, jedis, planets, lightsabers, starfighters, etc. When programming in Python, these collections of things are usually represented as lists, sets and dictionaries. Oftentimes, what I want to do with collections is to transform them in various ways. Comprehensions is a powerful syntax for doing just that. I use them extensively, and it's one of the things that keep me coming back to Python. Let me show you a few examples of the incredible usefulness of comprehensions.

Code for Keras plays catch blog post

Train

python qlearn.py

Test

Generate figures

	"""Short and sweet LSTM implementation in Tensorflow.

	Motivation:
	When Tensorflow was released, adding RNNs was a bit of a hack - it required
	building separate graphs for every number of timesteps and was a bit obscure
	to use. Since then TF devs added things like `dynamic_rnn`, `scan` and `map_fn`.
	Currently the APIs are decent, but all the tutorials that I am aware of are not
	making the best use of the new APIs.

	Advantages of this implementation:

	""" Trains an agent with (stochastic) Policy Gradients on Pong. Uses OpenAI Gym. """
	import numpy as np
	import cPickle as pickle
	import gym

	# hyperparameters
	H = 200 # number of hidden layer neurons
	batch_size = 10 # every how many episodes to do a param update?
	learning_rate = 1e-4
	gamma = 0.99 # discount factor for reward