Wayne sigpro

A Tour of PyTorch Internals (Part I)

The fundamental unit in PyTorch is the Tensor. This post will serve as an overview for how we implement Tensors in PyTorch, such that the user can interact with it from the Python shell. In particular, we want to answer four main questions:

How does PyTorch extend the Python interpreter to define a Tensor type that can be manipulated from Python code?
How does PyTorch wrap the C libraries that actually define the Tensor's properties and methods?
How does PyTorch cwrap work to generate code for Tensor methods?
How does PyTorch's build system take all of these components to compile and generate a workable application?

Extending the Python Interpreter

PyTorch defines a new package torch. In this post we will consider the ._C module. This module is known as an "extension module" - a Python module written in C. Such modules allow us to define new built-in object types (e.g. the Tensor) and to call C/C++ functions.

Peter Thiel's CS183: Startup - Class 1 Notes Essay

Here is an essay version of my class notes from Class 1 of CS183: Startup. Errors and omissions are my own. Credit for good stuff is Peter’s entirely.

CS183: Startup—Notes Essay—The Challenge of the Future

Purpose and Preamble

	"""
	Author: Awni Hannun

	This is an example CTC decoder written in Python. The code is
	intended to be a simple example and is not designed to be
	especially efficient.

	The algorithm is a prefix beam search for a model trained
	with the CTC loss function.

	# Hello, and welcome to makefile basics.
	#
	# You will learn why `make` is so great, and why, despite its "weird" syntax,
	# it is actually a highly expressive, efficient, and powerful way to build
	# programs.
	#
	# Once you're done here, go to
	# http://www.gnu.org/software/make/manual/make.html
	# to learn SOOOO much more.

	#!/usr/bin/env python
	# -- coding: utf-8 --

	# This is a simplified implementation of the LSTM language model (by Graham Neubig)
	#
	# LSTM Neural Networks for Language Modeling
	# Martin Sundermeyer, Ralf Schlüter, Hermann Ney
	# InterSpeech 2012
	#
	# The structure of the model is extremely simple. At every time step we