Peter Goldsborough goldsborough

Large Scale Distributed Deep Networks

Introduction

In machine learning, accuracy tends to increase with an increase in the number of training examples and number of model parameters.
For large data, training becomes slow on even GPU (due to increase CPU-GPU data transfer).
Solution: Distributed training and inference - DistBelief
Link to paper

DistBelief

Preface

This is an overview over the inner workings of Detexify. Extended knowlege in pattern recognition or machine learning is not necessary as I will explain some basics but understanding of linear algebra will definitely help. I have to note that I am not an expert, either. I more or less stumbled into this because of this project. Experts in this field may safely skip the first section.

	## Makefile to compile a typical, simple C++14 project.
	## Author : Bart Groeneveld
	## License : GPLv3
	## Latest update: Saturday, May 21st, 2016
	## You are probably only interested in the first line of this Makefile.

	# Libraries on which TARGET depends
	LIBS = \
	-lpthread
	# Name of the executable

	"""
	Minimal character-level Vanilla RNN model. Written by Andrej Karpathy (@karpathy)
	BSD License
	"""
	import numpy as np

	# data I/O
	data = open('input.txt', 'r').read() # should be simple plain text file
	chars = list(set(data))
	data_size, vocab_size = len(data), len(chars)

	/**
	* More info?
	* [email protected]
	* http://aspyct.org
	*
	* Hope it helps :)
	*/

	#include <stdio.h>
	#include <stdlib.h>

	/*
	author: jbenet
	os x, compile with: gcc -o testo test.c
	linux, compile with: gcc -o testo test.c -lrt
	*/

	#include <time.h>
	#include <sys/time.h>
	#include <stdio.h>

Peter Goldsborough goldsborough

Large Scale Distributed Deep Networks

Introduction

DistBelief

Preface

What's the problem?