Adam shaboi

layout

title

description

Purpose

The Batch Normalization paper describes a method to address the various issues related to training of Deep Neural Networks. It makes normalization a part of the architecture itself and reports significant improvements in terms of the number of iterations required to train the network.

Issues With Training Deep Neural Networks

Internal Covariate shift

Covariate shift refers to the change in the input distribution to a learning system. In the case of deep networks, the input to each layer is affected by parameters in all the input layers. So even small changes to the network get amplified down the network. This leads to change in the input distribution to internal layers of the deep network and is known as internal covariate shift.

It is well established that networks converge faster if the inputs have been whitened (ie zero mean, unit variances) and are uncorrelated and internal covariate shift leads to just the opposite.

FWIW: I (@rondy) am not the creator of the content shared here, which is an excerpt from Edmond Lau's book. I simply copied and pasted it from another location and saved it as a personal note, before it gained popularity on news.ycombinator.com. Unfortunately, I cannot recall the exact origin of the original source, nor was I able to find the author's name, so I am can't provide the appropriate credits.

Effective Engineer - Notes

By Edmond Lau
Highly Recommended 👍
http://www.theeffectiveengineer.com/

What's an Effective Engineer?

Big List of Real Estate APIs

Listings / Property Data

####Rets Rabbit http://www.retsrabbit.com

Rets Rabbit removes the nightmare of importing thousands of real estate listings and photos from RETS or ListHub and gives you an easy to use import and Web API server so you can focus on building your listing search powered website or app.

	domain,name,keywords,description,twitter
	walmartstores.com,Wal-Mart Stores,"SEC filing,Walmart photos,walmart stock,sustainability index,Wal-Mart,walmart annual reports,Walmart responsibility,executive speeches,walmart suppliers,global responsibility,walmart global sustainability report,walmart investors,Walmart interactive map,walmart history,privacy policy,financial reports,walmart news,Wal Mart,walmart sustainability,Walmart locations,Walmart videos,walmart story,Walmart,Walmart stores,walmart board of directors,community giving,walmart careers,Walmart jobs,sam walton","Find Walmart executive speeches, financial reports, press releases, downloadable photos and videos, and see an interactive map of our locations around the world.",walmart
	gm.com,General Motors,,"General Motors is home to Buick, Cadillac, GMC and Chevrolet. Find the latest news about GM automotive innovations, investor relations and more. ",GM
	ge.com,General Electric,,,generalelectric
	chevron.com,ChevronTexaco,"cvx, chevrontexaco, cheveron,

	#! /usr/bin/env python
	import logging
	import datetime
	import fleming
	import pytz
	import os
	from itertools import islice
	import sys


	"""Persistence strategies comparison script.

	This script compute the speed, memory used and disk space used when dumping and
	loading arbitrary data. The data are taken among:
	- scikit-learn Labeled Faces in the Wild dataset (LFW)
	- a fully random numpy array with 10000x10000 shape
	- a dictionary with 1M random keys/values
	- a list containing 10M random value
	The compared persistence strategies are:

	import pandas as pd
	import itertools
	import time
	import multiprocessing
	from typing import Callable, Tuple, Union

	def groupby_parallel(
	groupby_df: pd.core.groupby.DataFrameGroupBy,
	func: Callable[[Tuple[str, pd.DataFrame]], Union[pd.DataFrame, pd.Series]],
	num_cpus: int = multiprocessing.cpu_count() - 1,

	import logging
	import os

	class Logger:

	def __init__(self, name):
	self.create_log_dir()
	self.logger = logging.getLogger('{}'.format(name))
	format = "%(asctime)s [%(levelname)s]: %(message)s"
	logging.basicConfig(format=format, level=logging.DEBUG)