Sayantan Das ucalyptus2

Multi-node-training on slurm with PyTorch

What's this?

A simple note for how to start multi-node-training on slurm scheduler with PyTorch.
Useful especially when scheduler is too busy that you cannot get multiple GPUs allocated, or you need more than 4 GPUs for a single job.
Requirement: Have to use PyTorch DistributedDataParallel(DDP) for this purpose.
Warning: might need to re-factor your own code.
Warning: might be secretly condemned by your colleagues because using too many GPUs.

Code and Data for the Social Sciences: A Practitioner’s Guide, Gentzkow and Shapiro.
Good enough practices in scientific computing, Wilson et al.
Best Practices for Scientific Computing, Wilson et al.
Principled Data Processing, Patrick Ball.
The Plain Person’s Guide to Plain Text Social Science, Healy.
Avoiding technical debt in social science research, Toor.

For background on Technical Debt: Wikipedia; D. Sculley et al., [Machine Learning: The High-Interest Credit Card of Technical Debt](https://ai.google/resear

How to download streaming video

Streaming just means a download that they don't want you to keep. But Chrome's developer tools make it easy to access what's really going on under the hood.

Open Developer Tools

From the page where you want to download some things, go into your chrome menu to open the developer tools. You can either:

1.  (On a mac): Command-option-J
2. (On a PC): Control-alt-J

	# WSL2 network port forwarding script v1
	# for enable script, 'Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope CurrentUser' in Powershell,
	# for delete exist rules and ports use 'delete' as parameter, for show ports use 'list' as parameter.
	# written by Daehyuk Ahn, Aug-1-2020

	# Display all portproxy information
	If ($Args[0] -eq "list") {
	netsh interface portproxy show v4tov4;
	exit;
	}

	// If the script does not work, you may need to allow same site scripting https://stackoverflow.com/a/50902950

	Linkedin = {
	config: {
	scrollDelay: 3000,
	actionDelay: 5000,
	nextPageDelay: 5000,
	// set to -1 for no limit
	maxRequests: -1,
	totalRequestsSent: 0,

	import ruamel.yaml
	import argparse

	parser = argparse.ArgumentParser(description='(anaconda) environment.yml -> (pip) requirements.txt')
	parser.add_argument('environment_yml_file', type=str, metavar='environment.yml', default='environment.yml', help='the input file')
	parser.add_argument('-o', type=str, metavar='requirements.txt', default=None, help='the output file')
	args = parser.parse_args()
	basepath = args.environment_yml_file.replace("\\",'/').rsplit('/', maxsplit=1)[0]
	out_file = args.o
	if out_file == None:

	#!/bin/bash
	#
	# script to extract ImageNet dataset
	# ILSVRC2012_img_train.tar (about 138 GB)
	# ILSVRC2012_img_val.tar (about 6.3 GB)
	# make sure ILSVRC2012_img_train.tar & ILSVRC2012_img_val.tar in your current directory
	#
	# https://github.com/facebook/fb.resnet.torch/blob/master/INSTALL.md
	#
	# train/

	#Install Gource in Ubuntu
	========================
	#Go to the folder.... and
	#see http://tylerfrankenstein.com/code/install-gource-ubuntu-1010-visualize-git-repo

	sudo apt-get update
	sudo apt-get install libsdl2-dev libsdl2-image-dev libpcre3-dev libfreetype6-dev libglew-dev libglm-dev libboost-filesystem-dev libpng12-dev libsdl1.2-dev libsdl-image1.2-dev libtinyxml-dev
	./configure
	make
	sudo make install

	var INCR = 16; // Number of messages per page
	var SENT_URL = '//www.linkedin.com/inbox/invitations/sent?startRow=';
	var loMessageLinks = []; // Main array for all contacts

	var fetchMessages = function(i, cb) {

	console.log('Fetching message page #' + (i+1));
	$.get(SENT_URL + (i*INCR), function(data){
	var $dom = $(data);