Skip to content

Instantly share code, notes, and snippets.

View jacobkahn's full-sized avatar

Jacob Kahn jacobkahn

426 followers · 1 following
View GitHub Profile
@jacobkahn
jacobkahn / gist:33f7c6731d653d4b35ae096e856099f8
Created July 23, 2019 23:35
NCCL Tests on 4 node p3dn.24xlarge + ethernet - all_reduce_perf -b 8 -e 1G -f 2 -g 1 -c 1 -n 100
#
# out-of-place in-place
# size count type redop time algbw busbw error time algbw busbw error
# (B) (elements) (us) (GB/s) (GB/s) (us) (GB/s) (GB/s)
8 2 float sum 218.6 0.00 0.00 2e-07 217.6 0.00 0.00 2e-07
16 4 float sum 218.3 0.00 0.00 0e+00 218.7 0.00 0.00 1e-07
32 8 float sum 217.1 0.00 0.00 1e-07 216.9 0.00 0.00 2e-07
64 16 float sum 218.6 0.00 0.00 2e-07 217.6 0.00 0.00 2e-07
128 32 float sum 220.0 0.00 0.00 2e-07 220.9 0.00 0.00 2e-07
256 64 float sum 223.9 0.00 0.00 2e-07 222.9 0.00 0.00 2e-07
@jacobkahn
jacobkahn / gist:25e87a2706e73647dd4d222e2bf5b354
Created July 23, 2019 23:25
NCCL Tests on 8 node p3dn.24xlarge + ethernet - all_reduce_perf -b 8 -e 1G -f 2 -g 1 -c 1 -n 100
#
# out-of-place in-place
# size count type redop time algbw busbw error time algbw busbw error
# (B) (elements) (us) (GB/s) (GB/s) (us) (GB/s) (GB/s)
8 2 float sum 440.1 0.00 0.00 2e-07 442.8 0.00 0.00 2e-07
16 4 float sum 444.4 0.00 0.00 0e+00 440.2 0.00 0.00 2e-07
32 8 float sum 442.1 0.00 0.00 2e-07 443.7 0.00 0.00 2e-07
64 16 float sum 440.8 0.00 0.00 2e-07 441.3 0.00 0.00 2e-07
128 32 float sum 445.8 0.00 0.00 2e-07 444.7 0.00 0.00 2e-07
256 64 float sum 450.6 0.00 0.00 2e-07 448.1 0.00 0.00 2e-07
@jacobkahn
jacobkahn / gist:3703e3bbfe44343cd8ef033aed6093c9
Created July 23, 2019 23:17
NCCL Tests on 16 node p3dn.24xlarge + ethernet - all_reduce_perf -b 8 -e 1G -f 2 -g 1 -c 1 -n 100
#
# out-of-place in-place
# size count type redop time algbw busbw error time algbw busbw error
# (B) (elements) (us) (GB/s) (GB/s) (us) (GB/s) (GB/s)
8 2 float sum 878.1 0.00 0.00 1e-06 876.8 0.00 0.00 2e-07
16 4 float sum 880.0 0.00 0.00 0e+00 875.6 0.00 0.00 2e-07
32 8 float sum 877.1 0.00 0.00 2e-07 879.3 0.00 0.00 5e-07
64 16 float sum 879.7 0.00 0.00 5e-07 882.2 0.00 0.00 5e-07
128 32 float sum 889.2 0.00 0.00 5e-07 889.8 0.00 0.00 5e-07
256 64 float sum 892.5 0.00 0.00 5e-07 896.4 0.00 0.00 5e-07
@jacobkahn
jacobkahn / gist:0ae35590b5fc5bc49acad48fb4f11315
Created July 23, 2019 22:37
NCCL Tests on 16 node p3dn.24xlarge + EFA - all_reduce_perf -b 8 -e 1G -f 2 -g 1 -c 1 -n 100
#
# out-of-place in-place
# size count type redop time algbw busbw error time algbw busbw error
# (B) (elements) (us) (GB/s) (GB/s) (us) (GB/s) (GB/s)
8 2 float sum 1238.6 0.00 0.00 1e-06 1241.8 0.00 0.00 0e+00
16 4 float sum 1406.9 0.00 0.00 2e-07 1246.5 0.00 0.00 2e-07
32 8 float sum 1235.6 0.00 0.00 2e-07 1241.7 0.00 0.00 2e-07
64 16 float sum 1242.5 0.00 0.00 5e-07 1244.2 0.00 0.00 5e-07
128 32 float sum 1248.3 0.00 0.00 5e-07 1241.6 0.00 0.00 5e-07
256 64 float sum 1243.4 0.00 0.00 5e-07 1244.1 0.00 0.00 5e-07
@jacobkahn
jacobkahn / gist:3150a1a38abba40f58d8263d9c55969f
Created July 23, 2019 22:03
NCCL Tests on 8 node p3dn.24xlarge + EFA - all_reduce_perf -b 8 -e 1G -f 2 -g 1 -c 1 -n 100
#
# out-of-place in-place
# size count type redop time algbw busbw error time algbw busbw error
# (B) (elements) (us) (GB/s) (GB/s) (us) (GB/s) (GB/s)
8 2 float sum 631.3 0.00 0.00 2e-07 632.3 0.00 0.00 0e+00
16 4 float sum 686.3 0.00 0.00 2e-07 631.1 0.00 0.00 2e-07
32 8 float sum 627.5 0.00 0.00 2e-07 628.7 0.00 0.00 2e-07
64 16 float sum 632.0 0.00 0.00 2e-07 629.6 0.00 0.00 1e-07
128 32 float sum 629.1 0.00 0.00 2e-07 628.8 0.00 0.00 2e-07
256 64 float sum 628.2 0.00 0.00 2e-07 628.3 0.00 0.00 2e-07
@jacobkahn
jacobkahn / gist:1fb3977e99056b7935a6c63dda1e145f
Created July 23, 2019 22:02
NCCL Tests on 4 node p3dn.24xlarge + EFA - all_reduce_perf -b 8 -e 1G -f 2 -g 1 -c 1 -n 100
#
# out-of-place in-place
# size count type redop time algbw busbw error time algbw busbw error
# (B) (elements) (us) (GB/s) (GB/s) (us) (GB/s) (GB/s)
8 2 float sum 319.4 0.00 0.00 2e-07 319.2 0.00 0.00 0e+00
16 4 float sum 320.9 0.00 0.00 1e-07 319.0 0.00 0.00 1e-07
32 8 float sum 317.2 0.00 0.00 1e-07 318.1 0.00 0.00 1e-07
64 16 float sum 321.3 0.00 0.00 1e-07 318.4 0.00 0.00 1e-07
128 32 float sum 319.6 0.00 0.00 1e-07 319.4 0.00 0.00 1e-07
256 64 float sum 318.1 0.00 0.00 1e-07 318.9 0.00 0.00 1e-07
@jacobkahn
jacobkahn / gist:a92f4425abf831ac0799fabe9079263d
Last active July 23, 2019 22:01
NCCL Tests on 2 node p3dn.24xlarge + EFA - all_reduce_perf -b 8 -e 1G -f 2 -g 1 -c 1 -n 100
#
# out-of-place in-place
# size count type redop time algbw busbw error time algbw busbw error
# (B) (elements) (us) (GB/s) (GB/s) (us) (GB/s) (GB/s)
8 2 float sum 163.4 0.00 0.00 2e-07 164.4 0.00 0.00 1e-07
16 4 float sum 162.9 0.00 0.00 1e-07 162.3 0.00 0.00 1e-07
32 8 float sum 164.1 0.00 0.00 1e-07 164.0 0.00 0.00 1e-07
64 16 float sum 163.6 0.00 0.00 1e-07 163.3 0.00 0.00 1e-07
128 32 float sum 164.1 0.00 0.00 1e-07 161.6 0.00 0.00 1e-07
256 64 float sum 162.5 0.00 0.00 1e-07 163.0 0.00 0.00 1e-07
@jacobkahn
jacobkahn / examParsing.py
Created March 5, 2019 22:04
CIS 320 Exam Parser
import uuid
from pyPdf import PdfFileWriter, PdfFileReader
for qq in range(1, 10):
inputpdf = PdfFileReader(open("in" + str(qq) + ".pdf", "rb"))
j = -1
for i in xrange(inputpdf.numPages):
print(i)
if i % 10 == 0:
@jacobkahn
jacobkahn / unsubscribe.js
Created January 31, 2019 16:23
Unsubscribe from Repositories in an Org
function unwatchOrgRepos(orgName, repoNameFirstFourChars) {
var repoItems = document.querySelectorAll(".repo-list > li")
repoItems.forEach(function (item) {
var repoOrgElement = item.querySelector(".repo-name").previousElementSibling;
var repoElement = item.querySelector(".repo-name");
var repoOrgName = repoOrgElement.innerText;
if (repoOrgName === orgName && repoElement.innerHTML.substring(0, 4) == repoNameFirstFourChars) {
var unwatchButton = item.querySelector(".js-unsubscribe-form > button");
unwatchButton.click();
var itemName = repoOrgElement.parentElement.innerText;
@jacobkahn
jacobkahn / regexReplace.cpp
Created November 12, 2018 05:28
C++-compatible regex replace with only glibc regex.h
#include <regex.h>
std::string regexReplace(
const std::string& inStr,
const std::string& regexStr,
const std::string& repl) {
int ret = 0;
// Create expr
regex_t regex;
if ((ret = regcomp(&regex, regexStr.c_str(), REG_EXTENDED))) {