Skip to content

Instantly share code, notes, and snippets.

View yzchen's full-sized avatar
🏠
Working from home

yzchenmonkey yzchen

🏠
Working from home
9 followers · 6 following
  • HeFei, Anhui, China
View GitHub Profile
@yzchen
yzchen / profile-disk-speed.sh
Created February 19, 2019 09:34
profile hard disk performance
time sh -c "dd if=/dev/zero of=/disk/path bs=64k count=125000 && sync"
@yzchen
yzchen / hdf5-env.sh
Last active February 22, 2019 12:11
Problems with hdf5 on Shaheen
module swap PrgEnv-cray PrgEnv-intel
module load hdf5/1.8.21
wget https://support.hdfgroup.org/ftp/HDF5/current/src/unpacked/examples/h5_crtdat.c
gcc h5_crtdat.c
##################################################################
#
#h5_crtdat.c:19:18: fatal error: hdf5.h: No such file or directory
# #include "hdf5.h"
@yzchen
yzchen / caffe-prerequirements.sh
Last active May 6, 2019 11:41
Compile Caffe on HPC System without Root Access
#!/bin/bash
module use -a /project/k1341/.usr/local/share/modulefiles
module load common
# clear previous build
LOCALROOT=/project/k1341/.usr/local
rm -rf $LOCALROOT/bin $LOCALROOT/lib $LOCALROOT/lib64 $LOCALROOT/include $LOCALROOT/share/OpenCV $LOCALROOT/doc $LOCALROOT/man
# protobuf
@yzchen
yzchen / gdb-caffe.md
Last active April 14, 2019 10:53

GDB for Caffe

  1. add -g in Makefile
COMMON_FLAGS += -std=c++11

COMMON_FLAGS += -std=c++11 -g
@yzchen
yzchen / resnet-50D.prototxt
Created April 3, 2019 12:15
Caffe Resnet-50 type D network definition
name: "ResNet-50D"
layer {
name: "data"
type: "Data"
top: "data"
top: "label"
include {
phase: TRAIN
}
transform_param {
@yzchen
yzchen / allreduce-bench.cc
Created April 3, 2019 14:10
Benchmark allreduce in MPI
#include <cstdio>
#include <cstdlib>
#include <iostream>
#include <vector>
#include <mpi.h>
#include <ctime>
#include <cmath>
using namespace std;
@yzchen
yzchen / count_total_skylake_nodes.sh
Last active April 18, 2019 10:12
How many skylake nodes are used on Ibex
sinfo --partition=batch --format="%n %f" | grep "intel_gold_6148" | wc -l
@yzchen
yzchen / get_total_filesize_of_several_files.sh
Last active April 18, 2019 11:01
find -name "resnet_50_1024_nodes_32k_batch_solver_iter_*" -type f -exec du -ch {} + | grep total$
@yzchen
yzchen / compute_date_difference.sh
Created April 21, 2019 09:47
var1="12:08:18.092077"
var2="12:08:18.405611"
# result : x.xxxx secs
difference=$(( $( date -d "$var2" "+%N" ) - $( date -d "$var1" "+%N" ) )); echo "scale=4; $difference/10^9" | bc
@yzchen
yzchen / compare.cc
Created April 28, 2019 13:46
Truncate elements during conversion
#include <iostream>
#include <string>
#include <vector>
using namespace std;
#define IndexType int
typedef struct Int1 { IndexType x[1]; } Int1;
typedef struct Int2 { IndexType x[2]; } Int2;