ontheklaud’s gists

ontheklaud / resolve-java-cacerts.sh

Last active June 10, 2019 04:09

Resolve untrusted middle SSL server connection for Java environment (especially bazelbuild)

	# get latest OpenJDK (10 at here)
	# from: http://jdk.java.net/10/
	#
	# Example (OpenJDK 10-10.0.2 / GA)
	# https://download.java.net/java/GA/jdk10/10.0.2/19aef61b38124481863b1413dce1855f/13/openjdk-10.0.2_linux-x64_bin.tar.gz

	# 0. Extract openjdk into shared directory
	# sudo if needed
	tar -xf <openjdk>.tar.gz -C /usr/local

ontheklaud / build_tensorflow_from_src.sh

Last active November 2, 2018 09:05

Bazelbuild commands for TensorFlow Build

	#!/bin/bash

	# for CUDA acceleration,
	bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

	# for Intel MKL acceleration,
	bazel build --config=opt --config=mkl //tensorflow/tools/pip_package:build_pip_package
	bazel --server_javabase=$HOME/opt/jdk build --config=opt --config=mkl //tensorflow/tools/pip_package:build_pip_package

	# for TensorFlow r1.11 w/ Intel nGraph acceleration,

ontheklaud / build_intel_tensorflow.sh

Last active October 28, 2018 14:33

Build optimized TensorFlow with Intel acceleration

	# reference for intel python repository
	# https://software.intel.com/en-us/articles/using-intel-distribution-for-python-with-anaconda
	conda config --add channels intel

	conda create -n mkldnn -c conda-forge intelpython3_core python=3.6 cartopy numpy matplotlib scipy opencv cython seaborn pandas geopy scikit-learn scikit-image netcdf4 h5py

	# reference for keras module dependency:
	# https://github.com/tensorflow/tensorflow/issues/21518#issuecomment-411875069
	pip install keras_applications keras_preprocessing

ontheklaud / gist:f3a993b40fb45e45acb1f7a2507a1d2f

Last active March 19, 2019 09:00

TensorFlow r1.12rc2 build

	# GCC 8.2.0 build
	# References:
	# https://gcc.gnu.org/install/configure.html
	# http://luiarthur.github.io/gccinstall
	tar xzf gcc-8.2.0.tar.gz
	cd gcc-8.2.0
	# edit <./contrib/download_prerequisites>, from ftp:// to http://
	./contrib/download_prerequisites
	cd ..
	mkdir objdir

ontheklaud / install-accessories-centos7.sh

Created November 6, 2018 14:53

Non-root guide for installing unix accessories for CentOS 7

	# Screen (screen-4.6.2)
	wget https://ftp.gnu.org/gnu/screen/screen-4.6.2.tar.gz
	tar -xf <>
	./autogen.sh
	./configure --prefix=${HOME}/opt/screen-4.6.2
	make && make install

	# Ncurses (ncurses-6.1) / for Htop
	wget https://ftp.gnu.org/pub/gnu/ncurses/ncurses-6.1.tar.gz
	tar -xf <>

ontheklaud / build_tf110.log

Created December 3, 2018 06:08

tf1.10.1 build on CentOS 7 for CPU/CUDA optimization

	[user@host tensorflow-1.10.1]$ source activate tf110
	(tf110) [user@host tensorflow-1.10.1]$ ./configure
	WARNING: An illegal reflective access operation has occurred
	WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/home/user/.cache/bazel/_bazel_user/install/792a28b07894763eaa2bd870f8776b23/_embedded_binaries/A-server.jar) to field java.lang.String.value
	WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
	WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
	WARNING: All illegal access operations will be denied in a future release
	WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
	You have bazel 0.17.2 installed.
	Please specify the location of python. [Default is /home/user/anaconda3/envs/tf110/bin/python]: <ENTER>

ontheklaud / setup-ignite-cluster.txt

Last active March 20, 2019 00:48

Apache Ignite integration for high-performance data cache delivery

	# OpenJDK 1.8 setup

	# Apache Maven setup

	# Build In-Memory Data Fabric release (without LGPL dependencies)
	$ mvn clean package -DskipTests

	# Javadoc generation (optional)
	$ mvn initialize -Pjavadoc

ontheklaud / Copy_Wallpapers.bat

Created January 7, 2019 07:08

Copy great Windows 10 wallpapers into somewhere

	C:\Windows\System32\xcopy.exe /Y \
	"C:\Users\<user>\AppData\Local\Packages\Microsoft.Windows.ContentDeliveryManager_cw5n1h2txyewy\LocalState\Assets\*" \
	"C:\Users\<user>\Pictures\Wallpaper"

ontheklaud / workaround.log

Created January 9, 2019 02:23

LTFS Mount Failure with ICU bug with HPE SOS

	$ sudo /usr/local/bin/ltfs -o /dev/st0 /mnt/lto5/
	480f LTFS14000I LTFS starting, HPE StoreOpen Software version 3.4.0, log level 2
	480f LTFS14058I LTFS Format Specification version 2.4.0
	480f LTFS14104I Launched by "/usr/local/bin/ltfs -o /dev/st0 /mnt/lto5/"
	480f LTFS14105I This binary is built for Linux (x86_64)
	480f LTFS14106I GCC version is 4.8.5 20150623 (Red Hat 4.8.5-11)
	480f LTFS17087I Kernel version: Linux version 3.10.0-862.el7.x86_64 ([email protected]) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC) ) #1 SMP Fri Apr 20 16:44:24 UTC 2018 i386
	480f LTFS17089I Distribution: CentOS Linux release 7.5.1804 (Core)
	480f LTFS17089I Distribution: NAME="CentOS Linux"
	480f LTFS17089I Distribution: CentOS Linux release 7.5.1804 (Core)

ontheklaud / setup_custom_tensorflow_at_nurion.sh

Last active June 12, 2020 01:52

Setup Custom Python/TensorFlow env with Miniconda3 at KISTI Nurion HPC (KNL/SKL)

	#!/bin/bash

	# 0. import required modules
	module add gcc/7.2.0 openmpi/3.1.0 htop/2.2.0

	# 1. install Miniconda3 (Miniconda3-4.5.4/Python 3.6.5)
	# refer: https://repo.continuum.io/miniconda/
	# add miniconda PATH in ~/.bashrc
	wget https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh
	chmod +x Miniconda3-4.5.4-Linux-x86_64.sh

Seungkyun Hong ontheklaud