Shaowei Su shaowei-su

Environments

CPU: Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
GPU: NVIDIA V100
Memory: 251GiB
OS: Ubuntu 16.04.6 LTS (Xenial Xerus)

Docker Images:

tensorflow/tensorflow:latest-gpu
tensorflow/serving:latest-gpu

Building GCC 9.2.0 on CentOS 7

Introduction

CentOS 7 distribution (as well as RHEL 7) ships with a somewhat outdated version of the GCC compiler (4.8.5 on CentOS 7.5), which may not be suitable to your compilation requirements. For example, C11 - which supersedes C99 - is fully supported only starting from GCC 4.9).

Additionally, recent versions of GCC (GCC6, GCC7, GCC8, GCC9) come with improvements which help detect issues at build time and offer suggestions on how to fix them. Sometimes, these are even actually helpful!

This note describes how to build the latest GCC (9.2.0 as of October 2019) from sources on CentOS 7. This should be applicable as is on RHEL 7. For other Linux distributions, adapt as needed.

Used to retrieve the allocatable resources of a Kubernetes cluster.

Assumes that this is being executed within the K8s cluster.

Tested using python 2.7 and requires the installation of two pip libraries:

pip install pint
pip install kubernetes

Thanks everyone for commenting/contributing! I made this in college for a class and I no longer really use the technology. I encourage you all to help each other, but I probably won't be answering questions anymore.

This article is also on my blog: https://emilykauffman.com/blog/install-anaconda-on-wsl

Note: $ denotes the start of a command. Don't actually type this.

Steps to Install Anaconda on Windows Ubuntu Terminal

Install WSL (Ubuntu for Windows - can be found in Windows Store). I recommend the latest version (I'm using 18.04) because there are some bugs they worked out during 14/16 (microsoft/WSL#785)
Go to https://repo.continuum.io/archive to find the list of Anaconda releases
Select the release you want. I have a 64-bit computer, so I chose the latest release ending in x86_64.sh. If I had a 32-bit computer, I'd select the x86.sh version. If you accidentally try to install the wrong one, you'll get a warning in the terminal. I chose `Anaconda3-5.2.0-Li

How to reopen a pull-request after a force-push?

Precodinitions

You need the rights to reopen pull requests on the repository.
The pull request hasn't been merged, just closed.

Instructions

Write down the current commit hash of your PR-branch git log --oneline -1 <PR-BRANCH>
Write down the latest commit hash on github before the PR has been closed.
git push -f origin :

In this article, I will share some of my experience on installing NVIDIA driver and CUDA on Linux OS. Here I mainly use Ubuntu as example. Comments for CentOS/Fedora are also provided as much as I can.

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

Clarify and agree on the scope of the system

User cases (description of sequences of events that, taken together, lead to a system doing something useful)
- Who is going to use it?
- How are they going to use it?

	#!/bin/bash
	## This gist contains step by step instructions to install cuda v10.1 and cudnn 7.6 in CentOS 7
	### steps ####
	# verify the system has a cuda-capable gpu
	# download and install the nvidia cuda toolkit and cudnn
	# setup environmental variables
	# verify the installation
	###

	### to verify your gpu is cuda enable check

	import boto3

	def pull_s3_prefix(dst_dir, bucket, prefix):
	client = boto3.client('s3')
	resource = boto3.resource('s3')
	download_dir(client, resource, prefix, prefix, dst_dir, bucket)

	def download_dir(client, resource, prefix, start_prefix, local, bucket ):
	paginator = client.get_paginator('list_objects')
	for result in paginator.paginate(Bucket=bucket, Delimiter='/', Prefix=prefix):

	Latency Comparison Numbers
	--------------------------
	L1 cache reference/hit 1.5 ns 4 cycles
	Floating-point add/mult/FMA operation 1.5 ns 4 cycles
	L2 cache reference/hit 5 ns 12 ~ 17 cycles
	Branch mispredict 6 ns 15 ~ 20 cycles
	L3 cache hit (unshared cache line) 16 ns 42 cycles
	L3 cache hit (shared line in another core) 25 ns 65 cycles
	Mutex lock/unlock 25 ns
	L3 cache hit (modified in another core) 29 ns 75 cycles