Bash script for local building TensorFlow on Mac/Linux with all CPU optimizations (default pip package has only SSE)

build_tf.sh

#!/usr/bin/env bash

# Author: Sasha Nikiforov

# source of inspiration
# https://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions

# Detect platform
if [ "$(uname)" == "Darwin" ]; then
	# MacOS
	raw_cpu_flags=`sysctl -a | grep machdep.cpu.features | cut -d ":" -f 2 | tr '[:upper:]' '[:lower:]'`
elif [ "$(uname)" == "Linux" ]; then
	# GNU/Linux
	raw_cpu_flags=`grep flags -m1 /proc/cpuinfo | cut -d ":" -f 2 | tr '[:upper:]' '[:lower:]'`
else
	echo "Unknown plaform: $(uname)"
	exit -1
fi

# check if VirtuelEnv activated
if [ -z "$VIRTUAL_ENV" ]; then
	echo "VirtualEnv is not activated"
	exit -1
fi

VENV_BIN=$VIRTUAL_ENV/bin
VENV_LIB=$VIRTUAL_ENV/lib

# bazel tf needs these env vars
export PYTHON_BIN_PATH=$VENV_BIN/python
export PYTHON_LIB_PATH=$VENV_LIB/`ls $VENV_LIB | grep python`

COPT="--copt=-march=native"

for cpu_feature in $raw_cpu_flags
do
	case "$cpu_feature" in
		"sse4.1" | "sse4.2" | "ssse3" | "fma" | "cx16" | "popcnt" | "maes")
		    COPT+=" --copt=-m$cpu_feature"
		;;
		"avx1.0")
		    COPT+=" --copt=-mavx"
		;;
		*)
			# noop
		;;
	esac
done

bazel clean
./configure
bazel build -c opt $COPT -k //tensorflow/tools/pip_package:build_pip_package
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

pip install --upgrade /tmp/tensorflow_pkg/`ls /tmp/tensorflow_pkg/ | grep tensorflow`

What/where is the ./configure script?

@phobrain you have to clone the tensorflow repo and run this script inside it, more details: https://www.tensorflow.org/install/install_sources

Thanks! I had a tree checked out or likely unpacked according to some instruction page, but it didn't have the configure script. Following the install_sources link led to a successful compile. Had to get latest keras from github with 5-day-old fix for compatibility. Looks like it's cranking a lot faster now. Even without that, it's worth it to see the warnings gone. :-)

For training a truncated Inceptionv3 (pics 299x299) siamese-style in keras, batch_size=32, train/validate sizes=40/40,

virtualenv version epochs (seconds): 153, 131, 128, 130, 128 (total runtime 12m 31s)
compiled version: 114, 86, 86, 85, 87 (total 9m 1s)

Macbook Pro 2012.

@rogeriochaves thx :)

@phobrain great, thx for the input

Tensorflow 1.4.1 did not work with bazel 0.9.0 for me. Had to downgrade bazel to 0.8.1

works with tag v1.11.0, commit c19e293 python 3, I wasnt able to build with python 2, but didnt spend much time to do it Note: python 3.7 wont work on MacOS, you need python 3.6 or mb 3.5. More details below
tensorflow/tensorflow#21441

Due to some problem with virtualenv small patch is needed to build TF on virtualenv.

diff --git a/configure.py b/configure.py
index 7edab53964..5987c083cc 100644
--- a/configure.py
+++ b/configure.py
@@ -155,7 +155,7 @@ def get_python_path(environ_cp, python_bin_path):
   try:
     library_paths = run_shell(
         [python_bin_path, '-c',
-         'import site; print("\\n".join(site.getsitepackages()))']).split('\n')
+         'from distutils.sysconfig import get_python_lib; print(get_python_lib())']).split('\n')
   except subprocess.CalledProcessError:
     library_paths = [run_shell(
         [python_bin_path, '-c',