macOS High Sierra 10.13.3 install TensorFlow 1.5.0 with GPU support

macOS High Sierra 10.13.3 install TensorFlow 1.5.0 with GPU support.md

New compile referrence for TensorFlow 1.11.0:

https://gist.github.com/geekcui/ce0bda3dc40d048666fe904c0a2f1017

https://github.com/geekcui/tensorflow-macos-egpu

macOS: 10.13.3

WebDriver: 387.10.10.10.25.156

CUDA Toolkit: 9.1.128

cuDNN: 7

NVDAEGPUSupport: 6

XCode: 8.2

Bazel: 0.9.0

OpenMP: latest

Python: 3.6

TensorFlow: 1.5.0

https://qiita.com/74th/items/fc6ebb684c23f3655e7c

http://paolino.me/tutorial/tensorflow/machine-learning/deep-learning/gpu/2017/11/18/installing-tensorflow-1.4.0-macos-cuda/

https://gist.github.com/smitshilu/53cf9ff0fd6cdb64cca69a7e2827ed0f

http://melonteam.com/posts/pei_zhi_tensorflow_gpu_ban_ben_tian_keng_lu/

eGPU part

https://egpu.io/forums/mac-setup/wip-nvidia-egpu-support-for-high-sierra/

https://egpu.io/forums/implementation-guides/2017-15-macbook-pro-radeon-pro-555-gtx1080ti32gbps-tb3-mantiz-venus-macos10-13-4-win10/

https://github.com/vulgo/webdriver.sh

1 Remove/undo any Info.plist modifications (they aren’t needed anymore and might conflict).

2 Disable SIP

• Restart holding command + r
• Execute 'csrutil disable’
• Restart

3 Install WebDriver with webdriver.sh

git clone https://github.com/vulgo/webdriver.sh.git
cd webdriver.sh
sudo ./webdriver.sh -u https://us.download.nvidia.com/Mac/Quadro_Certified/387.10.10.10.25.156/WebDriver-387.10.10.10.25.156.pkg

4 Enable SIP

• Restart holding command + r
• Execute 'csrutil enable --without kext’
• Restart

5 Install nvidia-egpu

• Download https://cdn.egpu.io/wp-content/uploads/wpforo/attachments/71/4376-NVDAEGPUSupport-v6.zip
• Unzip 4376-NVDAEGPUSupport-v6.zip
• Install package NVDAEGPUSupport-v6.pkg

6 Check eGPU status

• Reboot
• Attach the egpu
• Login
• Check whether the GPU is recognized

CUDA & cuDNN

1 Download packages

#cudadriver_387.128_macos.dmg

http://us.download.nvidia.com/Mac/cuda_387/cudadriver_387.128_macos.dmg

#cuda_9.1.128_mac.dmg

https://developer.nvidia.com/cuda-toolkit-archive

#cudnn-9.1-osx-x64-v7-ga.tgz

https://developer.nvidia.com/rdp/cudnn-download

2 Install packages

1. Install cuda_9.1.128_mac.dmg with default options

2. Install cudadriver_387.128_macos.dmg with default options

3. Install cuDNN

tar -zxf cudnn-9.1-osx-x64-v7-ga.tgz
cd cuda
sudo cp -RPf include/* /Developer/NVIDIA/CUDA-9.1/include/
sudo cp -RPf lib/* /Developer/NVIDIA/CUDA-9.1/lib/
sudo ln -s /Developer/NVIDIA/CUDA-9.1/lib/libcudnn* /usr/local/cuda/lib/

3 Add environment variables

vim ~/.zshrc
#if you use bash, this should be ~/.bash_profile
  export CUDA_HOME=/usr/local/cuda
  export DYLD_LIBRARY_PATH=$CUDA_HOME/lib:$CUDA_HOME/extras/CUPTI/lib
  export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
  export PATH=$CUDA_HOME/bin:$PATH
source ~/.zshrc

3 Verify CUDA works fine

cd /Developer/NVIDIA/CUDA-9.1/samples/1_Utilities/deviceQuery
sudo make 
./deviceQuery

Homebrew part

1 Install homebrew

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

2 Install coreutils

brew install coreutils

3 Install OpenMP

https://clang-omp.github.io

http://antonmenshov.com/2017/09/09/clang-openmp-setup-in-xcode/

https://iscinumpy.gitlab.io/post/omp-on-high-sierra/

brew install cliutils/apple/libomp

4 Install bazel

cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core
git checkout bd8bba7e7 Formula/bazel.rb
brew remove bazel —force
brew install bazel
bazel version

5 Install anaconda

brew install anaconda
vim ~/.zshrc
#If you use bash, this should be ~/.bash_profile
  export PATH=/usr/local/anaconda3/bin:$PATH
source ~/.zshrc

Install XCode 8.2

• Download https://download.developer.apple.com/Developer\_Tools/Xcode\_8.2/Xcode\_8.2.xip
• Extract it, and rename Xcode.app to Xcode8.2.app
• Drag Xcode8.2.app to Applications

sudo xcode-select -s /Applications/Xcode8.2.app

You need restore xcode configuration or remove Xcode8.2.app later, since it will break homebrew

sudo xcode-select -s /Applications/Xcode.app

Install TensorFlow

1 Add virtualenv and activate virtualenv

conda create --p egpu python=3.6
source activate egpu
pip install six numpy wheel

2 Clone TensorFlow code

git clone https://github.com/tensorflow/tensorflow.git -b v1.5.0

3 Modify the code, to make it compatible with macOS

cd tensorflow
sed -i -e "s/ __align__(sizeof(T))//g" tensorflow/core/kernels/concat_lib_gpu_impl.cu.cc
sed -i -e "s/ __align__(sizeof(T))//g" tensorflow/core/kernels/depthwise_conv_op_gpu.cu.cc
sed -i -e "s/ __align__(sizeof(T))//g" tensorflow/core/kernels/split_lib_gpu.cu.cc

4 Compile the code

./configure
  #Please specify the location of python.: Accept the default option
	#Please input the desired Python library path to use.:  Accept the default option
	#Do you wish to build TensorFlow with Google Cloud Platform support? [Y/n]: n
	#Do you wish to build TensorFlow with Hadoop File System support? [Y/n]: n
	#Do you wish to build TensorFlow with Amazon S3 File System support? [Y/n]: n
	#Do you wish to build TensorFlow with XLA JIT support? [y/N]: n
	#Do you wish to build TensorFlow with GDR support? [y/N]: n
	#Do you wish to build TensorFlow with VERBS support? [y/N]: n
	#Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n
	#Do you wish to build TensorFlow with CUDA support? [y/N]: y
	#Please specify the CUDA SDK version you want to use, e.g. 7.0.: 9.1
	#Please specify the location where CUDA 9.1 toolkit is installed.: Accept the default option
	#Please specify the cuDNN version you want to use.: 7
	#Please specify the location where cuDNN 7 library is installed.: Accept the default option
	##Please specify a list of comma-separated Cuda compute capabilities you want to build with.
	##You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus. (GTX10X0: 6.1, GTX9X0: 5.2)
	#Please note that each additional compute capability significantly increases your build time and binary size.: 6.1
	#Do you want to use clang as CUDA compiler? [y/N]: n
	#Please specify which gcc should be used by nvcc as the host compiler.: Accept the default option
	#Do you wish to build TensorFlow with MPI support? [y/N]: n
	#Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified:  Accept the default option
	#Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: n

export CUDA_HOME=/usr/local/cuda
export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:/usr/local/cuda/extras/CUPTI/lib 
export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
export PATH=$DYLD_LIBRARY_PATH:$PATH

#bazel clean --expunge
bazel build --config=cuda --config=opt --action_env PATH --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH //tensorflow/tools/pip_package:build_pip_package

5 Build the wheel and install

bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
sudo pip install /tmp/tensorflow_pkg/tensorflow-1.5.0-cp36-cp36m-macosx_10_7_x86_64.whl

ERRORS

1 CUDA_ERROR_OUT_OF_MEMORY

https://stackoverflow.com/questions/39465503/cuda-error-out-of-memory-in-tensorflow

https://stackoverflow.com/questions/43467586/tensorflow-cuda-error-out-of-memory-always-happen

https://stackoverflow.com/questions/45546737/cuda-error-out-of-memory-how-to-activate-multiple-gpus-from-keras-in-tensorflow

#TensorFlow
gpu_options = tf.GPUOptions(allow_growth=True) 
with tf.Session(config=tf.ConfigProto(gpu_options=gpu_options)) as sess:

#Keras
import keras.backend as K
config = K.tf.ConfigProto()
config.gpu_options.allow_growth = True
session = K.tf.Session(config=config)

2 PyCharm Library not loaded: @rpath/libcudnn.7.dylib

https://stackoverflow.com/questions/37933890/tensorflow-gpu-setup-error-with-cuda-on-pycharm

Add environment variables to python default configuration:

CUDA_HOME=/usr/local/cuda

DYLD_LIBRARY_PATH=/usr/local/cuda/lib:/usr/local/cuda/extras/CUPTI/lib

LD_LIBRARY_PATH=/usr/local/cuda/lib:/usr/local/cuda/extras/CUPTI/lib

Hello! During the final bazel build, I get this:
"Closure Rules requires Bazel >=0.4.5 but was 0.11.1-homebrew".

I figure this may be an issue with 0.11, rather than the 0.9 that you used? Unfortunately, brew search bazel does not have any old versions to download.

Is there something else I'm missing? Thank you very much!

(I should point out that this command did not work: git checkout bd8bba7e7 Formula/bazel.rb)

@andrediamant, this should answer your question...
tensorflow/tensorflow#16654 (comment)

@aaronpenne Thank you for your help! This certainly enabled me to begin the compilation step.

Unfortunately, the compilation process itself now fails. Along with a wealth of warnings, this is the final error text. Any chance it is meaningful to you?

`INFO: From Compiling tensorflow/contrib/rnn/kernels/gru_ops_gpu.cu.cc:
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/CUDA/Half.h(508): error: explicit specialization of class "std::__1::numeric_limitsEigen::half" must precede its first use (
(388): here)

1 error detected in the compilation of "/var/folders/vh/rw8my97n7w763hqfd3v6c3fr0000gn/T//tmpxft_00008e9d_00000000-6_gru_ops_gpu.cu.cpp1.ii".
ERROR: /Users/andrediamant/tensorflow/tensorflow/contrib/rnn/BUILD:240:1: output 'tensorflow/contrib/rnn/_objs/python/ops/_gru_ops_gpu/tensorflow/contrib/rnn/kernels/gru_ops_gpu.cu.pic.o' was not created
ERROR: /Users/andrediamant/tensorflow/tensorflow/contrib/rnn/BUILD:240:1: not all outputs were created or valid
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 387.735s, Critical Path: 30.51s
FAILED: Build did NOT complete successfully`

It might worth mentioning that you'd probably need to disable SIP during compilation.

the download link for nvidia egpu support is broken. The link to the original post is here

@andrediamant I bumped into the same issue with you but there doesn't seem to be a solution yet though. Please refer to tensorflow/tensorflow/issues/17493

Epic, thank you so much. this got my system working finally. 2012 MacBook Pro

During compilation, I am getting a lot of errors like this:

./tensorflow/core/util/gpu_device_functions.h(868): error: calling a constexpr __host__ function("imag") from a __device__ function("operator*") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

where can I set the '--expt-relaxed-constexpr' flag?