Skip to content

Instantly share code, notes, and snippets.

@smitshilu

smitshilu/Tensorflow_Build_GPU.md

Last active June 9, 2020 18:27
Show Gist options
  • Save smitshilu/53cf9ff0fd6cdb64cca69a7e2827ed0f to your computer and use it in GitHub Desktop.
Save smitshilu/53cf9ff0fd6cdb64cca69a7e2827ed0f to your computer and use it in GitHub Desktop.
Download ZIP
Tensorflow 1.4 Mac OS High Sierra 10.13 GPU Support
Raw
Tensorflow_Build_GPU.md

Tensorflow

System information

  • OS - High Sierra 10.13
  • Tensorflow - 1.4
  • Xcode command line tools - 8.2 (Download from here: Xcode - Support - Apple Developer & Switch to different clang version: sudo xcode-select --switch/Library/Developer/CommandLineTools & check version: clang -v)
  • Cmake - 3.7
  • Bazel - 0.7.0
  • CUDA - 9
  • cuDNN - 7

Requirements

  • sudo pip install six numpy wheel
  • brew install coreutils

Step-by-step guide

Remove all align(sizeof(T)) from following files:

  • tensorflow/core/kernels/depthwise_conv_op_gpu.cu.cc

  • tensorflow/core/kernels/split_lib_gpu.cu.cc

  • tensorflow/core/kernels/concat_lib_gpu.impl.cu.cc

    For example, extern shared __align(sizeof(T))__ unsigned char smem[]; => extern shared unsigned char smem[];

Steps:

  • Disable SIP (in recovery mode enter command: csrutil disable)
  • ./configure (Find CUDA compute value from https://developer.nvidia.com/cuda-gpus) 
    Smit-Shilu:tensorflow-build smitshilu$ ./configure
You have bazel 0.7.0-homebrew installed.
Please specify the location of python. [Default is /Users/smitshilu/anaconda3/bin/python]:

Found possible Python library paths:
  /Users/smitshilu/anaconda3/lib/python3.6/site-packages
Please input the desired Python library path to use.  Default is [/Users/smitshilu/anaconda3/lib/python3.6/site-packages]

Do you wish to build TensorFlow with Google Cloud Platform support? [Y/n]: n
No Google Cloud Platform support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Hadoop File System support? [Y/n]: n
No Hadoop File System support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Amazon S3 File System support? [Y/n]: n
No Amazon S3 File System support will be enabled for TensorFlow.

Do you wish to build TensorFlow with XLA JIT support? [y/N]: n
No XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with GDR support? [y/N]: n
No GDR support will be enabled for TensorFlow.

Do you wish to build TensorFlow with VERBS support? [y/N]: n
No VERBS support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL support? [y/N]: n
No OpenCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.

Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 8.0]: 9.0

Please specify the location where CUDA 9.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:

Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 6.0]: 7

Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:

Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 3.5,5.2]6.1

Do you want to use clang as CUDA compiler? [y/N]:
nvcc will be used as CUDA compiler.

Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:

Do you wish to build TensorFlow with MPI support? [y/N]:
No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:

Add "--config=mkl" to your bazel command to build with MKL support.
Please note that MKL on MacOS or windows is still not supported.
If you would like to use a local MKL instead of downloading, please set the environment variable "TF_MKL_ROOT" every time before build.
Configuration finished
  • Add following paths:
    • export CUDA_HOME=/usr/local/cuda
    • export DYLD_LIBRARY_PATH=/Users/USERNAME/lib:/usr/local/cuda/lib:/usr/local/cuda/extras/CUPTI/lib (Replace USERNAME with your machine username)
    • export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
    • export PATH=$DYLD_LIBRARY_PATH:$PATH
  • Start build 
    bazel build --config=cuda --config=opt --action_env PATH --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH //tensorflow/tools/pip_package:build_pip_package
  • Generate a wheel for installation 
    bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
  • Install tensorflow wheel 
    sudo pip install /tmp/tensorflow_pkg/tensorflow-1.4.0rc1-cp36-cp36m-macosx_10_7_x86_64.whl (File name depends on tensorflow version and python version)
@xtr33me
Copy link

xtr33me commented Feb 2, 2018 •
edited
Loading

@ur001 I am getting the same error. Were you ever able to find a resolution?

--EDIT--
I'm just posting this here incase anyone else runs into the same issue @ur001 and I have had. Through my search I came across another link found here that was similar to the above instructions but for TF1.5. https://tweakmind.com/tensorflow-1-5-macos-10-13-2/

Now when I followed the instructions there I was having a different issue and unable to fully complete it due to another issue. However I noticed that I did have the latest version of xcode 9 that they referred to and it will cause the same error I was having and I noticed that both of these installs referenced 8.x. So I went down this path and followed the instructions that were advised when receiving the clang issue. After completing this, my install went through fine.

If Latest Xcode is installed:

/usr/bin/cc --version
Apple LLVM version 9.0.0 (clang-900.0.39.2)

You will get the following error:
nvcc fatal : The version ('90000') of the host compiler ('Apple clang') is not supported

To fix, install Xcode 8.3.3 and set as default
https://download.developer.apple.com/Developer_Tools/Xcode_8.3.3/Xcode8.3.3.xip

Double Click to Install, Archive Utility will Expand (takes some time)

cd ~/Downloads
mv Xcode.app Xcode-8.3.3.app
sudo mv Xcode-8.3.3.app /Applications
sudo xcode-select -s /Applications/Xcode-8.3.3.app
sudo xcodebuild -license

-- AFTER THE WHEEL IS INSTALLED THEN SET BACK ORIGINAL XCODE INSTALL --
Switch back to Current Xcode
sudo xcode-select -s /Applications/Xcode.app

@1371030
Copy link

1371030 commented Feb 5, 2018 •
edited
Loading

macOS:10.13.2
CUDA:9.1
cudnn:7
python:2.7.10

INFO: From Compiling tensorflow/core/kernels/scatter_nd_op_gpu.cu.cc:
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/CUDA/Half.h(508): error: explicit specialization of class "std::__1::numeric_limitsEigen::half" must precede its first use (
(388): here)

./tensorflow/core/lib/core/status.h(32): warning: attribute "warn_unused_result" does not apply here

1 error detected in the compilation of "/private/var/tmp/_bazel_root/f605337e84baed8e30662c78b8d6ecc6/execroot/org_tensorflow/tmp23b_3197947ae751be7b/tmpxft_00015dd2_00000000-6_scatter_nd_op_gpu.cu.cpp1.ii".
ERROR: /Users/admin/tensorflow-macOS-gpu-1.5.0-rc1/tensorflow/core/kernels/BUILD:3945:1: output 'tensorflow/core/kernels/_objs/scatter_nd_op_gpu/tensorflow/core/kernels/scatter_nd_op_gpu.cu.pic.o' was not created
ERROR: /Users/admin/tensorflow-macOS-gpu-1.5.0-rc1/tensorflow/core/kernels/BUILD:3945:1: not all outputs were created or valid
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 50.497s, Critical Path: 11.66s
FAILED: Build did NOT complete successfully

@smitshilu
Copy link
Author

@webprogrammingsolutions are you importing tensorflow from same directory? You should try import outside of tensorflow dir

@1371030
Copy link

1371030 commented Feb 7, 2018 •
edited
Loading

macOS:10.13.2
CUDA:9.1
cudnn:7
python:2.7.10
tensorflow:master

ERROR: /Users/admin/tensorflow/tensorflow/python/BUILD:1417:1: Executing genrule //tensorflow/python:sdca_ops_pygenrule failed (Aborted): bash failed: error executing command /bin/bash bazel-out/darwin-opt/genfiles/tensorflow/python/sdca_ops_pygenrule.genrule_script.sh
dyld: Library not loaded: @rpath/libcudart.9.1.dylib
Referenced from: /private/var/tmp/_bazel_root/73efb106b0ed525079e168cddc3c8e2c/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/python/gen_sdca_ops_py_wrappers_cc
Reason: image not found

Copy link

ghost commented Feb 8, 2018

Remove all align(sizeof(T)) from following files:
tensorflow/core/kernels/depthwise_conv_op_gpu.cu.cc

tensorflow/core/kernels/split_lib_gpu.cu.cc

tensorflow/core/kernels/concat_lib_gpu.impl.cu.cc

For example, extern shared align(sizeof(T)) unsigned char smem[]; => extern shared unsigned char smem[];

example just simple #
how i can do?

thanks

@smoothdvd
Copy link

@ur001 same error. How to resolve it?

@ngaurav
Copy link

ngaurav commented Feb 18, 2018

@xtr33me @ur001 @smoothdvd @1371030 Same error. Let me know if there is a solution to get past the following error:

INFO: From Compiling tensorflow/core/kernels/scatter_nd_op_gpu.cu.cc:
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/CUDA/Half.h(508): error: explicit specialization of class "std::__1::numeric_limitsEigen::half" must precede its first use (
(388): here)

./tensorflow/core/lib/core/status.h(32): warning: attribute "warn_unused_result" does not apply here

@uliano
Copy link

uliano commented Feb 18, 2018

@ngaurav & others with "eigen problem"
I was able to build a working TF 1.5 on Anaconda python 3.6 with OSX 10.13.3 Cuda 9.1 just by falling back to the 8.1 clang found in "Command Line Tools (macOS Sierra version 10.12).pkg" downloaded from Apple

@ngaurav
Copy link

ngaurav commented Feb 18, 2018

@uliano Thanks a lot. Switching to CommandLineToolsforXcode8.3.2.dmg worked.

@anubhavjaiswal03
Copy link

anubhavjaiswal03 commented Feb 19, 2018 •
edited
Loading

@smitshilu, I followed the instructions to the letter on My Mac yet i keep getting the same errors:

macOS:10.13.3
CUDA:9.1
cudnn:7
python:3.6
TF 1.4.0 (Failed to compile)
TF 1.5.0 (Failed to compile)

looks like there might be a problem with bazel,
There seems to be something trivial which i am missing. Would much appreciate your help :)
Keep getting this error:
Current Bazel version is 0.10.1, expected at least 0.5.4
ERROR: Error evaluating WORKSPACE file

Anubhavs-MacBook-Pro:tensorflow-build anubhavjaiswal$ bazel build --config=cuda --config=opt --action_env PATH --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH //tensorflow/tools/pip_package:build_pip_package
.
ERROR: /Users/anubhavjaiswal/Developer/tensorflow-build/WORKSPACE:15:1: Traceback (most recent call last):
	File "/Users/anubhavjaiswal/Developer/tensorflow-build/WORKSPACE", line 15
		closure_repositories()
	File "/private/var/tmp/_bazel_anubhavjaiswal/ac71edd2ca3b9531177a94f0fef11c2d/external/io_bazel_rules_closure/closure/repositories.bzl", line 69, in closure_repositories
		_check_bazel_version("Closure Rules", "0.4.5")
	File "/private/var/tmp/_bazel_anubhavjaiswal/ac71edd2ca3b9531177a94f0fef11c2d/external/io_bazel_rules_closure/closure/repositories.bzl", line 172, in _check_bazel_version
		fail(("%s requires Bazel >=%s but was...)))
Closure Rules requires Bazel >=0.4.5 but was 0.10.1
ERROR: Error evaluating WORKSPACE file
ERROR: /Users/anubhavjaiswal/Developer/tensorflow-build/WORKSPACE:41:1: Traceback (most recent call last):
	File "/Users/anubhavjaiswal/Developer/tensorflow-build/WORKSPACE", line 41
		tf_workspace()
	File "/Users/anubhavjaiswal/Developer/tensorflow-build/tensorflow/workspace.bzl", line 48, in tf_workspace
		check_version("0.5.4")
	File "/Users/anubhavjaiswal/Developer/tensorflow-build/tensorflow/workspace.bzl", line 38, in check_version
		fail("\nCurrent Bazel version is {}, ...))

Current Bazel version is 0.10.1, expected at least 0.5.4
ERROR: Error evaluating WORKSPACE file
ERROR: Skipping '//tensorflow/tools/pip_package:build_pip_package': error loading package 'external': Package 'external' contains errors
WARNING: Target pattern parsing failed.
ERROR: error loading package 'external': Package 'external' contains errors
INFO: Elapsed time: 1.582s
FAILED: Build did NOT complete successfully (0 packages loaded)

@anirudhmurali
Copy link

@ghost gcc: error: unrecognized command line option '-fobjc-link-runtime'
I see that you got this error as well, I got it during the bazel build of another tool. Did you find a fix?

@smitshilu
Copy link
Author

@anubhavjaiswal03 I would suggest installing bazel again and try .7 or .9 version.

@MichaelScript
Copy link

MichaelScript commented Feb 26, 2018 •
edited
Loading

@anubhavjaiswal03 Install bazel 0.7 (Compile from the bazel release and uninstall the homebrew version because homebrew forces you to use the latest xcode and there isn't an official .7 on tap via homebrew, if there was then you could just run sudo xcode-select --switch /Library/Developer/CommandLineTools/ to switch back to the current version of xcode)

@bahadir60
Copy link

bahadir60 commented Feb 28, 2018 •
edited
Loading

@smitshilu
thanks

@ghltshubh
Copy link

I got tensorflow compiled successfully but getting error while generating a wheel for installation.

Fri Mar 2 23:18:42 CST 2018 : === Using tmpdir: /var/folders/5p/bhsb4zjs41lblf2nd_bft5cm0000gp/T/tmp.XXXXXXXXXX.U08YYfRs
~/tensorflow/bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles ~/tensorflow
~/tensorflow
/var/folders/5p/bhsb4zjs41lblf2nd_bft5cm0000gp/T/tmp.XXXXXXXXXX.U08YYfRs ~/tensorflow
Fri Mar 2 23:18:48 CST 2018 : === Building wheel
Traceback (most recent call last):
  File "setup.py", line 26, in <module>
    from setuptools.command.install import install as InstallCommandBase
ModuleNotFoundError: No module named 'setuptools.command'

@orpcam
Copy link

orpcam commented Mar 3, 2018 •
edited
Loading

I got tensorflow compiled successfully but getting no output when using it.

Python 3.6.3 (default, Jan 31 2018, 13:53:45) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.__version__
'1.5.0-rc0'
>>> session = tf.Session()

The last line won't be executed. No output, no new interpreter line... Any ideas?

Update: It was my own fault. Something went wrong with my CUDA installation.

I wrote a short blog post about my successful Tensorflow 1.6-rc1 build. https://byai.io/howto-tensorflow-1-6-on-mac-with-gpu-acceleration/

@mjuchli
Copy link

mjuchli commented Mar 8, 2018

@anubhavjaiswal03 @MichaelScript i was unable to install bazel 0.7. For me the trick was to comment out line 69 in .../external/io_bazel_rules_closure/closure/repositories.bzl: # _check_bazel_version("Closure Rules", "0.4.5")

@mjuchli
Copy link

mjuchli commented Mar 8, 2018 •
edited
Loading

Running in new error:

env: python: No such file or directory
Target //tensorflow/tools/pip_package:build_pip_package failed to build

python is definitely within the PATH and can also be called with /usr/bin/env python.
Looks like this is related: tensorflow/tensorflow#15618
Any ideas?

@smitshilu
Copy link
Author

Try running ./configure again and set python environment variables properly

@vihanb
Copy link

vihanb commented Jun 7, 2018

I'm getting error:

ERROR: Inconsistent crosstool configuration; no toolchain corresponding to 'local_darwin' found for cpu 'darwin'

however I have installed brew install gcc@7 and ln -s /usr/local/bin/gcc-7 /usr/bin/gcc so not sure why still using clang instead of GCC, perhaps another issue?

@swathishesh19
Copy link

@smitshilu.. thank you!
Will this fix also work for AMD GPUs? or do I have to switch to an NVIDIA GPU

@smitshilu
Copy link
Author

@swathishesh19 it will work with NVIDIA only. For now tensorflow supports officially NVIDIA GPUs only

Copy link

ghost commented Jul 24, 2019

@coocoorhin what path are you passing while you are running ./configuration? If you check in this line
Please specify the location where CUDA 9.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Defult path is /usr/local/cuda so export the same path even default installation path is different

i did just as he did, instead of leaving default i changed it to Developers.......

@Hawk0826
Copy link

macOS:10.13.2
CUDA:9.1
cudnn:7
python:2.7.10

INFO: From Compiling tensorflow/core/kernels/scatter_nd_op_gpu.cu.cc:
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/CUDA/Half.h(508): error: explicit specialization of class "std::__1::numeric_limitsEigen::half" must precede its first use (
(388): here)

./tensorflow/core/lib/core/status.h(32): warning: attribute "warn_unused_result" does not apply here

1 error detected in the compilation of "/private/var/tmp/_bazel_root/f605337e84baed8e30662c78b8d6ecc6/execroot/org_tensorflow/tmp23b_3197947ae751be7b/tmpxft_00015dd2_00000000-6_scatter_nd_op_gpu.cu.cpp1.ii".
ERROR: /Users/admin/tensorflow-macOS-gpu-1.5.0-rc1/tensorflow/core/kernels/BUILD:3945:1: output 'tensorflow/core/kernels/_objs/scatter_nd_op_gpu/tensorflow/core/kernels/scatter_nd_op_gpu.cu.pic.o' was not created
ERROR: /Users/admin/tensorflow-macOS-gpu-1.5.0-rc1/tensorflow/core/kernels/BUILD:3945:1: not all outputs were created or valid
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 50.497s, Critical Path: 11.66s
FAILED: Build did NOT complete successfully

@1371030 same error. How to resolve it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment