-
-
Save AlexJoz/1670baf0b32573ca7923 to your computer and use it in GitHub Desktop.
### | |
# !!! THIS IS NOT A BASH SCRIPT !!! | |
### | |
# named .sh just so Github does correct syntax highlighting | |
# Inspired by https://gist.github.com/erikbern/78ba519b97b440e10640 | |
# | |
# This setup is available as a public AMI in US-East(N. Virginia): ami-9d0f3ff7 | |
# Add repos for cmake and gcc | |
sudo add-apt-repository ppa:ubuntu-toolchain-r/test | |
sudo add-apt-repository ppa:george-edison55/cmake-3.x | |
sudo apt-get update | |
### | |
# Install dependencies for tensorflow and opencv (only for pictures, not video) | |
### | |
sudo apt-get install -y build-essential cmake gcc-4.9 wget curl git libtbb-dev libfreetype6-dev libpng12-dev libzmq3-dev pkg-config python3-dev python3-numpy python3-pip software-properties-common swig zip zlib1g-dev openjdk-8-jdk openjdk-8-jre-headless | |
sudo apt-get upgrade | |
# blacklist noubeau to avoid cudasetup error | |
echo -e "blacklist nouveau\nblacklist lbm-nouveau\noptions nouveau modeset=0\nalias nouveau off\nalias lbm-nouveau off\n" | sudo tee /etc/modprobe.d/blacklist-nouveau.conf | |
echo options nouveau modeset=0 | sudo tee -a /etc/modprobe.d/nouveau-kms.conf | |
sudo update-initramfs -u | |
sudo reboot | |
sudo apt-get install -y linux-image-extra-virtual | |
sudo reboot | |
sudo apt-get install -y linux-source linux-headers-`uname -r` | |
### | |
# Download and install cuda driver | |
### | |
wget http://developer.download.nvidia.com/compute/cuda/7_0/Prod/local_installers/cuda_7.0.28_linux.run | |
# verify checksum | |
cs=$(md5sum cuda_7.0.28_linux.run | cut -d' ' -f1) | |
if [ "$cs" != "312aede1c3d1d3425c8caa67bbb7a55e" ]; then echo "WARNING: Unverified MD5 hash"; fi | |
chmod +x cuda_7.0.28_linux.run | |
./cuda_7.0.28_linux.run -extract=`pwd`/nvidia_installers | |
pushd nvidia_installers | |
sudo ./NVIDIA-Linux-x86_64-346.46.run | |
# When prompted you'll need to: | |
# * Agree to the license terms | |
# * Accept the X Library path and X module path | |
# * Accept that 32-bit compatibility files will not be installed | |
# * Review and accept the libvdpau and libvdpau_trace libraries notice | |
# * Choose `Yes` when asked about automatically updating your X configuration file | |
# * Verify successful installation by choosing `OK` | |
sudo modprobe nvidia | |
sudo ./cuda-linux64-rel-7.0.28-19326674.run | |
# When prompted, you'll need to: | |
# * Accept the license terms (long scroll, page down with `f`) | |
# * Use the default installation path | |
# * Answer `n` to desktop shortcuts | |
# * Answer `y` to create a symbolic link | |
### | |
# Install cuDNN 6.5 | |
### | |
# Note: You’ll need to register into the NVIDIA Accelerated Computing Developer Program | |
# and download the cuDNN v2 Library for Linux to your local workstation. | |
# You have to get the approval of your regestration from nvidia, it make take up to some days! | |
# You can upload library to host via gui of winscp. Settings for amazon: https://winscp.net/eng/docs/guide_amazon_ec2 | |
tar -xzf cudnn-6.5-linux-x64-v2.tgz | |
sudo cp cudnn-6.5-linux-x64-v2/libcudnn* /usr/local/cuda/lib64 | |
sudo cp cudnn-6.5-linux-x64-v2/cudnn.h /usr/local/cuda/include/ | |
### | |
# Install Bazel | |
### | |
cd | |
git clone https://github.com/bazelbuild/bazel.git | |
cd bazel | |
git checkout tags/0.1.5 | |
./compile.sh | |
sudo cp output/bazel /usr/bin | |
#export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64" | |
#export CUDA_HOME=/usr/local/cuda | |
echo 'export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64"' >> ~/.bashrc | |
echo 'export CUDA_HOME=/usr/local/cuda' >> ~/.bashrc | |
### | |
# Install Tensorflow | |
### | |
cd | |
git clone --recurse-submodules https://github.com/tensorflow/tensorflow | |
cd tensorflow | |
git checkout r0.7 | |
TF_UNOFFICIAL_SETTING=1 ./configure | |
# When prompted, you'll need to: | |
# * type path for python 3: /usr/bin/python3 | |
# * Build with GPU support - yes | |
# * Accept the defaults for Cuda SDK and Cudnn versions and locations. Just click Enter... | |
# * Specify Cuda compute device capability of `3.0` | |
# !!! Don't accept the default of `3.5,5.2` !!! | |
bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package | |
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/pip | |
sudo pip3 install --upgrade /tmp/pip/tensorflow-*.whl | |
### | |
# Install OpenCV | |
### | |
cd | |
git clone https://github.com/Itseez/opencv | |
git checkout tags/3.1.0 | |
# Switch off cuda!, OpenCV doesn't support the GPU module with Python. That means that you must write wrappers yourself to use it. | |
cmake -D CMAKE_BUILD_TYPE=RELEASE \ | |
-D CMAKE_INSTALL_PREFIX=/usr/local \ | |
-DWITH_CUDA=OFF \ | |
-DINSTALL_C_EXAMPLES=OFF \ | |
-DINSTALL_PYTHON_EXAMPLES=OFF \ | |
-D WITH_TBB=ON .. | |
make -j8 | |
sudo make install | |
sudo /bin/bash -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf' | |
sudo ldconfig | |
echo 'export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig' >> ~/.bashrc |
while trying to build tensorflow on an AWS EC2 instance, I am getting the following error:
ERROR: /home/ubuntu/Downloads/tensorflow/google/protobuf/BUILD:272:1: Linking of rule '//google/protobuf:protoc' failed: crosstool_wrapper_driver_is_not_gcc failed: error executing command third_party/gpus/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -o bazel-out/host/bin/google/protobuf/protoc ... (remaining 15 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
Do you know how can I solve this ?
Thanks for your help
@santteegt
I am seeing the same error. Did you solve it?
Published new AMI in N. Virginia with 0.8.0 support: ami-1e19ee73
Rather than
$ git clone --recurse-submodules https://github.com/tensorflow/tensorflow
$ cd tensorflow
$ git checkout r0.7
When you run git checkout r0.7
, git will remove the google/protobuf
submodule.
You can more simply do it like so
$ git clone --branch r0.7 --recurse-submodules https://github.com/tensorflow/tensorflow && pushd tensorflow
Sorry guys, there were no notifications about comments =(
Made new setup with tf 0.9, but without opencv for now, also available as public ami:
https://gist.github.com/AlexJoz/e84eb137bc4cb9389b22e92ef8f03153
also for everyone building this on an aws g2 you'll run out of memory at line:
bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
Right before this you should delete the nvidia_installer folder on your home directory, which will free up ~1gb.