Skip to content

Instantly share code, notes, and snippets.

@koki0702
Created August 7, 2018 03:55
Show Gist options
  • Save koki0702/1b5ee259ba688cd2aee7b644190e5813 to your computer and use it in GitHub Desktop.
Save koki0702/1b5ee259ba688cd2aee7b644190e5813 to your computer and use it in GitHub Desktop.
FROM nvidia/cuda:9.0-base-ubuntu16.04
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
cuda-command-line-tools-9-0 \
cuda-cublas-dev-9-0 \
cuda-cudart-dev-9-0 \
cuda-cufft-dev-9-0 \
cuda-curand-dev-9-0 \
cuda-cusolver-dev-9-0 \
cuda-cusparse-dev-9-0 \
curl \
git \
libcudnn7=7.1.4.18-1+cuda9.0 \
libcudnn7-dev=7.1.4.18-1+cuda9.0 \
libnccl2=2.2.13-1+cuda9.0 \
libnccl-dev=2.2.13-1+cuda9.0 \
libcurl3-dev \
libfreetype6-dev \
libhdf5-serial-dev \
libpng12-dev \
libzmq3-dev \
pkg-config \
python-dev \
rsync \
software-properties-common \
unzip \
zip \
zlib1g-dev \
wget \
&& \
rm -rf /var/lib/apt/lists/* && \
find /usr/local/cuda-9.0/lib64/ -type f -name 'lib*_static.a' -not -name 'libcudart_static.a' -delete && \
rm /usr/lib/x86_64-linux-gnu/libcudnn_static_v7.a
# Link NCCL libray and header where the build script expects them.
RUN mkdir /usr/local/cuda-9.0/lib && \
ln -s /usr/lib/x86_64-linux-gnu/libnccl.so.2 /usr/local/cuda/lib/libnccl.so.2 && \
ln -s /usr/include/nccl.h /usr/local/cuda/include/nccl.h
RUN curl -fSsL -O https://bootstrap.pypa.io/get-pip.py && \
python get-pip.py && \
rm get-pip.py
RUN pip --no-cache-dir install \
Pillow \
h5py \
ipykernel \
jupyter \
matplotlib \
mock \
numpy \
scipy \
sklearn \
pandas \
&& \
python -m ipykernel.kernelspec
# RUN ln -s -f /usr/bin/python3 /usr/bin/python#
# Set up our notebook config.
COPY jupyter_notebook_config.py /root/.jupyter/
# Jupyter has issues with being run directly:
# https://github.com/ipython/ipython/issues/7062
# We just add a little wrapper script.
COPY run_jupyter.sh /
# Set up Bazel.
# Running bazel inside a `docker build` command causes trouble, cf:
# https://github.com/bazelbuild/bazel/issues/134
# The easiest solution is to set up a bazelrc file forcing --batch.
RUN echo "startup --batch" >>/etc/bazel.bazelrc
# Similarly, we need to workaround sandboxing issues:
# https://github.com/bazelbuild/bazel/issues/418
RUN echo "build --spawn_strategy=standalone --genrule_strategy=standalone" \
>>/etc/bazel.bazelrc
# Install the most recent bazel release.
ENV BAZEL_VERSION 0.15.0
WORKDIR /
RUN mkdir /bazel && \
cd /bazel && \
curl -H "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36" -fSsL -O https://github.com/bazelbuild/bazel/releases/download/$BAZEL_VERSION/bazel-$BAZEL_VERSION-installer-linux-x86_64.sh && \
curl -H "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36" -fSsL -o /bazel/LICENSE.txt https://raw.githubusercontent.com/bazelbuild/bazel/master/LICENSE && \
chmod +x bazel-*.sh && \
./bazel-$BAZEL_VERSION-installer-linux-x86_64.sh && \
cd / && \
rm -f /bazel/bazel-$BAZEL_VERSION-installer-linux-x86_64.sh
# Download and build TensorFlow.
WORKDIR /tensorflow
#RUN git clone --branch=r1.9 --depth=1 https://github.com/tensorflow/tensorflow.git .
RUN git clone --branch=r1.8 --depth=1 https://github.com/tensorflow/tensorflow.git .
# Configure the build for our CUDA configuration.
ENV CI_BUILD_PYTHON python
ENV LD_LIBRARY_PATH /usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH
ENV TF_NEED_CUDA 1
ENV TF_CUDA_COMPUTE_CAPABILITIES=3.5,5.2,6.0,6.1,7.0
ENV TF_CUDA_VERSION=9.0
ENV TF_CUDNN_VERSION=7
# NCCL 2.x
ENV TF_NCCL_VERSION=2
RUN gzip -d /usr/share/doc/libnccl2/NCCL-SLA.txt.gz
RUN cp /usr/share/doc/libnccl2/NCCL-SLA.txt ../tensorflow/third_party/nccl
RUN chmod a+w /usr/share/doc/libnccl2/NCCL-SLA.txt
RUN ls -al /usr/share/doc/libnccl2/NCCL-SLA.txt
RUN cp /usr/share/doc/libnccl2/NCCL-SLA.txt /usr
RUN cp /usr/share/doc/libnccl2/NCCL-SLA.txt /usr/local/cuda
RUN cp /usr/share/doc/libnccl2/NCCL-SLA.txt /opt/cuda
RUN mkdir -p /usr/local/cuda-9.0/nccl
RUN cp /usr/share/doc/libnccl2/NCCL-SLA.txt /usr/local/cuda-9.0/nccl
RUN ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/libcuda.so.1 && \
LD_LIBRARY_PATH=/usr/local/cuda/lib64/stubs:${LD_LIBRARY_PATH} \
tensorflow/tools/ci_build/builds/configured GPU \
bazel build -c opt --copt=-mavx --config=cuda \
--cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" \
tensorflow/tools/pip_package:build_pip_package && \
rm /usr/local/cuda/lib64/stubs/libcuda.so.1 && \
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/pip && \
pip --no-cache-dir install --upgrade /tmp/pip/tensorflow-*.whl && \
rm -rf /tmp/pip && \
rm -rf /root/.cache
# Clean up pip wheel and Bazel cache when done.
RUN pip install cython opencv-python easydict scipy transforms3d
RUN apt update && apt install -y libsm6 libxext6
RUN git clone https://github.com/eigenteam/eigen-git-mirror.git
RUN mkdir /usr/include/eigen3
RUN cp -r eigen-git-mirror/Eigen /usr/include/eigen3/Eigen
WORKDIR /
RUN git clone https://github.com/yuxng/PoseCNN.git
WORKDIR PoseCNN/lib
RUN ln -s /usr/local/cuda/lib64/stubs/libcuda.so /usr/local/cuda/lib64/stubs/libcuda.so.1
#RUN ln -s /usr/local/cuda/lib64/stubs/libcublas.so /usr/local/cuda/lib64/stubs/libcublas.so.1.0
ENV LD_LIBRARY_PATH=/usr/local/cuda/lib64/stubs/:$LD_LIBRARY_PATH
ENV CPATH=/usr/include/eigen3
RUN make
#RUN sh make.sh
# download VGG-16.npy
RUN curl -c /tmp/cookie.txt -s -L "https://drive.google.com/uc?export=download&id=1UdmOKrr9t4IetMubX-y-Pcn7AVaWJ2bL" |grep confirm | sed -e\
"s/^.*confirm=\(.*\)&id=.*$/\1/" | xargs -I{} curl -b /tmp/cookie.txt -L -o vgg16.npy "https://drive.google.com/uc?confirm={}&export=d\
ownload&id=1UdmOKrr9t4IetMubX-y-Pcn7AVaWJ2bL"
RUN mv vgg16.npy /PoseCNN/data/imagenet_models/
# dwonload model
RUN curl -c /tmp/cookie.txt -s -L "https://drive.google.com/uc?export=download&id=1UNJ56Za6--bHGgD3lbteZtXLC2E-liWz" |grep confirm | sed -e\
"s/^.*confirm=\(.*\)&id=.*$/\1/" | xargs -I{} curl -b /tmp/cookie.txt -L -o demo_models.zip "https://drive.google.com/uc?confirm={}&ex\
port=download&id=1UNJ56Za6--bHGgD3lbteZtXLC2E-liWz"
RUN mv demo_models.zip /PoseCNN/data/demo_models
RUN pip install pillow pyyaml
RUN mkdir /PoseCNN/data/LOV/data
RUN apt-get install -y libxrender-dev
WORKDIR /PoseCNN
ENV PYTHONPATH=/PoseCNN/lib
CMD ./experiments/scripts/demo.sh 0
@saurabheights
Copy link

FYI: There is one part missing, this will not run ICP at the end. It requires compiling of lib/synthesize module and changing config file, the one provided in demo.sh. I am not too familiar with docker and thus taking me longer to work on this.
P.S. - Did you run this on docker or nvidia-docker?

@atabakd
Copy link

atabakd commented Dec 19, 2018

I appreciate the effort of the docker, especially for Bazel part. But Unfortunately, I couldn't make it work. There are errors when running makeand it won't do python setup.py build_ext --inplace and even when I fixed these, it still complained.

@rsisto
Copy link

rsisto commented Feb 13, 2019

Hi, tried this dockerfile but it failed at copying jupyter files:
Successfully installed MarkupSafe-1.1.0 Pillow-5.4.1 Send2Trash-1.5.0 backports-abc-0.5 backports.functools-lru-cache-1.5 backports.shutil-get-terminal-size-1.0.0 bleach-3.1.0 configparser-3.7.1 cycler-0.10.0 decorator-4.3.2 defusedxml-0.5.0 entrypoints-0.3 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.2.0 h5py-2.9.0 ipaddress-1.0.22 ipykernel-4.10.0 ipython-5.8.0 ipython-genutils-0.2.0 ipywidgets-7.4.2 jinja2-2.10 jsonschema-2.6.0 jupyter-1.0.0 jupyter-client-5.2.4 jupyter-console-5.2.0 jupyter-core-4.4.0 kiwisolver-1.0.1 matplotlib-2.2.3 mistune-0.8.4 mock-2.0.0 nbconvert-5.4.1 nbformat-4.4.0 notebook-5.7.4 numpy-1.16.1 pandas-0.24.1 pandocfilters-1.4.2 pathlib2-2.3.3 pbr-5.1.2 pexpect-4.6.0 pickleshare-0.7.5 prometheus-client-0.5.0 prompt-toolkit-1.0.15 ptyprocess-0.6.0 pygments-2.3.1 pyparsing-2.3.1 python-dateutil-2.8.0 pytz-2018.9 pyzmq-17.1.2 qtconsole-4.4.3 scandir-1.9.0 scikit-learn-0.20.2 scipy-1.2.1 simplegeneric-0.8.1 singledispatch-3.4.0.3 six-1.12.0 sklearn-0.0 subprocess32-3.5.3 terminado-0.8.1 testpath-0.4.2 tornado-5.1.1 traitlets-4.3.2 wcwidth-0.1.7 webencodings-0.5.1 widgetsnbextension-3.4.2
Installed kernelspec python2 in /usr/local/share/jupyter/kernels/python2
Removing intermediate container 07160b1e5baf
---> 1fd3c1b29db3
Step 6/53 : COPY jupyter_notebook_config.py /root/.jupyter/
COPY failed: stat /var/lib/docker/tmp/docker-builder003884375/jupyter_notebook_config.py: no such file or directory

Where can I find the missing files for this project?
I appreciate your help! Thanks!

@huckl3b3rry87
Copy link

@rsisto I would just comment out that line in the Dockerfile, you do not have his jupyter notebook

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment