Skip to content

Instantly share code, notes, and snippets.

@sub-mod
Created July 9, 2019 15:41
Show Gist options
  • Save sub-mod/4b797561d1c38968322bb2bdc7ca0a7f to your computer and use it in GitHub Desktop.
Save sub-mod/4b797561d1c38968322bb2bdc7ca0a7f to your computer and use it in GitHub Desktop.
kubeflow nb image
git checkout remotes/origin/v0.5-branch
cd components/tensorflow-notebook-image
./build_image.sh ./Dockerfile submod/tensorflow-1.13.1-notebook-cpu v0.5.0 ./versions/1.13.1/version-config.json
./build_image.sh ./Dockerfile submod/tensorflow-1.13.1-notebook-cpu v0.5.0 ./versions/1.13.1/version-config.json
+ DOCKERFILE=./Dockerfile
++ dirname ./Dockerfile
+ CONTEXT_DIR=.
+ IMAGE=submod/tensorflow-1.13.1-notebook-cpu
+ TAG=v0.5.0
+ CONFIG_FILE=./versions/1.13.1/version-config.json
++ jq -r .BASE_IMAGE ./versions/1.13.1/version-config.json
+ BASE_IMAGE=ubuntu:18.04@sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378
++ jq -r .TF_PACKAGE ./versions/1.13.1/version-config.json
+ TF_PACKAGE=https://files.pythonhosted.org/packages/77/63/a9fa76de8dffe7455304c4ed635be4aa9c0bacef6e0633d87d5f54530c5c/tensorflow-1.13.1-cp36-cp36m-manylinux1_x86_64.whl
++ jq -r .TF_PACKAGE_PY_27 ./versions/1.13.1/version-config.json
+ TF_PACKAGE_PY_27=https://files.pythonhosted.org/packages/d2/ea/ab2c8c0e81bd051cc1180b104c75a865ab0fc66c89be992c4b20bbf6d624/tensorflow-1.13.1-cp27-cp27mu-manylinux1_x86_64.whl
++ jq -r .TF_SERVING_VERSION ./versions/1.13.1/version-config.json
+ TF_SERVING_VERSION=null
++ jq -r .TFMA_VERSION ./versions/1.13.1/version-config.json
+ TFMA_VERSION=0.13.0
++ jq -r .TFDV_VERSION ./versions/1.13.1/version-config.json
+ TFDV_VERSION=0.13.1
+ '[' ubuntu:18.04@sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378 == null ']'
+ '[' 0.13.0 == null ']'
+ '[' 0.13.1 == null ']'
+ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
+ docker build --pull --build-arg BASE_IMAGE=ubuntu:18.04@sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378 --build-arg TF_PACKAGE=https://files.pythonhosted.org/packages/77/63/a9fa76de8dffe7455304c4ed635be4aa9c0bacef6e0633d87d5f54530c5c/tensorflow-1.13.1-cp36-cp36m-manylinux1_x86_64.whl --build-arg TF_PACKAGE_PY_27=https://files.pythonhosted.org/packages/d2/ea/ab2c8c0e81bd051cc1180b104c75a865ab0fc66c89be992c4b20bbf6d624/tensorflow-1.13.1-cp27-cp27mu-manylinux1_x86_64.whl --build-arg TF_SERVING_VERSION=null --build-arg TFMA_VERSION=0.13.0 --build-arg TFDV_VERSION=0.13.1 -t submod/tensorflow-1.13.1-notebook-cpu:v0.5.0 -f ./Dockerfile .
Sending build context to Docker daemon 19.91MB
Step 1/49 : ARG BASE_IMAGE=ubuntu:18.04@sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378
Step 2/49 : FROM $BASE_IMAGE
sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378: Pulling from library/ubuntu
Digest: sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378
Status: Image is up to date for ubuntu:18.04@sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378
---> cd6d8154f1e1
Step 3/49 : ARG TF_PACKAGE=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.7.0-cp36-cp36m-linux_x86_64.whl
---> Using cache
---> a1b18a85f56c
Step 4/49 : ARG TF_PACKAGE_PY_27=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.7.0-cp27-none-linux_x86_64.whl
---> Using cache
---> 6d2f2fd6096e
Step 5/49 : ARG TF_SERVING_VERSION=0.0.0
---> Using cache
---> dd432d1253f0
Step 6/49 : ARG TFMA_VERSION
---> Using cache
---> e4f84164ded0
Step 7/49 : ARG TFDV_VERSION
---> Using cache
---> 029209823b1b
Step 8/49 : ARG PIPELINE_SDK_PACKAGE=https://storage.googleapis.com/ml-pipeline/release/0.1.8/kfp.tar.gz
---> Using cache
---> 02650ceaa267
Step 9/49 : USER root
---> Using cache
---> 29d89d4486a6
Step 10/49 : ENV DEBIAN_FRONTEND noninteractive
---> Using cache
---> a2b18f365973
Step 11/49 : ENV NB_USER jovyan
---> Using cache
---> a34897c71bad
Step 12/49 : ENV NB_UID 1000
---> Using cache
---> 26248d522b65
Step 13/49 : ENV HOME /home/$NB_USER
---> Using cache
---> c5bd90e2b20e
Step 14/49 : ENV NB_PREFIX /
---> Using cache
---> 24f9dc6b7dd5
Step 15/49 : ENV CONDA_DIR=/opt/conda
---> Using cache
---> 1090bc6f7459
Step 16/49 : ENV PATH $CONDA_DIR/bin:$PATH
---> Using cache
---> 327cb873db7d
Step 17/49 : ENV TF_PACKAGE $TF_PACKAGE
---> Using cache
---> 694f0688f1c4
Step 18/49 : ENV TF_PACKAGE_27 $TF_PACKAGE_PY_27
---> Using cache
---> 6c8f15e3304f
Step 19/49 : ENV TF_SERVING_VERSION $TF_PACKAGE_PY_27
---> Using cache
---> 25f703be99a6
Step 20/49 : ENV TFMA_VERSION $TFMA_VERSION
---> Using cache
---> 09e842030748
Step 21/49 : ENV TFDV_VERSION $TFDV_VERSION
---> Using cache
---> b74ee8dde192
Step 22/49 : ENV PIPELINE_SDK_PACKAGE $PIPELINE_SDK_PACKAGE
---> Using cache
---> 4694a6064b86
Step 23/49 : SHELL ["/bin/bash", "-c"]
---> Using cache
---> f2aaf96d45d7
Step 24/49 : RUN apt-get update && apt-get install -yq --no-install-recommends apt-transport-https build-essential bzip2 ca-certificates curl g++ git gnupg graphviz locales lsb-release openssh-client sudo unzip vim wget zip && apt-get clean && rm -rf /var/lib/apt/lists/*
---> Using cache
---> 458463af44ad
Step 25/49 : ENV DOCKER_CREDENTIAL_GCR_VERSION=1.4.3
---> Using cache
---> 9e23c0b1522e
Step 26/49 : RUN curl -LO https://github.com/GoogleCloudPlatform/docker-credential-gcr/releases/download/v${DOCKER_CREDENTIAL_GCR_VERSION}/docker-credential-gcr_linux_amd64-${DOCKER_CREDENTIAL_GCR_VERSION}.tar.gz && tar -zxvf docker-credential-gcr_linux_amd64-${DOCKER_CREDENTIAL_GCR_VERSION}.tar.gz && mv docker-credential-gcr /usr/local/bin/docker-credential-gcr && rm docker-credential-gcr_linux_amd64-${DOCKER_CREDENTIAL_GCR_VERSION}.tar.gz && chmod +x /usr/local/bin/docker-credential-gcr
---> Using cache
---> e9ea9a76953f
Step 27/49 : RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen && locale-gen
---> Using cache
---> 948b55dd02f0
Step 28/49 : ENV LC_ALL en_US.UTF-8
---> Using cache
---> 3803ea64d1b2
Step 29/49 : ENV LANG en_US.UTF-8
---> Using cache
---> 85d1c6fb8cba
Step 30/49 : ENV LANGUAGE en_US.UTF-8
---> Using cache
---> 798de6ed1edd
Step 31/49 : RUN useradd -M -s /bin/bash -N -u $NB_UID $NB_USER && chown -R ${NB_USER}:users /usr/local/bin && mkdir -p $HOME
---> Using cache
---> 7345e7ab07e7
Step 32/49 : RUN export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)" && echo "deb https://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" > /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && apt-get update && apt-get install -y google-cloud-sdk kubectl
---> Using cache
---> d32405ebf295
Step 33/49 : RUN cd /tmp && wget --quiet https://github.com/krallin/tini/releases/download/v0.18.0/tini && echo "12d20136605531b09a2c2dac02ccee85e1b874eb322ef6baf7561cd93f93c855 *tini" | sha256sum -c - && mv tini /usr/local/bin/tini && chmod +x /usr/local/bin/tini
---> Using cache
---> 02eda068326b
Step 34/49 : ENV MINICONDA_VERSION 4.5.4
---> Using cache
---> git checkout remotes/origin/v0.5-branch
cd components/tensorflow-notebook-image
./build_image.sh ./Dockerfile submod/tensorflow-1.13.1-notebook-cpu v0.5.0 ./versions/1.13.1/version-config.json
./build_image.sh ./Dockerfile submod/tensorflow-1.13.1-notebook-cpu v0.5.0 ./versions/1.13.1/version-config.json
+ DOCKERFILE=./Dockerfile
++ dirname ./Dockerfile
+ CONTEXT_DIR=.
+ IMAGE=submod/tensorflow-1.13.1-notebook-cpu
+ TAG=v0.5.0
+ CONFIG_FILE=./versions/1.13.1/version-config.json
++ jq -r .BASE_IMAGE ./versions/1.13.1/version-config.json
+ BASE_IMAGE=ubuntu:18.04@sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378
++ jq -r .TF_PACKAGE ./versions/1.13.1/version-config.json
+ TF_PACKAGE=https://files.pythonhosted.org/packages/77/63/a9fa76de8dffe7455304c4ed635be4aa9c0bacef6e0633d87d5f54530c5c/tensorflow-1.13.1-cp36-cp36m-manylinux1_x86_64.whl
++ jq -r .TF_PACKAGE_PY_27 ./versions/1.13.1/version-config.json
+ TF_PACKAGE_PY_27=https://files.pythonhosted.org/packages/d2/ea/ab2c8c0e81bd051cc1180b104c75a865ab0fc66c89be992c4b20bbf6d624/tensorflow-1.13.1-cp27-cp27mu-manylinux1_x86_64.whl
++ jq -r .TF_SERVING_VERSION ./versions/1.13.1/version-config.json
+ TF_SERVING_VERSION=null
++ jq -r .TFMA_VERSION ./versions/1.13.1/version-config.json
+ TFMA_VERSION=0.13.0
++ jq -r .TFDV_VERSION ./versions/1.13.1/version-config.json
+ TFDV_VERSION=0.13.1
+ '[' ubuntu:18.04@sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378 == null ']'
+ '[' 0.13.0 == null ']'
+ '[' 0.13.1 == null ']'
+ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
+ docker build --pull --build-arg BASE_IMAGE=ubuntu:18.04@sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378 --build-arg TF_PACKAGE=https://files.pythonhosted.org/packages/77/63/a9fa76de8dffe7455304c4ed635be4aa9c0bacef6e0633d87d5f54530c5c/tensorflow-1.13.1-cp36-cp36m-manylinux1_x86_64.whl --build-arg TF_PACKAGE_PY_27=https://files.pythonhosted.org/packages/d2/ea/ab2c8c0e81bd051cc1180b104c75a865ab0fc66c89be992c4b20bbf6d624/tensorflow-1.13.1-cp27-cp27mu-manylinux1_x86_64.whl --build-arg TF_SERVING_VERSION=null --build-arg TFMA_VERSION=0.13.0 --build-arg TFDV_VERSION=0.13.1 -t submod/tensorflow-1.13.1-notebook-cpu:v0.5.0 -f ./Dockerfile .
Sending build context to Docker daemon 19.91MB
Step 1/49 : ARG BASE_IMAGE=ubuntu:18.04@sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378
Step 2/49 : FROM $BASE_IMAGE
sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378: Pulling from library/ubuntu
Digest: sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378
Status: Image is up to date for ubuntu:18.04@sha256:de774a3145f7ca4f0bd144c7d4ffb2931e06634f11529653b23eba85aef8e378
---> cd6d8154f1e1
Step 3/49 : ARG TF_PACKAGE=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.7.0-cp36-cp36m-linux_x86_64.whl
---> Using cache
---> a1b18a85f56c
Step 4/49 : ARG TF_PACKAGE_PY_27=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.7.0-cp27-none-linux_x86_64.whl
---> Using cache
---> 6d2f2fd6096e
Step 5/49 : ARG TF_SERVING_VERSION=0.0.0
---> Using cache
---> dd432d1253f0
Step 6/49 : ARG TFMA_VERSION
---> Using cache
---> e4f84164ded0
Step 7/49 : ARG TFDV_VERSION
---> Using cache
---> 029209823b1b
Step 8/49 : ARG PIPELINE_SDK_PACKAGE=https://storage.googleapis.com/ml-pipeline/release/0.1.8/kfp.tar.gz
---> Using cache
---> 02650ceaa267
Step 9/49 : USER root
---> Using cache
---> 29d89d4486a6
Step 10/49 : ENV DEBIAN_FRONTEND noninteractive
---> Using cache
---> a2b18f365973
Step 11/49 : ENV NB_USER jovyan
---> Using cache
---> a34897c71bad
Step 12/49 : ENV NB_UID 1000
---> Using cache
---> 26248d522b65
Step 13/49 : ENV HOME /home/$NB_USER
---> Using cache
---> c5bd90e2b20e
Step 14/49 : ENV NB_PREFIX /
---> Using cache
---> 24f9dc6b7dd5
Step 15/49 : ENV CONDA_DIR=/opt/conda
---> Using cache
---> 1090bc6f7459
Step 16/49 : ENV PATH $CONDA_DIR/bin:$PATH
---> Using cache
---> 327cb873db7d
Step 17/49 : ENV TF_PACKAGE $TF_PACKAGE
---> Using cache
---> 694f0688f1c4
Step 18/49 : ENV TF_PACKAGE_27 $TF_PACKAGE_PY_27
---> Using cache
---> 6c8f15e3304f
Step 19/49 : ENV TF_SERVING_VERSION $TF_PACKAGE_PY_27
---> Using cache
---> 25f703be99a6
Step 20/49 : ENV TFMA_VERSION $TFMA_VERSION
---> Using cache
---> 09e842030748
Step 21/49 : ENV TFDV_VERSION $TFDV_VERSION
---> Using cache
---> b74ee8dde192
Step 22/49 : ENV PIPELINE_SDK_PACKAGE $PIPELINE_SDK_PACKAGE
---> Using cache
---> 4694a6064b86
Step 23/49 : SHELL ["/bin/bash", "-c"]
---> Using cache
---> f2aaf96d45d7
Step 24/49 : RUN apt-get update && apt-get install -yq --no-install-recommends apt-transport-https build-essential bzip2 ca-certificates curl g++ git gnupg graphviz locales lsb-release openssh-client sudo unzip vim wget zip && apt-get clean && rm -rf /var/lib/apt/lists/*
---> Using cache
---> 458463af44ad
Step 25/49 : ENV DOCKER_CREDENTIAL_GCR_VERSION=1.4.3
---> Using cache
---> 9e23c0b1522e
Step 26/49 : RUN curl -LO https://github.com/GoogleCloudPlatform/docker-credential-gcr/releases/download/v${DOCKER_CREDENTIAL_GCR_VERSION}/docker-credential-gcr_linux_amd64-${DOCKER_CREDENTIAL_GCR_VERSION}.tar.gz && tar -zxvf docker-credential-gcr_linux_amd64-${DOCKER_CREDENTIAL_GCR_VERSION}.tar.gz && mv docker-credential-gcr /usr/local/bin/docker-credential-gcr && rm docker-credential-gcr_linux_amd64-${DOCKER_CREDENTIAL_GCR_VERSION}.tar.gz && chmod +x /usr/local/bin/docker-credential-gcr
---> Using cache
---> e9ea9a76953f
Step 27/49 : RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen && locale-gen
---> Using cache
---> 948b55dd02f0
Step 28/49 : ENV LC_ALL en_US.UTF-8
---> Using cache
---> 3803ea64d1b2
Step 29/49 : ENV LANG en_US.UTF-8
---> Using cache
---> 85d1c6fb8cba
Step 30/49 : ENV LANGUAGE en_US.UTF-8
---> Using cache
---> 798de6ed1edd
Step 31/49 : RUN useradd -M -s /bin/bash -N -u $NB_UID $NB_USER && chown -R ${NB_USER}:users /usr/local/bin && mkdir -p $HOME
---> Using cache
---> 7345e7ab07e7
Step 32/49 : RUN export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)" && echo "deb https://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" > /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && apt-get update && apt-get install -y google-cloud-sdk kubectl
---> Using cache
---> d32405ebf295
Step 33/49 : RUN cd /tmp && wget --quiet https://github.com/krallin/tini/releases/download/v0.18.0/tini && echo "12d20136605531b09a2c2dac02ccee85e1b874eb322ef6baf7561cd93f93c855 *tini" | sha256sum -c - && mv tini /usr/local/bin/tini && chmod +x /usr/local/bin/tini
---> Using cache
---> 02eda068326b
Step 34/49 : ENV MINICONDA_VERSION 4.5.4
---> Using cache
---> 165504fafac5
Step 35/49 : RUN cd /tmp && mkdir -p $CONDA_DIR && wget --quiet https://repo.continuum.io/miniconda/Miniconda3-${MINICONDA_VERSION}-Linux-x86_64.sh && echo "a946ea1d0c4a642ddf0c3a26a18bb16d *Miniconda3-${MINICONDA_VERSION}-Linux-x86_64.sh" | md5sum -c - && /bin/bash Miniconda3-${MINICONDA_VERSION}-Linux-x86_64.sh -f -b -p ${CONDA_DIR} && rm Miniconda3-${MINICONDA_VERSION}-Linux-x86_64.sh && conda config --system --prepend channels conda-forge && conda config --system --set auto_update_conda false && conda config --system --set show_channel_urls true && conda update --all && conda update conda && conda clean -tipsy
---> Using cache
---> 797f08850d56
Step 36/49 : RUN pip install --upgrade pip==19.0.1 && pip --no-cache-dir install ${TF_PACKAGE} jupyter jupyter-console==6.0.0 jupyterhub jupyterlab xgboost git+https://github.com/kubeflow/fairing@bd8313d9a4e4dd3b69a840e7cf0ea0d2326d720e ${PIPELINE_SDK_PACKAGE} && conda clean -tipsy
---> Running in 8da64b80af76
Collecting pip==19.0.1
Downloading https://files.pythonhosted.org/packages/46/dc/7fd5df840efb3e56c8b4f768793a237ec4ee59891959d6a215d63f727023/pip-19.0.1-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
Found existing installation: pip 19.1.1
Uninstalling pip-19.1.1:
Successfully uninstalled pip-19.1.1
Successfully installed pip-19.0.1
tensorflow-1.13.1-cp36-cp36m-manylinux1_x86_64.whl is not a supported wheel on this platform.
You are using pip version 19.0.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
The command '/bin/bash -c pip install --upgrade pip==19.0.1 && pip --no-cache-dir install ${TF_PACKAGE} jupyter jupyter-console==6.0.0 jupyterhub jupyterlab xgboost git+https://github.com/kubeflow/fairing@bd8313d9a4e4dd3b69a840e7cf0ea0d2326d720e ${PIPELINE_SDK_PACKAGE} && conda clean -tipsy' returned a non-zero code: 1
Step 35/49 : RUN cd /tmp && mkdir -p $CONDA_DIR && wget --quiet https://repo.continuum.io/miniconda/Miniconda3-${MINICONDA_VERSION}-Linux-x86_64.sh && echo "a946ea1d0c4a642ddf0c3a26a18bb16d *Miniconda3-${MINICONDA_VERSION}-Linux-x86_64.sh" | md5sum -c - && /bin/bash Miniconda3-${MINICONDA_VERSION}-Linux-x86_64.sh -f -b -p ${CONDA_DIR} && rm Miniconda3-${MINICONDA_VERSION}-Linux-x86_64.sh && conda config --system --prepend channels conda-forge && conda config --system --set auto_update_conda false && conda config --system --set show_channel_urls true && conda update --all && conda update conda && conda clean -tipsy
---> Using cache
---> 797f08850d56
Step 36/49 : RUN pip install --upgrade pip==19.0.1 && pip --no-cache-dir install ${TF_PACKAGE} jupyter jupyter-console==6.0.0 jupyterhub jupyterlab xgboost git+https://github.com/kubeflow/fairing@bd8313d9a4e4dd3b69a840e7cf0ea0d2326d720e ${PIPELINE_SDK_PACKAGE} && conda clean -tipsy
---> Running in 8da64b80af76
Collecting pip==19.0.1
Downloading https://files.pythonhosted.org/packages/46/dc/7fd5df840efb3e56c8b4f768793a237ec4ee59891959d6a215d63f727023/pip-19.0.1-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
Found existing installation: pip 19.1.1
Uninstalling pip-19.1.1:
Successfully uninstalled pip-19.1.1
Successfully installed pip-19.0.1
tensorflow-1.13.1-cp36-cp36m-manylinux1_x86_64.whl is not a supported wheel on this platform.
You are using pip version 19.0.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
The command '/bin/bash -c pip install --upgrade pip==19.0.1 && pip --no-cache-dir install ${TF_PACKAGE} jupyter jupyter-console==6.0.0 jupyterhub jupyterlab xgboost git+https://github.com/kubeflow/fairing@bd8313d9a4e4dd3b69a840e7cf0ea0d2326d720e ${PIPELINE_SDK_PACKAGE} && conda clean -tipsy' returned a non-zero code: 1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment