Skip to content

Instantly share code, notes, and snippets.

@rsimd

rsimd/Dockerfile

Last active June 13, 2019 13:55
Show Gist options
  • Save rsimd/be6b23f027bc0b7641d699268843ab53 to your computer and use it in GitHub Desktop.
Save rsimd/be6b23f027bc0b7641d699268843ab53 to your computer and use it in GitHub Desktop.
Download ZIP
Error of my Dockerfile for CuArrays.jl and Flux.jl
Raw
_info
(base) kr@lacia:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS"
(base) kr@lacia:~$ uname -a
Linux lacia 4.15.0-51-generic #55-Ubuntu SMP Wed May 15 14:27:21 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
(base) kr@lacia:~$ sudo lshw -class processor
*-cpu
description: CPU
product: AMD Ryzen 7 2700X Eight-Core Processor
vendor: Advanced Micro Devices [AMD]
physical id: 14
bus info: cpu@0
version: AMD Ryzen 7 2700X Eight-Core Processor
serial: Unknown
slot: AM4
size: 3700MHz
width: 64 bits
clock: 100MHz
capabilities: x86-64 fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate sme ssbd ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca
configuration: cores=8 enabledcores=8 threads=16
(base) kr@lacia:~$ lspci | grep -i nvidia
1d:00.0 VGA compatible controller: NVIDIA Corporation Device 1f07 (rev a1)
1d:00.1 Audio device: NVIDIA Corporation Device 10f9 (rev a1)
1d:00.2 USB controller: NVIDIA Corporation Device 1ada (rev a1)
1d:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device 1adb (rev a1)
Raw
Dockerfile
FROM nvidia/cuda:10.0-cudnn7-devel-ubuntu18.04
#FROM nvidia/cuda:10.0-cudnn7-devel-ubuntu16.04
#FROM nvidia/cuda:8.0-cudnn7-devel-ubuntu16.04
#FROM pytorch/pytorch:1.1.0-cuda10.0-cudnn7.5-devel
ENV LANG C.UTF-8
ENV NVIDIA_VISIBLE_DEVICES 0
ENV NVIDIA_DRIVER_CAPABILITIES utility,compute
#ENV CUDA_PATH /usr/local/cuda
ENV CUDA_ROOT /usr/local/cuda
#ENV CUDA_HOME /uer/local/cuda
ENV LD_LIBRARY_PATH /usr/local/cuda/lib64:/usr/local/cuda/lib:$LD_LIBRARY_PATH
ENV PATH /usr/local/cuda/bin:$PATH
## installation
RUN apt-get clean && \
apt-get update && \
apt-get install --yes \
build-essential \
curl \
ca-certificates \
software-properties-common \
wget \
git && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# for julia
RUN mkdir -p /opt/julia/usr && \
curl -s -L https://julialang-s3.julialang.org/bin/linux/x64/1.0/julia-1.0.4-linux-x86_64.tar.gz | tar -C /opt/julia/usr -x -z --strip-components=1 -f -
WORKDIR /
RUN ln -s /opt/julia/usr/bin/julia /usr/bin/julia && \
ln -s /opt/julia/usr/bin/julia-debug /usr/bin/julia-debug
# for juliagpu
RUN julia -e 'using Pkg; Pkg.add("CuArrays"); Pkg.build("CuArrays");Pkg.add("Flux"); Pkg.add("BSON")'
Raw
logfile
(base) kr@lacia:~/Documents$ docker build ./ -t jgpu-ub18-10-7
Sending build context to Docker daemon 3.072kB
Step 1/12 : FROM nvidia/cuda:10.0-cudnn7-devel-ubuntu18.04
---> 074d6f612d7b
Step 2/12 : ENV LANG C.UTF-8
---> Using cache
---> b9f6e19ca8a9
Step 3/12 : ENV NVIDIA_VISIBLE_DEVICES 0
---> Using cache
---> d9a9b73812c4
Step 4/12 : ENV NVIDIA_DRIVER_CAPABILITIES utility,compute
---> Using cache
---> 3dc5d3fe3bbd
Step 5/12 : ENV CUDA_ROOT /usr/local/cuda
---> Using cache
---> 6170e7de1211
Step 6/12 : ENV LD_LIBRARY_PATH /usr/local/cuda/lib64:/usr/local/cuda/lib$LD_LIBRARY_PATH
---> Using cache
---> e1f6a30f61d6
Step 7/12 : ENV PATH /usr/local/cuda/bin:$PATH
---> Using cache
---> 4b515d39fc1c
Step 8/12 : RUN apt-get clean && apt-get update && apt-get install --yes build-essential curl ca-certificates software-properties-common wget git && apt-get clean && rm -rf /var/lib/apt/lists/*
---> Using cache
---> 09073328579c
Step 9/12 : RUN mkdir -p /opt/julia/usr && curl -s -L https://julialang-s3.julialang.org/bin/linux/x64/1.0/julia-1.0.4-linux-x86_64.tar.gz | tar -C /opt/julia/usr -x -z --strip-components=1 -f -
---> Using cache
---> c9eda5587f57
Step 10/12 : WORKDIR /
---> Using cache
---> 96bbb9aa41ff
Step 11/12 : RUN ln -s /opt/julia/usr/bin/julia /usr/bin/julia && ln -s /opt/julia/usr/bin/julia-debug /usr/bin/julia-debug
---> Using cache
---> 7e245b2a428c
Step 12/12 : RUN julia -e 'using Pkg; Pkg.add("CuArrays"); Pkg.build("CuArrays");Pkg.add("Flux"); Pkg.add("BSON")'
---> Running in e058c493835b
Cloning default registries into /root/.julia/registries
Cloning registry General from "https://github.com/JuliaRegistries/General.git"
Resolving package versions...
Installed Reexport ─────────── v0.2.0
Installed Requires ─────────── v0.5.2
Installed BinaryProvider ───── v0.5.4
Installed CUDAnative ───────── v2.1.3
Installed LLVM ─────────────── v1.1.1
Installed DataStructures ───── v0.15.0
Installed StaticArrays ─────── v0.11.0
Installed CuArrays ─────────── v1.0.2
Installed TimerOutputs ─────── v0.5.0
Installed OrderedCollections ─ v1.1.0
Installed GPUArrays ────────── v0.7.1
Installed VersionParsing ───── v1.1.3
Installed CUDAdrv ──────────── v3.0.1
Installed NNlib ────────────── v0.6.0
Installed CUDAapi ──────────── v0.6.3
Installed FFTW ─────────────── v0.2.4
Installed JSON ─────────────── v0.20.0
Installed Crayons ──────────── v4.0.0
Installed Tokenize ─────────── v0.5.4
Installed Adapt ────────────── v0.4.2
Installed AbstractFFTs ─────── v0.4.1
Installed CSTParser ────────── v0.6.0
Installed Compat ───────────── v2.1.0
Installed FillArrays ───────── v0.6.3
Installed MacroTools ───────── v0.5.0
Installed Conda ────────────── v1.3.0
Updating `~/.julia/environments/v1.0/Project.toml`
[3a865a2d] + CuArrays v1.0.2
Updating `~/.julia/environments/v1.0/Manifest.toml`
[621f4979] + AbstractFFTs v0.4.1
[79e6a3ab] + Adapt v0.4.2
[b99e7846] + BinaryProvider v0.5.4
[00ebfdb7] + CSTParser v0.6.0
[3895d2a7] + CUDAapi v0.6.3
[c5f51814] + CUDAdrv v3.0.1
[be33ccc6] + CUDAnative v2.1.3
[34da2185] + Compat v2.1.0
[8f4d0f93] + Conda v1.3.0
[a8cc5b0e] + Crayons v4.0.0
[3a865a2d] + CuArrays v1.0.2
[864edb3b] + DataStructures v0.15.0
[7a1cc6ca] + FFTW v0.2.4
[1a297f60] + FillArrays v0.6.3
[0c68f7d7] + GPUArrays v0.7.1
[682c06a0] + JSON v0.20.0
[929cbde3] + LLVM v1.1.1
[1914dd2f] + MacroTools v0.5.0
[872c559c] + NNlib v0.6.0
[bac558e1] + OrderedCollections v1.1.0
[189a3867] + Reexport v0.2.0
[ae029012] + Requires v0.5.2
[90137ffa] + StaticArrays v0.11.0
[a759f4b9] + TimerOutputs v0.5.0
[0796e94c] + Tokenize v0.5.4
[81def892] + VersionParsing v1.1.3
[2a0f44e3] + Base64
[ade2ca70] + Dates
[8bb1440f] + DelimitedFiles
[8ba89e20] + Distributed
[b77e0a4c] + InteractiveUtils
[76f85450] + LibGit2
[8f399da3] + Libdl
[37e2e46d] + LinearAlgebra
[56ddb016] + Logging
[d6f4376e] + Markdown
[a63ad114] + Mmap
[44cfe95a] + Pkg
[de0858da] + Printf
[3fa0cd96] + REPL
[9a3f8284] + Random
[ea8e919c] + SHA
[9e88b42a] + Serialization
[1a1011a3] + SharedArrays
[6462fe0b] + Sockets
[2f01184e] + SparseArrays
[10745b16] + Statistics
[8dfed614] + Test
[cf7118a7] + UUIDs
[4ec0a83e] + Unicode
Building CUDAdrv ───→ `~/.julia/packages/CUDAdrv/WVU1H/deps/build.log`
Building LLVM ──────→ `~/.julia/packages/LLVM/ViliQ/deps/build.log`
┌ Error: Error building `CUDAdrv`:
│ Could not find CUDA driver library.
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1096
Building CUDAnative → `~/.julia/packages/CUDAnative/ytV2j/deps/build.log`
Building Conda ─────→ `~/.julia/packages/Conda/kLXeC/deps/build.log`
┌ Error: Error building `CUDAnative`:
│ Dependent package CUDAdrv.jl has not been built successfully.
│
│ This is not a fatal error, but GPU functionality will be unavailable.
│ If you expected this to work, please open a thread on
│ https://discourse.julialang.org/c/domain/gpu
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1096
Building FFTW ──────→ `~/.julia/packages/FFTW/p7sLQ/deps/build.log`
Building CuArrays ──→ `~/.julia/packages/CuArrays/PwSdF/deps/build.log`
Building CUDAdrv ───→ `~/.julia/packages/CUDAdrv/WVU1H/deps/build.log`
┌ Error: Error building `CUDAdrv`:
│ Could not find CUDA driver library.
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1096
Building LLVM ──────→ `~/.julia/packages/LLVM/ViliQ/deps/build.log`
Building CUDAnative → `~/.julia/packages/CUDAnative/ytV2j/deps/build.log`
┌ Error: Error building `CUDAnative`:
│ Dependent package CUDAdrv.jl has not been built successfully.
│
│ This is not a fatal error, but GPU functionality will be unavailable.
│ If you expected this to work, please open a thread on
│ https://discourse.julialang.org/c/domain/gpu
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1096
Building Conda ─────→ `~/.julia/packages/Conda/kLXeC/deps/build.log`
Building FFTW ──────→ `~/.julia/packages/FFTW/p7sLQ/deps/build.log`
Building CuArrays ──→ `~/.julia/packages/CuArrays/PwSdF/deps/build.log`
Resolving package versions...
Installed Media ──────────────── v0.5.0
Installed DiffResults ────────── v0.0.4
Installed Tracker ────────────── v0.2.2
Installed SpecialFunctions ───── v0.7.2
Installed ZipFile ────────────── v0.8.3
Installed Flux ───────────────── v0.8.3
Installed ColorTypes ─────────── v0.8.0
Installed SortingAlgorithms ──── v0.3.1
Installed Colors ─────────────── v0.9.5
Installed Missings ───────────── v0.4.1
Installed TranscodingStreams ─── v0.9.4
Installed CommonSubexpressions ─ v0.2.0
Installed BinDeps ────────────── v0.8.10
Installed Juno ───────────────── v0.7.0
Installed CodecZlib ──────────── v0.5.2
Installed FixedPointNumbers ──── v0.6.1
Installed URIParser ──────────── v0.4.0
Installed ForwardDiff ────────── v0.10.3
Installed AbstractTrees ──────── v0.2.1
Installed NaNMath ────────────── v0.3.2
Installed DiffRules ──────────── v0.0.10
Installed StatsBase ──────────── v0.30.0
Updating `~/.julia/environments/v1.0/Project.toml`
[587475ba] + Flux v0.8.3
Updating `~/.julia/environments/v1.0/Manifest.toml`
[1520ce14] + AbstractTrees v0.2.1
[9e28174c] + BinDeps v0.8.10
[944b1d66] + CodecZlib v0.5.2
[3da002f7] + ColorTypes v0.8.0
[5ae59095] + Colors v0.9.5
[bbf7d656] + CommonSubexpressions v0.2.0
[163ba53b] + DiffResults v0.0.4
[b552c78f] + DiffRules v0.0.10
[53c48c17] + FixedPointNumbers v0.6.1
[587475ba] + Flux v0.8.3
[f6369f11] + ForwardDiff v0.10.3
[e5e0dc1b] + Juno v0.7.0
[e89f7d12] + Media v0.5.0
[e1d29d7a] + Missings v0.4.1
[77ba4419] + NaNMath v0.3.2
[a2af1166] + SortingAlgorithms v0.3.1
[276daf66] + SpecialFunctions v0.7.2
[2913bbd2] + StatsBase v0.30.0
[9f7883ad] + Tracker v0.2.2
[3bb67fe8] + TranscodingStreams v0.9.4
[30578b45] + URIParser v0.4.0
[a5390f91] + ZipFile v0.8.3
[9abbd945] + Profile
Building ZipFile ─────────→ `~/.julia/packages/ZipFile/oD4uG/deps/build.log`
Building SpecialFunctions → `~/.julia/packages/SpecialFunctions/fvheQ/deps/build.log`
Building CodecZlib ───────→ `~/.julia/packages/CodecZlib/9jDi1/deps/build.log`
Resolving package versions...
Installed BSON ─ v0.2.3
Updating `~/.julia/environments/v1.0/Project.toml`
[fbb218c0] + BSON v0.2.3
Updating `~/.julia/environments/v1.0/Manifest.toml`
[fbb218c0] + BSON v0.2.3
Removing intermediate container e058c493835b
---> 936280d1866c
Successfully built 936280d1866c
Successfully tagged jgpu-ub18-10-7:latest
Raw
logfile2
(base) kr@lacia:~/Documents$ docker run -it jgpu-ub18-10-7 /bin/bash
root@e63d89619b08:/# julia
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.0.4 (2019-05-16)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
julia> using Flux, CuArrays
[ Info: Precompiling Flux [587475ba-b771-5e3f-ad9e-33799f191a9c]
[ Info: Precompiling CuArrays [3a865a2d-5b23-5a0f-bc46-62713ec82fae]
ERROR: LoadError: LoadError: LoadError: CUDAdrv.jl has not been configured. Please run Pkg.build("CUDAdrv") and restart Julia.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] version() at /root/.julia/packages/CUDAdrv/WVU1H/src/version.jl:10
[3] top-level scope at none:0
[4] include at ./boot.jl:317 [inlined]
[5] include_relative(::Module, ::String) at ./loading.jl:1044
[6] include at ./sysimg.jl:29 [inlined]
[7] include(::String) at /root/.julia/packages/CuArrays/PwSdF/src/blas/CUBLAS.jl:1
[8] top-level scope at none:0
[9] include at ./boot.jl:317 [inlined]
[10] include_relative(::Module, ::String) at ./loading.jl:1044
[11] include at ./sysimg.jl:29 [inlined]
[12] include(::String) at /root/.julia/packages/CuArrays/PwSdF/src/CuArrays.jl:3
[13] top-level scope at none:0
[14] include at ./boot.jl:317 [inlined]
[15] include_relative(::Module, ::String) at ./loading.jl:1044
[16] include(::Module, ::String) at ./sysimg.jl:29
[17] top-level scope at none:2
[18] eval at ./boot.jl:319 [inlined]
[19] eval(::Expr) at ./client.jl:393
[20] top-level scope at ./none:3
in expression starting at /root/.julia/packages/CuArrays/PwSdF/src/blas/libcublas_types.jl:95
in expression starting at /root/.julia/packages/CuArrays/PwSdF/src/blas/CUBLAS.jl:10
in expression starting at /root/.julia/packages/CuArrays/PwSdF/src/CuArrays.jl:47
ERROR: Failed to precompile CuArrays [3a865a2d-5b23-5a0f-bc46-62713ec82fae] to /root/.julia/compiled/v1.0/CuArrays/7YFE0.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1203
[3] _require(::Base.PkgId) at ./loading.jl:960
[4] require(::Base.PkgId) at ./loading.jl:858
[5] require(::Module, ::Symbol) at ./loading.jl:853
(v1.0) pkg> build CUDAdrv
Building CUDAdrv → `~/.julia/packages/CUDAdrv/WVU1H/deps/build.log`
┌ Error: Error building `CUDAdrv`:
│ Could not find CUDA driver library.
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1096
julia> using Pkg;Pkg.installed()
Dict{String,Union{Nothing, VersionNumber}} with 3 entries:
"BSON" => v"0.2.3"
"CuArrays" => v"1.0.2"
"Flux" => v"0.8.3"
julia>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment