Skip to content

Instantly share code, notes, and snippets.

@mizar

mizar/Dockerfile_fukauraou

Last active August 20, 2022 14:55
Show Gist options
  • Save mizar/d049939b5948af1847404cf241ca200c to your computer and use it in GitHub Desktop.
Save mizar/d049939b5948af1847404cf241ca200c to your computer and use it in GitHub Desktop.
Download ZIP
Raw
README.md

Windows11 + Docker + dlshogi/やねうら王/ふかうら王 環境構築

Docker Desktop for Windows を併用して、最新の dlshogi/ふかうら王 TensorRT対応エンジンを動かす環境を構築する試みです。

DeepLearningShogi 第31回世界コンピュータ将棋選手権バージョン, YouTube動画 【最新版】　dlshogi導入手順　【藤井二冠始め、多くの棋士が導入する将棋AI】 では、CUDA・cuDNN・TensorRTのインストールと環境変数の設定が必要でしたが、この部分を他の手順を使って構築してみたものです。

主な違い

  • ○ CUDA・cuDNN・TensorRT・環境変数の設定はDockerイメージに含めるため不要
  • × Docker Desktop for Windows のインストールが必要
  • × CUDA on WSL 対応グラフィックドライバのインストールが必要
  • × Dockerイメージのビルドが必要
  • ○ 最新のGitHub上のソースファイルから最新の思考エンジンをビルドして使える
  • × 過去の環境を再現することが不得手
  • × CUDA, TensorRT環境や dlshogi, ふかうら王 の今後の更新に伴い、インストール手順の更新が必要になる可能性
  • × インターネットからダウンロードする必要のあるデータ量が多い

PCに必要なもの

構築手順

  1. 事前準備・環境の確認
    • このページの右上隅近くにある「Download ZIP」ボタンより、このGistのZIPファイルをダウンロード。
    • ダウンロードしたZIPファイルを右クリックし、コンテキストメニューから「プロパティ」を選択する。
    • 「セキュリティ」の欄で「許可する」にチェックを入れ、「OK」ボタンを押す。
    • ダウンロードしたZIPファイルを展開し、展開したファイルについて以下の手順を実行する。
    • x00a_remove_zoneid.bat の実行（カレントディレクトリ以下のファイルに対するセキュリティ警告フラグの削除）
      • 前の手順でダウンロードしたZIPファイルのブロック設定を解除する前にZIPファイルを展開してしまった場合、この手順が必要になります。その場合は以下のようなダイアログが表示されるので、「詳細情報」→「実行」を選択して下さい。
      • image
      • image
    • x00b_os_version.bat の実行
      • Windows11 が動作している事を確認
    • x00c_get_device_display.bat の実行
      • NVIDIA GeForce RTX グラフィックが搭載されている事を確認
  2. WSLの有効化とアップデート
    • x01_enable_windowsoptionalfeature.bat を実行（「仮想マシン プラットフォーム」「Linux 用 Windows サブシステム」の有効化）、必要に応じて再起動
    • x01_wsl_update.bat を実行（「Linux 用 Windows サブシステム」の更新）
  3. x02_docker_desktop_win_installer.bat を実行、もしくは https://hub.docker.com/editions/community/docker-ce-desktop-windows を開き、ブラウザより Docker Desktop for Windows をダウンロードしてインストール、再起動
  4. x03_cuda_on_wsl_driver_installer.bat を実行、もしくは https://developer.nvidia.com/cuda/wsl/download を開き、ブラウザより Geforce Driver をダウンロードしてインストール、再起動
  5. x04_suisho4kai_setup.bat を実行（GPUを使わない、やねうら王+水匠4改 のdockerイメージ構築）
    • x04b_yaneuraou_nocache.bat は やねうら王 最新ビルドの再構築が必要な時用、初回は実行不要。
    • ダウンロード高速化のため、一部Ubuntuパッケージは近隣のミラーサイトからダウンロードする設定をしていますが、稀にダウンロードが失敗する場合があるようです。その際はお手数ですが再実行を試してみて下さい。
  6. x05_dlshogi_with_gct_wcsc31_setup.bat を実行（GPUを使う、dlshogi/ふかうら王 のdockerイメージ構築、「DeepLearningShogi 第31回世界コンピュータ将棋選手権バージョン」のダウンロード）
    • x05b_fukauraou_nocache.bat は dlshogi/ふかうら王 最新ビルドの再構築が必要な時用、初回は実行不要。
    • ダウンロード高速化のため、一部Ubuntuパッケージは近隣のミラーサイトからダウンロードする設定をしていますが、稀にダウンロードが失敗する場合があるようです。その際はお手数ですが再実行を試してみて下さい。
  7. GUIへのエンジンの登録
    • 将棋所, ShogiGUI などの USIプロトコルに対応したGUI のインストール
    • GUIに engine_suisho4kai-YaneuraOu-clang.bat をエンジンとして登録（CPU用、やねうら王 engine + 水匠4改 eval）
    • GUIに engine_dlshogi_with_gct_wcsc31-dlshogi-clang.bat をエンジンとして登録（CPU+GPU用、 dlshogi engine + dlshogi_wcsc31 model）
    • GUIに engine_dlshogi_with_gct_wcsc31-FukauraOu-clang.bat をエンジンとして登録（CPU+GPU用、 ふかうら王 engine + dlshogi_wcsc31 model）
    • ShogiGUI に .bat ファイルをエンジンとして登録する際、ファイル形式を 実行ファイル (*.exe) から すべてのファイル (*.*) に変更する必要があります。
    • dlshogi/ふかうら王に 第2回世界将棋AI電竜戦エキシビジョンバージョン のモデルファイルを適用したい方は、 第2回世界将棋AI電竜戦エキシビジョンバージョン ライセンス条項 に同意した上で、別途モデルファイルのダウンロードと個別のエンジン設定をお願いします。
Raw
Dockerfile_fukauraou
FROM nvcr.io/nvidia/cuda:11.4.2-cudnn8-devel-ubuntu20.04 AS builder
SHELL ["/bin/bash", "-c"]
RUN \
export DEBIAN_FRONTEND=noninteractive &&\
sed -i.bak -r 's!(deb|deb-src) http://archive\.ubuntu\.com/\S+!\1 mirror://mirrors.ubuntu.com/mirrors.txt!' /etc/apt/sources.list &&\
apt-get update &&\
apt-get -y -o DPkg::options::="--force-confdef" -o DPkg::options::="--force-confold" install \
curl gpg &&\
curl "https://keyserver.ubuntu.com/pks/lookup?search=0x1e9377a2ba9ef27f&op=get" | gpg --no-default-keyring --keyring /usr/share/keyrings/ubuntu-toolchain-r.gpg --import - &&\
echo "deb [signed-by=/usr/share/keyrings/ubuntu-toolchain-r.gpg] http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu focal main" | tee /etc/apt/sources.list.d/ubuntu-toolchain-r-test-focal.list &&\
curl "https://apt.llvm.org/llvm-snapshot.gpg.key" | gpg --no-default-keyring --keyring /usr/share/keyrings/llvm-snapshot.gpg --import - &&\
echo "deb [signed-by=/usr/share/keyrings/llvm-snapshot.gpg] https://apt.llvm.org/focal/ llvm-toolchain-focal main" | tee /etc/apt/sources.list.d/llvm-toolchain-focal.list &&\
curl "https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin" -o /etc/apt/preferences.d/cuda-repository-pin-600 &&\
curl "https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub" | gpg --no-default-keyring --keyring /usr/share/keyrings/cuda.gpg --import - &&\
echo "deb [signed-by=/usr/share/keyrings/cuda.gpg] https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /" | tee /etc/apt/sources.list.d/cuda.list &&\
apt-get update &&\
apt-get -y -o DPkg::options::="--force-confdef" -o DPkg::options::="--force-confold" install \
build-essential clang-14 libomp-14-dev g++-11 libopenblas-dev git ruby unzip vim-tiny \
libnvinfer-dev libnvinfer-plugin-dev libnvonnxparsers-dev libnvparsers-dev &&\
apt-get -y -o DPkg::options::="--force-confdef" -o DPkg::options::="--force-confold" upgrade &&\
apt-get clean &&\
rm -rf /var/lib/apt/lists/* &&\
rm /tmp/* -rf &&\
mkdir -p /workspace &&\
echo "2021_10_16_0000"
#COPY yaneuraou.diff /workspace/yaneuraou.diff
#COPY dlshogi.diff /workspace/dlshogi.diff
WORKDIR /workspace
RUN \
echo "shallow clone : shogi-server" &&\
git clone https://github.com/mizar/shogi-server.git /workspace/shogi-server -b wcsc2021 --depth 1 &&\
echo "shallow clone : YaneuraOu master" &&\
git clone https://github.com/yaneurao/YaneuraOu.git /workspace/YaneuraOu -b master --depth 1 &&\
echo "shallow clone : dlshogi master" &&\
git clone https://github.com/TadaoYamaoka/DeepLearningShogi.git /workspace/DeepLearningShogi -b master --depth 1 &&\
echo "build : FukauraOu" &&\
JOBS=`grep -c ^processor /proc/cpuinfo 2>/dev/null` &&\
cd /workspace/YaneuraOu/source &&\
# git apply /workspace/yaneuraou.diff &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_DEEP_TENSOR_RT TARGET_CPU=AVX2 COMPILER=g++-11 EXTRA_CPPFLAGS='-I/usr/local/cuda/include' EXTRA_LDFLAGS='-L/usr/local/cuda/lib64' normal >& >(tee /workspace/FukauraOu-gcc.log) &&\
mv YaneuraOu-by-gcc /workspace/FukauraOu-gcc &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_DEEP_TENSOR_RT clean &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_DEEP_TENSOR_RT TARGET_CPU=AVX2 COMPILER=clang++-14 EXTRA_CPPFLAGS='-I/usr/local/cuda/include' EXTRA_LDFLAGS='-L/usr/local/cuda/lib64' normal >& >(tee /workspace/FukauraOu-clang.log) &&\
mv YaneuraOu-by-gcc /workspace/FukauraOu-clang &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_DEEP_TENSOR_RT clean &&\
echo "build : YaneuraOu" &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE TARGET_CPU=AVX2 COMPILER=g++-11 normal >& >(tee /workspace/YaneuraOu-gcc-normal.log) &&\
mv YaneuraOu-by-gcc /workspace/YaneuraOu-gcc-normal &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE clean &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE TARGET_CPU=AVX2 COMPILER=clang++-14 normal >& >(tee /workspace/YaneuraOu-clang-normal.log) &&\
mv YaneuraOu-by-gcc /workspace/YaneuraOu-clang-normal &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE clean &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE TARGET_CPU=AVX2 COMPILER=g++-11 tournament >& >(tee /workspace/YaneuraOu-gcc-tournament.log) &&\
mv YaneuraOu-by-gcc /workspace/YaneuraOu-gcc-tournament &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE clean &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE TARGET_CPU=AVX2 COMPILER=clang++-14 tournament >& >(tee /workspace/YaneuraOu-clang-tournament.log) &&\
mv YaneuraOu-by-gcc /workspace/YaneuraOu-clang-tournament &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE clean &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE TARGET_CPU=AVX2 COMPILER=g++-11 evallearn >& >(tee /workspace/YaneuraOu-gcc-evallearn.log) &&\
mv YaneuraOu-by-gcc /workspace/YaneuraOu-gcc-evallearn &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE clean &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE TARGET_CPU=AVX2 COMPILER=clang++-14 evallearn >& >(tee /workspace/YaneuraOu-clang-evallearn.log) &&\
mv YaneuraOu-by-gcc /workspace/YaneuraOu-clang-evallearn &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE clean &&\
echo "build : dlshogi" &&\
cd /workspace/DeepLearningShogi/usi &&\
# git apply /workspace/dlshogi.diff &&\
make -j${JOBS} CC=g++-11 >& >(tee /workspace/dlshogi-gcc.log) &&\
mv bin/usi /workspace/dlshogi-gcc &&\
make clean &&\
make -j${JOBS} CC=clang++-14 >& >(tee /workspace/dlshogi-clang.log) &&\
mv bin/usi /workspace/dlshogi-clang &&\
make clean &&\
make -j${JOBS} CC=g++-11 make_book >& >(tee /workspace/make_book-gcc.log) &&\
mv bin/make_book /workspace/make_book-gcc &&\
make clean &&\
make -j${JOBS} CC=clang++-14 make_book >& >(tee /workspace/make_book-clang.log) &&\
mv bin/make_book /workspace/make_book-clang &&\
make clean &&\
echo "2021_10_16_0000"
FROM nvcr.io/nvidia/cuda:11.4.2-cudnn8-runtime-ubuntu20.04 AS runner
SHELL ["/bin/bash", "-c"]
RUN \
export DEBIAN_FRONTEND=noninteractive &&\
sed -i.bak -r 's!(deb|deb-src) http://archive\.ubuntu\.com/\S+!\1 mirror://mirrors.ubuntu.com/mirrors.txt!' /etc/apt/sources.list &&\
apt-get update &&\
apt-get -y -o DPkg::options::="--force-confdef" -o DPkg::options::="--force-confold" install \
curl gpg &&\
curl "https://keyserver.ubuntu.com/pks/lookup?search=0x1e9377a2ba9ef27f&op=get" | gpg --no-default-keyring --keyring /usr/share/keyrings/ubuntu-toolchain-r.gpg --import - &&\
echo "deb [signed-by=/usr/share/keyrings/ubuntu-toolchain-r.gpg] http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu focal main" | tee /etc/apt/sources.list.d/ubuntu-toolchain-r-test-focal.list &&\
curl "https://apt.llvm.org/llvm-snapshot.gpg.key" | gpg --no-default-keyring --keyring /usr/share/keyrings/llvm-snapshot.gpg --import - &&\
echo "deb [signed-by=/usr/share/keyrings/llvm-snapshot.gpg] https://apt.llvm.org/focal/ llvm-toolchain-focal main" | tee /etc/apt/sources.list.d/llvm-toolchain-focal.list &&\
curl "https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin" -o /etc/apt/preferences.d/cuda-repository-pin-600 &&\
curl "https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub" | gpg --no-default-keyring --keyring /usr/share/keyrings/cuda.gpg --import - &&\
echo "deb [signed-by=/usr/share/keyrings/cuda.gpg] https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /" | tee /etc/apt/sources.list.d/cuda.list &&\
apt-get update &&\
apt-get -y -o DPkg::options::="--force-confdef" -o DPkg::options::="--force-confold" install \
git ruby unzip vim-tiny libstdc++6 libgomp1 libomp5-14 libopenblas0 libnvinfer8 libnvinfer-plugin8 libnvonnxparsers8 libnvparsers8 &&\
apt-get -y -o DPkg::options::="--force-confdef" -o DPkg::options::="--force-confold" upgrade &&\
apt-get clean &&\
rm -rf /var/lib/apt/lists/* &&\
rm /tmp/* -rf &&\
mkdir -p /workspace/volume &&\
echo "2021_10_16_0000"
RUN \
echo "download : suisho4kai" &&\
mkdir -p /workspace/suisho4kai/eval &&\
cd /workspace &&\
FILE_ID=1uDezE_TBIT1m0Zxz0vJbB0pdCzpT-D6U &&\
FILE_NAME=suisho4kai_20210826.zip &&\
curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=${FILE_ID}" > /dev/null &&\
CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)" &&\
curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=${FILE_ID}" -o ${FILE_NAME} &&\
unzip -j ${FILE_NAME} **/engine_options.txt -d /workspace/suisho4kai &&\
unzip -j ${FILE_NAME} **/nn.bin -d /workspace/suisho4kai/eval &&\
rm ${FILE_NAME} &&\
echo "2021_10_16_0000"
RUN \
echo "shallow clone : shogi-server" &&\
git clone https://github.com/mizar/shogi-server.git /workspace/shogi-server -b wcsc2021 --depth 1 &&\
echo "shallow clone : YaneuraOu master" &&\
git clone https://github.com/yaneurao/YaneuraOu.git /workspace/YaneuraOu -b master --depth 1 &&\
echo "shallow clone : dlshogi master" &&\
git clone https://github.com/TadaoYamaoka/DeepLearningShogi.git /workspace/DeepLearningShogi -b master --depth 1 &&\
echo "2021_10_16_0000"
WORKDIR /workspace/volume
COPY --from=builder \
/workspace/FukauraOu-gcc \
/workspace/FukauraOu-gcc.log \
/workspace/FukauraOu-clang \
/workspace/FukauraOu-clang.log \
/workspace/YaneuraOu-gcc-normal \
/workspace/YaneuraOu-gcc-normal.log \
/workspace/YaneuraOu-clang-normal \
/workspace/YaneuraOu-clang-normal.log \
/workspace/YaneuraOu-gcc-tournament \
/workspace/YaneuraOu-gcc-tournament.log \
/workspace/YaneuraOu-clang-tournament \
/workspace/YaneuraOu-clang-tournament.log \
/workspace/YaneuraOu-gcc-evallearn \
/workspace/YaneuraOu-gcc-evallearn.log \
/workspace/YaneuraOu-clang-evallearn \
/workspace/YaneuraOu-clang-evallearn.log \
/workspace/dlshogi-gcc \
/workspace/dlshogi-gcc.log \
/workspace/dlshogi-clang \
/workspace/dlshogi-clang.log \
/workspace/make_book-gcc \
/workspace/make_book-gcc.log \
/workspace/make_book-clang \
/workspace/make_book-clang.log \
/workspace/
Raw
Dockerfile_yaneuraou_suisho4kai
FROM ubuntu:20.04 AS builder
SHELL ["/bin/bash", "-c"]
RUN \
export DEBIAN_FRONTEND=noninteractive &&\
sed -i.bak -r 's!(deb|deb-src) http://archive\.ubuntu\.com/\S+!\1 mirror://mirrors.ubuntu.com/mirrors.txt!' /etc/apt/sources.list &&\
apt-get update &&\
apt-get -y -o DPkg::options::="--force-confdef" -o DPkg::options::="--force-confold" install \
curl gpg &&\
curl "https://keyserver.ubuntu.com/pks/lookup?search=0x1e9377a2ba9ef27f&op=get" | gpg --no-default-keyring --keyring /usr/share/keyrings/ubuntu-toolchain-r.gpg --import - &&\
echo "deb [signed-by=/usr/share/keyrings/ubuntu-toolchain-r.gpg] http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu focal main" | tee /etc/apt/sources.list.d/ubuntu-toolchain-r-test-focal.list &&\
curl "https://apt.llvm.org/llvm-snapshot.gpg.key" | gpg --no-default-keyring --keyring /usr/share/keyrings/llvm-snapshot.gpg --import - &&\
echo "deb [signed-by=/usr/share/keyrings/llvm-snapshot.gpg] https://apt.llvm.org/focal/ llvm-toolchain-focal main" | tee /etc/apt/sources.list.d/llvm-toolchain-focal.list &&\
apt-get update &&\
apt-get -y -o DPkg::options::="--force-confdef" -o DPkg::options::="--force-confold" install \
build-essential clang-14 libomp-14-dev g++-11 libopenblas-dev git ruby unzip vim-tiny &&\
apt-get -y -o DPkg::options::="--force-confdef" -o DPkg::options::="--force-confold" upgrade &&\
apt-get clean &&\
rm -rf /var/lib/apt/lists/* &&\
rm /tmp/* -rf &&\
mkdir -p /workspace &&\
echo "2021_10_16_0000"
#COPY yaneuraou.diff /workspace/yaneuraou.diff
SHELL ["/bin/bash", "-c"]
WORKDIR /workspace
RUN \
echo "shallow clone : shogi-server" &&\
git clone https://github.com/mizar/shogi-server.git /workspace/shogi-server -b wcsc2021 --depth 1 &&\
echo "shallow clone : YaneuraOu master" &&\
git clone https://github.com/yaneurao/YaneuraOu.git /workspace/YaneuraOu -b master --depth 1 &&\
echo "build : YaneuraOu" &&\
JOBS=`grep -c ^processor /proc/cpuinfo 2>/dev/null` &&\
cd /workspace/YaneuraOu/source &&\
# git apply /workspace/yaneuraou.diff &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE TARGET_CPU=AVX2 COMPILER=g++-11 normal >& >(tee /workspace/YaneuraOu-gcc-normal.log) &&\
mv YaneuraOu-by-gcc /workspace/YaneuraOu-gcc-normal &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE clean &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE TARGET_CPU=AVX2 COMPILER=clang++-14 normal >& >(tee /workspace/YaneuraOu-clang-normal.log) &&\
mv YaneuraOu-by-gcc /workspace/YaneuraOu-clang-normal &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE clean &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE TARGET_CPU=AVX2 COMPILER=g++-11 tournament >& >(tee /workspace/YaneuraOu-gcc-tournament.log) &&\
mv YaneuraOu-by-gcc /workspace/YaneuraOu-gcc-tournament &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE clean &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE TARGET_CPU=AVX2 COMPILER=clang++-14 tournament >& >(tee /workspace/YaneuraOu-clang-tournament.log) &&\
mv YaneuraOu-by-gcc /workspace/YaneuraOu-clang-tournament &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE clean &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE TARGET_CPU=AVX2 COMPILER=g++-11 evallearn >& >(tee /workspace/YaneuraOu-gcc-evallearn.log) &&\
mv YaneuraOu-by-gcc /workspace/YaneuraOu-gcc-evallearn &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE clean &&\
make -j${JOBS} YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE TARGET_CPU=AVX2 COMPILER=clang++-14 evallearn >& >(tee /workspace/YaneuraOu-clang-evallearn.log) &&\
mv YaneuraOu-by-gcc /workspace/YaneuraOu-clang-evallearn &&\
make YANEURAOU_EDITION=YANEURAOU_ENGINE_NNUE clean &&\
echo "2021_10_16_0000"
FROM ubuntu:20.04 AS runner
SHELL ["/bin/bash", "-c"]
RUN \
export DEBIAN_FRONTEND=noninteractive &&\
sed -i.bak -r 's!(deb|deb-src) http://archive\.ubuntu\.com/\S+!\1 mirror://mirrors.ubuntu.com/mirrors.txt!' /etc/apt/sources.list &&\
apt-get update &&\
apt-get -y -o DPkg::options::="--force-confdef" -o DPkg::options::="--force-confold" install \
curl gpg &&\
curl "https://keyserver.ubuntu.com/pks/lookup?search=0x1e9377a2ba9ef27f&op=get" | gpg --no-default-keyring --keyring /usr/share/keyrings/ubuntu-toolchain-r.gpg --import - &&\
echo "deb [signed-by=/usr/share/keyrings/ubuntu-toolchain-r.gpg] http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu focal main" | tee /etc/apt/sources.list.d/ubuntu-toolchain-r-test-focal.list &&\
curl "https://apt.llvm.org/llvm-snapshot.gpg.key" | gpg --no-default-keyring --keyring /usr/share/keyrings/llvm-snapshot.gpg --import - &&\
echo "deb [signed-by=/usr/share/keyrings/llvm-snapshot.gpg] https://apt.llvm.org/focal/ llvm-toolchain-focal main" | tee /etc/apt/sources.list.d/llvm-toolchain-focal.list &&\
apt-get update &&\
apt-get -y -o DPkg::options::="--force-confdef" -o DPkg::options::="--force-confold" install \
git ruby unzip vim-tiny libstdc++6 libgomp1 libomp5-14 libopenblas0 &&\
apt-get -y -o DPkg::options::="--force-confdef" -o DPkg::options::="--force-confold" upgrade &&\
apt-get clean &&\
rm -rf /var/lib/apt/lists/* &&\
rm /tmp/* -rf &&\
mkdir -p /workspace/volume &&\
echo "2021_10_16_0000"
RUN \
echo "download : suisho4kai" &&\
mkdir -p /workspace/suisho4kai/eval &&\
cd /workspace &&\
FILE_ID=1uDezE_TBIT1m0Zxz0vJbB0pdCzpT-D6U &&\
FILE_NAME=suisho4kai_20210826.zip &&\
curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=${FILE_ID}" > /dev/null &&\
CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)" &&\
curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=${FILE_ID}" -o ${FILE_NAME} &&\
unzip -j ${FILE_NAME} **/engine_options.txt -d /workspace/suisho4kai &&\
unzip -j ${FILE_NAME} **/nn.bin -d /workspace/suisho4kai/eval &&\
echo "Suisho4kai/YaneuraOu-docker" | tee /workspace/suisho4kai/engine_name.txt &&\
echo "yaneurao, tayayan" | tee -a /workspace/suisho4kai/engine_name.txt &&\
rm ${FILE_NAME} &&\
echo "2021_10_16_0000"
RUN \
cd /workspace &&\
echo "shallow clone : shogi-server" &&\
git clone https://github.com/mizar/shogi-server.git /workspace/shogi-server -b wcsc2021 --depth 1 &&\
echo "shallow clone : YaneuraOu master" &&\
git clone https://github.com/yaneurao/YaneuraOu.git /workspace/YaneuraOu -b master --depth 1 &&\
echo "shallow clone : dlshogi master" &&\
git clone https://github.com/TadaoYamaoka/DeepLearningShogi.git /workspace/DeepLearningShogi -b master --depth 1 &&\
echo "2021_10_16_0000"
WORKDIR /workspace/suisho4kai
COPY --from=builder \
/workspace/YaneuraOu-gcc-normal \
/workspace/YaneuraOu-gcc-normal.log \
/workspace/YaneuraOu-clang-normal \
/workspace/YaneuraOu-clang-normal.log \
/workspace/YaneuraOu-gcc-tournament \
/workspace/YaneuraOu-gcc-tournament.log \
/workspace/YaneuraOu-clang-tournament \
/workspace/YaneuraOu-clang-tournament.log \
/workspace/YaneuraOu-gcc-evallearn \
/workspace/YaneuraOu-gcc-evallearn.log \
/workspace/YaneuraOu-clang-evallearn \
/workspace/YaneuraOu-clang-evallearn.log \
/workspace/
Raw
engine_dlshogi_with_gct_wcsc31-dlshogi-clang.bat
@docker run -i --rm --gpus all -v "%~dp0\\dlshogi_with_gct_wcsc31":/workspace/volume fukauraou:cuda11.4.2 ../dlshogi-clang %*
Raw
engine_dlshogi_with_gct_wcsc31-FukauraOu-clang.bat
@docker run -i --rm --gpus all -v "%~dp0\\dlshogi_with_gct_wcsc31":/workspace/volume fukauraou:cuda11.4.2 ../FukauraOu-clang %*
Raw
engine_suisho4kai-YaneuraOu-clang.bat
@docker run -i --rm yaneuraou:suisho4kai ../YaneuraOu-clang-normal %*
Raw
x00a_remove_zoneid.bat
@powershell.exe "Get-ChildItem -Recurse -File | Where-Object { $_ | Get-Item -Stream Zone.Identifier -ErrorAction Ignore } | Remove-Item -Stream Zone.Identifier;"
Raw
x00b_os_version.bat
@powershell.exe "Get-CimInstance Win32_OperatingSystem | Select-Object Caption, OSArchitecture, Version"
@pause
Raw
x00c_get_device_display.bat
@powershell.exe "Get-PnpDevice -Class Display"
@pause
Raw
x01_enable_windowsoptionalfeature.bat
@powershell.exe -NoProfile -ExecutionPolicy unrestricted -Command "Start-Process PowerShell.exe 'Write-Host \"* Enable VirtualMachinePlatform\"; Enable-WindowsOptionalFeature -Online -FeatureName VirtualMachinePlatform -All -NoRestart; Write-Host \"* Enable Microsoft-Windows-Subsystem-Linux\"; Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux -All -NoRestart; pause' -Verb runas"
Raw
x01_wsl_update.bat
@powershell.exe -NoProfile -ExecutionPolicy unrestricted -Command "Start-Process PowerShell.exe 'wsl.exe --update; pause' -Verb runas"
Raw
x02_docker_desktop_win_installer.bat
@start https://hub.docker.com/editions/community/docker-ce-desktop-windows
Raw
x03_cuda_on_wsl_driver_installer.bat
@start https://developer.nvidia.com/cuda/wsl/download
Raw
x04_suisho4kai_setup.bat
@echo off
cd %~dp0
docker build . -t yaneuraou:suisho4kai -f Dockerfile_yaneuraou_suisho4kai
Raw
x04b_yaneuraou_nocache.bat
@echo off
cd %~dp0
docker pull ubuntu:20.04
docker build . -t yaneuraou:suisho4kai -f Dockerfile_yaneuraou_suisho4kai --nocache
Raw
x05_dlshogi_with_gct_wcsc31_setup.bat
@powershell.exe -Exec bypass -File x_dlshogi_with_gct_wcsc31.ps1
Raw
x05b_fukauraou_nocache.bat
@echo off
cd %~dp0
docker pull nvcr.io/nvidia/cuda:11.4.2-cudnn8-devel-ubuntu20.04
docker pull nvcr.io/nvidia/cuda:11.4.2-cudnn8-runtime-ubuntu20.04
docker build . -t fukauraou:cuda11.4.2 -f Dockerfile_fukauraou --nocache
powershell.exe -Exec bypass -File x_dlshogi_with_gct_wcsc31.ps1
Raw
x_dlshogi_with_gct_wcsc31.ps1
$ProgressPreference = "SilentlyContinue";
Push-Location $PSScriptRoot;
Invoke-WebRequest "https://github.com/TadaoYamaoka/DeepLearningShogi/releases/download/wcwc31/dlshogi_with_gct_wcsc31.zip" -OutFile "dlshogi_with_gct_wcsc31.zip";
Expand-Archive dlshogi_with_gct_wcsc31.zip -Force;
Copy-Item "dlshogi_with_gct_wcsc31/model-0000225kai.onnx" "dlshogi_with_gct_wcsc31/model.onnx" -Force;
Copy-Item "dlshogi_with_gct_wcsc31/book_model-0000223_225kai_4m.bin" "dlshogi_with_gct_wcsc31/book.bin" -Force;
New-Item -Path "dlshogi_with_gct_wcsc31" -Name "engine_name.txt" -ItemType File -Force -Value @"
dlshogi_with_gct_wcsc31/FukauraOu-docker
yaneurao, TadaoYamaoka, k kanou
"@ | Out-Null;
New-Item -Path "dlshogi_with_gct_wcsc31" -Name "engine_options.txt" -ItemType File -Force -Value @"
option name EvalDir type string default .
option name BookDir type string default .
option name BookFile type combo default book.bin var no_book var standard_book.db var yaneura_book1.db var yaneura_book2.db var yaneura_book3.db var yaneura_book4.db var user_book1.db var user_book2.db var user_book3.db var book.bin
option name UCT_Threads1 type spin default 4 min 0 max 256
option name UCT_Threads2 type spin default 4 min 0 max 256
option name UCT_Threads3 type spin default 4 min 0 max 256
option name UCT_Threads4 type spin default 4 min 0 max 256
option name UCT_Threads5 type spin default 4 min 0 max 256
option name UCT_Threads6 type spin default 4 min 0 max 256
option name UCT_Threads7 type spin default 4 min 0 max 256
option name UCT_Threads8 type spin default 4 min 0 max 256
option name DNN_Model1 type string default model-0000225kai.onnx
"@ | Out-Null;
& docker.exe build . -t fukauraou:cuda11.4.2 -f Dockerfile_fukauraou;
& docker.exe run -i --rm --gpus all -v `"$(Join-Path $PSScriptRoot 'dlshogi_with_gct_wcsc31')`":/workspace/volume fukauraou:cuda11.4.2 ../FukauraOu-clang usi `, isready `, quit ;
Remove-Item "dlshogi_with_gct_wcsc31/model.onnx.*.serialized" -ErrorAction SilentlyContinue;
& docker.exe run -i --rm --gpus all -v `"$(Join-Path $PSScriptRoot 'dlshogi_with_gct_wcsc31')`":/workspace/volume fukauraou:cuda11.4.2 ../dlshogi-clang isready ;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment