Birch-san

Installing & running cene's imagen-pytorch

Install Anaconda via command-line arm64 installer

Install PyTorch nightly (to get M1 GPU support)

conda update -n base -c defaults conda
conda create -n torch-nightly python=3.9
pip install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu

1400x2000 -> 45x64 -> 180x256

(Downscale via traditional algorithm, upscale via AI)

Upscaling of images via Real-ESRGAN, compared by the algorithm via which they were downscaled:

Bilinear:

VSCode launched dalle-playground's backend/app.py Python process like so:

dalle-playground git:(main) ✗  cd /Users/birch/git/dalle-playground
/usr/bin/env /Users/birch/anaconda3/envs/torch-nightly/bin/python /Users/birch/.vscode/extensions/ms-python.python-2022.6.3/pythonFiles/lib/python/debugpy/launcher 52337 -- /Users/birch/git/dalle-playground/backend/app.py 8080

Using the following launch.json:

	from iree import compiler

	CODE = """
	#loc0 = loc(unknown)
	module @jit__threefry_split.11 {
	func.func public @main(%arg0: tensor<2xui32> loc(unknown)) -> tensor<2x2xui32> {
	%0 = "mhlo.iota"() {iota_dimension = 0 : i64} : () -> tensor<4xui32> loc(#loc1)
	%1 = "mhlo.slice"(%arg0) {limit_indices = dense<1> : tensor<1xi64>, start_indices = dense<0> : tensor<1xi64>, strides = dense<1> : tensor<1xi64>} : (tensor<2xui32>) -> tensor<1xui32> loc(#loc2)
	%2 = "mhlo.reshape"(%1) : (tensor<1xui32>) -> tensor<ui32> loc(#loc3)
	%3 = "mhlo.slice"(%arg0) {limit_indices = dense<2> : tensor<1xi64>, start_indices = dense<1> : tensor<1xi64>, strides = dense<1> : tensor<1xi64>} : (tensor<2xui32>) -> tensor<1xui32> loc(#loc4)

	# see https://github.com/google/iree/issues/9361,
	# https://github.com/google/iree/pull/9378

	from iree import compiler, runtime as rt
	rt.flags.parse_flags("--task_topology_group_count=8")

	CODE = """
	#loc0 = loc(unknown)
	module @jit_prim_fun.12 {
	func.func public @main(%arg0: tensor<1x1xi32> loc(unknown), %arg1: tensor<1xi32> loc(unknown), %arg2: tensor<1xi32> loc(unknown)) -> tensor<1x1xi32> {

	# https://google.github.io/iree/building-from-source/getting-started/#reference-pages
	# https://github.com/google/iree/blob/main/docs/developers/get_started/cmake_options_and_variables.md
	git clone --branch scatter --depth 1 https://github.com/Birch-san/iree.git
	git submodule update --init
	cd iree
	pip install pybind11
	export CMAKE_OSX_ARCHITECTURES="arm64"
	export MACOSX_DEPLOYMENT_TARGET=11.0
	export IREE_ENABLE_CPUINFO=OFF
	export IREE_HAL_DRIVER_VULKAN=ON

	import torch as th

	from itertools import tee

	from PIL import Image
	from IPython.display import display

	from imagen_pytorch.model_creation import create_model_and_diffusion as create_model_and_diffusion_imagen
	from imagen_pytorch.model_creation import model_and_diffusion_defaults as model_and_diffusion_defaults_imagen
	from imagen_pytorch.train_all import _fix_path

	Error invoking IREE compiler tool iree-compile
	Diagnostics:
	/Users/birch/anaconda3/envs/torch-nightly/lib/python3.9/site-packages/transformers/models/bart/modeling_flax_bart.py:926:1: error: failed to legalize operation 'mhlo.scatter' that was explicitly marked illegal
	input_ids = input_ids.at[(..., -1)].set(self.config.eos_token_id)
	^
	compilation failed


	Invoked with:
	iree-compile /Users/birch/anaconda3/envs/torch-nightly/lib/python3.9/site-packages/iree/compiler/tools/../_mlir_libs/iree-compile - --iree-input-type=mhlo --iree-vm-bytecode-module-output-format=flatbuffer-binary --iree-hal-target-backends=vulkan --iree-mlir-to-vm-bytecode-module --iree-llvm-embedded-linker-path=/Users/birch/anaconda3/envs/torch-nightly/lib/python3.9/site-packages/iree/compiler/tools/../_mlir_libs/iree-lld --mlir-print-debuginfo --mlir-print-op-on-diagnostic=false --iree-llvm-target-triple=arm64-apple-darwin21.5.0 --iree-flow-demote-i64-to-i32 --iree-vulkan-target-triple=m1-moltenvk-macos --iree-llvm-target-cpu-featur