chmod
docker-kernel.py
to make it executable (typicallychmod +x docker-kernel.py
).- Place
docker-kernel.py
somewhere in your PATH. Or alternatively, modifykernel.json
to use an absolute path todocker-kernel.py
. The latter (absolute path) option will be necessary in cases where Jupyter isn't exposed to your PATH environment variable. - If you wish to hard code the docker image to be used, modify
kernel.json
such that the third parameter ofargv
(currently set to "IMAGE_FROM_ENV") points to the required image. The default setting ("IMAGE_FROM_ENV") is a special token which directs the script to instead use a user specified environment variable (JUPYTER_KERNEL_DOCKER_IMAGE
) for image specification. Note that any image you wish to use must have a compatibleipykernel_launcher
module installed. - If you wish to hard code extra
docker run
parameters, modifykernel.json
such that the fourth parameter ofargv
(currently set to "MAP_CWD") specifies the parameters (as a string). Alternatively, you can at any time add parameters by setting theJUPYTER_DOCKER_EXTRA_PARAMETERS
environment variable. In particular, you will specify any volume mapping requirements via this option. The default setting ("MAP_CWD") is a special token which directs the script to map the current working directory (usually where Jupyter was launched from) to a directory within the container. Parameters defined by environment variable take precedence over other options. - Move
kernel.json
into some new directory (for examplesome_kernel_directory
). The name doesn't really matter as it is only used for Jupyter's internal purposes. If you wish to change the actual kernel name displayed within Jupyter, your will need to manually modify thedisplay_name
parameter withinkernel.json
. - Execute
jupyter kernelspec install --user some_kernel_directory
to install the kernel. - Launch
jupyter notebook
.
Last active
March 12, 2019 00:00
-
-
Save jmansour/c42c203f0596f6bed6138a440818516a to your computer and use it in GitHub Desktop.
Jupyter kernel via containerised Python environment
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python | |
import sys | |
import subprocess | |
import json | |
import os | |
# map from connection file to ipykernel_launcher args | |
argmap = { | |
"stdin_port" : "--stdin", | |
"ip" : "--ip", | |
"control_port" : "--control", | |
"hb_port" : "--hb", | |
"signature_scheme" : "--Session.signature_scheme", | |
"key" : "--Session.key", | |
"shell_port" : "--shell", | |
"transport" : "--transport", | |
"iopub_port" : "--iopub" | |
} | |
container_vals = { | |
"stdin_port" : 6000, | |
"control_port" : 6001, | |
"hb_port" : 6002, | |
"shell_port" : 6003, | |
"iopub_port" : 6004, | |
"ip" : "0.0.0.0" | |
} | |
# get connection file | |
connection_file = sys.argv[1] | |
image_name = sys.argv[2] | |
if image_name == "IMAGE_FROM_ENV": | |
IMAGE_KEY = "JUPYTER_KERNEL_DOCKER_IMAGE" | |
if IMAGE_KEY in os.environ: | |
image_name = os.environ[IMAGE_KEY] | |
else: | |
raise RuntimeError("You must set the 'JUPYTER_KERNEL_DOCKER_IMAGE' environment variable to specify your image.") | |
extra_parameters = sys.argv[3] | |
if extra_parameters == "MAP_CWD": | |
tmpdir = "/tmp/kernelmapdir" | |
cwd = os.getcwd() | |
extra_parameters = "-w {} -v {}:{}".format(tmpdir, cwd, tmpdir) | |
PARAMS_KEY = "JUPYTER_DOCKER_EXTRA_PARAMETERS" | |
if PARAMS_KEY in os.environ: | |
extra_parameters = os.environ[PARAMS_KEY] | |
with open(connection_file, 'r') as fp: | |
kjson = json.load(fp) | |
docker_args = ['docker', 'run', '--rm', | |
'-p', '{}:{}'.format( kjson[ 'stdin_port'] , container_vals[ 'stdin_port'] ), | |
'-p', '{}:{}'.format( kjson['control_port'] , container_vals['control_port'] ), | |
'-p', '{}:{}'.format( kjson[ 'hb_port'] , container_vals[ 'hb_port'] ), | |
'-p', '{}:{}'.format( kjson[ 'shell_port'] , container_vals[ 'shell_port'] ), | |
'-p', '{}:{}'.format( kjson[ 'iopub_port'] , container_vals[ 'iopub_port'] ), | |
] | |
docker_args += [extra_parameters,image_name] | |
command_line = "python -m ipykernel_launcher".split() | |
# overwrite json with values required by container | |
kjson.update(container_vals) | |
kjson["key"] = "'" + 'b"' + kjson["key"] + '"' + "'" # this guy needs to be described as bytes for python 3 ipykernel_launcher | |
for key in argmap.keys(): | |
command_line.append( "{}={}".format( argmap[key], kjson[key] ) ) | |
strguy = " ".join(docker_args+command_line) | |
subprocess.check_call(strguy, shell=True) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file just for gist title |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"display_name": "Docker Kernel", | |
"language": "python", | |
"argv": [ "docker-kernel.py", "{connection_file}", "IMAGE_FROM_ENV", "MAP_CWD" ] | |
} | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment