-
-
Save korakot/ae95315ea6a3a3b33ee26203998a59a3 to your computer and use it in GitHub Desktop.
# run this once, then reload, and then skip this | |
!apt install rustc | |
!gdown --id 1PULtTc-2e9z4bswh_SQqL5oy_4JpfV7c | |
!chmod +x evcxr_jupyter | |
!./evcxr_jupyter --install | |
// install dependency | |
:dep cmd_lib | |
use cmd_lib::run_cmd as sh; |
# just to use with python | |
!apt install rustc | |
%env USER=korakot |
This seems to be a current alternative:
Simply run the following IPython code in a newly created instance.
After completion the connection should re-establish with now a rust kernel.
(Make sure you changed the kernal name to "rust" manually by opening the notebook as text)
# run this once, wait for the reconnection, and then skip this
!curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
!/root/.cargo/bin/rustup component add rust-src
!curl -L https://github.com/evcxr/evcxr/releases/download/v0.14.1/evcxr_jupyter-v0.14.1-x86_64-unknown-linux-gnu.tar.gz | tar -zxC ./
!chmod +x evcxr_jupyter-v0.14.1-x86_64-unknown-linux-gnu/evcxr_jupyter
!./evcxr_jupyter-v0.14.1-x86_64-unknown-linux-gnu/evcxr_jupyter --install
!jupyter-kernelspec list
!bash -c "killall jupyter-notebook ; sleep 3 ; source ~/.cargo/env && jupyter notebook --ip=172.28.0.12 --port=9000" </dev/null>/dev/null &
The problem:
The Jupyter-Notebook is unable to find rustc even after installation as it seems to would require a restart.
I also instead opted to use rustup.
The last line kills the currently running jupyter-notebook process and starts another one in a bash instance, that has sourced the new environment variables needed for rust to work and especially to find rustc.
I'm new to google colab and jupyter, but this approach seems to work afterall.
Hey @Panakotta00, thanks for the reply. Unfortunately, your workaround doesn't work anymore. I am getting "Unable to connect to the runtime"
+1 i am unable to make this work.
output for the original bash solution:
await connected: disconnected
Error: await connected: disconnected
at https://ssl.gstatic.com/colaboratory-static/common/af761f603211b36e46e9dcbfd0e11687/external_polymer_binary.js:6836:372
output for the last mentioned workaround (by Panakotta00):
Could not fetch resource at https://colab.research.google.com/tun/m/m-s-1zbq2mk57c0qt/api/kernelspecs?backend_version=next&authuser=0: 403 Forbidden
FetchError: Could not fetch resource at https://colab.research.google.com/tun/m/m-s-1zbq2mk57c0qt/api/kernelspecs?backend_version=next&authuser=0: 403 Forbidden
at lG.DC [as constructor] (https://ssl.gstatic.com/colaboratory-static/common/af761f603211b36e46e9dcbfd0e11687/external_polymer_binary.js:1582:2512)
at new lG (https://ssl.gstatic.com/colaboratory-static/common/af761f603211b36e46e9dcbfd0e11687/external_polymer_binary.js:2004:186)
at ua.program_ (https://ssl.gstatic.com/colaboratory-static/common/af761f603211b36e46e9dcbfd0e11687/external_polymer_binary.js:6870:133)
at xa (https://ssl.gstatic.com/colaboratory-static/common/af761f603211b36e46e9dcbfd0e11687/external_polymer_binary.js:21:57)
at ua.next_ (https://ssl.gstatic.com/colaboratory-static/common/af761f603211b36e46e9dcbfd0e11687/external_polymer_binary.js:19:195)
at laa.next (https://ssl.gstatic.com/colaboratory-static/common/af761f603211b36e46e9dcbfd0e11687/external_polymer_binary.js:22:48)
at b (https://ssl.gstatic.com/colaboratory-static/common/af761f603211b36e46e9dcbfd0e11687/external_polymer_binary.js:22:309)
I've updated the commands to use the latest version of evcxr
. Here's the new commands:
# Install Rust using rustup
!curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
# Add the Rust source component
!/root/.cargo/bin/rustup component add rust-src
# Download and install the Evcxr kernel
!curl -LO https://github.com/evcxr/evcxr/releases/download/v0.15.1/evcxr_jupyter-v0.15.1-x86_64-unknown-linux-gnu.tar.gz
!tar -xzf evcxr_jupyter-v0.15.1-x86_64-unknown-linux-gnu.tar.gz
!chmod +x evcxr_jupyter-v0.15.1-x86_64-unknown-linux-gnu/evcxr_jupyter
!./evcxr_jupyter-v0.15.1-x86_64-unknown-linux-gnu/evcxr_jupyter --install
# Start or restart the Jupyter Notebook server
!pgrep jupyter-notebook && echo "No Jupyter Notebook instances running" || (killall jupyter-notebook ; sleep 3 ; jupyter notebook --ip=0.0.0.0 --port=9000)
However, after some investigation, it seems that certain actions we're trying to perform are restricted in the current Colab environment. It seems that Google Colab has security restrictions in place, and some actions, like modifying kernel specifications or kernel installation, are not allowed.
status update
As of today (22-Sep-2023), it still appears that none of the strategies work. As "wiseaidev" noted: Google Colab has tightened their security posture.
personal motivation
I just want a really good "I can code from my tablet" experience. In the past, Google Colab made a really solid "crank something out & share with controlled group" platform -- it still does for the languages it supports. But for Rust (and Go and many other languages), it appears to be a dead-end.
possible alternatives
colab piped through another computer for computation
Colab can pipe through a local kernel, so one could tinker with making a Colab notebook that has a gcloud
script to launch a GCP Compute Engine (GCE) to host a personal Rust Colab kernel, then pipe through that "local" runtime. But I picture that getting very messy when you want to block unauth'd users from connecting to your GCE & charging up CPU-minutes; then you'd have to manually auth those who you do want to share with (code collaborators)... sounds like an opportunity for you to mess up your own stuff. In addition, this negates one of the benefits of Google Colab: free compute.
rustpad
And if doing all that "pipe Colab through GCE" stuff, then one might as well just have a personal GCE that's running a private rustpad, auth people one place (can/cannot connect to GCE), and not worry about bouncing back-n-forth between Colab & the GCE. Granted, that solution is drifting "off topic" (if the topic is strictly "using Rust in Google Collab"), but it is still a "Rust in cloud" alternative.
github codespace (premium)
And the best solution I've used (but it isn't free) is having a premium github account, which allows one to visit any repo and just press .
to launch a cloud-hosted editor. That editor works well through my tablet's browser, too. Then inside that cloud-hosted VS-Code editor, I can launch a codespace terminal (that launches a server, which burns your cloudspace minutes). In there, just use the rustup.rs install command and you now have a cloud-hosted Rust environment, working on your git repo. This is the best "edit/run/etc the repo in my tablet's browser when I'm at the coffee shop" solution that I've found. But it is not free.
And if you just want a playground, then make a new repo at the main github.com, after checking a default readme, gitignore, and license, then you press the "click me to make it" button (or whatever the label is). After that, then you can press .
to launch the codespace, install rustup, do a cargo init
(you're already inside the repo root), and now you're running.
closing notes
I have zero professional affiliation with any of the referenced tools. I'm just a user of those tools -- nothing for me to gain by ref'ing them.
And I hope that my share helps others. I'm also sub'ing to this gist in case somebody has a Google Colab breakthrough, too. ...it would be nice to stop torching my github codespace minutes. 😄
One last comment I should have shared: those codespaces will auto-load your personal dotfiles
repo, too: Github doc pages re loading dotfiles. If you use dotfiles
repos to manage your configs, this is a real win. .....I'll stop now. I realize I've drifted way off-topic, but I didn't want to miss details that others may need, should they try this out for themselves.
Managed to run evcxr by proxying the IPC connection to TCP, as it expects. It needs both cargo and rustc installed, and they can be installed via apt, which is faster. Below is code which uses nix instead of apt to install most of the stuff: run once, change runtime and restart.
!rm -rf /root/.local/share/jupyter/kernels
# Nix install
!curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install linux --init none --no-confirm
!ln -s /root/.nix-profile/bin /opt/bin
# Install evcxr
!nix-env -f '<nixpkgs>' -iA cargo rustc evcxr sccache
!evcxr_jupyter --install
# Configure evcxr
!mkdir -p /root/.config/evcxr
!printf ":timing\n:sccache 1" > /root/.config/evcxr/init.evcxr
# IPC Proxy (https://stackoverflow.com/a/74821762)
!wget -qO- https://gist.github.com/SpencerPark/e2732061ad19c1afa4a33a58cb8f18a9/archive/b6cff2bf09b6832344e576ea1e4731f0fb3df10c.tar.gz | tar xvz --strip-components=1
!python install_ipc_proxy_kernel.py --quiet --kernel=rust --implementation=ipc_proxy_kernel.py > /dev/null
# To avoid confusion between kernel names
!sed -i 's/"display_name": "Rust"/"display_name": "Rust-TCP"/g' /root/.local/share/jupyter/kernels/rust_tcp/kernel.json
Many thanks to @mateusvmv efforts and friends here in this discussion. I put together a bash script that overwrites the current kernel and spins up a new Rust kernel without manual config:
# This script sets up and spins up a Jupyter Notebook environment with a Rust kernel using Nix and IPC Proxy.
!wget -qO- https://gist.github.com/wiseaidev/2af6bef753d48565d11bcd478728c979/archive/3f6df40db09f3517ade41997b541b81f0976c12e.tar.gz | tar xvz --strip-components=1
!bash setup_evcxr_kernel.sh
Any progress on getting it working again?