This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# You can use any branch but this article been tested with 0.1.2 only | |
git clone https://github.com/gclouduniverse/gcp-notebook-executor.git --branch v0.1.2 | |
cd gcp-notebook-executor | |
source utils.sh | |
INPUT_NOTEBOOK="./demo.ipynb" | |
# Should be existing bucket | |
GCP_BUCKET="gs://b0noi-tmp/test-execution/" | |
IMAGE_FAMILY_NAME="tf-latest-gpu" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
export IMAGE_FAMILY="tf-latest-gpu-experimental" # or tf-latest-cpu-experimental | |
export ZONE="us-west1-b" | |
export INSTANCE_NAME="my-instance" | |
export INSTANCE_TYPE="n1-standard-8" | |
gcloud compute instances create $INSTANCE_NAME \ | |
--zone=$ZONE \ | |
--image-family=$IMAGE_FAMILY \ | |
--image-project=deeplearning-platform-release \ | |
--maintenance-policy=TERMINATE \ | |
--accelerator='type=nvidia-tesla-p100,count=1' \ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
export INSTANCE_NAME="my-instance" | |
gcloud compute instances describe "${INSTANCE_NAME}" --format json | jq '.metadata.items[] | select(.key=="proxy-url") | .value' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
export IMAGE_FAMILY="tf-latest-gpu" # or put any required | |
export ZONE="us-west1-b" | |
export INSTANCE_NAME="my-instance" | |
export INSTANCE_TYPE="n1-standard-8" | |
gcloud compute instances create $INSTANCE_NAME \ | |
--zone=$ZONE \ | |
--image-family=$IMAGE_FAMILY \ | |
--image-project=deeplearning-platform-release \ | |
--maintenance-policy=TERMINATE \ | |
--accelerator='type=nvidia-tesla-v100,count=8' \ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
export IMAGE_FAMILY="tf-2-0-cu100-experimental" # or tf-2-0-cpu-experimental if no GPU is needed | |
export ZONE="us-west1-b" | |
export INSTANCE_NAME="my-instance" | |
export INSTANCE_TYPE="n1-standard-8" | |
gcloud compute instances create $INSTANCE_NAME \ | |
--zone=$ZONE \ | |
--image-family=$IMAGE_FAMILY \ | |
--image-project=deeplearning-platform-release \ | |
--maintenance-policy=TERMINATE \ | |
--accelerator='type=nvidia-tesla-p100,count=1' \ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# usage: start_dlvm_for_user tf-latest-cpu [email protected] my-awesome-instance my-project us-west1-b | |
function start_dlvm_for_user() { | |
export IMAGE_FAMILY=$1 | |
export USER_GOOGLE_ACCOUNT_MAIL=$2 | |
export INSTANCE_NAME=$3 | |
export PROJECT_ID=$4 | |
export ZONE=$5 | |
export INSTANCE_TYPE="n1-standard-8" | |
SERVICE_ACCOUNT_NAME="${INSTANCE_NAME}-sa" | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
export IMAGE_FAMILY="tf-latest-cpu" # or put any required | |
export ZONE="us-west1-b" | |
export INSTANCE_NAME="my-instance-wtih-limited-access" | |
export INSTANCE_TYPE="n1-standard-8" | |
gcloud compute instances create $INSTANCE_NAME \ | |
--zone=$ZONE \ | |
--image-family=$IMAGE_FAMILY \ | |
--image-project=deeplearning-platform-release \ | |
--machine-type=$INSTANCE_TYPE \ | |
--boot-disk-size=120GB \ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PROJECT_ID="" # PUT YOUR PROJECT ID | |
SERVICE_ACCOUNT_NAME="" # Service account name | |
GOOGLE_ACCOUNT_MAIL="" # user mail | |
gcloud iam service-accounts add-iam-policy-binding \ | |
"${SERVICE_ACCOUNT_NAME}@${PROJECT_ID}.iam.gserviceaccount.com" \ | |
--member="user:${GOOGLE_ACCOUNT_MAIL}" --role="roles/iam.serviceAccountUser" | |
# Replace osAdminLogin with osLogin if you do NOT want the user to have an | |
# ability to act as root. | |
gcloud projects add-iam-policy-binding ${PROJECT_ID} \ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gcloud iam service-accounts create slava-dlvm-access-account |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from PIL import Image | |
import numpy as np | |
import json | |
import codecs | |
img = Image.open("image.jpg").resize((240, 240)) | |
img_array=np.array(img) | |
result = { |