Skip to content

Instantly share code, notes, and snippets.

View swiftdiaries's full-sized avatar

Adhita Selvaraj swiftdiaries

View GitHub Profile
@swiftdiaries
swiftdiaries / Create a GKE cluster
Created November 13, 2018 14:43
Create a GKE cluster for Pipeline
gcloud auth login
gcloud config set project [your-project-id]
# Set your default zone.
gcloud config set compute/zone us-central1-a
# Create a cluster
CLUSTER_NAME=[YOUR-CLUSTER-NAME]
gcloud container clusters create $CLUSTER_NAME \
--zone us-central1-a \
@swiftdiaries
swiftdiaries / Install Istio on k8s
Last active December 19, 2018 19:07
Install instructions from knative
kubectl apply --filename https://gist.githubusercontent.com/swiftdiaries/04847b5dc194df0e1357441054bd240d/raw/2f84e8b9addb2b605a4d4742cc33aef820e2177e/Istio%2520installation%2520YAML
# Create a kubeflow namespace
kubectl create ns kubeflow
# Set label to enable istio-injection on the kubeflow namespace
kubectl label namespace kubeflow istio-injection=enabled
# Ensure Istio pods are Running or Completed.
kubectl get pods -n istio-system --watch
@swiftdiaries
swiftdiaries / Install Pipelines
Last active January 16, 2019 01:38
Install instructions for Kubeflow Pipelines
# Istio is required for pipelines
# [Install istio](https://gist.github.com/swiftdiaries/45be0f72c420b3b7ffd7671efa9e5d3c)
# Grab the (latest release of the pipelines)[https://github.com/kubeflow/pipelines/releases]
export PIPELINE_VERSION=0.1.6
# Download the bootstrapper job.
curl https://storage.googleapis.com/ml-pipeline/release/$PIPELINE_VERSION/bootstrapper.yaml --output bootstrapper.yaml
# Update additional parameters in bootstrapper.yaml
kubectl apply --filename https://gist.githubusercontent.com/swiftdiaries/04847b5dc194df0e1357441054bd240d/raw/2f84e8b9addb2b605a4d4742cc33aef820e2177e/Istio%2520installation%2520YAML
# Create a kubeflow namespace
kubectl create ns kubeflow
# Set label to enable istio-injection on the kubeflow namespace
kubectl label namespace kubeflow istio-injection=enabled
ks init pipeline-ks-app --env on-prem --namespace kubeflow --skip-default-registries
cd pipeline-ks-app
---
# PATCH #1: Creating the istio-system namespace.
apiVersion: v1
kind: Namespace
metadata:
name: istio-system
labels:
istio-injection: disabled
# PATCH #1 ends.
---
# Copyright 2018 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: nfs
namespace: kubeflow
spec:
accessModes:
- ReadWriteMany
storageClassName: nfs
resources:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: default-workspace
spec:
accessModes:
- ReadWriteOnce
storageClassName: nfs
resources:
requests:
# For clusters with the default Storage Class named 'standard'
# This is true for Cisco Container Platform and Google Kubernetes Engine clusters
helm init --service-account=tiller
helm repo update
helm install stable/nfs-server-provisioner --name kf-nfs-storageclass \
--set=persistence.enabled=true,persistence.storageClass=standard,persistence.size=200Gi
# For clusters with the default Storage Class named 'default'
# This is for Azure clusters
helm init --service-account=tiller
helm repo update
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: bolts-launcher
spec:
template:
metadata:
labels:
app: "notebook-launcher"
spec: