Jiaxin Shan Jeffwan

Software Engineer @ Bytedance

Jeffwan / ray-xgboost-auto.py

Created April 8, 2021 21:10

ray-xgboost-auto.py

	import os
	import ray

	from xgboost_ray import RayDMatrix, RayParams, train
	from sklearn.datasets import load_breast_cancer

	ray.init(address="auto")

	train_x, train_y = load_breast_cancer(return_X_y=True)
	train_set = RayDMatrix(train_x, train_y)

Jeffwan / raydp-spark-remote.py

Last active April 8, 2021 22:38

raydp-spark-remote.py

	import os
	import ray
	import raydp


	HEAD_SERVICE_IP_ENV = "EXAMPLE_CLUSTER_RAY_HEAD_SERVICE_HOST"
	HEAD_SERVICE_CLIENT_PORT_ENV = "EXAMPLE_CLUSTER_RAY_HEAD_SERVICE_PORT_CLIENT"

	head_service_ip = os.environ[HEAD_SERVICE_IP_ENV]
	client_port = os.environ[HEAD_SERVICE_CLIENT_PORT_ENV]

Jeffwan / ray-tf.py

Last active April 9, 2021 05:40

	import argparse
	import tensorflow as tf
	from tensorflow.keras.models import Sequential
	from tensorflow.keras.layers import Dense
	import numpy as np

	import os
	import ray
	from ray import tune
	from ray.util.sgd.tf.tf_trainer import TFTrainer, TFTrainable

Jeffwan / ray-xgboost.py

Last active April 7, 2021 23:13

ray-xgboost.py

	import os
	import ray

	from xgboost_ray import RayDMatrix, RayParams, train
	from sklearn.datasets import load_breast_cancer

	HEAD_SERVICE_IP_ENV = "EXAMPLE_CLUSTER_RAY_HEAD_SERVICE_HOST"
	HEAD_SERVICE_CLIENT_PORT_ENV = "EXAMPLE_CLUSTER_RAY_HEAD_SERVICE_PORT_CLIENT"

	head_service_ip = os.environ[HEAD_SERVICE_IP_ENV]

Jeffwan / raydp-spark.py

Last active April 8, 2021 22:34

raydp-spark.py

	import os
	import ray
	import raydp

	HEAD_SERVICE_IP_ENV = "EXAMPLE_CLUSTER_RAY_HEAD_SERVICE_HOST"
	HEAD_SERVICE_CLIENT_PORT_ENV = "EXAMPLE_CLUSTER_RAY_HEAD_SERVICE_PORT_CLIENT"

	head_service_ip = os.environ[HEAD_SERVICE_IP_ENV]
	client_port = os.environ[HEAD_SERVICE_CLIENT_PORT_ENV]
	ray.util.connect(f"{head_service_ip}:{client_port}")

Jeffwan / gist:5f11afe8593950ed444c959160eb2822

Created August 28, 2020 17:00

Your access to this site has been restricted

	```
	curl -v https://github.com/kubeflow/manifests/archive/v1.0.2.tar.gz 09:55:14
	* Trying 192.30.255.113...
	* TCP_NODELAY set
	* Connected to github.com (192.30.255.113) port 443 (#0)
	* ALPN, offering h2
	* ALPN, offering http/1.1
	* successfully set certificate verify locations:
	* CAfile: /etc/ssl/cert.pem
	CApath: none

Jeffwan / gist:e1cda3864f4a3fd7191a3fc95ebb0fe8

Created July 27, 2020 22:50



	```
	WARN[0007] Encountered error applying application application: (kubeflow.error): Code 500 with message: Apply.Run : unable to recognize "/tmp/kout799748204": no matches for kind "Application" in version "app.k8s.io/v1beta1" filename="kustomize/kustomize.go:266"
	WARN[0007] Will retry in 3 seconds. filename="kustomize/kustomize.go:267"

	```

Jeffwan / Dockerfile

Created May 20, 2020 17:11

Dockerfile

	ARG BASE_IMAGE=763104351884.dkr.ecr.us-west-2.amazonaws.com/tensorflow-training:1.15.2-cpu-py36-ubuntu18.04

	FROM $BASE_IMAGE

	ARG NB_USER=jovyan

	ARG KUBEFLOW_FAIRING_SDK_VERSION=0.7.1
	ARG KUBEFLOW_METADATA_SDK_VERSION=0.3.1
	ARG KUBEFLOW_KFP_SDK_VERSION=0.5.0
	ARG KUBEFLOW_KFSERVING_SDK_VERSION=0.3.0.1

Jeffwan / dependency.sh

Created May 12, 2020 22:58

kubernetes dependency resolver shell scripts

	#!/bin/sh
	set -euo pipefail

	VERSION=${1#"v"}
	if [ -z "$VERSION" ]; then
	echo "Must specify version!"
	exit 1
	fi
	MODS=($(
	curl -sS https://raw.githubusercontent.com/kubernetes/kubernetes/v${VERSION}/go.mod \|

Jeffwan / override.go

Last active February 20, 2022 17:48

Abstract class and interface in golang

	package mxnet

	import "fmt"

	type ControllerInterface interface {
	// Returns the Controller name
	Method1() string

	// Returns the GroupVersionKind of the API
	Method2() string