Skip to content

Instantly share code, notes, and snippets.

View korkridake's full-sized avatar
💙
print("Hello World!")

Korkrid Kyle Akepanidtaworn korkridake

💙
print("Hello World!")
37 followers · 114 following
View GitHub Profile
@korkridake
korkridake / mlops-train-experiment-compute.py
Created May 29, 2020 13:06
MLOps Ep.4 Productionizing Training Script (Create an experiment and validate if AMLCompute is already provisioned)
# Create an experiment
experiment_name = 'experiment-regression'
experiment = Experiment(workspace = ws, name = experiment_name)
# Choose a name for your CPU cluster
cpu_cluster_name = "kyledevpc1"
# Verify that cluster does not exist already
try:
cpu_cluster = ComputeTarget(workspace=ws, name=cpu_cluster_name)
@korkridake
korkridake / mlops-train-configure-workspace.py
Created May 29, 2020 13:01
MLOps Ep.4 Productionizing Training Script (Configure Workspaces)
ws = Workspace.from_config()
print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\n')
@korkridake
korkridake / mlops-train-import-libs.py
Created May 29, 2020 12:59
MLOps Ep.4 Productionizing Training Script (Import Libraries)
####################################################################
# Import Python Libaries
####################################################################
import os
import sys
import numpy as np
import pandas as pd
import argparse
import json
import urllib
@korkridake
korkridake / aws-sagemaker-xgboost-ep01.py
Created December 31, 2019 18:21
AWS SageMaker Estimators (https://sagemaker.readthedocs.io/en/stable/estimators.html)
#####################################################################################
# Initatiate the Session
#####################################################################################
sess = sagemaker.Session()
#####################################################################################
# Ensure that you specify the right locations of the XGBoost algorithm containers
#####################################################################################
xgb = sagemaker.estimator.Estimator(container,
role,
@korkridake
korkridake / aws-sagemaker-data-splitting-ep01.py
Created December 31, 2019 18:09
AWS Cloud Journey - Analyzing to Deploying Models with Amazon SageMaker (Ep. 1)
###############################################################
# Split into training, validation, and testing data
###############################################################
train_data, validation_data, test_data = np.split(model_data.sample(frac=1, random_state=1729), [int(0.7 * len(model_data)), int(0.9 * len(model_data))])
train_data.to_csv('train.csv', header=False, index=False)
validation_data.to_csv('validation.csv', header=False, index=False)
###############################################################
# Upload these files to S3
###############################################################
@korkridake
korkridake / aws-sagemaker-import-ep01.py
Created December 31, 2019 17:58
AWS Cloud Journey - Analyzing to Deploying Models with Amazon SageMaker (Ep. 1)
################################################
# Import the typical Python libraries
################################################
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import io
import os
import sys
import time
@korkridake
korkridake / postman-exchange-rate-ep-1.json
Created November 9, 2019 17:24
REST API In Action - BoT Average Exchange Rate API via Postman (Ep. 1)
{
"result": {
"timestamp": "2019-11-10 00:23:33",
"api": "Daily Weighted-average Interbank Exchange Rate - THB / USD",
"data": {
"data_header": {
"report_name_eng": "Rates of Exchange of Commercial Banks in Bangkok Metropolis (2002-present)",
"report_name_th": "อัตราแลกเปลี่ยนเฉลี่ยของธนาคารพาณิชย์ในกรุงเทพมหานคร (2545-ปัจจุบัน)",
"report_uoq_name_eng": "(Unit: Baht / 1 Unit of Foreign Currency)",
"report_uoq_name_th": "(หน่วย: บาท ต่อ 1 หน่วยเงินตราต่างประเทศ)",
@korkridake
korkridake / polynote-python-dependencies.sh
Created November 4, 2019 13:46
Polynote Python Dependencies and Run Polynote
polynoteuser02@polynotevm02:~/download$ sudo apt-get install build-essential
polynoteuser02@polynotevm02:~/download$ sudo apt install python3-pip
polynoteuser02@polynotevm02:~/download$ pip3 install jep
polynoteuser02@polynotevm02:~/download$ pip3 install jedi
polynoteuser02@polynotevm02:~/download$ pip3 install pyspark
polynoteuser02@polynotevm02:~/download$ pip3 install virtualenv
polynoteuser02@polynotevm02:~/download$ pip3 install numpy
polynoteuser02@polynotevm02:~/download$ pip3 install pandas
# Let's run Polynote
@korkridake
korkridake / polynote-environment-setup.sh
Last active November 4, 2019 13:35
Set up some environment variables for the proper functioning of Polynote
polynoteuser02@polynotevm02:~/download$ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
polynoteuser02@polynotevm02:~/download$ export SPARK_HOME=/home/polynoteuser02/download/spark-2.4.4-bin-hadoop2.7
polynoteuser02@polynotevm02:~/download$ export PATH = "$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbin"
polynoteuser02@polynotevm02:~/download$ echo $JAVA_HOME
polynoteuser02@polynotevm02:~/download$ echo $SPARK_HOME
polynoteuser02@polynotevm02:~/download$ echo $PATH
polynoteuser02@polynotevm02:~/download$ java -version
# If the below command can run, you should see usage information.
polynoteuser02@polynotevm02:~/download$ spark-submit
@korkridake
korkridake / polynote-linux-download.sh
Last active November 4, 2019 13:29
Download Apache Spark, Polynote, and Java
polynoteuser02@polynotevm02:~$ mkdir download
polynoteuser02@polynotevm02:~$ cd download
polynoteuser02@polynotevm02:~/download$ wget https://github.com/polynote/polynote/releases/download/0.2.11/polynote-dist.tar.gz
polynoteuser02@polynotevm02:~/download$ wget https://www-eu.apache.org/dist/spark/spark-2.4.4/spark-2.4.4-bin-hadoop2.7.tgz
polynoteuser02@polynotevm02:~/download$ sudo apt install -y curl
polynoteuser02@polynotevm02:~/download$ wget sudo apt install openjdk-8-jdk
polynoteuser02@polynotevm02:~/download$ tar -zxvpf polynote-dist.tar.gz
polynoteuser02@polynotevm02:~/download$ tar -zxvpf spark-2.4.4-bin-hadoop2.7.tgz