Korkrid Kyle Akepanidtaworn korkridake
💙
korkridake
/ automl-h20-simple-flow.R
Created
June 3, 2020 08:41
Automated machine learning H20.ai in R
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Install H2O packages | |
| if ("package:h2o" %in% search()) { detach("package:h2o", unload=TRUE) } | |
| if ("h2o" %in% rownames(installed.packages())) { remove.packages("h2o") } | |
| pkgs <- c("RCurl","jsonlite") | |
| for (pkg in pkgs) { | |
| if (! (pkg %in% rownames(installed.packages()))) { install.packages(pkg) } | |
| } | |
| install.packages("h2o", type="source", repos=(c("http://h2o-release.s3.amazonaws.com/h2o/latest_stable_R"))) | |
| library(h2o) | |
| localH2O = h2o.init() |
korkridake
/ mlops-train-register-dataset.py
Created
May 29, 2020 13:13
MLOps Ep.4 Productionizing Training Script (Register a dataset in Azure Machine Learning Datastore)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Get the datastore to upload prepared data | |
| datastore = ws.get_default_datastore() | |
| # Upload the local file from src_dir to the target_path in datastore | |
| datastore.upload(src_dir='../aml-kaggle/datasets/', target_path='ds_regression') | |
| # Create a dataset referencing the cloud location | |
| dataset = Dataset.Tabular.from_delimited_files(datastore.path('ds_regression/autos.csv')) | |
| # Register a dataset |
korkridake
/ mlops-train-experiment-compute.py
Created
May 29, 2020 13:06
MLOps Ep.4 Productionizing Training Script (Create an experiment and validate if AMLCompute is already provisioned)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Create an experiment | |
| experiment_name = 'experiment-regression' | |
| experiment = Experiment(workspace = ws, name = experiment_name) | |
| # Choose a name for your CPU cluster | |
| cpu_cluster_name = "kyledevpc1" | |
| # Verify that cluster does not exist already | |
| try: | |
| cpu_cluster = ComputeTarget(workspace=ws, name=cpu_cluster_name) |
korkridake
/ mlops-train-configure-workspace.py
Created
May 29, 2020 13:01
MLOps Ep.4 Productionizing Training Script (Configure Workspaces)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ws = Workspace.from_config() | |
| print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\n') |
korkridake
/ mlops-train-import-libs.py
Created
May 29, 2020 12:59
MLOps Ep.4 Productionizing Training Script (Import Libraries)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #################################################################### | |
| # Import Python Libaries | |
| #################################################################### | |
| import os | |
| import sys | |
| import numpy as np | |
| import pandas as pd | |
| import argparse | |
| import json | |
| import urllib |
korkridake
/ aws-sagemaker-xgboost-ep01.py
Created
December 31, 2019 18:21
AWS SageMaker Estimators (https://sagemaker.readthedocs.io/en/stable/estimators.html)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ##################################################################################### | |
| # Initatiate the Session | |
| ##################################################################################### | |
| sess = sagemaker.Session() | |
| ##################################################################################### | |
| # Ensure that you specify the right locations of the XGBoost algorithm containers | |
| ##################################################################################### | |
| xgb = sagemaker.estimator.Estimator(container, | |
| role, |
korkridake
/ aws-sagemaker-data-splitting-ep01.py
Created
December 31, 2019 18:09
AWS Cloud Journey - Analyzing to Deploying Models with Amazon SageMaker (Ep. 1)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ############################################################### | |
| # Split into training, validation, and testing data | |
| ############################################################### | |
| train_data, validation_data, test_data = np.split(model_data.sample(frac=1, random_state=1729), [int(0.7 * len(model_data)), int(0.9 * len(model_data))]) | |
| train_data.to_csv('train.csv', header=False, index=False) | |
| validation_data.to_csv('validation.csv', header=False, index=False) | |
| ############################################################### | |
| # Upload these files to S3 | |
| ############################################################### |
korkridake
/ aws-sagemaker-import-ep01.py
Created
December 31, 2019 17:58
AWS Cloud Journey - Analyzing to Deploying Models with Amazon SageMaker (Ep. 1)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ################################################ | |
| # Import the typical Python libraries | |
| ################################################ | |
| import pandas as pd | |
| import numpy as np | |
| import matplotlib.pyplot as plt | |
| import io | |
| import os | |
| import sys | |
| import time |
korkridake
/ postman-exchange-rate-ep-1.json
Created
November 9, 2019 17:24
REST API In Action - BoT Average Exchange Rate API via Postman (Ep. 1)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "result": { | |
| "timestamp": "2019-11-10 00:23:33", | |
| "api": "Daily Weighted-average Interbank Exchange Rate - THB / USD", | |
| "data": { | |
| "data_header": { | |
| "report_name_eng": "Rates of Exchange of Commercial Banks in Bangkok Metropolis (2002-present)", | |
| "report_name_th": "อัตราแลกเปลี่ยนเฉลี่ยของธนาคารพาณิชย์ในกรุงเทพมหานคร (2545-ปัจจุบัน)", | |
| "report_uoq_name_eng": "(Unit: Baht / 1 Unit of Foreign Currency)", | |
| "report_uoq_name_th": "(หน่วย: บาท ต่อ 1 หน่วยเงินตราต่างประเทศ)", |
korkridake
/ polynote-python-dependencies.sh
Created
November 4, 2019 13:46
Polynote Python Dependencies and Run Polynote
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| polynoteuser02@polynotevm02:~/download$ sudo apt-get install build-essential | |
| polynoteuser02@polynotevm02:~/download$ sudo apt install python3-pip | |
| polynoteuser02@polynotevm02:~/download$ pip3 install jep | |
| polynoteuser02@polynotevm02:~/download$ pip3 install jedi | |
| polynoteuser02@polynotevm02:~/download$ pip3 install pyspark | |
| polynoteuser02@polynotevm02:~/download$ pip3 install virtualenv | |
| polynoteuser02@polynotevm02:~/download$ pip3 install numpy | |
| polynoteuser02@polynotevm02:~/download$ pip3 install pandas | |
| # Let's run Polynote |