Skip to content

Instantly share code, notes, and snippets.

View jaeyow's full-sized avatar

JO Reyes jaeyow

View GitHub Profile
@jaeyow
jaeyow / getLabelsOfImage.py
Last active March 7, 2023 03:18
python method to call an Amazon Rekognition label detection
def getLabelsOfImage(bucket_name, number_labels, image):
# Use boto3 call detect_labels to get Amazon Rekognition labels
client = boto3.client("rekognition")
response = client.detect_labels(
Image={"S3Object": {"Bucket": bucket_name, "Name": image}},
MaxLabels=number_labels,
MinConfidence=98
)
return response["Labels"]
@jaeyow
jaeyow / sagemaker-metaflow-step.py
Created August 14, 2022 01:13
A Metaflow step for training a model using AWS SageMaker
@step
def model_training(self):
"""
Model training
- now training starts, first we specify the Docker image for the required algorithm, in this case linear learner
- create an estimator with the specified parameters,
- set the static hyperparameters, and SageMaker will automatically calculate those set as 'auto'
- calling fit() starts the training process, upto the specified number of epochs
- the save the model name and location for the next steps
- take note that we have to specify an instance for training, which may be different from the endpoint instance
@jaeyow
jaeyow / metaflow-comet-fanout-not-working.py
Created July 25, 2022 22:59
Comet ML and Metaflow - how to handle parallel writes to single experiment
from metaflow import FlowSpec, step, current
from comet_ml import API, Experiment
import os
import random
try:
from dotenv import load_dotenv
load_dotenv(verbose=True, dotenv_path='.env')
except:
print("No dotenv package")
@jaeyow
jaeyow / f1-mlops.yml
Created January 3, 2022 02:14
Using GitHub Actions as a cheap (and free) MLOps tool alternative
# Pre-process and model building
name: F1 Prediction MLOps
on:
workflow_dispatch
jobs:
build:
runs-on: ubuntu-latest