This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def expected_steps(df): | |
Q = df.drop( | |
['Null', 'Activation'], axis=1).drop(['Null', 'Activation'], axis=0) | |
I = np.identity(Q.shape[1]) | |
N = np.linalg.inv(I - Q.to_numpy()) | |
t = np.sum(N, axis=1) | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
python -m predict.test_predict \ | |
--csv-data-path gs://${PROJECT_ID}-black-friday-demo-bucket/data/test.csv \ | |
--model-name black_friday_forecast \ | |
--project-id ${PROJECT_ID} \ | |
--version-name $VERSION |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from setuptools import setup, find_packages | |
REQUIREMENTS = [ | |
"scikit-learn", | |
"pandas", | |
"numpy", | |
"tensorflow==1.15.0", | |
"gcsfs", | |
"google-api-python-client" | |
] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PROJECT_ID=<PROJECT_ID> | |
BUCKET_NAME="${PROJECT_ID}-black-friday-demo-bucket" | |
MODEL_NAME=black_friday_forecast | |
REGION=us-central1 | |
VERSION='v'`date '+%Y%m%d_%H%M%S'` | |
MODEL_PATH=gs://$BUCKET_NAME/models/$VERSION/model | |
PACKAGE_PATH=gs://$BUCKET_NAME/models/$VERSION/package/black_friday_forecast-0.0.1.tar.gz | |
# Package solution | |
python setup.py sdist |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# -*- coding: utf-8 -*- | |
import argparse | |
import datetime | |
import pandas as pd | |
import os | |
from sklearn import preprocessing | |
from sklearn.metrics import mean_squared_error, make_scorer |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PROJECT_ID=<PROJECT_ID> | |
BUCKET_NAME="${PROJECT_ID}-black-friday-demo-bucket" | |
MODEL_NAME=black_friday_forecast | |
REGION=us-central1 | |
VERSION='v'`date '+%Y%m%d_%H%M%S'` | |
MODEL_PATH=gs://$BUCKET_NAME/models/$VERSION/model | |
gcloud ai-platform jobs submit training black_friday_forecast_$(date +"%Y%m%d_%H%M%S") \ | |
--job-dir gs://$BUCKET_NAME/models/$VERSION/job_dir \ | |
--package-path ./train \ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import googleapiclient.discovery | |
import argparse | |
import pandas as pd | |
import json | |
import logging | |
logging.basicConfig() | |
logger = logging.getLogger(__name__) | |
logger.setLevel(logging.INFO) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import pandas as pd | |
import numpy as np | |
from sklearn.base import BaseEstimator | |
from sklearn import preprocessing | |
import json | |
class BlackFridayPreprocess(BaseEstimator): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from sklearn.ensemble import RandomForestRegressor | |
from train.models.custom_transformers import BlackFridayPreprocess, BlackFridayIdTransformer, BlackFridayLabelEncoder | |
from sklearn.pipeline import Pipeline | |
def random_forest(): | |
pipeline = Pipeline( | |
[ | |
('preprocess', BlackFridayPreprocess()), | |
('id_filter', BlackFridayIdTransformer( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Variable | Definition | |
---|---|---|
User_ID | User ID | |
Product_ID | Product ID | |
Gender | Sex of User | |
Age | Age in bins | |
Occupation | Occupation (Masked) | |
City_Category | Category of the City (A,B,C) | |
Stay_In_Current_City_Years | Number of years stay in current city | |
Marital_Status | Marital Status | |
Product_Category_1 | Product Category (Masked) |