Skip to content

Instantly share code, notes, and snippets.

View GabrielSGoncalves's full-sized avatar
🏀
Data Engineer @ Perform-Billtrust

GabrielSGoncalves GabrielSGoncalves

🏀
Data Engineer @ Perform-Billtrust
View GitHub Profile
# Define scaling method and values
scaling_method = "min_max"
scaling_method_data = {"min": 0, "max": 1000}
# Instatiate and fit Scorecard
scorecard = Scorecard(
target='TARGET',
binning_process=binning_process,
estimator=logreg,
scaling_method=scaling_method,
# Define the feature list from dataset (including categorical and numerical)
list_features = df_application_train.drop(columns=['TARGET']).columns.values
# Define categorical features list
list_categorical = df_application_train.select_dtypes(include=['object', 'category']).columns.values
# Define selection criteria for BinningProcess
selection_criteria = {"iv": {"min": 0.005, 'max':0.5, "strategy": "highest"}}
# Instatiate BinningProcess
import pandas as pd
from optbinning import Scorecard, BinningProcess
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
# Load the train dataset
df_application = pd.read_csv('./home-credit-default-risk/application_train.csv', low_memory=True)
df_application.set_index('SK_ID_CURR', inplace=True)
# Split the dataset into train and test
@GabrielSGoncalves
GabrielSGoncalves / plotly_line_and_bar.py
Created November 19, 2020 21:07
Code for plotting a line chart and barplot using Plotly.
import plotly.graph_objects as go
from plotly.subplots import make_subplots
import plotly.express as px
import plotly.graph_objects as
fig = make_subplots(rows=1, cols=2)
colors = px.colors.qualitative.Set1
text_position = [
"top center",
"bottom center",
@GabrielSGoncalves
GabrielSGoncalves / plotly_multiplot.py
Created August 20, 2020 14:55
Create a multiplot scatters using Plotly, redefine axis values and ticks and also add values next to markers.
import plotly.graph_objects as go
from plotly.subplots import make_subplots
fig = make_subplots(rows=2, cols=1,)
fig.add_trace(
go.Scatter(
x=df_analysis.index,
y=df_analysis.fpd_by_amount,
mode="lines+markers+text",
@GabrielSGoncalves
GabrielSGoncalves / compressed_pickle.py
Created June 5, 2020 14:29 — forked from cwidmer/compressed_pickle.py
save/load compressed pickled objects in python
import cPickle
import bz2
def save(filename, myobj):
"""
save object to file using pickle
@param filename: name of destination file
@type filename: str
@GabrielSGoncalves
GabrielSGoncalves / parsing_lambda_logs.py
Created January 9, 2020 18:25
Function to parse Lambda logs on CloudWatch using boto3
def parsing_lambda_logs(RequestId, GroupName , StreamName):
"""
Function that parses Lambda Logs on CloudWatch using boto3
Parameters:
===========
RequestId (str): Unique identifier for each AWS Lambda call
GroupName (str): Name of the Lambda Function group on CloudWatch
StreamName (str): Name of the log stream for the Function container
@GabrielSGoncalves
GabrielSGoncalves / mafft_lambda_function.py
Created December 19, 2019 17:15
Amazon Lambda function to run MAFFT aligner
import json
import os
import boto3
def load_file_from_S3(key, bucket):
"""Download file from S3 to /tmp/ folder"""
local_path = key.split('/')[-1]
print(key)
print(local_path)
filename = f'/tmp/{local_path}'
@GabrielSGoncalves
GabrielSGoncalves / papermill_python_api.py
Last active March 1, 2020 22:44
executing papermill using its Python API
import papermill as pm
pm.execute_notebook('weather_forecast_using_pyowm.ipynb',
'weather_forecast_using_pyowm_output.ipynb',
parameters={'city':'Sao Paulo,BR'},
kernel_name='papermill-tutorial')
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.