Skip to content

Instantly share code, notes, and snippets.

// Copyright 2007, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
@seahrh
seahrh / fast_predict.py
Last active February 13, 2021 03:33
Faster than Keras model.predict. If you are using tf.keras.Model and predicting via the functional call (as done in Jane Street: Neural Network Starter and other Notebooks), try using:
model.call = tf.function(model.call, experimental_relax_shapes=True)
for (test_df, pred_df) in tqdm(env.iter_test()):
pred = model(data, training = False).numpy()
@seahrh
seahrh / ls_summarize.sh
Created February 4, 2021 06:46
aws s3 -Rlah recursive list files with file size
aws s3 ls --summarize --human-readable --recursive s3://mybucket/mydir/
CREATE OR REPLACE PROCEDURE sp_update_ddl()
AS $$
BEGIN
alter table myschema.mytable add column newcolumn bigint;
EXCEPTION
WHEN OTHERS THEN
raise 'exception in filename.sql';
END
$$ LANGUAGE plpgsql
;
@seahrh
seahrh / bitbucket-pipelines.yml
Created January 13, 2021 09:47
Python starter bitbucket pipelines
definitions:
steps:
- step: &tests
image: python:3.7.9
script:
- pip install ."[tests]"
- mypy src
- pytest -vv --cov=src
caches:
- pip
@seahrh
seahrh / librosa_visualize_audio.py
Created January 12, 2021 10:16
1. Prints information about an audio singal, 2. plots the waveform, and 3. Creates player (Taken from https://www.audiolabs-erlangen.de/resources/MIR/FMP/B/B_PythonAudio.html)
import os
import numpy as np
from matplotlib import pyplot as plt
import IPython.display as ipd
import librosa
import pandas as pd
%matplotlib inline
def print_plot_play(x, Fs, text=''):
"""1. Prints information about an audio singal, 2. plots the waveform, and 3. Creates player
@seahrh
seahrh / librosa_visualize_audio.py
Created January 12, 2021 10:16
1. Prints information about an audio singal, 2. plots the waveform, and 3. Creates player
import os
import numpy as np
from matplotlib import pyplot as plt
import IPython.display as ipd
import librosa
import pandas as pd
%matplotlib inline
def print_plot_play(x, Fs, text=''):
"""1. Prints information about an audio singal, 2. plots the waveform, and 3. Creates player
@seahrh
seahrh / clip.py
Created December 11, 2020 00:55
Keras clipping custom metric. Adapted from https://neptune.ai/blog/keras-metrics
def recall(y_true, y_pred):
y_true = K.ones_like(y_true)
true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
all_positives = K.sum(K.round(K.clip(y_true, 0, 1)))
recall = true_positives / (all_positives + K.epsilon())
return recall
def precision(y_true, y_pred):
y_true = K.ones_like(y_true)
@seahrh
seahrh / featureimportance.py
Created November 6, 2020 09:41
Put feature importance scores in dataframe
feature_important = clf.get_booster().get_score(importance_type="weight")
keys = list(feature_important.keys())
values = list(feature_important.values())
data = pd.DataFrame(data=values, index=keys, columns=["score"]).sort_values(by = "score", ascending=False)
# Top 10 features
data.head(20)
@seahrh
seahrh / showsortkey.sql
Created November 5, 2020 08:52
redshift show sortkey columns
-- see https://docs.aws.amazon.com/redshift/latest/dg/r_PG_TABLE_DEF.html
select "column", type, encoding, distkey, sortkey
from pg_table_def where schemaname='public' and tablename='mytable' and sortkey!=0;