Skip to content

Instantly share code, notes, and snippets.

View shaboi's full-sized avatar
🎱

Adam shaboi

🎱
  • @populi-ai
  • Philly
View GitHub Profile
@ThriceGood
ThriceGood / logger.py
Last active February 26, 2018 15:51
Python Logging wrapper for easy logging
import logging
import os
class Logger:
def __init__(self, name):
self.create_log_dir()
self.logger = logging.getLogger('{}'.format(name))
format = "%(asctime)s [%(levelname)s]: %(message)s"
logging.basicConfig(format=format, level=logging.DEBUG)
@dmyersturnbull
dmyersturnbull / groupyby_parallel.py
Last active February 6, 2024 00:43
Performs a Pandas groupby operation in parallel
import pandas as pd
import itertools
import time
import multiprocessing
from typing import Callable, Tuple, Union
def groupby_parallel(
groupby_df: pd.core.groupby.DataFrameGroupBy,
func: Callable[[Tuple[str, pd.DataFrame]], Union[pd.DataFrame, pd.Series]],
num_cpus: int = multiprocessing.cpu_count() - 1,
@aabadie
aabadie / strategies_comparison.py
Last active September 17, 2020 17:36
Persistence strategies comparison
"""Persistence strategies comparison script.
This script compute the speed, memory used and disk space used when dumping and
loading arbitrary data. The data are taken among:
- scikit-learn Labeled Faces in the Wild dataset (LFW)
- a fully random numpy array with 10000x10000 shape
- a dictionary with 1M random keys/values
- a list containing 10M random value
The compared persistence strategies are:
@robdmc
robdmc / csv_logger.py
Last active December 13, 2022 09:57
Useful CSV logging with python
#! /usr/bin/env python
import logging
import datetime
import fleming
import pytz
import os
from itertools import islice
import sys
@shagunsodhani
shagunsodhani / Batch Normalization.md
Last active July 25, 2023 18:07
Notes for "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift" paper

The Batch Normalization paper describes a method to address the various issues related to training of Deep Neural Networks. It makes normalization a part of the architecture itself and reports significant improvements in terms of the number of iterations required to train the network.

Issues With Training Deep Neural Networks

Internal Covariate shift

Covariate shift refers to the change in the input distribution to a learning system. In the case of deep networks, the input to each layer is affected by parameters in all the input layers. So even small changes to the network get amplified down the network. This leads to change in the input distribution to internal layers of the deep network and is known as internal covariate shift.

It is well established that networks converge faster if the inputs have been whitened (ie zero mean, unit variances) and are uncorrelated and internal covariate shift leads to just the opposite.

@fredbenenson
fredbenenson / kickstarter_sql_style_guide.md
Last active October 14, 2025 15:32
Kickstarter SQL Style Guide
layout title description tags
default
SQL Style Guide
A guide to writing clean, clear, and consistent SQL.
data
process

Purpose

@mbejda
mbejda / Fortune-1000-Company-Twitter-Accounts.csv
Last active November 5, 2025 03:34
Fortune 1000 companies Twitter Accounts. Twitter,domain, keywords, and description
We can make this file beautiful and searchable if this error is corrected: Unclosed quoted field in line 5.
domain,name,keywords,description,twitter
walmartstores.com,Wal-Mart Stores,"SEC filing,Walmart photos,walmart stock,sustainability index,Wal-Mart,walmart annual reports,Walmart responsibility,executive speeches,walmart suppliers,global responsibility,walmart global sustainability report,walmart investors,Walmart interactive map,walmart history,privacy policy,financial reports,walmart news,Wal Mart,walmart sustainability,Walmart locations,Walmart videos,walmart story,Walmart,Walmart stores,walmart board of directors,community giving,walmart careers,Walmart jobs,sam walton","Find Walmart executive speeches, financial reports, press releases, downloadable photos and videos, and see an interactive map of our locations around the world.",walmart
gm.com,General Motors,,"General Motors is home to Buick, Cadillac, GMC and Chevrolet. Find the latest news about GM automotive innovations, investor relations and more. ",GM
ge.com,General Electric,,,generalelectric
chevron.com,ChevronTexaco,"cvx, chevrontexaco, cheveron,
@phobson
phobson / Lognormref.ipynb
Last active September 17, 2020 16:35
Lognormal python ref
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@GaelVaroquaux
GaelVaroquaux / mutual_info.py
Last active June 18, 2023 12:25
Estimating entropy and mutual information with scikit-learn: visit https://github.com/mutualinfo/mutual_info
'''
Non-parametric computation of entropy and mutual-information
Adapted by G Varoquaux for code created by R Brette, itself
from several papers (see in the code).
This code is maintained at https://github.com/mutualinfo/mutual_info
Please download the latest code there, to have improvements and
bug fixes.
@ulope
ulope / autolog.py
Last active November 20, 2017 15:17
Call logging decorator
# Written by Brendan O'Connor, [email protected], www.anyall.org
# * Originally written Aug. 2005
# * Posted to gist.github.com/16173 on Oct. 2008
# Copyright (c) 2003-2006 Open Source Applications Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#