Skip to content

Instantly share code, notes, and snippets.

@bokwoon95
bokwoon95 / vars.sql
Last active August 8, 2020 15:11
How to use psql, plpgsql and session variables together in one file
-- psql variables can only be used outside functions
-- plpgsql variables can only be used inside functions
-- session variables can be used both outside and inside functions, bridging the gap between the two types of variables
\set func_name vars
\echo '[':func_name'.sql]'
CREATE OR REPLACE FUNCTION :func_name (arg_execute TEXT)
RETURNS VOID AS $$ DECLARE
BEGIN
END;
def low_pass_filter_anomaly_detection(df,
column_name,
number_of_stdevs_away_from_mean):
"""
Implement a low-pass filter to detect anomalies in a time series, and save the filter outputs
(True/False) to a new column in the dataframe.
Arguments:
df: Pandas dataframe
column_name: string. Name of the column that we want to detect anomalies in
number_of_stdevs_away_from_mean: float. Number of standard deviations away from
@liquidgenius
liquidgenius / Query.py
Last active May 15, 2019 01:51
A simple class that utilizes chained methods to query a MySQL table for designated columns. All records are returned. Inspired by pandas.
import pymysql
from pprint import pprint as pprint
__author__ = "Liquidgenius"
class Query:
def __init__(self, host, port, user, password, schema):
self.host = host
self.port = port
self.user = user
@npearce
npearce / install-docker.md
Last active April 26, 2025 04:27
Amazon Linux 2 - install docker & docker-compose using 'sudo amazon-linux-extras' command

UPDATE (March 2020, thanks @ic): I don't know the exact AMI version but yum install docker now works on the latest Amazon Linux 2. The instructions below may still be relevant depending on the vintage AMI you are using.

Amazon changed the install in Linux 2. One no-longer using 'yum' See: https://aws.amazon.com/amazon-linux-2/release-notes/

Docker CE Install

sudo amazon-linux-extras install docker
sudo service docker start
@lobstrio
lobstrio / linkedin_extract.py
Created April 24, 2018 11:05
Scraping Linkedin profiles information through Selenium Python
# python package
import csv
import time
import random
import sys
import os
# selenium package
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
@liquidgenius
liquidgenius / dataframe_dataset_dataframe_roundtrip.py
Last active April 24, 2018 04:14
In Python, save a Pandas DataFrame to Sqlite using Dataset module and retrieve into Dataframe utilizing only Dataset function all().
import dataset
import pandas as pd
# create dataframe
df = pd.DataFrame()
names = ['Bob', 'Jane', 'Alice', 'Ricky']
ages = [31, 30, 31, 30]
df['names'] = names
df['ages'] = ages
from selenium import webdriver
from selenium.webdriver.chrome.options import DesiredCapabilities
from selenium.webdriver.common.proxy import Proxy, ProxyType
import time
co = webdriver.ChromeOptions()
co.add_argument("log-level=3")
co.add_argument("--headless")
@rdinse
rdinse / Google_Colaboratory_backup.py
Created March 13, 2018 22:55
Simple Google Drive backup script with automatic authentication for Google Colaboratory (Python 3)
# Simple Google Drive backup script with automatic authentication
# for Google Colaboratory (Python 3)
# Instructions:
# 1. Run this cell and authenticate via the link and text box.
# 2. Copy the JSON output below this cell into the `mycreds_file_contents`
# variable. Authentication will occur automatically from now on.
# 3. Create a new folder in Google Drive and copy the ID of this folder
# from the URL bar to the `folder_id` variable.
# 4. Specify the directory to be backed up in `dir_to_backup`.
@WillKoehrsen
WillKoehrsen / submit_assignment.py
Created March 10, 2018 20:26
Automation of Assignment Submisison
# selenium for web driving
import selenium
from selenium import webdriver
# time for pausing between navigation
import time
# Datetime for recording time of submission
import datetime
@xiaostrong
xiaostrong / qqmessage.py
Created February 21, 2018 08:57
QQ空间留言脚本
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import requests
import time
import re
import json
import time
def login():
options = webdriver.ChromeOptions()