Skip to content

Instantly share code, notes, and snippets.

View duncangh's full-sized avatar
💭
👨‍💻

Graham Duncan duncangh

💭
👨‍💻
View GitHub Profile
@duncangh
duncangh / .bash_profile
Created September 19, 2019 01:03 — forked from natelandau/.bash_profile
Mac OSX Bash Profile
# ---------------------------------------------------------------------------
#
# Description: This file holds all my BASH configurations and aliases
#
# Sections:
# 1. Environment Configuration
# 2. Make Terminal Better (remapping defaults and adding functionality)
# 3. File and Folder Management
# 4. Searching
# 5. Process Management
@duncangh
duncangh / type_as_string.py
Created September 18, 2019 22:03
Python 'type' object to string.
# https://stackoverflow.com/questions/5008828/convert-a-python-type-object-to-a-string
type_variety_list : list = [
10,
'x',
lambda x: x,
set(),
dict(),
[]
@duncangh
duncangh / picblast.sh
Created September 2, 2019 15:21 — forked from bwhitman/picblast.sh
Make an audio collage out of your live photos
mkdir /tmp/picblast; cd ~/Pictures/Photos\ Library.photoslibrary; for i in `find . | grep jpegvideocompl`;do ffmpeg -i $i /tmp/picblast/${i:(-8)}.wav; done; cd /tmp/picblast; ffmpeg -safe 0 -f concat -i <( for f in *.wav; do echo "file '$(pwd)/$f'"; done ) ~/Desktop/picblast.wav; rm -rf /tmp/picblast
@duncangh
duncangh / deque.md
Created July 8, 2019 00:42
Data Structures Python deque

Recent Emojis

if the implementation is recent emojis, then we could use a deque. The deque will be instantiated with a max length of 30 which is consistent with what Apple uses.

from collections import deque

recent_emojis = deque([], maxlen=30)

def update_recent_emojis(emoji_used, recent_emojis : deque):
    if emoji_used in recent_emojis:
        recent_emojis.remove(emoji_used)
var get_numeric_table_value = function(id) {
let element = document.getElementById(id);
return Number(element.textContent)
};
let ask = get_numeric_table_value('ask0');
let bid = get_numeric_table_value('bid0');
let avg = (bid + ask) / 2 ;
from ftplib import FTP
from datetime import datetime

start = datetime.now()
ftp = FTP('your-ftp-domain-or-ip')
ftp.login('your-username','your-password')

# Get All Files
files = ftp.nlst()
@duncangh
duncangh / controversy.js
Created May 9, 2019 19:45
Twitter find controversial replies
var root_tweet;
var tweets = new Array();
var scrollInterval = null;
var lastScrollHeight = 0;
function autoScroll() {
var sh = document.documentElement.scrollHeight;
@duncangh
duncangh / ec2_instance_connections.py
Last active May 12, 2019 00:22
RDS SQLAlchemy Connection Strings (if all of your databases are postgreSQL)
# I'm a one trick pony, but I can do ec2 conns too
import boto3
ec2 = boto3.client('ec2')
instances : list = ec2.describe_instances()['Reservations']
def _make_string_from_instance(instance):
key_name = instance['KeyName']
public_dns_name = instance['NetworkInterfaces'][0]['Association']['PublicDnsName']
@duncangh
duncangh / pandas_s3_streaming.py
Created April 7, 2019 03:03 — forked from uhho/pandas_s3_streaming.py
Streaming pandas DataFrame to/from S3 with on-the-fly processing and GZIP compression
def s3_to_pandas(client, bucket, key, header=None):
# get key using boto3 client
obj = client.get_object(Bucket=bucket, Key=key)
gz = gzip.GzipFile(fileobj=obj['Body'])
# load stream directly to DF
return pd.read_csv(gz, header=header, dtype=str)
def s3_to_pandas_with_processing(client, bucket, key, header=None):
@duncangh
duncangh / s3_to_pandas.py
Created April 7, 2019 03:01 — forked from jaklinger/s3_to_pandas.py
Read CSV (or JSON etc) from AWS S3 to a Pandas dataframe
import boto3
import pandas as pd
from io import BytesIO
bucket, filename = "bucket_name", "filename.csv"
s3 = boto3.resource('s3')
obj = s3.Object(bucket, filename)
with BytesIO(obj.get()['Body'].read()) as bio:
df = pd.read_csv(bio)