Skip to content

Instantly share code, notes, and snippets.

@elliottcordo
elliottcordo / package_tldextract_athena.sh
Created February 21, 2024 17:38
Example of how to package a Python package for Athena Spark
mkdir tldextract
cd tldextract
mkdir unpacked
pip install -t $PWD/unpacked tldextract cd unpacked
zip -r9 ../tldextract *
@elliottcordo
elliottcordo / athena_runner.py
Created February 21, 2024 17:02
Simple Athena Spark Runner
from time import sleep
import boto3
client = boto3.client('athena')
calculation_response = client.start_session(
Description='job_session',
WorkGroup='aws-meetup',
EngineConfiguration={
'CoordinatorDpuSize': 1,
'DefaultExecutorDpuSize': 1,
import networkx as nx
Graphtype = nx.Graph()
with open('result.csv', "r") as data:
# skip the header
next(data, None)
# parse into the graph
G = nx.parse_edgelist(data, delimiter=',', create_using=Graphtype)
use database "CRM_POC_DB";
use schema "CUST_CLEANUP";
drop table match_results;
create temporary table match_results as
with cust as (
select
ACCT_CP,
COMPANY_CUST,
COMPANY_CODE,
@elliottcordo
elliottcordo / Translate-a-tron.py
Last active January 27, 2020 20:01
Translate-a-tron.py
import boto3
import os
import slack
from pprint import pprint
SLACK_API_TOKEN = os.environ["SLACK_API_TOKEN"]
BOT_ID = '<@UT4N8STGX>'
FLAG_MAP = {
':flag-mx:': 'es',
':flag-cn:': 'zh',
@elliottcordo
elliottcordo / redshift_connection_killer.py
Last active June 23, 2016 18:59
Redshift Connection Killer
#!/usr/bin/env python
import sys, os
import json
import argparse
from classes.Config import Config
from classes.pg_tools import PGInteraction
from classes.Logger import Logger as l
from pprint import pprint
from datetime import datetime, timedelta
@elliottcordo
elliottcordo / crontab.md
Last active May 18, 2016 11:51
crontab for @reboot actions

Make sure you setup your crontab as su.. crontabs are user specific and you don't want the same jobs scheduled twice!

sudo -s
export VISUAL=nano; crontab -e

Make entries and when done control-x and enter yes for saving changes

0 23   *   *   *    /sbin/shutdown -r +5
@reboot python /usr/local/robopager/robopager/robopager.py
[general]
recording_length=10
wait_time=120
archive_path=/Users/elliottcordo/projects/pinary/archive
[email]
from_email=xxx
to_email=xxx
password=xxx
@elliottcordo
elliottcordo / README.md
Last active October 16, 2015 15:03
simple python config file encrypter with sample config file and config file reader

#simple example of encryption usage with python config

  • yeah, base64 isn't very secure
  • you probably also wanna compile to an executable to obscure the code..
@elliottcordo
elliottcordo / rpi_led.py
Last active October 9, 2015 13:17
raspberry pi basic IO
import RPi.GPIO as GPIO
from time import sleep
GPIO.setmode(GPIO.BOARD)
GPIO.setup(7, GPIO.OUT)
while True:
sleep(.5)
GPIO.output(7,True)
sleep(.5)
GPIO.output(7,False)