Skip to content

Instantly share code, notes, and snippets.

View alanwill's full-sized avatar
🤔
Thinking

Alan Williams alanwill

🤔
Thinking
View GitHub Profile
@alanwill
alanwill / delete-multiple-rows.py
Created August 21, 2016 08:59
Delete Multiple DynamoDB rows
import boto3
getCleanTable = taskStatus.query(
KeyConditionExpression=Key('requestId').eq('a3248585-8ac0-4889-b1fb-976d9113f7f2') & Key('eventTimestamp').gt('1')
)
for i in getCleanTable['Items']:
cleanupTable = boto3.client('dynamodb').batch_write_item(
RequestItems={
'talr-taskStatus': [
@alanwill
alanwill / invoke_apigateway.js
Created November 10, 2016 21:12
Swagger Controller to invoke API Gateway
'use strict';
/*
'use strict' is not required but helpful for turning syntactical errors into true errors in the program flow
http://www.w3schools.com/js/js_strict.asp
*/
/*
Modules make it possible to import JavaScript files into your application. Modules are imported
using 'require' statements that give you a reference to the module.
It is a good idea to list the modules that your application depends on in the package.json in the project root
@alanwill
alanwill / verify
Last active January 4, 2017 07:23
onename verify
Verifying that "alanwill.id" is my Blockstack ID. https://onename.com/alanwill
@alanwill
alanwill / transform.py
Created May 7, 2018 22:57
Copies files to a date centric directory structure based on value of a json field within the file
import os, json, shutil
from datetime import datetime
rootdir ='./input'
for subdir, dirs, files in os.walk(rootdir):
#print(subdir, dirs, files)
for file in files:
print(file)
f=open(subdir + '/'+ file,'r')
@alanwill
alanwill / glue-json2parquet.py
Last active March 26, 2025 19:30
AWS Glue JSON to Parquet transformation script
import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job
## @params: [JOB_NAME]
args = getResolvedOptions(sys.argv, ['JOB_NAME'])
bucketpathparam = getResolvedOptions(sys.argv, ['s3_path'])