This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import sys | |
from awsglue.transforms import * | |
from awsglue.utils import getResolvedOptions | |
from pyspark.context import SparkContext | |
from awsglue.context import GlueContext | |
from awsglue.job import Job | |
## @params: [JOB_NAME] | |
args = getResolvedOptions(sys.argv, ['JOB_NAME']) | |
bucketpathparam = getResolvedOptions(sys.argv, ['s3_path']) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os, json, shutil | |
from datetime import datetime | |
rootdir ='./input' | |
for subdir, dirs, files in os.walk(rootdir): | |
#print(subdir, dirs, files) | |
for file in files: | |
print(file) | |
f=open(subdir + '/'+ file,'r') |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Verifying that "alanwill.id" is my Blockstack ID. https://onename.com/alanwill |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
'use strict'; | |
/* | |
'use strict' is not required but helpful for turning syntactical errors into true errors in the program flow | |
http://www.w3schools.com/js/js_strict.asp | |
*/ | |
/* | |
Modules make it possible to import JavaScript files into your application. Modules are imported | |
using 'require' statements that give you a reference to the module. | |
It is a good idea to list the modules that your application depends on in the package.json in the project root |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import boto3 | |
getCleanTable = taskStatus.query( | |
KeyConditionExpression=Key('requestId').eq('a3248585-8ac0-4889-b1fb-976d9113f7f2') & Key('eventTimestamp').gt('1') | |
) | |
for i in getCleanTable['Items']: | |
cleanupTable = boto3.client('dynamodb').batch_write_item( | |
RequestItems={ | |
'talr-taskStatus': [ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aws lambda update-function-code --function my-lambda-function --zip-file fileb://lambda-package.zip |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"body" : $input.json('$'), | |
"headers": { | |
#foreach($header in $input.params().header.keySet()) | |
"$header": "$util.escapeJavaScript($input.params().header.get($header))" #if($foreach.hasNext),#end | |
#end | |
}, | |
"method": "$context.httpMethod", | |
"params": { |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
apt-get update -y | |
apt-get install apt-transport-https ca-certificates -y | |
apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D | |
rm -f /etc/apt/sources.list.d/docker.list | |
echo "deb https://apt.dockerproject.org/repo ubuntu-trusty main" | tee /etc/apt/sources.list.d/docker.list | |
apt-get update -y |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
curl -s 'https://sks-keyservers.net/pks/lookup?op=get&search=0xee6d536cf7dc86e2d7d56f59a178ac6c6238f52e' | apt-key add --import | |
apt-get update -y && apt-get install apt-transport-https -y | |
echo "deb https://packages.docker.com/1.10/apt/repo ubuntu-trusty main" | tee /etc/apt/sources.list.d/docker.list | |
apt-get update -y && sudo apt-get install docker-engine -y | |
service docker start | |
usermod -a -G docker ubuntu |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// Use Gists to store code you would like to remember later on | |
console.log(window); // log the "window" object to the console |
NewerOlder