Skip to content

Instantly share code, notes, and snippets.

@EricLondon
EricLondon / s3_sns_lambda.js
Last active September 19, 2018 12:39
AWS Lambda S3 Notification to SNS with S3 MetaData
'use strict';
// ENV vars
const AWS_REGION_STRING = process.env.AWS_REGION || 'us-east-1';
const AWS_ACCOUNT_ID = process.env.AWS_ACCOUNT_ID;
const SNS_TOPIC_NAME = process.env.SNS_TOPIC_NAME;
const SNS_TOPIC_ARN = `arn:aws:sns:${AWS_REGION_STRING}:${AWS_ACCOUNT_ID}:${SNS_TOPIC_NAME}`;
const AWS = require('aws-sdk');
AWS.config.update({
@EricLondon
EricLondon / 1_usage.sh
Created September 12, 2018 18:33
Ruby stream command output
./parent.rb
command: ./child.rb 1
2018-09-12 14:32:09: stdout
2018-09-12 14:32:09: stderr
2018-09-12 14:32:11: stdout
2018-09-12 14:32:11: stderr
pid 97509 exit 1
97509
false
command: ./child.rb 0
@EricLondon
EricLondon / .1_readme.txt
Last active October 16, 2018 14:27
Export Elasticsearch spark driver log messages
# usage
nvm use .
npm install
chmod +x export-spark-driver-logs.sh
./export-spark-driver-logs.sh $ES_IP:$ES_PORT driver-##############-####
@EricLondon
EricLondon / MapGroupSortLimit.java
Created May 30, 2018 18:29
Java 8 group Map by value, sort desc, and limit 10
Map<String, Long> counted = list.stream()
.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));
Map<String, Long> sorted = counted.entrySet().stream()
.sorted(Collections.reverseOrder(Map.Entry.comparingByValue()))
.limit(10)
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue, (e1, e2) -> e1, LinkedHashMap::new));
@EricLondon
EricLondon / chainable.js
Created April 9, 2018 14:56
javascript chainable promise example
let method1 = function() {
console.log("method1")
return true
}
let method2 = function() {
console.log("method2")
return true
}
@EricLondon
EricLondon / brew-specific-version.sh
Last active January 10, 2018 14:14
Brew install specific version of package
# check if a specifc version is avaiable to install
brew search python@
==> Searching local taps...
python@3 [email protected] ipython@6
[email protected] ipython@5 python@2
# confirm semantic/minor version of package
brew info python@3 | head -1
python3: stable 3.6.4 (bottled), devel 3.7.0a3, HEAD
# ^ in this case, I want 3.5.x
@EricLondon
EricLondon / spark-scala-snippets.scala
Last active May 2, 2018 17:40
Spark Scala Snippets
////////////////////////////////////////////////////////////////////////////////////////////////////
// Custom UDF
////////////////////////////////////////////////////////////////////////////////////////////////////
// define method
def isNumeric: (String => Boolean) = {
case null => false
case value => value forall Character.isDigit
}
val ISNUMERIC = udf(isNumeric)
@EricLondon
EricLondon / emr-zeppelin-run.rb
Created October 31, 2017 15:20
EMR/Zeppelin set interpreter and run all notebooks
#!/usr/bin/env ruby
require 'json'
require 'net/http'
require 'open-uri'
require 'pp'
require 'uri'
ZEP_HOST = 'localhost'
ZEP_PORT = 8890
@EricLondon
EricLondon / Dockerfile
Created October 17, 2017 15:32
Docker build arguments
FROM busybox
ARG MESSAGE
RUN echo $MESSAGE > message.txt
@EricLondon
EricLondon / emr-zeppelin-status.rb
Last active October 5, 2017 14:14
EMR/Zeppelin API notebook status check
#!/usr/bin/env ruby
require 'open-uri'
require 'json'
require 'pp'
ZEP_HOST = 'localhost'
ZEP_PORT = 8890
ZEP_ADDR = "http://#{ZEP_HOST}:#{ZEP_PORT}"