Skip to content

Instantly share code, notes, and snippets.

View Jammink2's full-sized avatar

John Hammink Jammink2

10 followers · 5 following
View GitHub Profile
@Jammink2
Jammink2 / gist:6fcf0ca97fc241d611f3
Last active August 29, 2015 14:11 — forked from wrburgess/gist:2170970
require 'csv'
require 'json'
csv_string = CSV.generate do |csv|
JSON.parse(File.open("filename.json").read).each do |hash|
data = hash[1]
theline = theline + " #{data},"
end
end
@Jammink2
Jammink2 / lambda.py
Last active April 5, 2016 00:19
import urllib, urllib2, base64, time, json, sys
import dateutil, dateutil.parser, datetime
td_database = 'kinesis'
td_table = 'events'
td_master_key = '<YOUR_TD_MASTER_API_KEY>'
td_endpoint = 'https://in.treasuredata.com/js/v3/event'
def upload_td(records):
# https://docs.treasuredata.com/articles/javascript-sdk#appendix-api-endpoint
@Jammink2
Jammink2 / lambda.json
Created April 4, 2016 23:31
{
"Records": [
{
"eventID": "shardId-000000000000:49545115243490985018280067714973144582180062593244200961",
"eventVersion": "1.0",
"kinesis": {
"partitionKey": "partitionKey-3",
"data": "eyJmaWVsZDEiOiAyLCAiZmllbGQyIjogImIifQ==",
"kinesisSchemaVersion": "1.0",
"sequenceNumber": "49545115243490985018280067714973144582180062593244200961"
@Jammink2
Jammink2 / gist:dbf6af2a04df7f4a346d9ac0882e777b
Last active April 12, 2016 23:16
Commands
wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-2.3.1.deb
sudo dpkg -i elasticsearch-2.3.1.deb
@Jammink2
Jammink2 / gist:65b75afef30f1e3824fa81751fed545f
Last active April 12, 2016 23:17
Edit Elasticsearch.yml
sudo nano /etc/elasticsearch/elasticsearch.yml
@Jammink2
Jammink2 / gist:2ececd79f85b97e42fbb2b3180e2892e
Last active April 8, 2016 22:05
Elasticsearch Settings to edit
cluster.name: elasticsearch
node.name: "integrations"
node.master: true
node.data: true
index.number_of_shards: 5
Index.number_of_replicas: 1
network.bind_host: <your_instance's_internal_IP> #from EC2 console
transport.tcp.port: 9300
http.port: 9200
@Jammink2
Jammink2 / gist:7a71f4b891e5d5c7a2e6379af4ad37ed
Last active April 12, 2016 23:17
Tailing Elasticsearch log
tail -f /var/log/elasticsearch/elasticsearch.log
@Jammink2
Jammink2 / gist:855dc4577bd54f3c130365f59cc5abf4
Last active April 12, 2016 23:22
Restarting Elasticsearch
sudo /etc/init.d/elasticsearch restart
@Jammink2
Jammink2 / accounts.yml
Created April 8, 2016 21:59
in:
type: sfdc
username: "USERNAME" # your salesforce username
password: "PASSWORD" # your Salesforce password
security_token: "SECURITY_TOKEN" # your Security Token
client_id: "CLIENT_ID" # your app's consumer key
client_secret: "CLIENT_SECRET" # your app's consumer secret
login_url: https://login.salesforce.com/
incremental: false # incremental = full data dump
target: Account # Salesforce Object to import
@Jammink2
Jammink2 / gist:ed5798b9a0f248496ec99e5628e8f70a
Last active April 12, 2016 23:18
Create DB and Table on Treasure Data
td db:create sfdc
td table:create sfdc account