Yuta Imai yuta-imai
yuta-imai
/ cognito-identity-sample.js
Last active
August 29, 2015 14:05
The sample code for using CognitoIndentityCredentials to retrieve AWS Credentials.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| //現状CognitoはUS_EAST_1にしか無いので一度こちらを設定 | |
| AWS.config.region = 'us-east-1'; | |
| AWS.config.credentials = new AWS.CognitoIdentityCredentials({ | |
| AccountId: 'YOUR_AWS_ACCOUNT_ID', | |
| IdentityPoolId: 'COGNITO_IDENTITIY_POOL_ID', | |
| RoleArn: 'IAM_ROLE_ARN' | |
| }); | |
| //その後、利用したいリージョンに設定を変更 | |
| AWS.config.region = 'ap-northeast-1'; |
yuta-imai
/ kinesis_data_extractor.py
Last active
August 29, 2015 14:04
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import time | |
| import boto.kinesis | |
| conn = boto.kinesis.connect_to_region('ap-northeast-1') | |
| stream_name = 'YOUR_STREAM_NAME' | |
| stream = conn.describe_stream(stream_name) | |
| shardid = stream['StreamDescription']['Shards'][0]['ShardId'] | |
| iterator_object = conn.get_shard_iterator(stream_name,shardid,'LATEST') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| CREATE EXTERNAL TABLE cflog ( | |
| dt STRING, | |
| tm STRING, | |
| edge STRING, | |
| bytes STRING, | |
| ip STRING, | |
| method STRING, | |
| host STRING, | |
| uri STRING, | |
| status STRING, |
yuta-imai
/ cflog-importer.sh
Last active
August 29, 2015 13:57
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| TARGET=$1 # for example '2014-03-10' or '2014-03-10-10' | |
| KEYPAIR=$2 # ec2 keypair name for ssh | |
| MASTER_INSTANCE_TYPE='m1.large' | |
| SLAVE_INSTANCE_TYPE='m1.large' | |
| NUM_INSTANCES='3' | |
| LOG_URI='' # s3://bucketname/path/to/emrlogdir/ This can be different bucket other than cflog | |
| S3ENDPOINT='s3-ap-northeast-1.amazonaws.com' # ap-northeast-1 is endpoint for Tokyo region. | |
| CFLOG='' # s3://bucketname/path/to/cflogdir/ |
yuta-imai
/ s3measurer.py
Last active
January 2, 2016 16:49
The script to measure total size of objects in S3 bucket.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/usr/bin/env python | |
| import boto | |
| import sys | |
| bucket_name = sys.argv[1] | |
| s3 = boto.connect_s3() | |
| bucket = s3.get_bucket(bucket_name) | |
| iterator = bucket.list() | |
| size = 0 |
yuta-imai
/ webserver.js
Last active
January 1, 2016 16:09
8080番ポートでHTTPリクエストを受け付け、受け取ったリクエストのヘッダとボディをJSONにして出力するサンプルコード。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| var http = require('http'); | |
| http.createServer(function(req,res){ | |
| var data = { | |
| RequestHeader: req.headers | |
| }; | |
| if(req.method == 'GET'){ | |
| response(res,data); |
yuta-imai
/ kinesis_put_record.rb
Created
December 4, 2013 23:46
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| require 'aws-sdk' | |
| require 'logger' | |
| require "json" | |
| require "base64" | |
| def hexdigest value | |
| digest = Digest::SHA256.new | |
| if value.respond_to?(:read) | |
| chunk = nil | |
| chunk_size = 1024 * 1024 # 1 megabyte |
yuta-imai
/ DynamoDBTransactionSample.java
Last active
December 24, 2015 11:09
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // DynamoDBクライアントの初期化 | |
| AmazonDynamoDB client = new AmazonDynamoDBClient(); | |
| // Transaction管理テーブルの作成 | |
| TransactionManager.verifyOrCreateTransactionTable( | |
| client, "Transactions" /*tableName*/, | |
| 10 /*readCapacityUnits*/, | |
| 10 /*writeCapacityUnits*/, | |
| 10 * 60 /*waitTimeSeconds*/); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| var http = require('http'); | |
| http.createServer(function(req,res){ | |
| res.writeHead(200,{'Content-Type:': 'application/json'}); | |
| res.end(JSON.stringify(req.headers)); | |
| }).listen(80); | |
| console.log('Server running at http://127.0.0.1:80/'); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import boto.ec2 | |
| import boto.sts | |
| import commands | |
| # fetch credential via sts | |
| # stsはus-eastにのみエンドポイントがある | |
| # EC2のIAM ROLEに紐付いた期限付きcredentialをSTSを使って取得する | |
| sts = boto.sts.connect_to_region("us-east-1") | |
| credential = sts.get_session_token() |