koolhead17 koolhead17
koolhead17
/ kafka
Created
February 10, 2017 10:30
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| "kafka": { | |
| "1": { | |
| "enable": true, | |
| "brokers": ["localhost:9092"], | |
| "topic": "bucketevents" | |
| } | |
| } |
koolhead17
/ listen-bucket.js
Created
January 7, 2017 06:15
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| var Minio = require('minio') | |
| // Instantiate the minio client with the endpoint | |
| // and access keys as shown below. | |
| var minioClient = new Minio.Client({ | |
| endPoint: 'play.minio.io', | |
| port: 9000, | |
| secure: true, | |
| accessKey: 'Q3AM3UQ867SPQQA43P2F', | |
| secretKey: 'zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG' |
koolhead17
/ pgessays.py
Created
December 29, 2016 21:01
— forked from olasitarska/pgessays.py
Builds epub book out of Paul Graham's essays.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # -*- coding: utf-8 -*- | |
| """ | |
| Builds epub book out of Paul Graham's essays: http://paulgraham.com/articles.html | |
| Author: Ola Sitarska <[email protected]> | |
| Copyright: Licensed under the GPL-3 (http://www.gnu.org/licenses/gpl-3.0.html) | |
| This script requires python-epub-library: http://code.google.com/p/python-epub-builder/ | |
| """ |
koolhead17
/ distributedMinioDocker
Last active
December 19, 2016 06:24
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Step to run Distributed Minio using Docker. | |
| Install Docker Toolbox for Windows (http://www.docker.com/products/docker-toolbox) | |
| After Docker Toolbox installation, open Docker Quickstart terminal and run these commads. | |
| ``` | |
| docker pull minio/minio | |
| curl https://raw.githubusercontent.com/minio/minio/master/docs/docker/docker-compose.yml > docker-compose.yml | |
| docker-compose.exe up | |
| ``` |
koolhead17
/ paperclip.rb
Created
October 28, 2016 15:18
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| config.paperclip_defaults = { | |
| storage: :s3, | |
| s3_protocol: ':https', | |
| s3_permissions: 'public', | |
| s3_region: 'us-east-1', #ENV.fetch("AWS_REGION"), | |
| s3_credentials: { | |
| bucket: 'kline', #ENV.fetch("AWS_BUCKET"), | |
| access_key_id: 'Q3AM3UQ867SPQQA43P2F',#ENV.fetch("AWS_ACCESS_KEY_ID"), | |
| secret_access_key: 'zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG', #ENV.fetch("AWS_SECRET_ACCESS_KEY"), | |
| }, |
koolhead17
/ boto.py
Created
October 27, 2016 19:33
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import boto3 | |
| from botocore.client import Config | |
| import requests | |
| # Get the service client. | |
| s3 = boto3.resource('s3', | |
| endpoint_url='http://localhost:9000', | |
| aws_access_key_id='BKIKJAA5BMMU2RHO6IBB ', | |
| aws_secret_access_key='V8f1CwQqAcwo80UEIJEjc5gVQUSSx5ohQ9GSrr12', |
koolhead17
/ alluxio-site.properties
Created
October 17, 2016 12:26
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ cat conf/alluxio-site.properties | |
| alluxio.underfs.address=s3a://kline/data | |
| aws.accessKeyId=T1MK61R4CT2W5C370LIS | |
| aws.secretKey=WzEF4TPa0+61haiyDUQxuyE7epSknq/mdWEfiqmK | |
| alluxio.underfs.s3.endpoint=127.0.0.1:9000 | |
| alluxio.underfs.s3a.secure.http.enabled=false | |
| alluxio.underfs.s3.disable.dns.buckets=false |
koolhead17
/ alluxio-1.3.0-logs-master.log
Created
October 17, 2016 12:22
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ cat /home/ubuntu/alluxio-1.3.0/logs/master.log | |
| 2016-10-17 17:45:20,798 ERROR logger.type (S3AUnderFileSystemFactory.java:create) - Failed to create S3AUnderFileSystem. | |
| com.amazonaws.services.s3.model.AmazonS3Exception: A header you provided implies functionality that is not implemented (Service: Amazon S3; Status Code: 501; Error Code: NotImplemented; Request ID: 3L137), S3 Extended Request ID: 3L137 | |
| at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1305) | |
| at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:852) | |
| at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:630) | |
| at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:405) | |
| at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:367) | |
| at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:318) |
koolhead17
/ nats.go
Last active
October 12, 2016 13:52
go program to listen to nats event notification.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package main | |
| // Import Go and NATS packages | |
| import ( | |
| "runtime" | |
| "log" | |
| "github.com/nats-io/nats" | |
| ) | |
| func main() { |
koolhead17
/ nats_log.sh
Last active
October 12, 2016 13:53
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ go run nats.go | |
| 2016/10/12 06:51:26 Connected | |
| 2016/10/12 06:51:26 Subscribing to subject 'bucketevents' | |
| 2016/10/12 06:51:33 Received message '{"EventType":"s3:ObjectCreated:Put","Key":"images/myphoto.jpg","Records":[{"eventVersion":"2.0","eventSource":"aws:s3","awsRegion":"us-east-1","eventTime":"2016-10-12T13:51:33Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"principalId":"minio"},"requestParameters":{"sourceIPAddress":"[::1]:57106"},"responseElements":{},"s3":{"s3SchemaVersion":"1.0","configurationId":"Config","bucket":{"name":"images","ownerIdentity":{"principalId":"minio"},"arn":"arn:aws:s3:::images"},"object":{"key":"myphoto.jpg","size":56060,"eTag":"1d97bf45ecb37f7a7b699418070df08f","sequencer":"147CCD1AE054BFD0"}}}],"level":"info","msg":"","time":"2016-10-12T06:51:33-07:00"} | |