Avinash avinash-mishra

How to create a MultiNode - MultiBroker Cluster for Kafka on AWS

PreRequisites :

Kafka Binary files : http://kafka.apache.org/downloads.html
Atleast 2 AWS machines : AWS EMR or EC2 will be preferable
A Kafka Manager Utility to watch up the Cluster : https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem

Cassandra 2.x Cluster on AWS EC2

Install java 8

# Remove java 7
sudo yum remove -y java

# Install basic packages
sudo yum install -y git

S3 Syncing

Export out all your S3 data
Source: http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html

# sync from bucket to local
aws s3 sync <bucket> <target_folder> <options>
aws s3 sync s3://mybucket . --acl public-read

aws s3 sync --region ap-northeast-1 s3://[移動元バケット名] s3://[移動先バケット名]

	#! /usr/bin/env python

	#
	# mysql2cassandra.py
	# Dump a MySQL result set to file and then import into a Cassandra column family
	#
	# Configuration
	# mysql_params [host, port, user, password, db] MySQL conenction parameters
	# mysql_columns [colname, colname2, ...] Columns for building MySQL query
	# The column that will hold values of the row key in the Cassandra column family must be first

Avinash avinash-mishra

How to create a MultiNode - MultiBroker Cluster for Kafka on AWS

PreRequisites :

Cassandra 2.x Cluster on AWS EC2

Install java 8

S3 Syncing

Machine Learning