Avinash avinash-mishra

Based on http://datascale.io/cassandra-partitioning-and-clustering-keys-explained/

Primary Keys

A single column Primary Key is also called a Partition Key.

When Cassandra is deciding where in the cluster to store this particular piece of data, it will hash the partition key. The value of that hash dictates where the data will reside and which replicas will be responsible for it.

Kafka frequent commands

Assuming that the following environment variables are set:

KAFKA_HOME where Kafka is installed on local machine (e.g. /opt/kafka)
ZK_HOSTS identifies running zookeeper ensemble, e.g. ZK_HOSTS=192.168.0.99:2181
KAFKA_BROKERS identifies running Kafka brokers, e.g. KAFKA_BROKERS=192.168.0.99:9092

Server

Start Zookepper and Kafka servers

Install MySQL on macOS Sierra

This procedure explains how to install MySQL using Homebrew on macOS Sierra 10.12

Install Homebrew

Installing Homebrew is effortless, open Terminal and enter :
$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Note: Homebrew will download and install Command Line Tools for Xcode 8.0 as part of the installation process.

Install MySQL

At this time of writing, Homebrew has MySQL version 5.7.15 as default formulae in its main repository :

Block Elements ## Headers 見出し先頭に`#`をレベルの数だけ記述します。 ```

見出し1

見出し2

見出し3

見出し4

見出し5

見出し6

## 見出し2
### 見出し3
#### 見出し4
##### 見出し5
###### 見出し6 ## Block 段落 空白行を挟むことで段落となります。 ```
段落1
(空行)
段落2
``` 段落1 段落2 ## Br 改行 改行の前に半角スペース` `を2つ記述します。 ```
hoge
fuga(スペース2つ)
piyo
``` hoge
fuga piyo ## Blockquotes 引用 先頭に`&gt;`を記述します。ネストは`&gt;`を多重に記述します。 ```
&gt; 引用 &gt; 引用
&gt;&gt; 多重引用
``` &gt; 引用 &gt; 引用
&gt;&gt; 多重引用 ## Code コード `` `バッククオート` `` 3つ、あるいはダッシュ`~`３つで囲みます。 ```
print 'hoge'
``` ```
print 'hoge'
``` ### インラインコード `` `バッククオート` `` で単語を囲むとインラインコードになります。 ```
これは `インラインコード`です。
``` これは `インラインコード`です。 ## pre 整形済みテキスト 半角スペース4個もしくはタブで、コードブロックをpre表示できます ``` class Hoge def hoge print 'hoge' end end
``` class Hoge def hoge print 'hoge' end end ## Hr 水平線 アンダースコア`_` 、アスタリスク`*`、ハイフン`-`などを3つ以上連続して記述します。 ```
hoge
***
hoge
___
hoge
---
``` hoge
***
hoge
___
hoge
--- # Lists ## Ul 箇条書きリスト ハイフン`-`、プラス`+`、アスタリスク`*`のいずれかを先頭に記

A curated list of AWS resources to prepare for the AWS Certifications

A curated list of awesome AWS resources you need to prepare for the all 5 AWS Certifications. This gist will include: open source repos, blogs & blogposts, ebooks, PDF, whitepapers, video courses, free lecture, slides, sample test and many other resources.

Kafka Topics

List existing topics

bin/kafka-topics.sh --zookeeper localhost:2181 --list

Purge a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000

... wait a minute ...

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --delete-config retention.ms

	#! /usr/bin/env python

	#
	# mysql2cassandra.py
	# Dump a MySQL result set to file and then import into a Cassandra column family
	#
	# Configuration
	# mysql_params [host, port, user, password, db] MySQL conenction parameters
	# mysql_columns [colname, colname2, ...] Columns for building MySQL query
	# The column that will hold values of the row key in the Cassandra column family must be first

	import nltk
	import gensim


	sample="""Renewed fighting has broken out in South Sudan between forces loyal to the president and vice-president. A reporter in the capital, Juba, told the BBC gunfire and large explosions could be heard all over the city; he said heavy artillery was being used. More than 200 people are reported to have died in clashes since Friday. The latest violence came hours after the UN Security Council called on the warring factions to immediately stop the fighting. In a unanimous statement, the council condemned the violence "in the strongest terms" and expressed "particular shock and outrage" at attacks on UN sites. It also called for additional peacekeepers to be sent to South Sudan.
	Chinese media say two Chinese UN peacekeepers have now died in Juba. Several other peacekeepers have been injured, as well as a number of civilians who have been caught in crossfire. The latest round of violence erupted when troops loyal to President Salva Kiir and first Vice-President Riek Machar began sho

	from cassandra.cluster import Cluster
	from cassandra.auth import PlainTextAuthProvider
	import pandas as pd

	def pandas_factory(colnames, rows):
	return pd.DataFrame(rows, columns=colnames)

	cluster = Cluster(
	contact_points=['127.0.0.1'],
	auth_provider = PlainTextAuthProvider(username='cassandra', password='cassandra')

	from sklearn.datasets import fetch_20newsgroups
	from sklearn.feature_extraction.text import TfidfVectorizer
	from sklearn.linear_model import LogisticRegression
	from sklearn.pipeline import Pipeline
	import pandas as pd
	import numpy as np

	# Grab just two categories from the 20 newsgroups dataset
	categories=['sci.space', 'rec.autos']