puma007

Kafka frequent commands

Assuming that the following environment variables are set:

KAFKA_HOME where Kafka is installed on local machine (e.g. /opt/kafka)
ZK_HOSTS identifies running zookeeper ensemble, e.g. ZK_HOSTS=192.168.0.99:2181
KAFKA_BROKERS identifies running Kafka brokers, e.g. KAFKA_BROKERS=192.168.0.99:9092

Start Zookepper and Kafka servers

国内从 Docker Hub 拉取镜像有时会遇到困难，此时可以配置镜像加速器。

Dockerized 实践 https://github.com/y0ngb1n/dockerized

	// Tensorflow Serving Go client for the inception model

	// go get github.com/golang/protobuf/ptypes/wrappers google.golang.org/grpc
	//
	// Compile the proto files:
	//
	// git clone https://github.com/tensorflow/serving.git
	// git clone https://github.com/tensorflow/tensorflow.git
	//
	// mkdir -p vendor

	'''
	spark/bin/spark-submit \
	--master local --driver-memory 4g \
	--num-executors 2 --executor-memory 4g \
	--packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.4.0 \
	sstreaming-spark-final.py
	'''
	from pyspark.sql import SparkSession
	from pyspark.sql.types import *
	from pyspark.sql.functions import expr