miguno

# List running app instances that currently manage (parts of) state store "top-five-songs"
$ http://localhost:7070/kafka-music/instances/top-five-songs

[
	{
		"host": "localhost",
 "port": 7070,

miguno

# List all running instances of this application
$ http://localhost:7070/kafka-music/instances

[
	{
		"host": "localhost",
 "port": 7070,

miguno

# Runs HLL experiments, using randomly generated strings as input data.
#
# Requirements
# ============
#
#   # https://github.com/ascv/HyperLogLog
#   $ pip install HLL
#
#
# How to use

miguno

package com.myorg.ksql.udaf;

import io.confluent.ksql.function.udaf.UdafFactory;
import io.confluent.ksql.function.udf.UdfDescription;

@UdfDescription(name = "totalStringLength", author = "Confluent", version = "1.0")
public class TotalStringLength {

  @UdafFactory(description = "sums the length of strings")
  public static Udaf<String, Long> createSumLengthString() {

miguno

CREATE STREAM orders (orderid VARCHAR, itemid VARCHAR, address STRUCT<city VARCHAR, state VARCHAR, zicode BIGINT>)
  WITH (kafka_topic = 'orders_topic', value_format =  'JSON', key = 'orderId');

miguno

$ docker container run --rm \
    --net streams_streams-net \
    -v /docker/host/path/to/app.jar:/docker/container/path/to/app.jar \
    openjdk:8-jre java -jar /docker/container/path/to/app.jar

miguno

Notes

• Good: very quick snapshotting even for large heap spaces (e.g. works if your JVM process would otherwise timeout and die)
• Bad: such a memory snapshot contains significantly less useful information than YourKit's "enhanced" memory snapshots.

Usage

# 1. Find the relevant JVM pid on the target machine
$ ps ...

miguno

Live Coding a KSQL Application

Introduction

miguno

SELECT user_id, page, action FROM clickstream c
  LEFT JOIN users u ON c.user_id = u.user_id
  WHERE u.level = 'Platinum';

miguno

ksql> EXPLAIN ctas_ip_sum;

Type                 : QUERY
SQL                  : CREATE TABLE IP_SUM as SELECT ip, SUM(bytes)/1024 as kbytes FROM CLICKSTREAM WINDOW SESSION (300 SECONDS) GROUP BY ip;

Local runtime statistics
------------------------
messages-per-sec:     104.38   total-messages:       14238     last-message: 12/14/17 4:30:42 PM GMT
 failed-messages:          0      last-failed:         n/a
(Statistics of the local KSQL Server interaction with the Kafka topic IP_SUM)