erikdw

Tuning Storm+Trident

Tuning a dataflow system is easy:

The First Rule of Dataflow Tuning:
* Ensure each stage is always ready to accept records, and
* Deliver each processed record promptly to its destination

erikdw

Adding 2 nodes to an existing 3-node ZooKeeper ensemble without losing the Quorum

Since many deployments may start out with 3 nodes and so little is known about how to grow a cluster from 3 memebrs to 5 members without losing the existing Quorum, here is an example of how this might be achieved.

In this example, all 5 nodes will be running on the same Vagrant host for the purpose of illustration, running on distinct configurations (ports and data directories) without the actual load of clients.

YMMV. Caveat usufructuarius.

Step 1: Have a healthy 3-node ensemble

erikdw

import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;

public class HelloCovariance {
  public static void main(String[] args) {
    ConcurrentHashMap<String, String> properties = new ConcurrentHashMap<>();
    Set<String> keySet = properties.keySet();
  }
}

erikdw

#!/bin/bash
IMAGE="mjanser/capifony"
CURRDIR=$(pwd)

if [ "$(uname)" == "Darwin" ]; then

    # under Mac OS X platform
    command -v jq >/dev/null 2>&1 || { echo >&2 "please install jq with \"brew install jq\" or \"port install jq\""; exit 1; }
    docker pull $IMAGE
    ENTRYPOINT=$(docker inspect $IMAGE | jq -r '.[0].ContainerConfig.Entrypoint[0]')

erikdw

Metaspace Research

What is PermGen?

• The Permanent Generation memory pool contains permanent class metadata and descriptors information when classes are loaded
  • PermGen space is always reserved for classes and items that are attached to them (i.e., static members)
• PermGem space is contiguous in memory to the Java heap and have the same rounds of Garbage Collection but it is not part of the Java heap

What is Metaspace?

• In JDK8, the PermGen space has been entirely replaced by Metaspace which is no longer contiguous to the Java heap and now exists in native memory

erikdw

javascript:
document.querySelectorAll('.load-diff-button').forEach(node => node.click());
document.querySelectorAll('.js-details-target').forEach(node => node.click());

// Both of those buttons should have the '.btn-link' class too, but I wasn't able to quickly figure out how to
// add an AND condition to that.  Tried the below, didn't work:
//document.querySelectorAll('[.btn-link][.load-diff-button]').forEach(node => node.click());

erikdw

# input vars
variable "environment" {
  default = "Dev"
}

# local vars
locals {
  name     = "hello"
  env      = "${lower(var.environment)}"
  lb_name  = "${join("-", list("TF", var.environment, local.name))}"

erikdw

-- Usage: tshark -X lua_script:statsd_dissector.lua -r capture.pcap
-- Usage: tshark -X lua_script:statsd_dissector.lua -T fields -e statsd.metric_name -e statsd.value -e statsd.metric_type -e stats.metric_tags -r capture.pcap

local statsd = Proto("statsd","Statsd Protocol")

local pf_metric_name = ProtoField.new("Metric Name", "statsd.metric_name", ftypes.STRING)
local pf_value = ProtoField.new("Value", "statsd.value", ftypes.STRING)
local pf_metric_type = ProtoField.new("Metric Type", "statsd.metric_type", ftypes.STRING)
local pf_metric_tags = ProtoField.new("Metric Tags", "statsd.metric_tags", ftypes.STRING)