Benchmark to measure the performance improvement of lock-less UID assignment in OpenTSDB

uidbench.java

import org.slf4j.Logger;
import net.opentsdb.uid.UniqueId;

import org.hbase.async.HBaseClient;

final class uidbench {

  private static void log(final String msg) {
    System.out.println("[" + Thread.currentThread().getName() + "] " + msg);
  }

  public static void main(final String[] args) throws Exception {
    log("Starting.");
    final HBaseClient client = new HBaseClient("localhost");
    final int numthreads = Integer.parseInt(args[0]);
    final String basename = args[1];
    final int iterations = 10000;

    final class Test extends Thread {
      final int id;
      Test(final int id) {
        super("Test" + id);
        this.id = id;
      }
      public void run() {
        log("Test thread starting...");
        final UniqueId uid = new UniqueId(client, "tsdb-uid".getBytes(),
                                          "test", 3);
        final long start = System.nanoTime();
        for (int i = 0; i < iterations; i++) {
          uid.getOrCreateId(basename + i + "-" + id);
        }
        final long finish = System.nanoTime();
        final long duration = (finish - start) / 1000000;
        log("Created " + iterations + " UIDs in " + duration + "ms => "
            + (iterations * 1000 / duration) + " UIDs allocated per sec");
      }
    }

    try {
      final Test[] tests = new Test[numthreads];
      for (int i = 0; i < numthreads; i++) {
        tests[i] = new Test(i);
      }
      log("Kicking off test threads");
      for (int i = 0; i < numthreads; i++) {
        tests[i].start();
      }
      for (int i = 0; i < numthreads; i++) {
        tests[i].join();
      }
    } finally {
      log("Shutting down.");
      client.shutdown().join();
    }
    log("Success!");
  }
}

Results, on a MacBook Pro with 8 hardware threads, a recent build of the HBase 0.94 branch. The baseline is OpenTSDB v1.1.0, and the lock-less implementation is based on an approach I suggested on the mailing list. This table shows the sum of the number of UID allocated per second for all threads.

# Threads	Baseline	Lock-less
1	760	1170
2	890	2534
3	936	3402
4	944	3912
8	920	5552

This shows that the new lock-less code is almost linearly scalable. There is still obviously some level of contention within HBase, to perform atomic increments.