randomstatistic

for x in `seq 1 20`; do strace -f -tt -o /tmp/st dig
www.google.com > /dev/null && grep -P '(send|recv)msg\(20'
/tmp/st | grep -v EAGAIN | awk '/sendmsg/ {f= substr($2,7)} /recvmsg/
{e= substr($2,7)} END {print e-f}'; done | sort -n

randomstatistic

#!/bin/bash

dir=$1
if [ "$1" == "" ]; then
    echo "Must provide a directory as an argument"
    exit 1
fi

ftypes=$(find $1 -type f -size +10c | grep -E ".*\.[a-zA-Z0-9]*$" | sed -e 's/.*\(\.[a-zA-Z0-9]*\)$/\1/' | sort | uniq)

randomstatistic

import java.util.concurrent.{TimeUnit, LinkedBlockingQueue}

import scala.annotation.tailrec
import scala.collection.JavaConverters._
import scala.concurrent.Future

/**
 * Thread-safe lock on Future generation. The put() method accepts futures without blocking so long as there are less than
 * $size futures that have been added via put() that are still alive. If more than $size futures are still running,
 * calling put() *blocks* the calling thread until some of the current futures finish.

randomstatistic

This was against a pretty specific index, using a very specialized query corpus, with lots of caveats. Be careful about comparisions.
Index stats: 85M docs/shard, 3 shards, 1 node.
Query stats: 142k queries. 78k use a simple facet.query, 27k do geospatial radius, 115 use CollapseQParser

CollapsingQParserPlugin: 
  22% of garbage by size 
  These were huge, perhaps a half-dozen allocations
  Lines 510,512 CollapsingQParserPlugin
SolrIndexSearcher: (getDocListAndSetNC)
  56% of garbage by size

randomstatistic

The idea is to add calculated fields to the “fl” of your query that match the components of your query. 
Solr 4.0+ allows calculated functions in your field list using a <field name>:<function> syntax, and one of the available functions is “query”.

This seems like a vaguely unpleasant way to determine which clauses matched, but I could see it working. I think it would go something like:

q=colA:foo OR colB:bar&fl=*,score,matchedColA:query({!standard q=“colA:foo”},0),matchedColB:query({!standard q=“colB:bar”},0)

Presumably the field matchedColA would be non-zero if colA:foo matched on that document, and matchedColB would be non-zero 
if colB:bar matched.
(I’m actually not sure if “standard” works as the name of the default query parser, but whatever, the idea is that it needs to match the relevant bit of your query.)

randomstatistic

Java:
  http://java-performance.com/
  
Linux:
  perf (https://perf.wiki.kernel.org/index.php/Main_Page)
  
  perf can be a bit finicky to set up though, One of the most interesting payoffs for the effort is getting cool flame graphs of cpu usage, like this example:
http://www.brendangregg.com/FlameGraphs/cpu-mixedmode-flamegraph-java.svg (from the blog post http://techblog.netflix.com/2015/07/java-in-flames.html)
 
Install perf:

randomstatistic

import org.apache.zookeeper.server.{NIOServerCnxn, ZooKeeperServer}
import java.net.{ServerSocket, InetSocketAddress}
import kafka.server.{KafkaServer, KafkaConfig}
import kafka.producer.{ProducerConfig, Producer}
import java.util.Properties
import kafka.serializer.{DefaultEncoder, StringEncoder}
import java.io.File
import scala.util.Random
import kafka.admin.AdminUtils

randomstatistic

public class FixedBitSetPool {

  public static final int poolSize = 10;
  private static ArrayBlockingQueue<FixedBitSet> pool = new ArrayBlockingQueue<FixedBitSet>(poolSize);

  // Ask for a FBS
  public static FixedBitSet request(int size) {
    FixedBitSet next = pool.poll();
    if (next == null || next.length() < size) {
      // if the size doesn't match, throw it away and return a new one of the requested size

randomstatistic

import java.util.concurrent.locks.ReentrantLock

import scala.concurrent.{ExecutionContext, Future, Promise}
import scala.concurrent.duration._

class LeakyBucket(dripEvery: FiniteDuration, maxSize: Int) {
  require(maxSize > 0, "A bucket must have a size > 0")
  private val dripEveryNanos = dripEvery.toNanos

  private val lock = new ReentrantLock()

randomstatistic

import io.gatling.commons.stats.{KO, OK}
import io.gatling.commons.util.TimeHelper._
import io.gatling.core.Predef._
import io.gatling.core.action.{Action, ExitableAction}
import io.gatling.core.action.builder.ActionBuilder
import io.gatling.core.stats.StatsEngine
import io.gatling.core.stats.message.ResponseTimings
import io.gatling.core.structure.ScenarioContext
import io.gatling.core.util.NameGen