Jeff Haynie jhaynie

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups command.

List the topics to which the group is subscribed

kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.

Reset the consumer offset for a topic (preview)

How to Change Open Files Limit on OS X and macOS

This text is the section about OS X Yosemite (which also works for macOS Sierra) from https://docs.basho.com/riak/kv/2.1.4/using/performance/open-files-limit/#mac-os-x

The last time i visited this link it was dead (403), so I cloned it here from the latest snapshot in Archive.org's Wayback Machine https://web.archive.org/web/20170523131633/https://docs.basho.com/riak/kv/2.1.4/using/performance/open-files-limit/

Mac OS X

To check the current limits on your Mac OS X system, run:

From time to time, Musk will send out an e-mail to the entire company to enforce a new policy or let them know about something that's bothering him. One of the more famous e-mails arrived in May 2010 with the subject line: Acronyms Seriously Suck:

There is a creeping tendency to use made up acronyms at SpaceX. Excessive use of made up acronyms is a significant impediment to communication and keeping communication good as we grow is incredibly important. Individually, a few acronyms here and there may not seem so bad, but if a thousand people are making these up, over time the result will be a huge glossary that we have to issue to new employees. No one can actually remember all these acronyms and people don't want to seem dumb in a meeting, so they just sit there in ignorance. This is particularly tough on new employees.

That needs to stop immediately or I will take drastic action - I have given enough warning over the years. Unless an acronym is approved by me, it should not enter the SpaceX glossary.

Scaling your API with rate limiters

The following are examples of the four types rate limiters discussed in the accompanying blog post. In the examples below I've used pseudocode-like Ruby, so if you're unfamiliar with Ruby you should be able to easily translate this approach to other languages. Complete examples in Ruby are also provided later in this gist.

In most cases you'll want all these examples to be classes, but I've used simple functions here to keep the code samples brief.

Request rate limiter

This uses a basic token bucket algorithm and relies on the fact that Redis scripts execute atomically. No other operations can run between fetching the count and writing the new count.

	package main [33/492]

	import (
	"fmt"
	"strings"

	"golang.org/x/crypto/openpgp"
	)

	func main() {

	#!/bin/bash

	# Copyright © 2017 Google Inc.
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#
	# Unless required by applicable law or agreed to in writing, software

	version: 0.1

	# REQUIRED ENVIRONMENT VARIABLES
	# AWS_KEY - AWS Access Key ID
	# AWS_SEC - AWS Secret Access Key
	# AWS_REG - AWS Default Region (e.g. us-west-2)
	# AWS_OUT - AWS Output Format (e.g. json)
	# AWS_PROF - AWS Profile name (e.g. central-account)
	# IMAGE_REPO_NAME - Name of the image repo (e.g. my-app)
	# IMAGE_TAG - Tag for the image (e.g. latest)

	/*
	To the extent possible under law, Konstantin Olkhovskiy has waived
	all copyright and related or neighboring rights to this snippet.

	CC0 license: http://creativecommons.org/publicdomain/zero/1.0/
	*/
	package main

	import (
	"crypto/x509"

	FROM lambdalinux/baseimage-amzn:latest

	CMD ["/sbin/my_init"]

	RUN yum update -y && yum install -y yum-plugin-ovl && yum install -y \
	gcc \
	make \
	cmake \
	git \
	libcurl-devel \

	(function() {
	let canvas = document.querySelector('canvas');
	// Optional frames per second argument.
	let stream = canvas.captureStream(25);
	var options = {mimeType: 'video/webm; codecs=vp9'};
	let recorder = new MediaRecorder(stream, options);
	let blobs = [];

	function download(blob) {
	var url = window.URL.createObjectURL(blob);