Ben Holley benolee

Scaling your API with rate limiters

The following are examples of the four types rate limiters discussed in the accompanying blog post. In the examples below I've used pseudocode-like Ruby, so if you're unfamiliar with Ruby you should be able to easily translate this approach to other languages. Complete examples in Ruby are also provided later in this gist.

In most cases you'll want all these examples to be classes, but I've used simple functions here to keep the code samples brief.

Request rate limiter

This uses a basic token bucket algorithm and relies on the fact that Redis scripts execute atomically. No other operations can run between fetching the count and writing the new count.

This is Japanese-English translation of my diary articles

These articles demonstration how to find out memory issues and tuning.

How to estimate memory consumption
How to use allocation_tracer gem to know object creation details
How to optimize this typical issue in VM optimization

break /home/shim/Documents/foreman/dynflow/lib/dynflow/utils.rb:60

Why do we need indifferent_hash while saving?

 "/home/shim/Documents/foreman/dynflow/lib/dynflow/utils.rb:60:in `indifferent_hash'",
 "/home/shim/Documents/foreman/dynflow/lib/dynflow/persistence_adapters/sequel.rb:268:in `extract_metadata'",
 "/home/shim/Documents/foreman/dynflow/lib/dynflow/persistence_adapters/sequel.rb:226:in `prepare_record'",
 "/home/shim/Documents/foreman/dynflow/lib/dynflow/persistence_adapters/sequel.rb:236:in `save'",
 "/home/shim/Documents/foreman/dynflow/lib/dynflow/persistence_adapters/sequel.rb:125:in `save_action'",

From https://github.com/GhostGroup/infrastructure/pull/180

Header verification

Use socat to eavesdrop on the Nginx -> Unicorn socket-based comms.

ubuntu@ip-10-0-191-247:/etc/nginx/sites-enabled$ SOCK=/var/www/shared/.unicorn.sock [49/144]

Debugging ruby performance - Aman Gupta

C, Linux, networks, cpu, and memory usage

lsof -nPp <pid>
tcpdump
write the data to a file, load it up in wireshark
strace, trace systems calls and signals (jump into kernelspace)
SIGVTALRM, ruby 1.8 every 10msec to switch threads (green threads)
posix-spawn instead of fork-exec, check out posix-spawn gem
ltrace

– ltrace -c ruby foo.rb # summary mode

Introduction

An introduction to curl using GitHub's API

The Basics

Makes a basic GET request to the specifed URI

curl https://api.github.com/users/caspyin

	# frozen_string_literal: true

	class SafeHash < Hash
	def initialize(namespace)
	@namespace = namespace
	@lock = Concurrent::ReadWriteLock.new
	super
	merge!({ @namespace => Concurrent::Map.new })
	end

	class TBQueue
	def initialize size
	tvar = Concurrent::TVar

	@read = tvar.new []
	@write = tvar.new []
	@rsize = tvar.new 0
	@wsize = tvar.new size
	end

	require 'fiber'
	require 'concurrent' # gem install concurrent-ruby
	Thread.abort_on_exception = true

	class Async
	def self.perform(&block)
	instance = new
	instance.instance_eval(&block)
	until instance.dispatched.value.zero? do
	unless instance.yields.empty?

	#!/bin/bash
	source mysql_access
	TODAY=$(date +"%Y-%m-%d_%H%M%S")

	CHECK_RETAINER=$(
	mysql api_test \
	-u$MYSQL_USER \
	-p$MYSQL_PASS \
	-h$SOURCE_HOST \
	--socket=$MYSQL_SOCK \