The double hijack trick, serving 65k connections from a single ruby process

c10k.ru

# This server demo does a socket hijack in Rack and then saves the socket to a global variable
# to prevent it from being GCed when the Puma thread ends. It will then write "BEEP" to each 
# socket every ten seconds to prevent the connection timing out. During testing, it easily 
# handled up to 65523 connections, after which it ran into the `ulimit` for open file descriptors.
# The bit with the waiting area is there because a normal `Set` is not thread safe and it would
# drop socket due to race conditions. The `Queue` is thread safe and will make sure all sockets
# are preserved.

# run with `rackup -q -p 8000 -o 0.0.0.0 c10k.ru`
# testing: install `ab` and then run `ab -c 20000 -n 20000 <ip adress of server>:8000/
# Make sure you have puma and rack installed globally, as there is no gemfile for this demo.

require 'puma'
require 'rack'
require 'set'

conn_waiting_area = Queue.new
conn_storage = Set.new

# "Reactor"
Thread.new do
  loop do
    begin
      sleep 3
      until conn_waiting_area.empty?
        conn = conn_waiting_area.deq
        conn_storage << conn
      end
      start_time = Time.now
      conn_storage.each do |c| 
        begin
          c << "BEEP\n"
        rescue
          conn_storage.delete(c)
        end
      end
      end_time = Time.now
      puts "Number of clients connected: #{conn_storage.length}, time for entire write run: #{end_time - start_time}"
    rescue StandardError => e 
      p e
    end # begin
  end # loop
end # Thread new

app = lambda do |env|
  response_headers = {}
  response_headers['Transfer-Encoding'] = 'binary'
  response_headers["Content-Type"] = "text/plain"
  response_headers["rack.hijack"] = lambda do |io|
    conn_waiting_area << io
  end
  [200, response_headers, nil]
end

run app

Very nice explanation of Rack and how to hijack it!

😍