igrigorik

http://www.snailinaturtleneck.com/blog/2010/05/05/with-a-name-like-mongo-it-has-to-be-good/

"The highlight, for me, was Eliot Horowitz’s talk on sharding. He set up a MongoDB cluster of 25
large EC2 instances and started hammering them... 8,000,000 operations per second... The whole
audience burst into applause."

8M op/s, across 25 machines
~320,000 op/s for single node
0.000003125 s/op or 3.125 µs/op (microseconds/op)

igrigorik

http://www.postrank.com/
http://apidocs.postrank.com/
http://apidocs.postrank.com/Feed-API
http://www.postrank.com/feed/f5e2f6c0fb48c7ce2f9efd6bf62870c6
http://apidocs.postrank.com/Feed-API
http://feeds.postrank.com/f5e2f6c0fb48c7ce2f9efd6bf62870c6?level=1&q=ipad&format=json
https://analytics.postrank.com/tour
http://briancray.com/2010/01/04/popular-posts-social-media-engagement-postrank-top-posts-api/
http://labs.postrank.com/
http://labs.postrank.com/realtime/map

igrigorik

(function () {
    var _UDS_CONST_LOCALE = 'en';
    var _UDS_CONST_SHORT_DATE_PATTERN = 'MDY';
    var _UDS_MSG_SEARCHER_IMAGE = ('Image');
    var _UDS_MSG_SEARCHER_WEB = ('Web');
    var _UDS_MSG_SEARCHER_BLOG = ('Blog');
    var _UDS_MSG_SEARCHER_VIDEO = ('Video');
    var _UDS_MSG_SEARCHER_LOCAL = ('Local');
    var _UDS_MSG_SEARCHCONTROL_SAVE = ('save');
    var _UDS_MSG_SEARCHCONTROL_KEEP = ('keep');

igrigorik

# http://github.com/eventmachine/eventmachine/blob/master/lib/em/iterator.rb
# pins the cpu if concurrency > number of urls

require 'em-synchrony'
require 'em-synchrony/iterator'
require 'em-synchrony/em-http'

urls = ['http://www.google.com/']

puts "Running fibered iterator"

igrigorik

# checkout: http://github.com/igrigorik/async-rails/

From bb2a78858cffa7c6937642986e9aca1a4f862c0d Mon Sep 17 00:00:00 2001
From: Ilya Grigorik <ilya@igvita.com>
Date: Thu, 10 Jun 2010 00:46:48 -0400
Subject: [PATCH] async rails3

---
 Gemfile                               |    6 ++++++
 app/controllers/widgets_controller.rb |    6 ++++++

igrigorik

# on remote server
jstatd -J-Djava.security.policy=tools.policy -p 1099

# on local machine, connection just hangs, no response...
# the port is open, so that's not it
jps -l -m -v remote_host:1099 

igrigorik

# minimal async sinatra app
# requires ruby 1.9.x

require 'sinatra'
require 'em-http'
require 'em-synchrony'
require 'em-synchrony/em-http'
require 'rack/fiber_pool'

use Rack::FiberPool

igrigorik

# MBP pro, production mode, thin server
# - each request spawns an internal http request back to thin
# - final destination does a 1s sleep inside of mysql
#
# concurrency       time
#   75              73.426
#   60              66.411
#   50              65.502
#   40              78.105
#   30              106.624

igrigorik

require 'em-http'
require 'addressable/uri'
require 'em-synchrony'
require 'em-synchrony/em-http'

class RobotCrawler
  def initialize(uri)
    @uri = Addressable::URI.parse(uri)
  end

igrigorik

require "date"
require "time"


# converts base10 integers into NewBase60
def num_to_sxg(num=nil)
  sxg        = ""
  vocabulary = "0123456789ABCDEFGHJKLMNPQRSTUVWXYZ_abcdefghijkmnopqrstuvwxyz"

  return 0 if num.nil? || num.zero?