The setup process to get jsbin live (in a clone environment)

jsbin-live-clone.md

Based on installing Ubuntu 10.04 (lucid) and cloning the live environment (calling local environment dev.jsbin.com)

Add core packages:

sudo apt-get update
sudo apt-get install libpcre3-dev build-essential libssl-dev git-core varnish sendmail
sudo su -
sendmailconfig
cd /opt/
wget http://nginx.org/download/nginx-1.0.0.tar.gz
tar -zxvf nginx*
cd /opt/nginx*/
./configure --prefix=/opt/nginx --user=nginx --group=nginx
make
make install
adduser --system --no-create-home --disabled-login --disabled-password --group nginx
echo "jsbin" > /etc/hostname
hostname -F /etc/hostname
apt-get install mysql-server
logout # from root

Install Node 0.8.1:

mkdir ~/src
cd ~/src
wget http://nodejs.org/dist/v0.8.1/node-v0.8.1.tar.gz
tar -zxvf node-v0.8.1.tar.gz
cd node-v0.8.1
./configure
make
sudo make install
sudo npm install -g forever
sudo npm install -g grunt

Edit /opt/nginx/conf/nginx.conf to:

worker_processes  1;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    keepalive_timeout  65;

    gzip on;
    gzip_http_version 1.1;
    gzip_comp_level 2;
    gzip_types    text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;
    gzip_min_length 10;

    server {
        listen       81;
        server_name  localhost;

        location / {
            root   /WWW/jsbin/public;
        }

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

Restart nginx:

/etc/init.d/nginx restart

Edit /etc/varnish/default.vcl config:

# node is the defalut
backend default {
   .host = "127.0.0.1";
   .port = "8000";
   .connect_timeout = 1s;
   .first_byte_timeout = 2s;
   .between_bytes_timeout = 60s;
   .max_connections = 800;
}

# nginx is running on port 80
backend nginx {
   .host = "127.0.0.1";
   .port = "81";
   .connect_timeout = 5s;
   .first_byte_timeout = 30s;
   .between_bytes_timeout = 60s;
   .max_connections = 800;
}

sub vcl_recv {
    if (req.http.host ~ "^dev-static.jsbin.com") {
       set req.backend = nginx;
       return (lookup);
    }

    if (req.http.Accept) {
       if (req.http.Accept ~ "text/event-stream") {
         return (pipe);
       }
    }


    return (pass);
}

sub vcl_miss {
  if ( req.backend != nginx ) {
    set bereq.first_byte_timeout = 1m;
  }
}

Edit /etc/default/varnish to:

# Configuration file for varnish
#
# /etc/init.d/varnish expects the variables $DAEMON_OPTS, $NFILES and $MEMLOCK
# to be set from this shell script fragment.
#

# Maximum number of open files (for ulimit -n)
NFILES=131072

# Maximum locked memory size (for ulimit -l)
# Used for locking the shared memory log in memory.  If you increase log size,
# you need to increase this number as well
MEMLOCK=82000

# Default varnish instance name is the local nodename.  Can be overridden with
# the -n switch, to have more instances on a single server.
INSTANCE=$(uname -n)

# This file contains 4 alternatives, please use only one.

## Alternative 1, Minimal configuration, no VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# content server on localhost:8080.  Use a 1GB fixed-size cache file.
#
# DAEMON_OPTS="-a :6081 \
#              -T localhost:6082 \
#        -b localhost:8080 \
#        -u varnish -g varnish \
#            -S /etc/varnish/secret \
#        -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"


## Alternative 2, Configuration with VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request.  Use a 1GB
# fixed-size cache file.
#
DAEMON_OPTS="-a :80 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,200M"


## Alternative 3, Advanced configuration
#
# See varnishd(1) for more information.
#
# # Main configuration file. You probably want to change it :)
# VARNISH_VCL_CONF=/etc/varnish/default.vcl
#
# # Default address and port to bind to
# # Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# # a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
# VARNISH_LISTEN_ADDRESS=
# VARNISH_LISTEN_PORT=6081
#
# # Telnet admin interface listen address and port
# VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
# VARNISH_ADMIN_LISTEN_PORT=6082
#
# # The minimum number of worker threads to start
# VARNISH_MIN_THREADS=1
#
# # The Maximum number of worker threads to start
# VARNISH_MAX_THREADS=1000
#
# # Idle timeout for worker threads
# VARNISH_THREAD_TIMEOUT=120
#
# # Cache file location
# VARNISH_STORAGE_FILE=/var/lib/varnish/$INSTANCE/varnish_storage.bin
#
# # Cache file size: in bytes, optionally using k / M / G / T suffix,
# # or in percentage of available disk space using the % suffix.
# VARNISH_STORAGE_SIZE=1G
#
# # File containing administration secret
# VARNISH_SECRET_FILE=/etc/varnish/secret
# 
# # Backend storage specification
# VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"
#
# # Default TTL used when the backend does not specify one
# VARNISH_TTL=120
#
# # DAEMON_OPTS is used by the init script.  If you add or remove options, make
# # sure you update this section, too.
# DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
#              -f ${VARNISH_VCL_CONF} \
#              -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
#              -t ${VARNISH_TTL} \
#              -w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \
#          -S ${VARNISH_SECRET_FILE} \
#              -s ${VARNISH_STORAGE}"
#


## Alternative 4, Do It Yourself
#
# DAEMON_OPTS=""

Restart varnish using:

mkdir -p /var/lib/varnish/`hostname`/
/etc/init.d/varnish restart

Add JS Bin & JS Bin database

mkdir /WWW
cd /WWW
chmod 777 .
git clone git://github.com/remy/jsbin.git
git checkout development
npm install
mysql -uroot -e 'create database jsbin'
mysql -uroot jsbin < build/jsbin.sql
mysql -uroot jsbin < build/v3.sql

Add config.local.json and set to:

{
  "env": "production",
  "url": {
    "host": "dev.jsbin.com",
    "prefix": "/",
    "static": "dev-static.jsbin.com"
  },
  "store": {
    "adapter": "mysql",
    "mysql": {
      "host": "localhost",
      "user": "root",
      "password": "[CHANGE TO YOUR PASSWORD]",
      "database": "jsbin"
    }
  },
  "session": {
    "secret": "[CHANGE TO SOME SECRET]"
  }
}

Note that we're using jsbin.dev and static.jsbin.dev to replicate online, so you'll need to add these records to your local /etc/host file too.

Add run.js to /WWW/jsbin/

var jsbin = require('./lib/app');

jsbin.on('before', function () {
  jsbin.use(function (req, res, next) {
    var v3betaurl = jsbin.set('url prefix') + '/3/',
        host = req.header('host', '');

    if (req.url.indexOf(v3betaurl) === 0) {
      res.redirect(301, req.url.substring(v3betaurl.length - 1));
    } else if (host.indexOf('www') === 0 || host.indexOf('3.') === 0) {
      res.redirect(301, jsbin.set('url full') + req.url);
    } else {
      next();
    }
  });
});

jsbin.connect();

Build latest JS Bin JavaScript source:

cd /WWW/jsbin/public
grunt

Run JS Bin:

cd /WWW/jsbin/
PORT=8000 forever start --append -l forever.log run.js

To restart all services:

/etc/init.d/nginx start
/etc/init.d/varnish start
/etc/init.d/mysql start
cd /WWW/jsbin/
PORT=8000 node run.js

Varnish isn't part of the stack anymore.

@remy: I thought I'd ask before I fiddle with it trying to figure this out myself... Can you point me to the latest doc or process to take an Ubuntu local install of JSBin to an existing mySQL (also) local server. I already have a reverse proxy working and just need the essentials to go production on the DB side of things and anything else you would advise. Alternatively, I could consider using your public service, but my use case isn't mainstream, but ping me if you want to discuss. Thanks much!