stympy

Terraforming API Gateway to SQS queue

Example of a bare-minimum terraform script to setup an API Gateway endpoint that takes records and puts them into an SQS queue.

SQS

Start by creating the SQS queue.

resource "aws_sqs_queue" "queue" {

stympy

# Place this in your .bashrc
# Covers several corner cases such as nested apostrophes, history extraction in screens/subshells, Slack being down, etc.
# Strings to replace with your own credentials:
## {your email address} (1 instance)
## {slack webhook url} (1 instance)
## {your computer name} (2 instances)

slack() {
    # Get slack message when a command exits
    # Example usage: python long_job.py; slack

stympy

# Ruby on Rails Development Environment
FROM ruby:2.5.0

# Set up Linux
RUN apt-get update
RUN apt-get install -y build-essential inotify-tools libpq-dev nodejs postgresql-client

WORKDIR /app
EXPOSE 3000

stympy

# A user session class, a simplified mix of the following code samples:
# * https://github.com/blog/1661-modeling-your-app-s-user-session
# * http://www.jonathanleighton.com/articles/2013/revocable-sessions-with-devise/
class Session < ActiveRecord::Base

  # Uncomment if you use Hobo Fields, else add these yourself
  # fields do
  #   session_id :string, :index => true, :unique => true
  #   accessed_at :datetime
  #   user_ip :string

stympy

A little background on my context:

We run multiple Solr JVMs per box, which live in directories like:

/mnt/solr_8983
/mnt/solr_8984
/mnt/solr_8985
...
/mnt/solr_${PORT}

stympy

CREATE KEYSPACE IF NOT EXISTS docstore WITH replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };

CREATE TABLE IF NOT EXISTS docstore.metadata (
    userid text,
    docid text,
    joinkey text,
    title text,
    body text,
    PRIMARY KEY(userid, docid));

stympy

WAL-E needs to be installed on all machines, masters and slaves.

How to install WAL-E

Only one machine, the master, writes WAL segments via continuous archiving. The configuration for the master postgresql.conf is:

archive_mode = on
archive_command = 'envdir /etc/wal-e.d/env wal-e wal-push %p'
archive_timeout = 60

stympy

.DS_Store
Gemfile.lock
*.pem
node.json
tmp/*
!tmp/.gitignore

stympy

require 'v8'
require 'open-uri'
require 'pp'
require 'ap'
require 'taka'
require 'ostruct'

#
# Monkey patch all elements to include a 'style' attribute
#

stympy

# encoding: utf-8
require 'base64'

#
# Utility class for rails applications: manually decode (session) cookies!
#
# Just drop this in you rails application's lib directory, run a rails console
# session, require the file, then create an instance in order to analyze your
# cookie header (either the "Set-Cookie" response or "Cookie" request header).
#