lwoodson

Dockerized Integration Testing of Catalog Processor

Catalog Processor is a platform consisting of 3 applications, their databases (mongodb, mysql) and service dependencies. For integration testing purposes, we are bringing these pieces up as docker containers within the Jenkins test job rather than hosting them traditionally in AWS. We thus are able to exercise our platform end-to-end in all of its distributed glory without having to provision, deploy, maintain or pay for these disparate pieces in AWS. This was accomplished using the existing Jenkins Docker slaves and required no new infrastructure. Discussion will include:

• A brief overview of our architecture
• The integration test suite
• The script to run the cluster of containers and test suite
• Jenkins job configuration

lwoodson

public class AttemptAssertion {
    public static void attempt(timeout, wait, Callable testBody) {
        int elapsed = 0;
        Exception exception = null;

        while (true) {
            Thread.sleep(wait * 1000);
            elapsed++;
            
            try {

lwoodson

mysql:
  image: mysql:5.6
  ports:
    - "3306:3306"
  environment:
    MYSQL_ALLOW_EMPTY_PASSWORD: true
    MYSQL_DATABASE: catalog_processor
mongo:
  image: mongo:3.0.5
  ports:

lwoodson

{
    "id": XXXXX,
    "client_name": "walgreens",
    "priority": "normal",
    "type": "count",
    "job_configs": [
      {
        "type": "chunked",
        "client_origin": "walgreens",
        "operations": [

lwoodson

import java.util.ArrayList;
import java.util.List;
import java.util.function.Consumer;

public class Test {
    private List<Consumer<Object>> callbacks;
    
    public Test() {
        callbacks = new ArrayList<Consumer<Object>>();
    }

lwoodson

# Proof that rsync does what it says it does regarding incremental updates

# terminal 1
mkdir source
cd source
for i in {1..1000}; do dd if=/dev/urandom of=source$i count=1024 bs=1024; done

# terminal 2 -- kill when above finishes
for i in {1..1000}; do rsync -rtvu source/ dest; sleep 1; done

lwoodson

ActiveRecord::Base.logger = Logger.new('/dev/null')

# function APP_TIME()
# The current time matching the timing of events in the database (UTC)
sql = <<-SQL
  CREATE OR REPLACE FUNCTION app_time()
  RETURNS timestamp as $$
    SELECT (localtimestamp AT TIME ZONE current_setting('TimeZone') AT TIME ZONE 'UTC')::timestamp
  $$ LANGUAGE SQL IMMUTABLE STRICT;
SQL

lwoodson

select 
  relname,
  greatest(last_vacuum, last_autovacuum) as timestamp,
  case
    when last_autovacuum is not null then 'autovacuum'
    else 'vacuum'
  end as type
from 
  pg_stat_user_tables 
where 

lwoodson

Upgrading from postgres 9.3 to 9.4

Outline

Subject to change after runs in staging

1. Install new postgres version in separate directory structure
2. Ensure pg_upgrade is installed for 9.3 and 9.4
3. Copy 9.3 conf files to 9.4 directories.
4. Dump database using pg_dump (just in case)
5. Shutdown application (app, worker, label, all). Ensure maintenance is up.

lwoodson

HTTP_LOG_PATH = File.expand_path("~/http.log")
FileUtils.rm_f(HTTP_LOG_PATH)

module NetInstrumentation
  def self.included(base)
    puts "Writing HTTP out to ~/http.log"
    base.send(:alias_method, :__original_request, :request)
    base.send(:alias_method, :request, :__logged_request)
  end