Integration (acceptance) tests that use the mediawiki action api and rest apis:

Take a long time to run (1m30s for #mobile-content-service)
- Locally it destroys any TDD cycle or rapid feedback cycle
Are brittle to failures and other production conditions on such external services for continuous integration or local development.

There are known solutions for these problems, knowingly recording external http requests by running the tests and caching them, so that subsequent test runs use the cached results. See https://www.ctl.io/developers/blog/post/http-apis-test-code for informative post.

Requirements for such tool:

Should run in pure node (to avoid depending on multiple platforms for the project, like ruby)
Should allow recording a test session http interactions into a fixtures folder with minimal disruption in the code files
Should allow testing by playing recorded fixtures, or real calls, with minimal effort (like environment variables for example, REPLAY=cached npm test, CLI flags, or other simple methods)
If possible it should rely on well known http mocking libraries like nock

##Research and options

##DIY: utils/record.js

Using [[ https://www.ctl.io/developers/blog/post/http-apis-test-code | this post ]] as inspiration, create a local test/utils/record.js and use it on the tests we want to record.

Pros

Simple to set up, we maintain it
Less dependencies on external projects (possibly unmaintained) Cons
Duplication across other services repos (solvable by publishing as a package)
NIH, not participating in OSS community API

var record = require('../../utils/record');

describe('summary service', function () {
  var recorder = record('summary service');
  before(recorder.before);

  // ... all your tests

  after(recorder.after);
});

Re-record fixtures by removing fixtures folders or using environment variable (something like NOCK_RECORD=1 npm test)

##LinkedIn's [[ https://github.com/linkedin/sepia | sepia ]]

LinkedIn has a library that does HTTP recording/playback, seems pretty well thought out and mature, and they claim to use it.

Pros

Backed by a big company
Mature library
Minimal API surface via ENV variables
Very configurable
Version > 2 Cons
Not recently released (a year ago)
- Not necessarily a problem if works fine and it is just stable API

require('sepia'); // Somewhere in the tests

Or mocha -r sepia

Then use env variables to run the tests in different modes:

VCR_MODE=record npm test   # record, use network
VCR_MODE=playback npm test # playback using fixtures
VCR_MODE=cache npm test    # cache-first mode

##[[ https://github.com/thiagofelix/mocha-vcr | mocha-vcr ]]

This is a extension to Mocha BDD DSL intended to enable recording your test suite's HTTP interactions and replay them during future test runs for fast, deterministic, accurate tests.

Pros

Recently released, should work fine with recent versions
[[ https://github.com/thiagofelix/mocha-vcr/tree/master/src | Really small codebase ]]
- Can be picked up, fixed, republished easily
Not configurable Cons
Really new
Not configurable API

describe('Proxy Client', function () {
  // Specify cassette name 'google.com'
  vcr('google.com', it('forward requests to google.com', function () {
    let site = proxy.request('google.com')
    expect(site.title).to.equal('google.com')
  }))

  // Auto generate cassete name from test title
  vcr(it('forward requests to github.com', function () {
    let site = proxy.request('github.com')
    expect(site.title).to.equal('github.com')
  })
})

And run mocha via mocha --require mocha-vcr --ui vcr

##[[ https://github.com/assaf/node-replay | replay ]]

Pros

Seems a mature library
Minimal API, not intrusive like sepia
Configurable if needed Cons
Last updated 10 months ago API

require('replay'); // Somewhere on the test

Or mocha -r replay

REPLAY=record node test.js # Use env variables for replaying mode

bloody : All requests go out, none get replayed
cheat : Replays recorded responses, and allow HTTP outbound requests. For writing new tests or changing code to make new, un-recorded HTTP requests you haven't quite settled on which requests to make
record : Replays recorded responses, or captures responses for future replay. Use this whenever you're writing new tests or code that makes new HTTP requests
replay : Replays recorded responses, does not allow outbound requests. This is the default mode

Other discarded options for various reasons are:

Next steps

Try the promising options on the repo
- Update task with findings
Engage in conversations about it
Decide on option and submit patch
Profit

joakin/record.md

Next steps