Skip to content

Instantly share code, notes, and snippets.

@jacksmith15
Last active January 7, 2020 11:06
Show Gist options
  • Save jacksmith15/57a601daa8d9063522220ad6bc2ae680 to your computer and use it in GitHub Desktop.
Save jacksmith15/57a601daa8d9063522220ad6bc2ae680 to your computer and use it in GitHub Desktop.

Large Payloads testing work overview

There are 5 Pull Requests, which I recommend should be merged in the following order:

  • Genomic Record Referral Search bugfix: PR 813 This is a very small one (~21 lines), but has its own ticket. It blocks the test from passing.
  • Config Test data fix: PR 115
    Extremely tiny (~4 lines) - should only take a minute.
  • Genomic Record Bulk data generation command: PR 815 Contains a large amount of the work (~900 lines), specifically enables us to generate sufficient data and then access it in the integration tests.
  • Biobank Messaging DB tear down script generation command: PR 588 Medium sized (~100 lines) - but very self contained. It's a helper for maintenance rather than anything functional.
  • Integration Tests PR 314 Quite large (~400 lines). This adds the tests for bulk messages and also introduces functionality for clearing down the Biobank database at the start of each test run. Depends on all of the above, as well as teh feature ticket for Biobank.

Highlights

The broad strokes of this work are as follows:

  1. Genomic Record generates a bulk load of test data via a management command specifically with GEL1001 in mind. Pieces of the generated data which are relevant to generating GEL1001 CSVs are persisted in the Genomic Record database under a transient table in the test schema. This allows Integration tests to construct GEL1001 messages for the correct Referral/Patient/Sample triples, as other records may fail GEL1001 validation.
  2. Integration tests now clear the Biobank database at the start of each test run. This is to avoid conflicts in the various identifiers required between subsequent test runs. The script to do this is maintained in the Integration Tests repo. This script validates the migration number before running, and should give an informative error if it needs updating. A new management command is added to Biobank (PR 588) for updating this script.
  3. All of the above is to support the new test case in Integration tests which submits a 1000-row GEL1001 CSV, and ensures submits successfully and within 30s.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment