Michael mikekaminsky

Testing for Analytics and Datascience

Introduction

Historically software development for research and for the sciences has not been well tested. This has been the case for a variety of reasons, including:

Scientists and researchers are not developers, and simply don't know about best practices
Non-deterministic functions are difficult to test
Set-returning functions (most of data munging) are very time-consuming to test because you must generate a lot of test data
It's really difficult to think through all of the possible edge cases like you might try in TDD

	SELECT
	col_a
	, col_b
	, col_c
	FROM table_a
	LEFT JOIN table_b
	ON table_a.id = table_b.a_id

	library(dplyr)
	library(progress)
	library(foreach)
	library(doParallel)
	library(tidyr)
	library(magrittr)

	##########################
	# Blog Post Example
	##########################

	CREATE OR REPLACE FUNCTION kaplan_meier(text)
	RETURNS TABLE (
	time_period int,
	pivot text,
	exposed int,
	events int,
	marginal_probability double precision,
	cumulative_probability double precision
	) AS
	$PROC$