Last active
December 22, 2024 15:21
-
-
Save mottalrd/7ddfd45d14bc7433dec2 to your computer and use it in GitHub Desktop.
Evan Miller source code for sample size from my blog post http://www.alfredo.motta.name/ab-testing-from-scratch/
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
function num_subjects(alpha, power_level, p, delta) { | |
var t_alpha2 = ppnd(1.0-alpha/2); | |
var t_beta = ppnd(power_level); | |
var sd1 = Math.sqrt(2 * p * (1.0 - p)); | |
var sd2 = Math.sqrt(p * (1.0 - p) + (p + delta) * (1.0 - p - delta)); | |
return (t_alpha2 * sd1 + t_beta * sd2) * (t_alpha2 * sd1 + t_beta * sd2) / (delta * delta); | |
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@sanjeevpe, I'm late but hope it helps. Below I present a customed Python function for estimating A/B testing sample size based on Evan Miller's sample size calculator. It includes the alternative to decide whether to used a two-tailed or one-tailed test or even relative or absolute delta.
Required modules
from scipy.stats import norm
import numpy as np
Evan Miller Sample Size Calculator
def evan_miller_sample_size(bcr, mde, sig_level = .05, power = .8, absolute_variation = True, two_tailed = True) -> int:
'''
This function retrieves the sample size required using Evan Miller's
methodology.