A-B tests.txt

1. Sleep study
    a. Set up test population, as many people as possible
        i. Survey candidates to see whether they currently use sleeping medication, have insomnia, etc. to ensure the group is as homogenous as possible
    b. Randomly assign people to control and test groups
        i. Check to see that the random assignment didn’t accidently assign a larger proportion of a particular category to one sample group, e.g. more men than women in the test group
    c. Generate hypothesis: “The medication will result in more restful sleep, as indicated by REM time and minimal movement.”
    d. Outcome: REM-time sleep will increase and there is a decrease in restlessness
    e. Other measures: Check to see that a suitable cross-section of people is represented, i.e. gender, age range, etc., unless the medication is designed for a particular group of people
2. Gym uniforms
    a. Test population can be both current members and potential members
    b. Control group could be current members and test group could be potential members
    c. Hypothesis: “The new uniform will increase the number of new member applications.”
    d. Outcome: Membership application submission will increase during the period of uniform testing
    e. Other measures: Retention on existing members will not decrease; a wider demographic range of applicants may occur
3. Pet rental
    a. Population could be existing customers
    b. Customers would randomly be directed to current web site or testing web site; that choice would be retained in a cookie on the user’s computer to ensure they see the same site during the test period
    c. Hypothesis: “The revised site will result in higher numbers of pet rentals among existing customers.”
    d. Outcome: The new site will see an increase in the number of pet rentals during the test period
    e. Other measures: Use existing customer information to identify that the demographics of the control and test groups are equally represented; cookies may be removed by customers, resulting in skewed results (could be remedied by a cookie tracking database to see how many cookies were removed).
4. Email subject
    a. Population is dependent upon the definition of “people”. Could be existing contacts or could be new contacts.
    b. Control and test groups would be randomly selected from population
    c. Hypothesis: “Adding ‘Read Me’ will increase the number of people reading the email”
    d. Outcome: More emails will be read, as determined by the number of “read receipt” returns
    e. Other measures: How many emails were ignored/deleted/marked as spam (based on measuring capabilities), how quickly emails were read upon receiving them