Skip to content

Instantly share code, notes, and snippets.

@rpietro
Last active August 29, 2015 13:56
Show Gist options
  • Save rpietro/9144987 to your computer and use it in GitHub Desktop.
Save rpietro/9144987 to your computer and use it in GitHub Desktop.
N-of_1 Design in Education

N-of-1 Design in Education

Ricardo Pietrobon, Uwe Heiss, Mauro Maldonato, Joao Vissoci, Bruno Melo, Jacinto Franco, Seiji Isotani

N-of-1 trials are cross-over trials where the randomization is conducted at the subject rather than the group level. This modification has a number of important implications, among them:

  1. Data from individual subjects can be analyzed on their own
  2. When putting together data from multiple N-of-1 trials, the synthesis is made following meta-analysis principles
  3. Similar to cross-over trials, the design will have to carefully consider issues such as carry-over effect, period effects, intra-subject correlation
  4. Since N-of-1 trials are randomized controlled trials, most of the principles applicable to parallel trials also apply to N-of-1, and so the CONSORT statement has to be observed, obviously with modifications.

Design issues

Carryover effect

Carryover effect and counterbalancing - this document has an excellent coverage of the issue of counterbalancing in N-of-1 trials, or to "test different subjects under the different conditions in different orders." Counterbalancing is essential in education since most educational interventions will have a carryover effect. The reason why counterbalancing can help is explained in the article above:

Why does counterbalancing help? In many cases, the carryover effect in one direction will simply cancel out the carryover effect in the other direction. Imagine that there is a practice effect in our noise experiment, so that subjects tend to be better at the concentration task under the second condition. If we counterbalance, then for some subjects the practice effect will boost their performance a bit in the noisy condition, and for others it will boost their performance a bit in the quiet condition. As a result, these two effects will cancel each other out when we aggregate the data across all the subjects.

Run-in and washout

Run-in period is a period before the trial starts when subjects are kept without an intervention that could potentially interfere with the efficacy evaluation (@pablos1998run). A washout period is identical to the run-in, but applied in between interventions in an N-of-1 trial. In an educational context, this will usually mean that the educational intervention should be stopped between the intervention periods.

Period effects

When the rate of a certain outcome varies over time independently from the intervention. In an educational context, this would be applicable to extended trials involving children, where their cognitive ability will improve over time independently from the educational intervention. The same would be applicable to, for example, an instructional program on how to use a medical record where students are actually exposed to the medical record for their daily activities. The latter means that their ability to use the medical record would increase independently from the educational intervention. Sometimes that exposure can occur at different rates since different groups might make more or less use of certain medical record features, therefore adding to the complexity of period effects.

Data visualization and modeling

The main functions in crossdes are:

  • get.plan: Menu-driven selection of crossover designs
  • allcombs: Construct crossover design with all possible treatment orders
  • williams: Construct a Williams design
  • williams.BIB: Construct a crossover design based on a combination of balanced incomplete block
  • designs and Williams designs.
  • des.MOLS: Construct crossover design based on mutually orthogonal Latin Squares
  • balmin.RMD: Construct balanced minimal repeated measurements crossover design
  • isCbalanced: Check whether a crossover design is balanced for first order carryover effects
  • isGYD: Check whether a crossover design is balanced

Another interesting article is "Graphical Insight and Data Analysis for the 2,2,2 Crossover Design" (@pikounis2001graphical)

Reporting guidelines for where N-of-1 can be used

The primary reporting guidelines for N-of-1 is the CONSORT extension for n-of-1 (CENT). Since CENT has not yet been published, below is an incomplete list:

  1. Trial design
    • planned number and duration of each period
    • run-in and wash out with rationale
    • Series of N- of-1 trials
  2. Ethics
    • IRB approval
  3. Participants
    • Eligibility
    • Settings and locations
  4. Interventions
  5. Outcomes
    • Primary
    • Secondary
  6. Sample size and stopping rules
  7. Randomization
    • Sequence generation
    • Allocation concealment
    • Blinding
  8. Data analysis
    • Assumption checking: carry-over effect, period effects, intra-subject correlation
    • Efficacy evaluation
    • Synthesis methods if more than one N-of-1 is being used
  9. Reproducicle research
    • scripts on github
    • data on github and figshare
    • storage of software packages
  10. N-of-1 data compilation

References

Books

  1. @jones2003design -- see full text made available by the library of the University of Wisconsin
  2. @bland2000statistical
  3. @montgomery1997design
  4. @vonesh1996linear

Articles in other fields with implications for Education

  1. @kunert1998sensory - overview of N-of-1 sensory trials
  2. @gabler2011n is a systematic review of the literature on N-of-1 trials
  3. @lillie2011n is a good overview of N-of-1 trial design and the issues that should be considered when applying them
  4. @cook1995randomized is an overview of N-of-1 trials in biomedical research
  5. @mahon1996randomised demonstrates that N-of-1 trial had a positive result on participants when compared with regular clinical practice
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment