Describing the relationship between t, r, and F for paired samples t-tests and repeated measures ANOVA with two factor levels.

pairedSamplest-testEffectSize.rmd

# Paired Sample t-tests: Yes, Virginia there is an effect size for that!

There are a plethora of options, Cohen’s d, a tweaked version of Glass’ $\Delta$ (1976): $\Delta_{RM}$ (Gibbons et al., 1993), dIGPP (Becker, 1988), generalized $\eta^2$ (Bakeman, 2005; Olejnik & Algina, 2003), and r (which in this context is a square rooted alias for partial eta squared).  Of these I prefer generalized $\eta^2$ or r.  Why r I hear you say.  I’ve never seen r used before in this context.  

A paired samples t is just a one sample t-test of the differences.  The advantage of using r is that it is well known and amenable to meta-analysis.  The interpretation does take a couple steps to think through, but I thought I’d try to help point you in the right direction.  You’ll recall that a paired samples t-test reduces to a within samples ANOVA with 1 df in the numerator.  Just as r is the square root of the proportion explained, so too is r (in this context) the square root of the proportion of variance explained by estimating the difference score mean over the total variance in the null model (with no estimate).    As a manual calculation using the formula for r the approach is a little counter intuitive. One has to divide the difference scores by 2 these values will be matched with the number 1 (in another column).  Then divide the difference scores by 2 and multiply by -1, these values will be matched (in the other column) with the number 0.  If you then look at the raw correlation coefficient for the vector of 1s and 0s with the transformed difference scores, you’ll find it matches the expectations indicated above, that is, it is r.  I think most of you don’t have a background in R, but for fun I mocked up a demo anyway.  I’ve commented the code with # marks so you might be able to follow along.  The function c is R’s way of concatenating vectors and rep is R’’s way of repeating numbers.  rnorm is how you can draw random numbers from a normal distribution in R. 

## Defining my SS formula
```{r}
SS <- function(x) {sum(x-mean(x)^2)}
```

## Defining the t to r function
```{r}
t2r <- function (t, df) 
{
  return(unname(sqrt(t^2/(t^2 + df))))
}
```

## Setting up data 
Setting random number seed.

```{r}
set.seed(1400) 
```

Selecting 20 numbers randomly from a normal distribution to reflect subject effects.

```{r}
(subeffect <- rnorm(20))
```

Now taking those subject effects, and adding another 20 random numbers from the normal distribution to reflect error.

```{r}
(t1 <- subeffect+rnorm(20))
```

Now doing the same as above, but adding in a true difference in scores of 1.5.

```{r}
(t2 <- subeffect+rnorm(20)+.3)
```


## Relationship between the t-test and a correlation
Calculating the t-test.

```{r}
(t.res <- t.test(t2,t1,paired=TRUE))
```

Converting t to r.

```{r}
(t2r(t.res[["statistic"]],t.res[["parameter"]]))
```

Showing that this r is the same as the correlation between the split in half across sign difference scores with a dummy array of 1s and 0s.  First, I have to set up the matrix.

```{r}
(mat <- cbind(c((t2-t1)/2,-(t2-t1)/2),c(rep(1,20),rep(0,20))))
```

The I have to run the correlation.

```{r} 
cor(mat)
```

## Relationship between t-test and ANOVA

Loading the `ez` package and setting contrasts to make for easier ANOVA calculation.

```{r}
suppressMessages(library(ez))
options("contrasts"=c("contr.sum","contr.poly"))
```

Transforming the data into a suitable format for ezANOVA

```{r}
dat <- data.frame(dv=c(t1,t2),iv=factor(c(rep(1,20),rep(0,20))),subjid=factor(rep(1:20,2)))
```

Calculating the ANOVA.
```{r}
ezANOVA(dv=.(dv),within=.(iv),wid=.(subjid),return_aov=TRUE,dat=dat)
```

Notice how F = $t^2$.  This is a relationship that only exists when the numerator degrees of freedom for the F statistic are 1 (indicating an equivelancy with the t-test).

```{r}
as.numeric(t.res[["statistic"]])^2
```

## Model Testing Approach
While we are on the subject, let's explore the relationship between t and F in a model testing framework.  First, we will calculate the variance under the null hypothesis.

```{r}
(null.model <- sum(((t2-t1)-0)^2))
```

Calculating the variance under the alternative hypothesis.

```{r}
(h1.model <- sum(((t2-t1)-mean(t2-t1))^2))
```

Calculating the proportion of variance explained as happens in the within subjects ANOVA.

```{r}
((null.model-h1.model)/null.model)
```

Squaring the effect size r to show how it relates to the proportion of variance explained

```{r}
(t2r(t.res[["statistic"]],t.res[["parameter"]])^2)
```

Using model testing approach
```{r} 
((null.model-h1.model)/((null.model-(null.model-h1.model))/19))
```

Notice how this is the same as $t^2$, which we already know is equal to F.

```{r}
as.numeric(t.res[["statistic"]])^2
```