(Recorded in 2013, but misplaced and not released until now.)

I work through an example of a one-sample t test on a mean, and (intentionally) make many false statements. Some of them might sound pretty reasonable. The lesson is: Get your statistics help from a reputable source!

Wrong statements, with explanations:

0:25: “Is the mean weight of cucumbers in my garden equal to 200 grams?” Not wrong, but it’s a bad example on a number of fronts:

-It’s not a question anybody would ask.

-The mean weight of cucumbers is in the garden is not going to equal exactly 200 grams, and we know that going in.

-A t test might provide evidence on whether the true mean weight differs from 200 grams, but it’s not going to tell us that the true mean equals 200 grams.

-If I really wanted to know the true mean weight (of the cucumbers that currently exist in my garden), I could probably pick them all and find out the true value. Or perhaps measure them on the vine with minimal error.

0:55: “H_0: X bar = 200 grams.” Hypotheses never involve statistics or the value of statistics.

1:25: “H_a” X bar greater than 200 grams.” The choice of alternative must never be based on the current sample’s data. It’s cheating to pick the alternative based on the observed data. And again, hypotheses never involve statistics or the values of statistics.

1:40: “…we just pick alpha, our significance level, to be 0.05.” While this is fairly common practice, it usually doesn’t make any sense and definitely doesn’t make any sense here. Here, no decision needs to be made, and we would simply assess the strength of the evidence against H_0 using the p-value. (Some will disagree with me on this front.)

2:00: “I could have picked any cucumbers in my garden, so every cucumber had the same chance of being picked.” While it is true that I could have picked any set of 4 cucumbers in the garden, that doesn’t imply they all had the same chance of being selected.

2:04: “Yes, this was a simple random sample.” Even though there was likely some randomness involved, that doesn’t make it a simple random sample. A SRS has a very specific meaning, one that was almost surely not the case here.

2:23: “Knowing what the first cucumber weighed tells me nothing about what the second cucumber weighs. They were weighed independently, and the observations are therefore independent.” This is just silly.

2:37: “This is not an important question for a t test.” One of the assumptions of the one-sample t test is normality, and that assumption is very important for small sample sizes.

2:51: “The t statistic has a t distribution, and we won’t need to concern ourselves with normality.” The t statistic has a t distribution only if H_0 is true and the assumptions (including normality) are true.

3:00: “The t statistic is X bar - mu” The hypothesized mean (mu_0) is subtracted in the numerator of the test statistic, not the true mean mu (we don’t know mu, and if we did know it we wouldn’t be carrying out the test!).

3:08: “Mu is the true mean weight of cucumbers in my garden, and that’s simply given here in the null hypothesis.” While it is the case that mu represents the true mean weight of cucumbers in the garden, the value of 200 in the null hypothesis is the hypothesized mean (mu_0) and not the true mean mu.

3:20: “Minus the true value of mu”. Again, we subtract the hypothesized mean, not the true mean.

3:43: “For a t test, the degrees of freedom are always n-1.” For one-sample t tests on a single mean, like the one in this video, the degrees of freedom are n-1. But there are many different types of t tests, and the degrees of freedom vary between them.

4:25: “The p-value is the probability of getting a sample mean that is at least as large as the observed sample mean of 214 grams.” This is a common misconception. If H_0 is true, and the assumptions are true, then the p-value is the probability of getting a t statistic at least as large as the one observed (1.12). For a t test like this one, this is not the same as the probability of getting a sample mean that is at least as large as 214 (even under Ho).

4:33: “If I were to go out to my garden and get another 4 cucumbers, the probability of getting a mean weight of 214 grams or more would be 0.17.” This is possibly my favourite line in the video, as it sounds pretty good but is simply not true. We don’t know what this probability is, as it depends on unknown parameters (mu, sigma).

4:46: “So we have pretty strong evidence that the null hypothesis is true, and that the true mean weight of cucumbers in my garden is actually 200 grams.” We found no meaningful evidence against the null hypothesis, but that most definitely doesn’t imply that it’s true. Lack of evidence against H_0 does not imply strong evidence for it. In addition, we essentially know going in that the true mean weight of cucumbers in the garden is not going to be exactly 200 grams.