The Tortuga Society
TheFundamentalsofInferentialStatistics
WhyITeachThisOrder
People learning statistics waste a ton of time memorizing definitions and formulas for hypothesis testing because they do not understand one concept - sampling distributions.
If you understand sampling distributions, you understand confidence intervals, hypothesis testing, and p-values in a way that many data professionals do not. This eliminates the need for brute force memorization. Instead you can reason about your work from first principles, and the formulas begin to make intuitive sense.
The reason people misunderstand sampling distributions is not that they are complicated They generally misunderstand them because they are treated as a building block to hypothesis testing and never revisited. Even worse, they're o en just given formulas and never tested on their understanding of this foundational concept.
While sampling distributions are fundamental to hypothesis testing, I also find that it's hard to appreciate them without the context of understanding hypothesis testing This means that your learning path should have you revisit them.
sampling distributions --> hypothesis testing --> sampling distributions
If all of this is meaningless to you, you ’ re fine I’m going to get you squared away You just have to pay attention and not half-ass the material.
Howtounderstandsamplingdistributions
Sampling distributions aren't immediately intuitive. Most people need to watch multiple videos and see multiple explanations before the information finally clicks.
There is one video series I know of which does a good job of revisiting sampling distributions as you go, which helps cement the material. We will be using this playlist as our core scaffolding for this series. I will also provide supplementary videos to reinforce the concepts that we cover in case something is unclear.
SamplingDistributions
Import note: the timestamps are not necessarily where you will find the answer. They are the timestamp at which you should have enough information to provide the correct answer.
Video1:SamplingDistributions
What is a Sampling Distribution? | Puppet Master of Statistics
(1:18) What is a sampling distribution? What are a statistic and a parameter (look this up on your own if you don't already know the answer)?
(2:12) Why can't we take the true population mean? What do we do instead?
(3:38) What is a standard deviation (you'll need to look this up on your own to refresh it)?
(4:25) What does "this sample mean is just one of many that we could have gotten by chance" mean? Why is this true?
(4:50) Looking at the chalkboard on screen, what is a sampling distribution? Describe it in the context of the image on the whiteboard.
(5:55) What is sampling variability?
(6:25) What is the standard error? How is it calculated? What does it tell us?
(6:55) How does standard error relate to the sample mean and the population mean?
(7:45) What is typically the shape of the sampling distribution? How does the distribution of the underlying data impact the shape of the sampling distribution? How does sample size impact this?
OptionalSupplements
Sampling distribution basics
Sampling distributions more concisely
Con denceIntervals
Confidence intervals are where the rubber meets the road for sampling distributions. It's our first chance to make a statement about a population from our sample using statistics techniques.
If you ' re anything like me, confidence intervals will be the first clue that you didn't quite understand sampling distributions as well as you thought you did. If you find yourself in that situation, pause the video and go back to the previous section. This way, your new material will reinforce the old material. The alternative is you keep watching, despite not understanding, and piss away more of your day while confusing yourself. Your choice
Remember, our learning path is:
sampling distributions --> hypothesis testing --> sampling distributions
Prerequisites
Please review the previous section before giving this part a read. I'll be expecting that you know each of the following:
What a sampling distribution is
What a standard deviation is
What a standard error is
Why a sampling distribution would have a standard deviation
What a parameter and a statistic are
Why we can't just generalize a statistic on to a parameter
If you can comfortably answer those, you ' re ready for this part.
Con denceIntervals
Some of these questions will feel redundant. It's because they are. Answer them anyway, and try to not reuse an old answer. Reapproaching the ideas will help you understand them.
Important note: the timestamps are not necessarily where you will find the answer They are the timestamps at which you should have enough information to provide the correct answer.
Video1:Con denceIntervals
What is a Confidence Interval? | Puppet Master of Statistics
(0:42) Why would relying on the sample statistic as a proxy for the population parameter be problematic?
(1:44) What is sampling variability?
(1:55) What is a sampling distribution?
(2:23) What is a standard error? Conceptually, what differentiates a standard error from a standard deviation?
(2:35) Why is the sample mean usually less than two standard errors away from the true mean? This is key, so take a moment to reason through this one.
(2:49) Why will two standard errors sometimes not capture the true mean?
(3:47) Why is your best guess for the true mean length 41cm in this example?
(4:00) Why do we do "plus or minus" two standard errors from our sample mean to create our margin of error?
(4:15) Is there a difference between a confidence interval and a margin of error? If so, what is it? (See Here)
(4:35) Why does the stand error depend on the sample size? Revisit the formula to see how " n " impacts the calculation.
(4:45) Intuitively (not using the calculation) why would changing the number of observations impact our margin of error?
(END OF VIDEO) Using everything that you ' ve learned so far, how does the confidence interval relate to the sampling distribution? Dig deep into this answer. This should summarize what you ' ve learned from the last two videos.
Video 2 (Recommended): Confidence intervals with math
What are confidence intervals? | Zed Statistics
This video revisits all of the topics in the previous video but goes a bit more into the math. It also has fewer nightmare-inducing puppets.
It will probably answer a few of the questions that you have a er the previous video.
HypothesisTestingandP-Values
If you complete this section, as it's laid out, you will have a better understanding of p-values than many academics. This concept is unnecessarily misunderstood. You can be part of the solution. At the very least you’ll be able to impress in the interview. This lesson is very heavy on questions that require you to reason about concepts before you have complete information. Do not skip these just because they are hard. It's okay if your answer is imperfect or even wrong. There is a robust body of evidence that reasoning about questions and arriving at the wrong answer still improves retention of the correct answer.
Prerequisites
I'll be expecting that you know each of the following:
What a sampling distribution is
What a standard deviation is
What a standard error is
What a parameter and a statistic are
What a confidence interval is
UnderstandingP-valuesandHypothesisTesting
Important note: the timestamps are not necessarily where you will find the answer. They are the timestamps at which you should have enough information to provide a useful answer.
Video 1: P-values and Hypothesis Testing
What is a Hypothesis Test and a P-Value? | Puppet Master of Statistics
(2:30) Given what you ' ve seen so far, explain what a hypothesis test is and what it's used for.
(2:55) Define sampling distribution and confidence interval.
(3:00) Given your definitions of sampling distribution and confidence interval, how would we test for compatibility between the mandated minimum mean of 35cm and our sample mean of 41cm? Refer to your answers from the last lesson if necessary.
(3:05) Why do we use hypothesis tests?
(3:23) What is a null hypothesis? Given our understanding of sampling distributions, confidence intervals, and sample means, why might we use a null hypothesis?
(3:32) What is the standard error used for, and how is it calculated?
(4:24) Define P-value
(4:42) Define P-value in terms of sampling distributions
(4:42) Given your definitions of P-value, what are two possible interpretations of a low P-value?
(5:55) Why is p<0.05 the standard in research?
(5:56) Make a strong case for why p<0.10 should replace p<0.05 as the standard for research.
(5:57) Make a strong case for why P<0.01 should replace p<0.05 as the standard for research.
(6:25) Perform the math that the instructor performed and see if you arrive at the same answer.
(6:35) How does a confidence interval differ from simply taking the p-value?
(7:00) Define the null hypothesis in terms of p-values and confidence intervals
Video 2 (Optional): Null and Alternative Hypotheses
Null and Alternate Hypothesis - Statistical Hypothesis Testing - Statistics Course
If you ' re unclear about what the null hypothesis is, this is a great video
WrappingUp
If you ’ ve made it this far, and engaged with the material, I want to congratulate you. If you skimmed the videos, skipped hard questions, or moved onto subsequent sections while still having only a shaky grasp of the material, I’m here to tell you that you fucked up. Not in a cute “oh you ’ re so bad teehee” sort of way, but in a “ you ’ re not going to make it” sort of way.
This material can be complicated. It sometimes involves stepping back, stretching your legs, and revisiting it again when you ’ re a little fresher If you found the material easy, that’s great. If you found the material challenging and still managed to learn, that’s even better. The ability to insist that you will, in spite of the voice that says you will not, is what makes winners. It’s what you ’ re going to rely on when you have to cram learning a new skill for your interviews or your next stacked job.
If you engaged honestly with this material, go relax You finished this first pass You’ll need to revisit the questions a couple of times to help make the concepts stick, but you can do that in a few days, and then in a week or two.
If you didn’t give this material its due time and attention - if you skipped the challenging parts - now is a good time to figure out if you ’ re cut out for this. It’s okay to be confused and to struggle with the concepts. It’s not okay to quit. If you ’ re going to quit here, you ’ re going to quit every other time things get challenging. I’ve seen it plenty. Either go back to the top and start again, or save yourself the trouble and find another job.