Activity Planning Worksheet: GROUP 5 – STATISTICAL POWER AND ERRORS

Summary:

This activity is designed to clarify the concept of statistical power and Type I/II errors. Students will conduct an in-class activity demonstrating how the difference between the hypothesized and actual values impacts power. They will then utilize a web-based applet to see how other factors (e.g., sample size, standard deviation, alpha, and whether the test is one or two tailed) also influence power.

1. Learning objectives

  • Define Power, Type I errors, and Type II errors and explain their meanings in context.
  • List the five factors that affect power. Explain how each affects power.

2. Context

  • Students should already have been exposed to sampling distributions, the binomial distribution, hypothesis testing with proportions, some exposure to Type I/II errors, and critical regions.

3. Mechanics

  • Students generate 30 random integers between 1 and 10 using either a 10-sided die or another random number generator (calculator or computer).
  • A result of 1 is considered a success
  • Each group tests their data against several different theoretical proportions (e.g., .05, .10, .15, .20, .50)
  • NOTE: Make sure you have some theoretical proportions near the actual value or there will not be any Type II errors.
  • Students write appropriate null and alternative hypotheses for each of the theoretical proportions

•H0: proportion = .05 H1 proportion ≠.05

•H0: proportion = .10 H1 proportion ≠.10

•H0: proportion = .15 H1 proportion ≠.15

•H0: proportion = .20 H1 proportion ≠.20

•H0: proportion = .50 H1 proportion ≠.50

  • Students record whether their data are significantly different from each of the theoretical proportions.
  • These hypothesis tests are based on varying differences between proportions, but leaving sample size and alpha constant.
  • Groups record their responses on the board and the proportion of significant results is calculated for each theoretical proportion
  • As the theoretical proportion gets further from the actual proportion (.10) the number of groups getting a significant result should go up.
  • This demonstrates how power increases as the difference theoretical proportions increase
  • This is where students can be told that power is the chance of finding a significant difference if a difference actually exists.
  • If any groups reject the null for a hypothesized proportion of .10, this can be used as an opportunity to discuss Type I errors
  • If any group fails to reject the null for the other hypothesized proportions, this can be used as an opportunity to discuss Type II errors.
  • APPLETS
  • Two applets can be used that allow the instructor/student to vary each of the factors that impact power
  • We recommend that the instructor demonstrate each of the following scenarios:
  • Increased distance between means or proportions (the in-class activity) increases power.
  • Increased standard deviation decreases power
  • Increased sample size increases power
  • Increased alpha increases power
  • One-tailed tests are more powerful than two-tailed tests

4. Variety

  • Students have to complete the steps involved in hypothesis tests with proportions
  • Students have to determine null and alternative hypotheses
  • Students have to use tables to find p values
  • Students have to do more abstract interpretation of the impact of various factors on power
  • Students get an example of the differences between “Fail to Reject H0” and “Accept H0”

5. Summary

  • Students have a handout where they answer the following questions.
  • The power of a test is ______as the sample size gets ______.
  • The power of a test is ______as the standard deviation gets ______.
  • The power of a test is ______as the alpha level gets ______.
  • The power of a ______-tailed test is larger than a ______-tailed test
  • The power of a test is ______as the difference between the means/proportions gets ______.

6. Follow-up

  • Power is mentioned in almost every lecture where we learn a new test.