Statistics 921:

Design and Analysis of Experiments and Observational Studies

Syllabus

Professor: Dylan Small

E-mail:

Office: 464 Huntsman Hall

Office Hours: Tuesday, 4:45-5:45; Wednesday, 9-10; Thursday, 4:45-5:45; by appointment.

Class Website: www-stat.wharton.upenn.edu/~dsmall/stat921-f07

Text: The Theory of the Design of Experiments, D.R. Cox and N. Reid (recommended). Additional readings will be handed out. A good reference for many of the topics in the course is P. Rosenbaum (2002), Observational Studies, 2nd Edition, New York: Springer-Verlag.

Course description: This course will cover statistical methods for the design and analysis of experiments and observational studies. The first half of the course will focus on the design and analysis of experiments and will cover The Theory of the Design of Experiments by Cox and Reid. The second half of the course will focus on observational studies and will cover readings from the statistical literature. Topics for the course will include the potential outcomes framework for causal inference; control of bias and haphazard variation in experiments; factorial experiments; matching, propensity score and regression methods for controlling confounding in observational studies; tests of hidden bias; sensitivity analysis; and instrumental variables.

Course Requirements: There will be regular homework assignments and a final project.

Computing software: We will make use of the freeware statistical computing software R. R can be downloaded from .

Course Prerequisites: Stat 541.

TentativeSchedule:

1. Week 1 (1 class)

Topics: The potential outcomes framework for causal inference in statistics.

Reading: Holland, P. Statistics and causal inference (with discussion). Journal of the American Statistical Association 1986; 81: 945-970.

2. Week 2

Topics: General concepts in designing experiments, the avoidance of bias in experiments and methods of inference for randomized experiments.

Reading: Cox, D.R. and Reid, N. The Theory of the Design of Experiments. Chapters 1-2.

Rosenbaum, P.R., Covariance adjustment in randomized experiments and observational studies. Statistical Science 1984; 17: 286-304. Sections 1-2.

Small, D.S., Ten Have, T.R. and Rosenbaum, P.R, Randomization inference in a group-randomized trial of treatments for depression: covariance adjustment, noncompliance and quantile effects. Journal of the American Statistical Association, in press. Sections 1, 2.1, 2.3, 3.1-3.3, 4 and 5.

3. Week 3

Topics: Control of haphazard variation in experiments through blocking (randomized block designs, balanced incomplete block designs, crossover designs).

Reading: Cox, D.R. and Reid, N. The Theory of the Design of Experiments. Chapters 3-4.

4. Week 4

Topics: Factorial designs. Fractional factorial designs.

Reading: Cox, D.R. and Reid, N. TheTheory of the Design of Experiments. Chapter 5.

5. Week 5

Topics: Confounding in designs, split plot designs, response surface methods.

Reading: Cox, D.R. and Reid, N. TheTheory of the Design of Experiments. Chapter 6.

6. Week 6

Topics: Optimal design, overflow on design of experiments.

Reading: Cox, D.R. and Reid, N. TheTheory of the Design of Experiments. Chapter 7.

7. Week 7 (1 class due to fall break)

Topics: Observational studies examples; the design of observational studies.

Reading: Rosenbaum, P.R. Observational study. In Encyclopedia of Statistics in Behavioral Sciences, eds. B.S. Everitt and D.S. Howell, 2005.

Rubin, D.B. The design versus the analysis of observational studies for causal effects: Parallels with the design of randomized trials. Statistics in Medicine 2007; 26: 20-36.

8. Week 8

Topics: Controlling for observed confounders in observational studies.

Readings:

(1) Rosenbaum, P.R. and Rubin, D.B. The central role of the propensity score in observational studies for causal effects. Biometrika 1983; 70: 41-55.

(2) Rosenbaum, P.R., Covariance adjustment in randomized experiments and observational studies. Statistical Science 1984; 17: 286-304. Sections 3-4, 7.

(3) Hansen, B.B. Full matching in an observational study of coaching for the SAT. Journal of the American Statistical Association 2004; 467: 609-618;

(4) Joffe, M.M., Ten Have, T.R., Feldman, H.I and Kimmel, S.E. Model selection, confounder control, and marginal structural models: review and new applications. The American Statistician 2004; 58: 272-279.

(5) Dehejia, R.H. and Wahba, S. Causal effects in nonexperimental studies: reevaluating the evaluation of training programs. Journal of the American Statistical Association 1994; 94: 1053-1062.

9. Week 9

Topics: Sensitivity analysis for unmeasured confounding; tests of ignorable treatment assignment; use of multiple control groups.

(1) Rosenbaum, P.R. Sensitivity analysis. In Encyclopedia of Statistics in Behavioral Sciences, eds. B.S. Everitt and D.S. Howell, 2005.

(2) Rosenbaum, P.R., Covariance adjustment in randomized experiments and observational studies. Statistical Science 1984; 17: 286-304. Section 5.

(3) Rosenbaum, P.R. From association to causation in observational studies: the role of tests of strongly ignorable treatment assignment. Journalof the American Statistical Association 1984; 79: 41-48.

(4) Rosenbaum, P.R. Dropping out of high school in the United States: an observational study. Journal of Educational Statistics 1986; 11: 207-224.

(5) Rosenbaum, P.R. The role of a second control group in an observational study. Statistical Science 1987; 292-316.

10. Week 10.

Topics: Adjusting for post-treatment variables; time-varying confounding.

Readings:

(1) Rosenbaum, P.R. The consequences of adjustment for a concomitant variable that has been affected by the treatment. Journal of the Royal Statistical Society, Series A 1984; 656-666.

(2) Hernán, M.A., Brumback, B.A. and Robins, J.M. Estimating the causal effect of zidovudine on CD4 count with a marginal structural model for repeated measures. Statistics in Medicine 2002; 21: 1689-1709.

11. Week 11.

Topics: Instrumental variables method for estimating causal effects.

Readings:

(1) Holland, P.W. (1988). Causal inference, path analysis and recursive structural equations models. Sociological Methodology, 18, 449-484.

(2) Angrist, J., Imbens, G. and Rubin, D. (1996). Identification of causal effects using instrumental variables (with discussion). Journal of the American Statistical Association 1996; 91: 444-472.

(3) Rosenbaum, P.R., Covariance adjustment in randomized experiments and observational studies. Statistical Science 1984; 17: 286-304. Sections 5.

(4) Small, D.S., Ten Have, T.R. and Rosenbaum, P.R, Randomization inference in a group-randomized trial of treatments for depression: covariance adjustment, noncompliance and quantile effects. Journal of the American Statistical Association, in press. Sections 2.2, 3.4.

(5) Small, D.S. and Rosenbaum, P.R., War and wages: the strength of instrumental variables and their sensitivity to unobserved biases. Journal of the American Statistical Association, in press. Sections 1-3.

12. Week 12

Topics: Design of observational studies.

(1) Rosenbaum, P.R., Choice as an alternative to control in observational studies (with discussion). Statistical Science 1999; 14: 259-304.

(2) Rosenbaum, P.R., Design sensitivity and observational studies. Biometrika 2004; 91: 153-164.

(3) Rosenbaum, P.R., Heterogeneity and causality: Unit heterogeneity and design sensitivity in observational studies. American Statistician 2005; 59: 147-152.

(4) Small, D.S. and Rosenbaum, P.R., War and wages: the strength of instrumental variables and their sensitivity to unobserved biases. Journal of the American Statistical Association, in press. Sections 4-6.

13. Week 13

Topics: Overflow; additional topics based on students’ interests.

14. Week 14

Topics: Students will make presentations on their final projects.