PY600K – Measurement 1
Psychometrics
(PSYC6207; OS8207)
Fall 2007Wednesdays 9:30AM to 12:15PMColvard 4039
INSTRUCTOR INFORMATION
Instructor: Eric D. Heggestad
Office: Colvard 4045Phone: 687-6061
Office Hours: By appointmentE-mail:
COURSE OBJECTIVES
The objective of this course is to familiarize you with basic concepts in psychometric theory. A strong emphasis will be placed on the fundamentals of reliability and validity. Within the context of our study of validity, we will give particular emphasis on the statistical tool of factor analysis. We will also address issues associated with scaling techniques, such as Guttman scaling, unfolding, and item response theory. Finally, we will discuss item analyses and scale construction.
READINGS
There are two sources of readings for this class – a textbook and a collection of supplemental readings by various authors from a variety of sources. The required textbook is:
Crocker, L. & Algina, J. (1986). Introduction to classical and modern test theory. Blemont, CA: Wadsworth.
The readings for this course are not easy… you are going to have to think about what you are reading. You’re not going to believe this, but I am going to recommend that you do the assigned readings after our meeting. Don’t put this off, you really will get the most out of the readings if you do them shortly after our class. Seriously, don’t put this off!
HOMEWORK
You will complete a number of homework assignments over the course of the semester. Although I haven’t created the assignments yet, I can tell you that they will be designed to get you some “hands on” experience with psychometric evaluation. I will be giving you data sets to do the analysis/evaluations on. I am assuming that you are all moderately proficient with SPSS. More on the assignments as the semester progresses.
EXAMS
There will be three exams in this class; they will cover all of the assigned reading and the information presented in class between the exams.I haven’t yet decided exactly what the exams will look like. You can expect that they will consist of some combination of short answer and long answer responses. I promise to give you information on the precise format of the exams sometime before the first exam.
Don’t miss the exams!! If, for some reason, you miss the exam, then you will need to take a make-up exam. The make-up exam is always harder than the original exam (it generally consists of the questions I thought were too hard for the original exam).
EVALUATION
Your grade will be based on your performance on each of the three exams.
We will go with a traditional scoring system. As such…
A = 90% and above
B = 80-89%
C = below 80%
WEEK 1 INTRODUCTIONAUGUST 22Crocker & Algina – Chapters 1, 2 & pp. 45-49
WEEK 2 RELIABILITY IAUGUST29Crocker & Algina – Chapters 6 & 7
Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98-104.
WEEK 3 RELIABILITY IISEPTEMBER5Schmidt, F. L., Le, H., & Ilies, R. (2003). Beyond alpha: An empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual differences constructs. Psychological Methods, 8, 206-224.
Dudek, F.J. (1979). The continuing misinterpretation of the standard error of measurement. Psychological Bulletin, 86, 335-337.
Baugh, F. (2002). Correcting effect sizes for score reliability: A reminder that measurement and substantive issues are linked inextricably. Educational and Psychological Measurement, 62, 254-263.
WEEK 4 REALIABILITY OF RATINGSSEPTEMBER 12Tinsley, H.E.A., & Weiss, D.J. (1975). Interrater reliability and agreement of subjective judgments. Journal of Counseling Psychology, 22, 358-376.
Shrout, P.E., & Fleiss, J.L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428.
James, L.R., Demaree, R.G., & Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69, 85-98.
Kozlowski, S.W.J., & Hattrup, K. (1992). A disagreement about within-group agreement: Disentangling issues of consistency versus consensus. Journal of Applied Psychology, 77, 161-167.
WEEK 5 GENERALIZABILITY THEORYSEPTEMBER19Homework #1 due
Crocker & Algina – Chapter 8
Strube, M. J. (2000). Reliability and generalizability theory. In L. G. Grimm and P. R. Yarnold (Eds.), Reading and understanding multivariate statistics (pp. 23-66). Washington, DC: American Psychological Association.
Demorest, M. E. & Bernstein, L. E. (1992). Sources of variability in speechreading sentences: A generalizability analysis. Journal of Speech & Hearing Research, 35, 876-891.
WEEK 6 EXAM ISEPTEMBER 26WEEK 7 VALIDITY IOCTOBER3
Homework #2 due
Crocker & Algina – Chapters 10 & 11
Bushman, B. J., & Wells, G. L. (1998). Trait aggressiveness and hockey penalties: Predicting hot tempers on the ice. Journal of Applied Psychology, 83, 969-974.
Cronbach, L.J., & Meehl, P.E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.
WEEK 8 VALIDITY IIOCTOBER 10Campbell. D.T., & Fiske, D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.
WEEK 9 FACTOR ANALYSIS IOCTOBER 17Crocker & Algina – Chapter 13
Bryant, F. B., & Yarnold, P. R. (2000). Principal-components analysis and exploratory and confirmatory factor analysis. In L. G. Grimm and P. R. Yarnold (Eds.), Reading and understanding multivariate statistics (pp. 99-136). Washington, DC: American Psychological Association.
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10, 1-9.
Fabrigar, L. R., Wegener, D.T., MacCallum, R.C., Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272-299.
WEEK 10 FACTOR ANALYSIS IIOCTOBER24Lee, J., Wong, C. T., Day, J. D., Maxwell, S. E., & Thorpe, P. (2000). Social and academic intelligences: a multi-trait multi-method study of their crystallized and fluid characteristics. Personality and Individual Differences, 29, 539-553.
WEEK 11 EXAM IIOCTOBER 31WEEK 12 ITEM ANALYSESNOVEMBER7
Homework #3 due
Crocker & Algina – Chapters 4, 5 & 14
WEEK 13SCALINGNOVEMBER 14Crocker & Algina – Chapter 3
WEEK 14THANKSGIVING BREAKNOVEMBER 21WEEK 15ITEM RESPONSE THEORYNOVEMBER28
Homework #4 due
Crocker & Algina – Chapter 15
Henard, D. H. (2000). Item response theory. In L. G. Grimm and P. R. Yarnold (Eds.), Readingand understanding multivariate statistics (pp. 23-66). Washington, DC: American Psychological Association.
WEEK 16BIAS IN TESTINGDECEMBER5Crocker Algina – Chapters 12 16.
Kaplan, R. M. & Saccuzzo, D. P. (2001). Psychological testing: Principles, Applications, and Issues. Belmont, CA: Wadsworth. Chapter 19 – Test Bias
Steele, C. M. & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology, 69, 797-811.
Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American Psychologist, 52, 613-629.
FINALWhenever we decide that we want it