MARSHALL SCHOOL OF BUSINESS

UNIVERSITY OF SOUTHERN CALIFORNIA

MOR-605: Research Methods in Organizational Behavior

Spring 2013, Monday02:00-04:50pm, HOH506

Instructor:Peter Carnevale

Office: 620 HOH, office hours by appointment

E-Mail:

This course provides doctoral students a foundation for conducting independent, scholarly, programmatic micro behavioral research (i.e., individuals or small groups as the primary unit of analysis). The course is designed around key steps in the research process, from the choice of a research question, formulating theory and hypotheses, design, measurement, data analysis and interpretation(with an emphasis on data graphs), data communication (writing as well as presenting), and so on; we’ll also deal with practical and important matters such as the IRB, and scientific integrity. A central focus of the course is the laboratory experiment, and this reflects the view that a valid statement about causality is a high achievement. Research is very much about choices and constraints that are driven by available resources (e.g., time, money, data collection sites), environmental pressures (e.g., discipline-wide or local norms about "appropriate" research topics, the "right" journals to publish in, or the "best" research strategies), a researcher's own distinct competencies (e.g., technical skills, social networks, access to samples). Students will make short presentations throughout the semester, and the last class will be devoted to student research presentations (conference-like, i.e., short, using Tuftean powerpoint). There will be an occasional “showcase” paper from a content domain to illustrate aspects of method. You will develop a “Researchers Checklist” and you are asked to make one.

Evaluation: Grades will derive from (1) a take-home examination due at the end of the semester (40%); (2) a research project/proposal that you write up and present (40%) (submit a < 250 word proposal in or before Week 8, and it requires my approval), and (3)written assignments due every week and participation in discussions (20%).

Required Readings to Purchase (Other readings available via Blackboard)

  • Gawande, A. (2010). The checklist manifesto – How to get things right. New York:

Metropolitan Books. ISBN: 0805091742

  • Abelson, R.P. (1995). Statistics as principled argument. Hillsdale, NJ: Erlbaum.
  • Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally.
  • Reis, H.T., & Judd, C.M. (Eds.) (2000). Handbook of research methods in social and personality psychology. Cambridge, UK: Cambridge University Press.
  • Tufte, E.R. (2001). The visual display of quantitative information. Graphics Press, 2001.
  • Publication Manual of the American Psychological Association, 6thEdition. Washington, DC:APA.

Background / Additional Reading

On writing:

  • Strunk, W. Jr., & White, E.B. (1979). The elements of style. New York: Macmillan

you can search on "possessive" etc.]

Philosophical aspects of science, and critique:

  • Hempel, C.G. (1966). Philosophy of natural science. Engelwood Cliffs, NJ: Prentice-Hall.
  • Kaplan, A. (1964). The conduct of inquiry: Methodology for behavioral science. Chandler Publishing Co.: San Francisco.
  • Kuhn, T.S. (1970). The structure of scientific revolutions. Chicago: Univ. of Chicago Press.
  • Nagel, E. (1961). The structure of science. New York: Harcourt, Brace, and World.
  • Sokal, A., & Bricmont, J.(1998).Fashionable nonsense.New York, NY: Picador USA

Other great books on method (there are many):

  • Aronson, E., Ellsworth, P.C., Carlsmith, J.M., & Gonzales, M.N. (1990). Methods of research in social psychology. 2nd Edition. Addison-Wesley.
  • Brinberg, D. & McGrath, J. E. (1985). Validity and the Research Process. Beverly Hills, CA: SAGE Publishing Co.
  • Locke, E.A. (Ed.) (1985), The generalizability of laboratory experiments: An inductive survey.Lexington, MA: Heath.
  • McGrath, J. E. Martin, J., & Kulka, R. A. (1982). Judgment Calls in Research. Beverly Hills, CA: SAGE Publications Inc.
  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston:Houghton Mifflin.
  • Rosenthal, R., & Rosnow, R. L. (Eds.). (1969). Artifact in behavioral research. New York: Academic Press.
  • Runkel, P. J. & McGrath, J. E. (1972). Research on Human Behavior: A Systematic Guide. New York: Holt, Rinehart & Winston.
  • Webb, E. J., Campbell, D. T., Schwartz, R. D., & Sechrest, L. (1999). Unobtrusive measures, Revised. Sage Publications, Inc.

Important Dates

Week 8:Submit a < 250 word abstract for final research project/proposal, via email.

April 29:Student research presentations

Week 10:Take-home final examination distributed.

April 29:Writeup of research project due (20p max, including everything, emailed)

May 6:Take-home final examination and “checklist” due, via email.

The Research Project (which forms the basis of the research presentation):

  • This is a write-up of a proposed program of research that incorporates at least 2 distinct methods (e.g., lab study, field study, case study, computer simulation).
  • By Week 8, submit a < 250 word proposal, via email, to me. Specify the question(s), why it is important, the design, measures, proposed analyses, and expected effect(s). I will provide feedback, and we should mutually agree on what you will do.
  • This should be written up using the Publication Manual of the American Psychological Association, 6th Edition as a guide. (Or it could involve a real study...let’s talk)
  • Note: Maximum 20 pages, including everything, title page, abstract, intro, method, results (expected), discussion, references, tables/figures, etc. Email to me on/before April 29.

Class Outline (highly susceptible to incessant modification)

Week 1, January 14:Introductions and overview

  • Some comments on method and this class
  • Presenting your work, and writing
  • Scientific integrity

National Research Council (2002). Integrity in Scientific Research: Creating an Environment That Promotes Responsible Conduct. Published by the Committee on Assessing Integrity in Research Environments, National Academies Press.

  • Assignments for next week:
  • Read the articles listed, lead discussion on one of them
  • Generate a research question, method, abstract, and a short presentation

Week 2, January 28: Asking the question, discovery, being interesting, having impact.

  1. Judge, T. A., Colbert, A., Cable, D. M., & Rynes, S. L. (2007). What causes a management article to be cited — article, author, or journal? Academy of Management Journal, 50, 491-506.
  2. Feynman, R.P. (1974). Cargo-Cult science: Some remarks on science, pseudoscience and learning how not to fool yourself. Engineering and Science, 1-13.
  3. Davis, M. S. (1971).That’s interesting!Toward a phenomenology of sociology and a sociology of phenomenology.Philosophy of the Social Sciences, 1, 309-344.
  4. Aronson, E. (1977). Research in social psychology as a leap of faith. Personality and Social Psychology Bulletin, 3, 190-195.
  5. Nisbett, R.E. (1990). The anti-creativity letters: Advice from a senior tempter to a junior tempter. American Psychologist, 45, 1078-1082.
  6. McGuire, W.J. (1997). Creative hypothesis generating in psychology. Annual Review of Psychology, 48, 1-30.
  7. McGrath, J. (2002). The joy of prose: Some comments about producing technical manuscripts in social science. [UIUC, unpublished manuscript]
  8. Rosenfield, J.R. (2003). Death by presentation, and 14 modes of resurrection (personal blog):
  9. Simon, H. A. (1992). What is an "explanation" of behavior?Psychological Science,3, 150-161.
  10. Jick, T. D. (1979). Mixing qualitative and quantitative methods: Triangulation in action Administrative Science Quarterly, 24, 602-611
  1. Showcase article & book:
  2. Triplett, N. (1897). The dynamogenic factors in pacemaking and competition. American Journal of Psychology, 9, 507-533.
  3. The Autobiography of Charles Darwin by Charles Darwin

Week 3, February 4

Types of experiments, validity, theory and design, the control group.

Constructing path diagrams of hypotheses and their operationalizations.

  1. Brewer, M. B.(2000).Research design and issues of validity (Chapter 1, pp. 3-16).In H. T. Reis & C. M. Judd(Eds.), Handbook of research methods in social and personality psychology.Cambridge, UK: Cambridge University Press.
  2. Smith, E. R.(2000).Research design (Chapter 2, pp. 17-39).In H. T. Reis & C. M. Judd(Eds.),Handbook of research methods in social and personality psychology.Cambridge, UK: Cambridge University Press.
  3. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally.
  4. Medin: A Dangerous Dichotomy: Basic and Applied Research (2012)
  1. Showcase article on the importance of the control group:
  2. Moskowitz, D.S. (2004).Does elevated power lead to approach and reduced power to inhibition?Comments on Keltner, Gruenfeld, and Anderson (2003).Psychological Review, 111(3), 808-811.
  1. Suggested, optional:
  • Cronbach, L. J. (1957). The two disciplines of scientific psychology. American Psychologist,12, 671-684.
  • Carnevale, P.J., & De Dreu, C.K.W.(2005). Laboratory experiments on negotiation and social conflict. In P.J. Carnevale & C.K.W. De Dreu (Eds.) (2006). Methods of negotiation research. Leiden, The Netherlands: Martinus Nijhoff Publishers.
  • Behling, O. (1978).Some problems in the philosophy of science of organizations.Academy of Management Review, 3, 193-201.
  • Sutton, R. I., & Staw, B. M. (1995). What theory is not. Administrative Science Quarterly, 40, 371-384.
  • Greenwald, A. G., Pratkanis, A. R., Leippe, M. R., & Baumgardner, M. H. (1986). Under what conditions does theory obstruct research progress? Psychological Review, 93, 216-229.
  • Bacharach, S. B. (1989).Organizational theories:Some criteria for evaluation.Academy of Management Review, 14, 496-515.
  • Meehl, P. (1978).Theory testing in psychology and physics:A methodological paradox. Philosophy of Science, 34, 103-115.
  • Folger, R., & Turillo, C. J. (1999).Theorizing as the thickness of thin abstraction.Academy of Management Review, 24, 742-758.
  • Solomon, R.L. (1949). An extension of control group design. Psychological Bulletin, 46, 137-150.
  • Proctor,R. W., & Capaldi,E. J.(2001). Empirical evaluationand justification of methodologies in psychological science. Psychological bulletin, 127, 759- 72.

Week 4, February 11

Formulating the question and the answer: Dilemmatics, Programmatics, the alternative explanation, and the mediating variable

  1. Platt, J.R. (1964). Strong inference. Science, 146, 347-353. [see also Hafner, E.M., & Presswood, S. (1965). Strong inference and weak interactions. Science, 149, 503-510.]
  2. Bem, D. J. (1987). Writing the empirical journal article. [in Z&D] There is an online-only article that is a revised and updated version of this, url listed here:
  3. Kerr, N. (1998). Harking: hypothesizing after the results are known. Personality and Social Psychology Review, 2, 196-217.
  4. McGrath, J. E. (1981). Dilemmatics: The study of research choices and dilemmas. American Behavioral Scientist, 25, 179-210.
  5. Bargh, J. A. & Chartrand, T. L.(2000). The mind in the middle: A practical guide to priming and automaticity research (Chapter 10, pp. 253-285). In H. T. Reis & C. M. Judd(Eds.), Handbook of research methods in social and personality psychology. Cambridge, UK: Cambridge University Press.
  6. Norbert L. Kerr, et al. (2000). Methods of small group research. In H. T. Reis & C. M. Judd(Eds.), Handbook of research methods in social and personality psychology. Cambridge, UK: Cambridge University Press. Chapter 7.
  7. Spencer, S. J., Zanna, M. P., & Fong, G. T. (2005). Establishing a causal chain: Why experiments are often more effective than mediational analyses in examining psychological processes.Journal of Personality and Social Psychology, 89,845-851.
  1. Showcase articles:
  • Zanna, M.P., Higgins, E.T., & Taves, P.A. (1976). Is dissonance phenomenologically aversive? Journal of Experimental Social Psychology, 12, 530-538.
  • Yamagishi, T., Hashimoto, H., & Schug, J. (2008). Preference vs. strategies as explanations for culture-specific behavior. Psychological Science, 19(6), 579-584

Week 5, February 25Outstanding in the field: External Validity

  1. Colquitt, J.A. (2008). From the editors: Publishing laboratory research in AMJ: A question of when, not if. Academy of Management Journal, 51, 616–620.
  2. Mook, D.G. (1983). In defense of external invalidity. American Psychologist, 38, 379-387.
  3. Berkowitz, L., & Donnerstein, E. (1982). External validity is more than skin deep: Some answers to criticisms of laboratory experiments. American Psychologist, 37, 245-257.
  4. Sears, D.O. (1986). College sophomores in the laboratory: Influences of a narrow data base on psychology’s view of human nature. Journal of Personality and Social Psychology, 51, 515-530.
  5. Anderson, C.A., Lindsay, J.J., & Bushman, B.J. (1999). Research in the psychological laboratory: Truth or triviality? Current Directions in Psychological Science, 8, 3-9.
  6. Webb, E. J. & Weick, K. E. (1979). Unobtrusive measures in organizational theory: Areminder.Administrative Science Quarterly,24, 650-659.
  1. Showcase articles: A true field experiment, and culture in the "experimental ethnography"
  • McGillicuddy, N. B., Welton, G. L., & Pruitt, D. G. (1987). Third party intervention: A field experiment comparing three different models. Journal of Personality and Social Psychology, 53, 104-12.
  • Cohen, D., Nisbett, R. E., Bowdle, B. F., & Schwarz, N. (1996).Insult, aggression, and the southern culture of honor: An "experimental ethnography."Journal of Personality & Social Psychology, 70, 945-960.
  1. Suggested, optional
  • Salancik, G.R. (1979). Field stimulations for organizational behavior research. Administrative Science Quarterly, 24, 638-649.
  • Reis, H. T. & Gable, S. L. Even-sampling and other methods for studying everyday experience (Chapter 8, pp. 190-222). In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology. Cambridge, UK: Cambridge University Press.
  • Campbell, J. P. (1986). Labs, fields, and straw issues. In E. Locke (Ed.),Generalizing from laboratory to field settings(pp 269-279). Toronto: DC Heath.
  • Dipboye, R. L., & Flanagan, M. F. (1979).Research settings in industrial and organizational psychology:Are findings in the field more generalizable than in the laboratory?American Psychologist, 34, 141-150.
  • Ilgen, D. R. (1986). Laboratory research: A question of when, not if. In E. Locke (Ed.),Generalizing from laboratory to field settings(pp 257-267). Toronto: DC Heath.
  • Seashore, S. E. (1974). Field experiments with formal organizations. In E. P. Cox, III (Ed.),Research for business decisions: An interdisciplinary approach(pp. 152-158). University of Texas at Austin: Bureau of Business Research. Reprinted fromHuman Organization,23(Summer 1964), 164-170.

Week 6, March 4:Do economists do it better?

  1. Hertwig, R. & A. Ortman. (2001). Experimental practices in economics: A methodological challenge for psychologists. Behavioral and Brain Sciences, 24: 383-451.
  2. Croson, R. (2006). The method of experimental economics. In P.J. Carnevale & C.K.W. De Dreu (Eds.) (2006). Methods of negotiation research. Leiden, The Netherlands: Martinus Nijhoff Publishers.
  1. Suggested, optional
  • Camerer, C.F., Hogarth, R. (1999). The effects of financial incentives in economics experiments: A review and capital-labor-production framework. Journal of Risk and Uncertainty, 7-42.
  • Commentary on Camerer Hogarth (1999), same journal issue.
  • Lee, J. (2007). Repetition and financial incentives in economics experiments. Journal of Economic Surveys, 21, 628-681.
  • Ortmann, A. & Hertwig, R. (2002), The costs of deception: Evidence from psychology. Experimental Economics, 5, 111-131.
  • Oczak, M., & Niedźwieńska, A. (2007). Debriefing in deceptive research: A proposed new procedure. Journal of Empirical Research on Human Research Ethics, 2, 49–59.
  • Bonetti, S. (1998). Experimental economics and deception.Journal of Economic Psychology, 19, 377-395.
  • Taylor, K. M., & Shepperd, J. A. (1966). Probing suspicion among participants in deception research.American Psychologist,51, 886-887.
  • Hey, J. (1998).Experimental economics and deception: A Comment.Journal of Economic Psychology, 19, 397-401.
  • McDaniel, T. & Starmer, C. (1998).Experimental economics and deception: A Comment.Journal of Economic Psychology, 19, 403-409.

Week 7, March 11Dealing with Data, the Results: From Design to Conclusion

  1. Abelson, R.P. (1995). Statistics as principled argument. Hillsdale, NJ: Erlbaum.
  1. Suggested, optional
  • Abelson, R. P. (1985).A variance explanation paradox:When a little is a lot.Psychological Bulletin, 97, 128-132.
  • Prentice, D. A., & Miller, D. T. (1992).When small effects are impressive.Psychological Bulletin, 112, 160-164.
  • Fichman, M. (1999). Variance explained: Why size does not (always) matter. Research in Organizational Behavior, 21, 295-331.
  • Wegener, D. T., & Fabrigar, L. R.(2000).Analysis and design for nonexperimental data: Addressing causal and noncausal hypotheses (Chapter 16, pp. 412-450).In H. T. Reis & C. M. Judd(Eds.), Handbook of research methods in social and personality psychology.Cambridge, UK: Cambridge University Press.
  • McClelland, G. H.(2000).Nasty data: unruly, ill-mannered observations can ruin your analysis, (Chapter 15, pp. 393).In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology.Cambridge, UK: Cambridge University Press.

Week 8, March 25More Dealing with Data, the Results: From Design to Conclusion

  1. Abelson, R.P. (1995). Statistics as principled argument. Hillsdale, NJ: Erlbaum.
  2. Cohen, J. (1994). The earth is round (p < .05).American Psychologist, 49, 997-1003.
  3. Spencer, S. J., Zanna, M. P., & Fong, G. T. (2005). Establishing a causal chain: Why experiments are often more effective than mediational analyses in examining psychological processes.Journal of Personality and Social Psychology, 89,845-851.
  1. Suggested, optional

More Results issues: Statistics for mediating process

  • Baron, R.M. & Kenny, D.A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173-1182.
  • MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., Sheets, V.(2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7, 83-104.
  • Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies: new procedures and recommendations. Psychological Methods, 7(4), 422-445.
  • Muller, D., Judd, C. M., & Yzerbyt, V. Y. (2005). When moderation is mediated and mediation is moderated. Journalof Personality and Social Psychology, 89, 852-863.
  • Kenny, D.A. (2003) mediation,
  • Lots of online resources, e.g.,

Week 9, April 1Dealing with Data: Graphs

  1. Grice, P. (1975). Logic and Conversation. In Peter Cole and Jerry L. Morgan (Eds), Syntax and Semantics, Vol. 3, Speech Acts (pp. 41–58; here 45–47). New York: Academic Press.
  2. Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1312.
  3. Wainer, H (1984). How to display data badly. The American Statistician, 38, 137-147.
  4. Klass:Presenting Data: Tabular and graphic display of social indicators
  1. Tufte, E.R. (2001). The visual display of quantitative information. Cheshire,CT: Graphics Press.
  2. Tufte, E. R. (2003). The cognitive style of PowerPoint. Cheshire,CT: Graphics Press.

Homework assignment:

With some data, make two figures, a bad one and a good one, using criteria from the above items.

Find a good one and a bad one from your favorite journal.

  1. Suggested, optional
  • Wainer H., & Thissen, D. (1981). Graphical data analysis, Annual Review of Psychology, 32, 199-202.

Week 10, April 8 The IRB and related matters: Some classics

  1. Wilson, D.W., & Donnerstein, E. (1976). Legal and ethical aspects of nonreactive social psychological research. American Psychologist, 31, 765-773
  2. Rosenthal, R. (1994). Science and ethics in conducting, analyzing, and reporting psychological research. Psychological Science, 5, 127-134.
  3. Pomerantz, J.R. (1994). On criteria for ethics in science: Commentary on Rosenthal.Psychological Science, 5, 135-136.
  4. Parkinson, S. (1994). Scientific or ethical quality. Psychological Science, 5, 137-138.
  5. Gurman, E.B. (1994). Debriefing for all concerned: Ethical treatment of human subjects. Psychological Science, 5, 139.
  6. Mann, T. (1994). Informed consent for psychological research: Do subjects comprehend consent forms and understand their legal rights? Psychological Science, 5, 140-143.
  7. Sears, D.O. (1994). On separating church and lab. Psychological Science, 5, 237-239.
  8. American Psychological Association (2002). Ethical principles of psychologists and code of conduct.American Psychologist,57, 1060-1073.
  9. Sharpe, D., Adair, J. G., & Roese, N. J. (1992). Twenty years of deception research: A decline in subjects' trust?Personality and Social Psychology Bulletin,18, 585-590.

Suggested, optional

  • Sieber, J. E. (1988). Planning ethically responsible research. In L. Bickman & S. J. Rog (Eds.), Handbook of Applied Social Research Methods, pp. 127-156. Thousand Oaks, CA: Sage Publications.
  • Oczak, M., & Niedźwieńska, A. (2007). Debriefing in deceptive research: A proposed new procedure. Journal of Empirical Research on Human Research Ethics, 2, 49–59.
  • Darley, J. M. (1999). Methods for the study of evil-doing actions. Personality and social psychology bulletin, 3, 269-275.
  • von Glinow, M. A., & Novelli, L. (1982). Ethical standards within organizational behavior. Academy of Management Journal, 25, 417-436.
  • Smith, S. S. & Richardson, D. (1983). Amelioration of deception and harm in psychological research.Journal of Personality and Social Psychology,44, 252-253.
  • Rubin, Z. (1985). Deceiving ourselves about deception: Comment on Smith and Richardson's "Amelioration of deception and harm in psychological research."Journal of Personality and Social Psychology,48, 252-253.
  • Smith, S. S. & Richardson, D. (1985). On deceiving ourselves about deception.Journal of Personality and Social Psychology,48, 254-255.
  • Wollins, L. (1962). Responsibility for raw data. American Psychologist, 657-658.
  • Ceci, S. J., Peters, D., & Plotkin, J.(1985).Human subjects review, personal values, and the regulation of social science research.American Psychologist, 40, 994-1002.

Assignment: Complete CII training through the IRB.

See:

See:

Week 10, Alternative, April 8 Research conduct/ replication / field-qualitative-cases

A. Replicability in Psychological Science: A Crisis of Confidence?

  1. Special Issue in PoPS (2012)
  2. Medin: A Science We Can Believe In
  1. Medin: Rigor Without Rigor Mortis: The APS Board Discusses Research Integrity

B. On Field and Qualitative Work:

  1. Cialdini, R. B. (2009). We have to break up. Perspectives on Psychological Science, 4(1), 5-6.
  2. Wall, J. (2006). The joys of field research. In P.J. Carnevale & C.K.W. De Dreu (Eds.) (2006). Methods of negotiation research. Leiden, The Netherlands: Martinus Nijhoff Publishers.
  3. Zartman, I. (2006). Comparative case studies. In P.J. Carnevale & C.K.W. De Dreu (Eds.) (2006). Methods of negotiation research. Leiden, The Netherlands: Martinus Nijhoff Publishers.
  4. Zbaracki, M. J. (1998). The rhetoric and reality of total quality management. Administrative Science Quarterly, 43, 602-636.

Week 11, April 15The New Laboratory: Online and virtual