Syllabus
EPE 798-401 Seminar on Institutional Research and Effectiveness
Wednesday, 4-6:30 Spring 2008
Rm. 245 Taylor Education Building
Dr. Karen Carey
143 Taylor Education Bldg. Office hours: Thurs. AM , or by appt
Cell: 913-5263 Office phone: 257-2330
Home: (502) 535-0134
The development of higher education as a field of study parallels the development of institutional research as a function within institutions of higher education. During that time, both have changed considerably as higher education has become increasingly complex. Although they are frequently described as two different groups, “researchers” and “practitioners,” both have contributed to the literature and to our understanding of higher education. Indeed, IR has been described by Wilensky, Terenzini and others as a form of “organizational intelligence,” and, as such, it has become a more widely recognized and valued function within colleges and universities and other higher education agencies.
This course is designed to give graduate students an understanding of the relationships of issues in higher education, broadly stated, to the functions of institutional research. Students will understand the relationship of institutional research to various internal and external organizational functions, to other aspects of the study of higher education, and to the emerging field of knowledge management. In addition, students will acquire the fundamental knowledge and some of the skills needed to succeed in this role and to help institutions address issues related to accountability and accreditation.
The challenge of institutional research is that one is expected to wear many hats and to be able to help others understand almost any issue that arises in the context of higher education, providing information that is always accurate and timely. Although they can be grouped in different ways, institutional researchers typically are interested in the following issues, as they are grouped in the Program Tracks of the Association for Institutional Research:
1. Student life: Research and practice related to students’ personal and student social development, student demographic and economic issues, campus and community engagement, student mobility and flow, and student satisfaction with their higher education experience. Enrollment management is sometimes included here.
2. Student learning and outcomes: Research and practice related to student outcomes assessment, student intellectual development, quantitative and qualitative measures of student learning, psychometrics and testing, academic program improvement resulting from assessment of student learning.
3. Academic Programs, curriculum, and faculty issues: Issues related to the development and management of academic departments, programs, curriculum, and faculty activities, i.e., the kind of information that a faculty member, department chair, dean or chief academic officer would use in evaluating the status of academic programs.
4. Institutional Management and planning: Campus-level planning, evaluation, and management functions -- focus on the types of information and analyses that would be of primary interest to senior campus-level administrators for campus-wide planning and improvement.
5. Higher Education collaborations, policy issues, and accountability: Issues that go beyond the campus, including accountability of individual institutions to external publics, as well as multi-institutional collaborations, such as data exchanges, learning consortia, and articulation agreements, system-level issues, and public policy related to higher education.
6. The Practice of Institutional Research: Theory, techniques, technologies, tools and ethics: the organizational, ethical, methodological and technological aspects of the profession.
We will explore the relationship of institutional research to various internal and external organizational functions, to other aspects of the study of higher education, and to the emerging field of knowledge management. In addition, students will acquire the fundamental knowledge and some of the skills needed to succeed in this role and to help institutions address issues related to accountability and accreditation. Please note that this is not a statistics course, although quantitative analytical skills are extremely important for institutional researchers.
Course requirements:
Texts: There is no single text for this course. Instead, a variety of journal articles will be assigned. To save trees and money, as many as possible will be available full-text, on-line through the library. Others will be either distributed in class or placed on on-line reserve in the library.
Class: I will assume that you have read the articles and are prepared to discuss them in class. Note: There are no dumb questions. Please come to class ready to ask, question, or critique anything that puzzles you. Our students come to class with a rich variety of experiences, and your questions will undoubtedly help everyone.
Please notify me ahead of time if you must miss a class.
Your grade will be based on your participation in class, the quality of your questions, and the following projects/papers:
Institutional Research Project
You will conduct a version of a typical IR project, and produce the report. This project can be done in groups or on your own. Detailed requirements will be distributed later, but choose your topic EARLY!
Context memo. This assignment is intended to provide practice in contextualizing your work for institutional decision-makers, and can be used for your project as well. Use at least three journal articles as well as locally relevant information to inform your perspective. If you are working as part of a group, each person will write her/his own memo, and may use different articles. Show that you know the institution and why the study has been requested or is needed.
Project calendar and plan: Submit an outline of your project plan, the steps involved, the resources you plan to use, the budget if any, and the timetable for carrying it out.
Critique/review: Write a brief critique of an institutional report or institutional plan, giving the author suggestions for making it more effective.
Benchmark study: Perform a benchmark study on a topic of your choice and write up the results.
Environmental scan journal: Keep an environmental scanning journal, identifying what each trend signifies and offering how you think the institution or unit could take advantage of and/or minimize the effects of the trend.
TENTATIVE Course outline and readings
During the semester, other readings may be introduced, depending on the interests of the class.
January 9 Course Overview
o Student learning goals; Review syllabus and course expectations
o How do current important issues in Higher Education play out at the institutional level?
o Who conducts research on issues in higher education? How are they trained?
o How is IR the same as and different from others?
o What is IR? What kinds of studies are conducted? How? Why?
o The Association for Institutional Research Code of Ethics; ASHE Reader
January 16 Institutional Research
o How IR functions within an organization; within different settings and types of institutions;
o History and evolution of IR functions
Read for 1/17 in New Directions for Institutional Research. What is Institutional Research All About? A Critical and Comprehensive Assessment of the Profession. J Fredericks Volkwein, Ed. Number 104, Winter, 1999 (full text on-line)
Volkwein, J. Fredericks, “The Four Faces of Institutional Research” (Chapter 1)
Terenzini, Patrick T., “On the Nature of Institutional Research and the Knowledge and Skills It Requires,” (Chapter 2)
Peterson, Marvin W., “The Role of Institutional Research: From Improvement to Redesign.” (Chapter 8)
Ehrenberg, Ronard G. “Why Universities Need Institutional Researchers and Institutional Researchers Need Faculty Members More Than Both Realize.” In Research in Higher Education, Vol. 46, No. 3, May 2005, p.349-363. Full text on-line.
Data management and administration is one important responsibility of Institutional Research. How does this responsibility fit into the readings for this week?
1. Information collection – IR’s interest v. transactional interests
2. Reporting -- standard production reports
3. Fact Books -- online examples
January 23 IR and its institutional context
Guest lecturers: Roger Sugarman, Director of Institutional Research, UK (and, possibly)
Judith Weckman, Director of Institutional Research and Assessment, Berea
o Defining Critical Issues for IR;
o Major Responsibilities
Pass the IRB certification and submit documentation
Readings
“The Presidency and Institutional Research at Liberal Arts Colleges,” Pagano and Peacock, 2001. (Handout)
“Learning by Association” AIR presidential address, Victor Borden, AIR Annual Forum, 2003.
“Common Assumptions about Organizations that Mislead Institutional Researchers and their Clients,” Frank A. Schmidtlein, Research in Higher Education, Vol. 40, no. 5, 1999. on-line.
January 30 IR and Enrollment Management
o Studying student flow and the enrollment pipeline; strategic enrollment management..
Rowley, D. J. et. Al, (Chapter 11) Essential Areas I: Enrollment Management and Program Planning, Strategic Change in Colleges and Universities. Jossey-Bass, 1997.
Kroc, R. and Hanson, G. “Enrollment Management” Ch. 5 in The Primer for Institutional Research, William E. Knight, ed. AIR, 2003, 75-102. (hand-out)
McLaughlin, G., Brozovsky, P. and McLaughlin, J. “Changing Perspectives on Student Retention: A Role for Institutional Research.” Research in Higher Education, vol. 39, no. 1, 1998, 1-17. On-line journal.
Terkla, Dawn Geronimo, Roscoe, Heather S, and Etish-Andrews, Jane, “Voices from around the World: International Undergraduate Student Experiences.” Paper presented at 45th Annual AIR Forum, San Diego, CA, May 30, 2005. (hand-out)
Gilbertson, Dawn, “University of Phoenix Pushes Ahead.” The Arizona Republic, September 15, 2004.
By now you should have decided on your project and have it approved.
February 6 Benchmarking
Context memo due.
o Tools for peer group comparisons, benchmarks and benchmarking
Teeter, Deborah and Brinkman, Paul. “Peer Institutions.” Chapter 6 in A Primer for Institutional Research, Wm. E. Knight, ed., Association for Institutional Research, 2003, 103-122.
Warne, T. R., et. al., “Peer Group Analysis: For Administrators Only?” paper presented at the AIR Forum, May, 2005.
Benchmarking in Higher Education, Alstete, Jeffrey. ERIC Clearinghouse on Higher Education, Washington, DC. ERIC Identifier: ED402800. http://www.ericfacility.net/databases/ERIC_Digests/ed402800.html
“Using Benchmarking to Influence Tuition and Fee Decisions” Chapter 3 in Using Benchmarking to Inform Practice in Higher Education, Bender, B and J. Schuh, eds., New Directions for Higher Education no. 118, 2002. on-line.
Epper, Rhonda M. “Applying Benchmarking to Higher Education: Some Lessons From Experience.” Change, Nov. 1999.
o The Benchmarking Code of Conduct
o Using IPEDS peer analysis system for IR
February 13 Faculty Data
Benchmark study due
o Comparing faculty within and across institutions
Michael F. Middaugh, “Academic Program and Faculty Issues,” Chapter 3 in Institutional Research: Decision Support in Higher Education, Rich Howard, ed., AIR, 2001. (handout)
Middaugh and Isaacs, “Chapter 3. Describing Faculty Activity and Productivity for Multiple Audiences,” in The Primer for Institutional Research, Wm. E. Knight, ed., 2003
(handout)
McLaughlin and Howard, “Chapter 4. Faculty Salary Analyses,” in The Primer for Institutional Research, Wm. E. Knight, ed., 2003 (electronic reserve)
Isaacs,Heather and Middaugh, Michael F. “Out-of-Classroom Faculty Activity Study: A look at Two Data Collection Cycles and a Verification Study” Paper presented at 45th Annual AIR Forum, May, 2005, San Diego.
Look at Delaware Study Data collection instrument.
February 20 Quantitative and qualitative approaches
Project plan due
o Questionnaires, survey research, focus groups;
o Using existing local data in IR
o Using on-line quantitative and qualitative approaches survey software
Read: in New Directions for Institutional Research #112, Winter 2001:
Pascarella, E. “In the Shadow of the Mountains” A report on the ACA alumni survey.
Ch. 2. “Qualitative and Quantitative Research; A Complementary Balance, Kenneth W. Borland, Jr., p. 5.
Ch. 3. “Using Qualitative and Quantitative Methods for Complementary Purposes: A Case Study” by McLaughlin, Josetta S.; McLaughlin, Gerald W.; Muffo, John A. p. 15.
Ch. 4. “Conceptual Models for Creating Useful Decision Support,” Howard, Richard D. p 45. Available on line
Suskie, Linda, What works in survey research.
February 27 IR Skills: Collecting data and providing information
Using the web for IR; national data sets, web sites,
“Using National Data Sets for Postsecondary Education Research” John H. Milam, Jr, HigherEd.org. available at http://highered.org/docs/milam-datasets.PDF
Examples:
Sax, Linda J., and Cassandra E. Harper, UCLA, “Origins of the Gender Gap: Pre-College and College Influences on Differences Between Men and Women” paper presented at the National AIR Forum, May 2005, San Diego.
Romano, Richard, and Wisniewski, Martin. “Tracking Community College Transfers Using National Student Clearinghouse Data” AIR Professional File number 94; Winter, 2005.
St. John, Paulsen and Carter. “Diversity, College Costs, and Postsecondary Opportunity: An Examination of the Financial Nexus between College Choice and Persistence for African Americans and Whites” in The Journal of Higher Education, Vol. 76, no. 5 (Sept/Oct 2005).
“Adelman, Clifford. “Geographic Mobility Among the ‘Swirlers”: Where Do They Come From? Where Do They Go?” U.S. Dept of Education/OVAE. Paper presented at the annual AIR Forum, May 2005, San Diego.
March 6 Assessing Student Learning
Chapters 1 and 2 from Learning that Lasts: Integrating Learning, Development and Performance in College and Beyond, Marcia Mentkowski and Associates, Jossey-Bass, 2000.
Bauer, Karen W., “Assessment for Institutional Research: Guidelines and Resources”
Ch. 2 in The Primer for Institutional Research, William E. Knight, ed AIR, Tallahassee, FL. 2003.
Terenzini, Patrick T. "Assessment with Open Eyes: Pitfalls in Studying Student Outcomes". Journal of Higher Education, Vol. 60, No 6 (November/December 1989) on-line journal.
Commission on Assessment for Student Development, Commission Clearing House, at http://www.acpa.nche.edu/comms/comm09/dragon-index.html.
Aloi, Susan and Jones, Elizabeth, West Virginia University. “Examining the Current State of Outcomes Assessment” paper presented at the Association for Institutional Research Forum, May, 2005, San Diego.
“Principles of Good Practice for Assessing Student Learning” AAHE Assessment Forum
March 19 Assessment vs. Effectiveness
Guest lecturer: Stacey W. Street, Assessment Coordinator, EKU
o multiple approaches to quality improvement: a quick trip through some of the possibilities and resistances.
Berdahl and McConnell, “Autonomy and Accountability” Ch. 3 in American Higher Education in the Twenty-first Century, Altbach, Berdahl and Gumport, eds., Johns Hopkins, 2001.
Muffo, John A., “Institutional Research Support of Accountability.” Ch. 1 in The Primer for Institutional Research, William E. Knight, ed. AIR, Tallahassee, FL 2003.
NSSE and other instruments.
March 26 IR and Accountability
Read the Kentucky Council for Postsecondary Education’s accountability reports summaries at the Planning and Accountabilty Page of their web site, http://cpe.ky.gov, or go directly to it at http://cpe.ky.gov/NR/rdonlyres/6EB12D9B-9A4D-4A70-BD99-C032AC8876F5/0/KeyIndicators.pdf