Выходные данные статьи:
Robert T. Blackburn; Mary Jo Clark, An Assessment of Faculty Performance: Some Correlates Between Administrator, Colleague, Student and Self-Ratings, Sociology of Education, Vol. 48, Issue 2 (Spring, 1975), pp. 242-256.
An Assessment of Faculty Performance: Some Correlates Between Administrator, Colleague, Student and Self-Ratings
Robert T. Blackburn
Center for the Study of Higher Education
The University of Michigan
Mary Jo Clark
Educational Testing Service
Princeton, New Jersey
This paper addresses the uncertainties surrounding the evaluation of faculty work performance and reviews the conflicting studies of the two principal professorial roles, teaching and research. It then presents the findings from a study of the faculty at a typical liberal arts college. The 45 full-time (85 percent of the population) faculty members were rated by administrators, faculty colleagues, students, and themselves on two performance measures: (1) teaching-effectiveness and (2) overall contribution to the college. While reasonably high agreement exists between faculty peers and students on their assessment of professors – and to a lesser (but still statistically significant) extent between administrators and these two role sets-the near zero correlations between the professor and each role set are causes for genuine concern. Related research supports a psychological explanation of self-misperceptions.
Faculty members complain more about the manner in which their work is judged and rewarded than about any other dimension of their professorial role (Guthrie, 1949; Theophilus, 1967). They worry about tenure, promotion, and merit increases. Often faculty members believe that deserved honors come too late, if at all.
The professor's anguish is not surprising. Most academics sincerely believe they are performing at levels higher than those for which they receive institutional and personal recognition. Furthermore, professors can document their frustrations with respect to the assessment of their worth–ignorance on the part of the evaluators (Gustad, 1967: 270). Those who pass judgment seldom witness a performance.
THE PROBLEM AND ITS BACKGROUND
Teaching
While a popular theory says the professor assigns a low value to teaching, studies reveal just the opposite is in fact the case. Faculty members give their teaching highest priority, as. does then-college (Cartter, 1967). In addition, teaching is their greatest source of pleasure (Gaff and Wilson, 1971: 195).
But how is a professor's pedagogy to be judged when there still is no acceptable definition of "good" teaching (McKeachie, 1967; 1970; Biddle and Ellena, 1964; Rothwell, n.d.)? Furthermore, neither deans, department chairperson, nor the professor's peers ever see him teach. Even if student evaluation forms are used and are available to administrators, deans will not publicly claim that those who are in the role of apprentices are qualified to judge those who have credentials as masters (Kent, 1967: 316). Some faculty sincerely and vociferously protest student evaluation (Bryant, 1967; Hildebrand, 3Jffl2). Objective data are rarely on hand when a judgment on teaching performance is rendered. The little available evidence linking student-judged teaching effectiveness and student learning has been positive, but not strong (McKeachie, 1969). However, M. Rodin and B. Rodin (1972) have reported a high inverse relationship (r = -.75).
More recently, Gessner (1973) obtained correlations reaching .77 between student-judged teaching effectiveness and performance on a national examination. The conflicting outcomes increase faculty uncertainties and heighten concerns about the validity and reliability of evidence being used when they are assessed.
Research
Scholarly output is supposedly a more objective dimension of professorial value. At least publications can be assessed, but the extent to which careful- critiques are actually performed remains debatable. Besides, inferring teaching effectiveness from research productivity is difficult. First of all, expert opinion differs widely. On the one side, Jencks and Riesman (1968: 532) believe research is essential for vital teaching, the view many faculty support. On the other hand, Fischer(1968: 10) claims the opposite, an opinion students vent when they say a professor's teaching is poor because the time he might have spent in preparation for class has gone into unrelated scholarship.
The Relationship of Teaching to Research
The studies conducted on the relationship between research and teaching show either no relationship or at best a slight positive association. In research at the University of Washington, Purdue, Kansas State, Carnegie-Mellon, Wayne State, and a midwest liberal arts college, very low correlations were found between student-judged teaching effectiveness and research activity (Voeks, 1962; Feldhusen and McDaniels, 1967; Hoyt, 1970; Hayes, 1971; Blackburn, 1972; Harry and Goldner, 1972; Sherman, 1973). Other contemporary investigations report small, positive correlations at Tufts, Illinois, California-Davis, and Purdue (Bresler, 1968; Stallings and Singhal, 1970; Hildebrand and Wilson, 1970; McDaniel and Feldhusen, 1970).
Hence, over the entire spectrum of high to low research-oriented college and university faculty, the relationship between teaching and research is at best slight and most often not significant. Predicting a faculty member's performance in either role from the other is mainly a matter of chance.[1]
Agreements Between Evaluators
Yet, administrators and faculty members persist in assuming a positive relationship between "Beaching and research as they make judgments about promotions, tenure, and salary increases (Maslow and Zimmerman, 1956; Luthans, 1967). Even the professor assumes a single conception of academic worth which specifies that if a colleague is doing research, his classes are ipso facto superior (Hammond, Meyer, and Miller, 1969). At the same time, the faculty will differ with administrators on what criteria are critical. For example, Hussain and Leestamper (1968) discovered that four of the ten (out of sixty) criteria students and faculty agreed to be most important for good teaching were not in the set of criteria administrators used on their merit rating form for assessing faculty performance whereas four of the ten criteria students and faculty believed least important were considered for merit ratings. Attitudes toward students are an example of the first; research and publication fall in the second. In a study by Crawford and Bradshaw (1968), each of ten subgroups–assistant professors and instructors, associate and full professors, department chairmen, deans, and six student groups divided by sex and three levels of ability–differed in a statistically significant way from all other subgroups in the rating given to the most important characteristics of effective university teaching. The thirteen teacher characteristics groups rated ranged from knowledge of the subject and organization of lectures to sense of humor and punctuality. Birnbaum (1966) found inconsistencies in faculty evaluation at the community-college level.
Promotion and Merit Raises
Finally, two studies underscore the faculty's genuine concern about how they are assessed and rewarded. Luthans (1967) found that while deans, department chairmen, and other administrators believe that teaching is the most important function of the faculty, and that the faculty agree, administrators confess that promotion is judged on other criteria, e.g., research. However, Luthans uncovered no relationship between research and promotion. Over half the administrators reported they sometimes approved promotions to full professor in the absence of a significant publication record. Similarly at Kansas State, Hoyt (1970) found no significant relationships between either rate of promotion or receipt of merit raises and either teaching effectiveness or publication regard.
Unpublished studies in three departments at the University of Michigan likewise failed to yield a promotion pattern with either publicly available student-judged teaching effectiveness or publication record. Time-in-rank appears to suppress any relationship. Quite likely a department finds it very difficult to 'promote an individual with less time-in-rank than a colleague with longer (or even equal) tenure, especially in a pool of able faculty members. The difference in performance would have to be dramatic to alter the order of promotion. Such views are apparent to the outsider but, of course, are seen differently by the individual who believes he deserves a promotion and/or merit increase irrespective of time on the job and/or how well a colleague is performing.
So, a definite basis exists for faculty members' complaints about matters of recognition for their efforts. Whether they are judged well or poorly matters, of course. Equally important, however, is their firm conviction that they are not judged properly.
Improving the assessment process is therefore extremely important. Sorting out fact from folklore as well as from unfounded belief is the first step.
While several studies of faculty teaching effectiveness have utilized student evaluation, and a few have used peer ratings, this research combines these two sources and introduces both administrative and self evaluations. This study also expands the notion of faculty work performance to include dimensions other than teaching effectiveness and publications. A global rating on overall contribution to the college serves as an independent measure. The findings provide insights into faculty beliefs. The unexpected results also have serious implications for behavior within colleges and universities.
SETTING AND METHODOLOGY
The study was conducted at "Midwest" college. Its fifteen hundred students encompass a full range of interests and academic qualifications. A moderately well-trained faculty staffs the typical departments. While participation from the approximately twenty-five part-time faculty members was respectable, other factors, led to restricting the analyses to the full-time faculty. Forty-five of the fifty-three faculty members (85 percent) in the latter category responded to all measures.
As Midwest grew, it experienced a separation from its founding church both in support and in control. While drawing its student body principally from its own and a contiguous state, Midwest how is attracting youth from the more heavily populated East and Northeast. Student body SAT scores are exactly at the national norm for college-bound youth. Like other liberal arts colleges of both higher and lower selectivity, the two largest groups of students are preparing for teaching and business careers. In these and other ways, Midwest is like many other American colleges and is almost average for the more than eight hundred private and church-related liberal arts colleges. And, since the principal faculty roles are teaching and contributing to the organization. Midwest is not unlike many emerging state colleges arid universities except, of course, with respect to size.
Having convinced themselves that self-analysis was necessary for major change. Midwest faculty members willingly participated in a series of self-studies. Among other things^ they rated their colleagues and themselves on teaching effectiveness and overall contribution to the college. Midwest's administrators also rated faculty members on both dimensions of performance. Student evaluations of teaching effectiveness were also obtained. A sample of faculty participated in a second set of ratings to provide an estimate of measurement reliability.
Specifically, each faculty member rated every other teacher in his curricular division and himself on a five-point scale of "teaching effectiveness." The faculty member was told to "consider those qualities which are important in the evaluations^ the skills and practices and products of a classroom teacher regardless of rank or experience or teaching of the person being rated." In a similar way, each faculty member judged himself and each colleague on a five-point scale concerning his "overall "contribution" to Midwest College. Administrators rated faculty in both categories. The rater was told to "take into account the person's total contribution, whether his own work or his stimulation of others, whether scholarly or administrative or in human relations; the person's overall usefulness in helping the college carry out its responsibilities."
Cartter's (1966) methodology was employed. An intensive analysis was conducted to establish the reliability and the validity of the technique. While expert value judgments are on quality of effectiveness of outcomes, not on specific behaviors, the validity requirement of logical relevance was still met. Also, the validity measure of the consistency of rating across raters with different sources of information and perspectives was satisfied. Reliability was ascertained by the test-retest method as well as by multifactor one-way analysis of variance to obtain estimates of person and error variance. Values ranged from .72 to .86 but were lower on self estimates, a single measure. Also, by using role theory and other data (demographic, selected psychological traits, job satisfaction, professional attitudes, and perceived stress), other required statistical tests were satisfied.
Student evaluations of teaching effectiveness were obtained from a standard 14-item five-point scale questionnaire that the college systematically employs to evaluate all courses each semester. (The high correlations that exist between different scales (e.g., see Sherman) made it unnecessary to intervene in the professor's classroom to obtain an additional measure of teaching effectiveness.) Responses to the question "How would you rate your instructor in teaching effectiveness?=were averaged across all courses taught by a faculty member during one semester for an index of his teaching performance as judged by students.
FINDINGS
The intercorrelations are collected in Table 1. Different ratings on the two performance dimensions demonstrate discrimination was made between teaching effectiveness and overall contribution to the college by the same group of raters. Secondly, teaching effectiveness as judged by the faculty correlated significantly with similar ratings by administrators and students. (The correlations between administrator and student ratings on teaching effectiveness, although lower (.47), is also statistically significant.) However, among self-ratings with other role sets on the same trait, only colleague ratings on overall contribution to the college demonstrate a significant relationship (.45). Self-ratings on teaching effectiveness have near-zero correlations with ratings for each of the other three groups of raters.
1
TABLE 1
Rated Teaching Effectiveness and Rated Overall Contribution to the College
As Evaluated by Professors, Administrators, Self, and Students in Classes
Professors / Administrators / Self / StudentsTeaching / Contribution / Teaching1 / Contribution / Teaching2 / Contribution / Teaching3
(N=45) / (N=45) / (N=45) / (N=45) / (N=45) / (N=41) / (N=45)
Professors
TeachingOverall Contribution / (.72)
.494 / (.86)
Administrators
Teaching1
Overall Contribution2 / .634
.17 / .24
.544 / ( )
.434 / ( )
Self
Teaching / .28 / .344 / .10 / .13 / ( )
Overall Contribution / .334 / .454 / .07 / .15 / .724 / ( )
Students
Teaching / .624 / .24 / .474 / -.04 / .19 / -.07 / ( )
1The administrator rating on teaching effectiveness is the mean of ratings by the appropriate division chairman and the academic dean.
2The administrator rating on overall contribution to the college is the mean of ratings by the president, academic dean, and assistant dean.