Position Paper on Faculty Activity Data for the AAU Institutional Data Committee

DRAFT

Position Paper on Faculty Activity Data for the AAU Institutional Data Committee

Prepared by the Association of American Universities Data Exchange

Dennis Hengstler, Bill Hayward, and Rana Glasgal

April 2005

Page

DRAFT

Executive SUMMARY

The AAU Executive Committee has charged the Institutional Data Committee (IDC) to evaluate university data collection efforts in the areas of Graduate Education, Undergraduate Education, and Faculty Activities and to identify what data collection procedures should be implemented to produce relevant and comparable data across AAU universities. This paper addresses Faculty Activities data, as companion papers will address the other areas of graduate and undergraduate education. Specifically, the paper discusses: a) the types of data being collected; b) the major issues and problems associated with the data collection efforts; and c) recommendations for future data collection activities. Among the topics covered are:

Number of Faculty and Faculty Profiles
Faculty Salaries and Benefits
Faculty Honors and Awards
Faculty Activity (Publications/Citations)
Faculty Patents and Licenses
Research Activity
Research Space
Post Doctorates
Faculty Surveys on Quality of Life, Campus Climate
Teaching Activity
Other

In reviewing the various data on faculty activities, the IDC will need to consider why it is important to collect such information and what questions could be answered with the data collected. It is assumed that the primary focus of the data collection efforts is to provide data to AAU member institutions for internal policy decisions and that the data could greatly facilitate the government relations and administration (e.g., member selection) of the AAU organization.

Other important questions, issues, and principles that will need to be addressed are:

Should the data be collected by discipline/program or by organizational unit? If so, what level of detailshould be collected (e.g., broad discipline, sub-discipline), and what taxonomy of disciplines (discipline classifications) should be reported (CIP, NRC, NSF, ISI, etc)? If discipline level data is desired the AAU Data exchange recommends using CIP codes and that appropriate crosswalks be developed between discipline taxonomies.
How should data be treated / released when there is a small number (N) to ensure the privacy of individuals and as well as the reliability / generality of the data? What constitutes a small N (e.g., 1, 3, 5, 10)?
Should historical data be included if comparable?
How should data be collected (unit-record data vs. aggregate or summary data), how often should they be collected, who should collect and check the data, how should the data be maintained (e.g., should the data reside the AAU Data Exchange data warehouse), what standard reports should be generated from the data, what principles and guidelines should be developed for downstream reporting of the data (e.g., who has access to what data;under what conditions can data be released and to whom)?
What infrastructure and funding will be necessary to support the desired data collection and reporting efforts?

Below is a summary of the recommendations for each of the above topics:

Page

DRAFT

RECOMMENDATONS:

1. Number of Faculty and Faculty Profiles

The AAUDE Faculty Profile Survey provides the most comprehensive and detailed information about faculty. This survey should be collected for all AAU institutions.

Questions the IDC will need to consider are: What guidelines should be developed for the release of faculty demographic characteristics when there are small Ns that potentially could identify a faculty member (e.g., cases where there is only one faculty member of a given race or gender)?
What level of detail is desired in the reporting of faculty by discipline?
Should reports include both the total faculty paid and the total budgeted FTE for the discipline (department)?
Should other useful characteristics of the faculty (e.g., citizenship, age) be reported?

2 Faculty Salary and Benefits

The AAUDE Faculty Salary Survey provides the most comprehensive and detailed salary information for faculty. This survey should be collected for all AAU institutions.

Questions the IDC will need to consider are:

What guidelines should be developed for the release of faculty salaries when there are small Ns that potentially could identify a faculty member (e.g., cases where there is only one faculty member).
What level of detail is desired in the reporting of faculty salaries by discipline?
Should salary data be reported by gender or other characteristics?

3 Faculty Honors and Awards

AAUDE has initiated a project to collect and maintain faculty awards and honors in the AAUDE Data Warehouse. It is recommended that faculty honors and awards continue to be collected and maintained in the AAUDE warehouse. Questions the IDC will need to consider are:

Which honors and awards should be collected and maintained?
Should the AAUDE database track the awards by the name of the recipient and their current institution?
Are there other fields (e.g., discipline, date and year) that should be included in the database?
How to fund the collection and maintenance cost associated with the faculty honors and awards database?

The IDC / AAU should also explore the possibility of working with the various awarding organizations to make their data available in a more user-friendly format, such as adding listings of awards by institution name and making their lists available for electronic download.

Another avenue of exploration is the work of the International Congress of Distinguished Award (ICDA). If some of the faculty awards tracked by ICDA are of interest, perhaps AAU or AAUDE could collaborate with this group to obtain their data.

4. Citations & Publications

The ISI dataset provides the most comprehensive and detailed information about faculty publications and citations. It is recommended that publication and citation data be collected for all AAU institutionsfrom the ISI dataset. Questions the IDC will need to consider are:

Which disciplines should to be reported and what taxonomy should be used (e.g., ISI taxonomy, CIP taxonomy, NRC taxonomy, other taxonomies). It becomes more expensive as the number of broad disciplines is expanded.
Should the reporting of publications and citations be normalized to account for the size of faculty at the AAU institutions? Raw (un-normalized) data would be collected from the ISI dataset and additional work would be required for the normalization.
Should AAU work with Thomson Scientific to address some of the issues listed above in order to improve their product? For example, to address the issue that ISI is not a good indicator in the Humanities --- perhaps Thomson Scientific could be persuaded to compile a database of books written by university faculty and published by the top publishing houses (e.g., Oxford University Press).
Should AAU work with Thomson Scientific to refine their list of “acceptable journals and publications?”

5. Faculty Patents / Licenses

Patent and licensure information is collected annually from a survey of the Association of University Technology Managers (AUTM). If patent and licensure information is desired, the IDC and AAU should try to encourage 100% participation in the AUTM survey. Universities that currently report data at the system level will need to report individual campus data in the AUTM survey or provide such data to the AAUDE representative. AAUDE could work with AUTM to receive special extracts of their database for incorporation into the warehouse. It is assumed that there would be a nominal charge to access this information from AUTM. IDC will also need to ascertain what specific types of information (e.g., patent applications filed, patents issued, licenses and options executed, license income received, start-up companies, etc.) from the AUTM survey should be routinely collected and maintained in the warehouse and what type of reports from the warehouse should be produced.

6. Research Activity

As recommended in the AAU Membership Taskforce report, the best measure of research activity at particular institutions is the NSF R&D Expenditure Survey. We recommend that this measure continue to be used and that refinements be made in standard reports to account for the research nature of individual campuses (i.e., the influence of medical and agricultural research in the total research support of a campus). AAU may wish to work with the NSF to expand the number of disciplines and/or sub-disciplines reported as the level of detail is fairly restricted.

7. Research Space

If the IDC desires to collect research space information, the NSF Scientific and Engineering Research Facilities survey is the only available source of information. This information could be housed in the AAUDE data warehouse. AAUDE would need to make arrangements with NSF to obtain the data in electronic format. If more detailed information is required (e.g., data for disciplines such as physics and chemistry), AAU may wish to pursue discussions with NSF to modify future surveys.

8. Post Doctorates

The postdoctoral data from the NSF Survey on Graduate Students and Postdoctorates in Science and Engineering provides the only source of information about postdocs. Because NSF postdoctoral data are also readily available, additional analyses by discipline, gender, and individual citizenship could be provided. For example, two potentially useful indicators calculated from these data would be the percentage of postdoctorates who are U.S. citizens and the ratio of postdoctorates to graduate students or faculty by discipline.

9. Faculty Surveys on Quality of Life, Campus Climate

Although AAU institutions participate in periodic surveys of their faculty, there is no systematic/consistent source of data on the Quality of Life/Campus Climate. If the IDC would like to collect survey information from faculty, it would be possible to maintain such data in the AAUDE data warehouse. Questions the IDC will need to consider are:

What information would AAU like to receive from faculty (i.e., what questions to ask on the survey)?
How often would the AAU like to receive such information (i.e., how often to administer the survey—one a year, once every two or three years)?
Who should coordinate the survey (i.e., should this be contracted out, should one of the AAUDE representatives be asked to coordinate the survey)?
What resources will be provided to support such a survey?

10. Teaching Activity

From time to time, AAUDE has discussed the idea of collecting Student Credit Hour (SCH) data by level, most recently at its 2005 meeting. Previous discussions have also included the idea of collecting faculty teaching workload data (i.e., number primary classes (excluding non-independent study and thesis/dissertation courses) taught per ladder faculty member. If the IDC wishes to collect such information, AAUDE could initiate formal discussions and develop recommendations for collecting this information

11. Other

Are there other data the IDC may wish to pursue such as faculty turnover, retention, and retirement rates, mandatory retirement and promotion and tenure policies, etc? (Note: faculty teaching activity, such as the number of classes taught per faculty and student credit hours per faculty, will be addressed in the companion paper on undergraduate education.)

Position Paper on Faculty Activity Data for the Institutional Data Committee

Prepared by the AAU Data Exchange (AAUDE)

Dennis Hengstler, Bill Hayward, Rana Glasgal

What data are currently collected, whether there are systematic flaws or redundancies in the data collected, and how any such problems could be rectified?
What data are not being collected that would be useful for both institutional and national planning and policy development, and what data collection procedures should be implemented to produce relevant and comparable data across AAU universities?

This paper will address the above questions related to Faculty Activities, as companion papers will address the other areas of graduate and undergraduate education. It is understood that the primary focus of the data collection efforts is to obtain a better understanding of the AAU institutions for use in internal planning and for legislative and lobbying activities.

The first section of this paper discussesthe various types of faculty activity data being collected. Specific attention will be devoted to: a) the data sources (who collects the data, how the data are similar/redundant or different); b) the major issues and problems associated with the data collection efforts; and c) recommendations for future data collection activities. Among the topics to be covered are:

Number of Faculty and Faculty Profiles
Faculty Salaries and Benefits
Faculty Honors and Awards
Faculty Activity (Publications/Citations)
Faculty Patents and Licenses
Research Activity
Research Space
Post Doctorates
Faculty Surveys on Quality of Life, Campus Climate
Teaching Activity
Other

In considering the various data on faculty activities, the IDC will need to address a number questions and issues: (e.g., why it is important to collect such information and what questions could be answered with the data that is collected?). It is assumed that the primary focus of the data collection efforts is to provide data to AAU member institutions for internal policy decisions and that the data could greatly facilitate the government relations and administration (e.g., member selection) of the AAU organization.

Other important questions, issues, and principles that will need to be addressed are:

Should the data be collected by discipline/program or by organizational unit? If so, what level of detail should be collected (e.g., broad discipline, sub-discipline), and what taxonomy of disciplines (discipline classifications) should be reported (CIP, NRC, NSF, ISI, etc)? If discipline level data is desired the AAU Data exchange recommends using CIP codes and that appropriate crosswalks be developed between discipline taxonomies.
How should data be treated / released when there are a small number (N)to ensure the privacy of individuals and as well as the reliability / generality of the data? What constitutes a small N (e.g., 1, 3, 5, 10)?
Should historical data be included if comparable?
How should data be collected (electronic vs. hardcopy; unit-record data vs. aggregate or summary data), how often should they be collected, who should collect and check the data, how should the data be maintained (e.g., should the data reside the AAU Data Exchange data warehouse), what standard reports should be generated from the data, what principles and guidelines should be developed for downstream reporting of the data (e.g., who has access to what data; under what conditions can data be released and to whom)?
What infrastructure and funding will be necessary to support the desired data collection and reporting efforts?

The remaining section of the paper addresses the second charge of the Institutional Data Committee and will be based on future discussion and recommendations of the group.

1. NUMBER OF FACULTY AND FACULTY PROFILES

Data Sources

A variety of surveys collect information on faculty. Most, however, collect faculty headcount data in conjunction with the reporting of faculty salaries (see section on Faculty Salaries). The following three surveys (two IPEDS surveys from the federal government and one from the AAU Data Exchange) collect faculty information independently of the faculty salary data: Only AAUDE collects faculty information by discipline.

IPEDS EAP (Employee by Assigned Position) Survey: This survey annually reports the total number (headcount) of faculty and staff employees on the payroll as of November 1. It excludes faculty on leave without pay, casual and contracted employees, hospital employees, undergraduate and work-study students. Information is reported by: Primary Function/Occupation Activity (instruction/research/public service, instruction, research, public service, executive, technical, clerical, etc), Faculty Status (tenure, tenure-track, no tenure track, without faculty status, graduate students), Full vs. Part Time Status; and Medical vs. Non-Medical affiliation.

IPEDS FALL STAFF SURVEY: This survey, conducted biennially reports the same information as in the IPEDS EAP survey (headcount of faculty and staff employees as of November 1) but in a different format. Information on total campus headcount is reported by: Primary Function/Occupation Activity (full-time faculty by 9/10-month, 11/12-month, and <9-month appointments, part-time faculty, executive, technical, clerical, etc); Ethnicity; Gender; and by Salary (<$30k, $30-39k, $40-49k, $50-64k, $65-79k, $80-99k, $100k +).

AAUDE FACULTY PROFILE SURVEY. This is a fairly recent survey implemented by the AAU Data Exchange (AAUDE). For the faculty included in the IPEDS EAP (Employee by Assigned Position) survey, the AAUDE Faculty Profile survey collects faculty headcount data by: discipline (6-digit CIP code), department name, academic rank, tenure status, full/part-time status, gender, ethnicity, medical school affiliation, EAP category (i.e., primarily instruction, research, public service, all three areas, other), and track (i.e., regular, research, clinical, public service, librarian, other). This and other AAUDE data are maintained in a data warehouse located on a secured server at MIT. At the present time, access to the warehouse is restricted to AAUDE members only.

Data Issues

AVAILABILITY: The IPEDS data are easily available via the web for all AAU institutions. Although relative new, the AAUDE Faculty Profile Survey elicited a 44% response rate for 2003-04 (10 AAU private institutions and 17 public AAU institutions).

COMPREHENSIVENESS: The number and profile of the faculty from the IPEDS and AAUDE Faculty Profile survey include only faculty who are being paid as of November 1. Faculty who are on unpaid leave, faculty who start in the spring semester, and faculty who are teaching by agreement (no salary) are excluded in the faculty counts. Total FTE by discipline (department) must be imputed from the full-time and part-time faculty. AAUDE institutions attempt to reconcile and match faculty for the various IPEDS and AAUDE surveys.