RSPH
CRITERION X – EVALUATION AND PLANNING
A. PROCEDURES AND PROCESS
X.A.1 Description of evaluation procedures and planning processes
The school systematically evaluates its overall efforts against its mission, goals and objectives, assesses its effectiveness in serving its various constituencies, and seeks both internal and external guidance as it plans to achieve its mission in the future.
Annual Evaluation and Planning Process
The school annually evaluates progress on existing objectives and revises its objectives in support of its mission and goals. Evaluation findings and objectives are contained in the Annual Report, which is shared with faculty members and historically has been presented to Emory University’s Provost, Executive Vice President for Health Affairs, and President in the fall of each year. Annual Reports are also posted on the RSPH website (www.sph.emory.edu/admin/academic) and are available on site in the resource file.
Data used for the evaluation portion of the Annual Report are derived from the following sources:
· Faculty and Department Annual Reports
Submitted in July of each year, department annual reports provide both faculty and department-level performance data that are used to evaluate overall RSPH progress on teaching, research, and service-related objectives. A sample of the faculty annual report format, which provides the information required to prepare the department annual report, is included in Appendix X.A.1.
· Data from the Office of Student Services
Student Services provides performance data on objectives related to student recruitment, diversity, academic performance, and student life. Criteria V.D and IX.A and B of this document provide several examples of this information and how it is used to evaluate the school’s success in enrolling a qualified, diverse student body that achieves the established learning objectives.
· Data from the Office of Administration and Finance
The office of Administration and Finance prepares reports assessing performance on objectives related to faculty recruitment, educational programs, financial resources, sponsored research, space and equipment, faculty and staff development, and staff performance. Examples of outcome measures based on data from this office may be found in Criteria IV.10, Resources, and VIII.A, Faculty.
· Survey and targeted evaluation data, including the Student Exit Survey, Survey of Recent Graduates and individual course evaluations
Information gained from student and alumni surveys and course evaluations is used to evaluate progress against educational programs and student life objectives. Feedback received may lead directly to improvements in program or support services, as demonstrated in Criterion IX.C. By monitoring the Exit Survey evaluations, the school identified opportunities to improve programs that support student life, and addressed them with personnel changes and an expansion in services.
Additional information used to inform the annual planning process is derived from these sources:
· Department annual reports
Department annual reports contain a summary of departmental strengths, weaknesses, goals and objectives for the following year, which are considered when setting school-wide goals and objectives. For example, department faculty recruitment objectives become part of the school-wide objective to “recruit and retain nationally and internationally known faculty members.” (See Criterion I, Goal II.)
· Recommendations and observations made at the annual faculty retreat
Faculty members attending the annual retreats may directly examine the school’s mission and goals, or may focus on particular objectives that have an impact on achieving the mission and goals. Participants at the September 19, 2003 RSPH Faculty Retreat recommended “paths to greatness” actions, identified potential measures of progress, and examined whether the school’s current mission and goals supported these actions. As a result, the school’s mission was expanded, a fourth goal was added, and recommended actions were used to inform the 2004 annual planning process. Specific actions subsequently taken during the 2003-04 academic year were reported and discussed at the October 1, 2004 annual retreat. In addition, faculty members generated new ideas about how to be a “destination school,” which will be considered for action during the 2004-05 academic year.
· Recommendations from school governance groups and standing committees
Groups involved in school governance provide recommendations that inform planning and objective setting for all school goals and objectives, as described in III.1 and III.2. Examples include recommendations from the Curriculum Committee to revise core courses, from the APT Committee to develop and adopt definitions of “adequacy” and “excellence” in service, and from the Faculty Senate on issues related to the faculty appointment process and fund raising.
· Recommendations from the Community Advisory Network
This recently formed advisory group provides information that helps RSPH forecast future academic needs, encouraging appropriate change that will strengthen the competitiveness of both the school and its graduates. A Community Advisory Network membership list is provided in Appendix III.2.
Materials from the summer 2004 meeting and a resulting “professional skills
checklist,” which is currently being developed and implemented, are available on site in the resource file.
· Recommendations made by external reviewers
Outside experts performing periodic department and school-level reviews assist the school in identifying future goals and the resources required to achieve them. The school strives to review each department externally approximately every five years, as described in Criterion VIII.A.3. Each evaluation results in an assessment of the quality of the department’s program and recommendations for improvement that are considered in the annual planning process. In 2001, a site visit team arranged by the Executive Vice President for Health Affairs reviewed the entire school and its leadership. The reports of all external evaluators are available on site in the resource file.
· Recommendations from the larger public health community
RSPH closely monitors emerging information and recommendations regarding local, national and global public health needs, and considers these needs in its planning processes so that it can contribute to the solutions. For example, the Curriculum Committee is responsible for assuring that learning objectives for the school’s core courses are consistent with recommendations of the Institute of Medicine in Who Will Keep the Public Healthy (2003).
The following timeline and process is used for evaluation, planning and initiating change. Information gathered during the evaluation and planning process is summarized in the school’s Annual Report:
Time / Action / Responsible Person/sOngoing / Make recommendations to chairs and deans at standing meetings / School governance groups and standing committees
Ongoing / Solicit and interpret recommendations from health sciences, university, and community constituents, such as university strategic planning groups and members of the Community Advisory Network / Dean, administrators, committee members
Ongoing / Implement assigned actions resulting from evaluation and planning activities / Administrators, chairs, faculty or staff, as appropriate
Continuation
Time / Action / Responsible Person/s
May / Conduct student and alum surveys / Executive Associate Dean for Academic Affairs
June / Prepare individual annual performance reports / Faculty
July / Prepare department-level annual performance reports / Chairs
July / Gather administrative performance data for the past academic year / Student Services;
Administration and Finance
August / Review and evaluate all performance data listed above, plus recommendations from governance groups, standing committees, external reviewers, the larger public health community, and faculty at the previous annual retreat. Revise objectives for upcoming academic year. Prepare Annual Report. Solicit input from deans, chairs and faculty. / Executive Associate Dean for Academic Affairs
September/October / Report performance and progress to faculty at annual retreat. Solicit additional input from faculty for upcoming academic year objectives and longer-term planning. / Dean
RSPH Strategic Planning
In addition to the annual evaluation and planning process outlined above, the school historically engaged in a periodic, full-scale strategic planning effort, most recently in 1998. That process included:
· a review of the mission, goals and objectives of the university and school
· a review of the university and school values platform
· an environmental scan focusing on the local, national and global context for higher education and public health
· benchmarking the RSPH with other peer institutions and to itself on available dimensions
· an assessment of strengths, weaknesses, opportunities and threats
A copy of this strategic plan is available in the resource file on site.
Health Sciences Center and University Strategic Planning
Under the leadership of Dr. Michael Johns, the Woodruff Health Sciences Center established a strategic planning office, and ongoing RSPH strategic planning is now done in concert with other units of the Health Sciences Center and the university. A 2004-2005 Emory University Strategic Planning Process is being co-led by the Woodruff Health Sciences Center CEO and the University Provost. An overview of this process and information contributed by RSPH are available on site in the resource file.
X.A.2 Measures for evaluating effectiveness of evaluation and planning activities
The school examines the effectiveness of its evaluation and planning activities through both internal assessments and external reviews.
Internal Measures of Evaluation and Planning Effectiveness
The school annually evaluates its progress on achieving its mission, goals and objectives through a variety of measures, as presented in Criteria IV through IX in this document. The evaluation results are summarized in the school’s Annual Report, described in X.A.1. Five key internal measures of progress discussed in each of the past four Annual Report Executive Summaries (Appendix X.A.1), include:
1. Faculty of excellence. The most important indicator of the school’s quality is its faculty, which leads it in its mission to “acquire, disseminate, and apply knowledge; and train leaders.”
· Measures: Key faculty appointments and faculty honors
· Performance: Successfully recruited high-quality faculty members to fill key positions, and capitalized on new opportunities to add outstanding faculty. Significant faculty honors include appointments to the IOM National Academy of Science (four faculty, five jointly-appointed faculty, and one emeritus faculty member); and two Woodruff Chairs, Emory’s highest faculty honor.
2. Excellence in teaching. To fulfill the RSPH mission, it is necessary to be viewed by students as an institution of excellence, offering a high-quality, faculty-intensive education.
· Measure: Student course evaluations
· Performance: Student evaluations of the “value” of all courses averaged 7 on a scale of 9, with 9 being excellent, between 2001 and 2004.
3. High-quality students. To successfully educate individuals for professional and research careers in public health, the school must recruit and admit qualified students.
· Measures: GRE scores and undergraduate GPA; diversity of student body
· Performance: The mean undergraduate GPA of admitted MPH and MSPH students held steady (3.39 to 3.29) between 2002 and 2004, and mean GRE scores increased from 1707 to 1762 during the same period. African-American student enrollment exceeds 20%, and is higher than all but one other accredited school of public health, according to the most recent available ASPH data.
4. Excellence in research. The quality and quantity of RSPH research are key indicators of the school’s ability to acquire and disseminate the knowledge necessary to promote health and prevent disease in human populations.
· Measures: External research funding and number of peer-reviewed publications
· Performance: Research awards increased from 37.2 million in FY 2002 to 52.5 million in FY 2004. Per capita authorships increased from 7.2 to 7.6 for tenure track faculty and from 2.7 to 2.9 for non-tenure track faculty during that same period of time.
5. Financial stability. Financial stability is a key measure of the school’s ability to fulfill its commitments to students, faculty, staff, and the community.
· Measure: Levels of endowment and overall operating income
· Performance: Endowments increased by 14.1% between 2002 and 2004. The school has operated in the black for each of the past 9 years.
External Measures of Evaluation and Planning Effectiveness:
In addition to internal assessment of evaluation and planning effectiveness, the school actively seeks external measures of progress against its mission-based goals and objectives in the following ways.
1. Five-Year Review. A panel of experts reviewed the school’s progress under the leadership of Dean James W. Curran.
· Measures: Overall strengths and weaknesses identified by outside experts
· Performance: Strengths – Rapid progress in becoming one of the strongest schools of public health, leadership, direction, and importance of RSPH to the university in facilitating its aspirations to become a global institution. Weakness – Need for additional resources to build to the next level of excellence.
2. Department-level reviews.
The school strives to externally review each department approximately every five years, as noted in Criterion VIII.A.3. Departments must conduct a self-study and collect relevant background information. The documents are reviewed by a team of three or more distinguished experts in the field who then conduct a site visit of two days. Site visitors prepare a report with their assessment of the quality of the program, recommendations for improvement, and responses to particular questions asked by the Dean. Recent evaluations include the Departments of Epidemiology (2001), Environmental and Occupational Health (2001), Biostatistics (2002), and International Health (2004). The Departments of Behavioral Sciences and Health Education and Health Policy and Management are slated for an evaluation in 2005. Reports are available on site.
3. CEPH Accreditation. Accreditation is the measure of the school’s ability to “prepare professional personnel to identify, prevent and solve community health problems,” and therefore to achieve its teaching, research, and service goals.
· Measure: Accreditation
· Performance: The school was accredited for 7 years in 1997.
4. Ranking among schools of public health. RSPH seeks to enhance its reputation among the top schools of public health in public media that address the subject. For example:
· Measure: Ranking among schools of public health by U.S. News & World Report
· Performance: The school’s current ranking is 9.
X.A.3 Assessment of the extent to which this criterion is met
Strengths
· The school has a process for annually evaluating progress on objectives.
· Department-level objectives are considered in the school-wide planning process.
· The school uses quantitative information for quality improvement and for revising its objectives.
· The school monitors the effectiveness of its evaluation and planning activities through both internal and external examination of its performance.