For Medical Education Programs

GUIDE TO THE

MEDICAL SCHOOL SELF-STUDY

FOR MEDICAL EDUCATION PROGRAMS

LEADING TO THE M.D. DEGREE

FOR SITE VISITS IN THE 2017-2018 ACADEMIC YEAR

(Published March 2016)

For further information, contact:

CACMS Secretariat

Committee on Accreditation of Canadian Medical Schools

Association of Faculties of Medicine of Canada

2733 Lancaster Road, Suite 100

Ottawa, Ontario, Canada K1B 0A9

Phone: 613-730-0687 Ext 225 Fax: 613-730-1196

Visit the CACMS website at:

https://www.afmc.ca/accreditation/committee-accreditation-canadian-medical-schools-cacms

Guide to the Medical School Self-Study

For medical education programs leading to the M.D. Degree

©Copyright March 2016 by the Committee on Accreditation of Canadian Medical Schools. All rights reserved. All material subject to this copyright may be reproduced, with citation, for the noncommercial purpose of scientific or educational advancement.

TABLE OF CONTENTS

OVERVIEW OF THE ACCREDITATION PROCESS 1

A. Purposes of Accreditation and Self-Study 1

B. Accreditation Standards 1

GENERAL STEPS IN THE ACCREDITATION PROCESS 2

A. Completion of the DCI and Compilation of Other Documents 3

B. Self-Study Analysis and Final Report Development 3

C. The Accreditation Visit and Preparation of the Visit Report 3

D. Accreditation Decisions and Follow-Up 4

TYPICAL SCHEDULE FOR A CACMS FULL ACCREDITATION REVIEW 5

A. Assistance from the CACMS Secretariat 7

COMPLETING THE DATA COLLECTION INSTRUMENT (DCI) 7

A. Supporting Documentation 7

B. Date Range 7

C. Updates 8

CONDUCTING THE MEDICAL SCHOOL SELF-STUDY 8

A. The Self-Study Task Force 8

B. Subcommittees of the Task Force 8

C. Preparation of the Final Medical School Self-Study Report 9

EVALUATION OF ELEMENTS 9

A. Instructions 9

B. Comment Field - Evidence to Support the Rating 10

C. Rating the Elements 11

D. Comment Field - Recommendations to Address Identified Problems 11

COMPONENTS OF THE SELF-STUDY REPORT 12

ii

OVERVIEW OF THE ACCREDITATION PROCESS

A.  PURPOSES OF ACCREDITATION AND SELF-STUDY

Obtaining accreditation from the Committee on Accreditation of Canadian Medical Schools (CACMS) and the Liaison Committee on Medical Education (LCME) ensures that medical education programs are in compliance with defined standards. The accreditation process has two general and related aims: to promote medical school self-evaluation and improvement, and to determine whether a medical education program meets prescribed standards.

As a process of evaluation, accreditation seeks to answer three general questions:

1.  Has the medical school clearly established its mission and goals for the educational program?

2.  Are the program's curriculum and resources organized to meet its mission and goals?

3.  What is the evidence that the program is currently achieving its mission and goals and is likely to continue to meet them in the future?

The medical school self-study process and the resulting findings are central to these aims. In the process of conducting its self-study, a medical school brings together representatives of the medical school administration, faculty, student body, and other constituencies to: 1) collect and review data about the medical school and its educational program, 2) identify areas that require improvement, and 3) define strategies to ensure that any problems are addressed effectively.

The report resulting from the self-study process provides an evaluation of the quality and effectiveness of the medical education program and the adequacy of resources to support it. The usefulness of the self-study as a guide for planning and change is enhanced when participation is broad and representative, when the results and conclusions are widely disseminated, and when the participants have engaged in a thoughtful process of analysis and reflection. Because of the time and resources required to conduct a self-study, schools should give careful thought to other purposes that may be served by the process. For example, the self-study might serve as a vehicle to familiarize a new dean, dean’s staff member or department chair with the environment and operation of the school; to initiate a curriculum review; and/or to provide the academic community at large with an opportunity to reaffirm the school’s educational mission and goals or set new strategic directions for the medical education program. A self-study process that serves multiple purposes and involves multiple constituencies is more likely to have a productive outcome related to medical school improvement than one that is conducted solely to satisfy accreditation requirements.

B.  ACCREDITATION STANDARDS

The self-study is directly linked to the standards for accreditation. The standards for accreditation of Canadian medical schools are contained in the CACMS publication CACMS Accreditation Standards and Elements (S&E).

Medical schools with accreditation visits during the 2017-2018 academic year will use the corresponding version of S&E. These standards and elements have been widely reviewed and endorsed by the medical education community, including the organizations that sponsor the CACMS.

Medical schools are expected to achieve compliance with each of the 12 standards. Compliance with a standard will be based on satisfactory performance in the elements associated with the standard. See “Accreditation Decisions and Follow-Up” below.

GENERAL STEPS IN THE ACCREDITATION PROCESS

Information provided by the medical school is considered by the medical school, the site visit team and lastly the CACMS in the context of accreditation standards. The general steps in the process are as follows:

1. Completion of the data collection instrument (DCI) and compilation of supporting documents.

2. Analysis of data from the Independent Student Analysis (ISA), the most recent graduation questionnaires, and the DCI, narrative responses and appendices, by a medical school self-study subcommittees and task force, development of self-study reports for each standard, and compilation of these updated reports into a final medical school self-study report.

3. Visit by an ad hoc site visit team and preparation of the site visit report for review by the CACMS.

4. Action on accreditation by the CACMS and LCME.

Each of the steps is summarized below and in the accompanying schedule, which shows the usual timetable for completion of each step.

Medical School Site Visit Personnel

The Dean must designate a core team of faculty and staff to manage the aspects of the site visit preparation process. The faculty accreditation lead manages the data collection and self-study processes; the site visit coordinator typically manages site visit logistics, and may assist with data collection. It is critical that both positions be staffed by individuals who have a deep understanding of the program and who will be able to work well with key individuals within and external to the medical school. Designated personnel will need the authority and experience to gather accurate information and garner widespread participation among faculty, staff, and students. Please refer to the full position descriptions below before making these designations. Schools must complete the site visit personnel designation form no later than six weeks following publication of the DCI for their respective site visit year. This will ensure that the appropriate school personnel receive updates and event notifications from the CACMS Secretariat.

Faculty Accreditation Lead

The faculty accreditation lead should be a senior faculty member, who may also hold a position of vice, associate or assistant dean or other leadership position in the medical school, who is knowledgeable about the medical school and its educational program. This individual should be able to locate medical school or university policies and information sources, explain medical school conventions, and ensure participation by members of the senior administration, faculty, and student body. Ideally, the faculty accreditation lead will be familiar with CACMS site visit processes, and will have served on a site visit team as the designated faculty fellow for his or her school.

The school must ensure that the faculty accreditation lead has appropriate administrative support, and release time from other duties in order to accomplish the responsibilities associated with this role. The faculty accreditation lead will be required to:

• Answer questions during DCI preparation

• Assign specific questions/sections of the DCI to individuals with the appropriate knowledge

• Ensure factual accuracy and typographical/grammatical clarity in the DCI

• Ensure that each aspect of multi-part DCI questions are fully-addressed

• Synthesize all narrative DCI responses into a cohesive, factually and stylistically-consistent document that accurately reflects the medical school

• Coordinate the activities of self-study subcommittees

• Staff the self-study task force

• Develop the site visit schedule in collaboration with the site visit team secretary

• Serve as the school’s primary point of contact for the CACMS Secretariat and site visit team secretary

Site Visit Coordinator

The site visit coordinator should be an experienced, senior administrative staff member who will manage the logistics of the site visit and other administrative functions such as formatting and submitting the site visit package. The site visit coordinator will normally make hotel reservations for the team, coordinate ground transportation for the visit, and schedule the necessary faculty and staff identified for sessions during the site visit.

A.  COMPLETION OF THE DCI AND COMPILATION OF OTHER DOCUMENTS

The questions in the DCI are directly linked to specific elements. The questions should be answered and the relevant documents compiled by the persons most knowledgeable about each of the topics. Care should be taken to ensure that the data and terminology are current, accurate, and consistent across the DCI (e.g., consistent abbreviations, consistent names and abbreviations for committees). The faculty accreditation lead who oversees the accreditation process at the school should ensure that the completed DCI undergoes a comprehensive review to identify any inaccuracies, missing items, or inconsistencies in reported information. The absence of a document, data, and/or information specifically requested in the DCI will be taken to mean that the document, data, and/or information do not exist.

Data from the independent student analysis, the most recent graduation questionnaire, and internal sources should be reviewed by the relevant self-study subcommittees and utilized in the development of the individual subcommittee reports and the final medical school self-study report.

While the DCI is being completed, medical students will carry out their own review of the educational program, student services, and other areas of relevance to students. While the administration may provide logistical support and assistance in analyzing the data, planning for the student survey and the interpretation of the results is a student responsibility. Students should be directed to the CACMS publication: Guide to the Independent Student Analysis (AY 2017-2018).

B.  SELF-STUDY ANALYSIS AND FINAL REPORT DEVELOPMENT

The medical school self-study task force and its subcommittees are responsible for conducting the self-study. The project as a whole should be guided by the faculty accreditation lead. Each subcommittee should review and analyze the ISA, the most recent graduation questionnaires, and the DCI data, narrative responses and appendices for the elements to which they have been assigned. In the majority of cases, the relevant documentation to evaluate the element has been requested in the DCI itself. The subcommittees carry out the first evaluation and rating of elements, provide evidence to support the rating, and develop recommendations with timelines to address areas of unsatisfactory performance. This process is described in more detail later in the guide. Subsequently, the task force analyzes the subcommittee reports, along with the supporting documentation and any new information, and revises as appropriate the individual subcommittee element ratings. The task force updates the evidence to support the rating and the final recommendations to address elements where performance is unsatisfactory and elements requiring monitoring. The task force creates the final self-study report that includes all of the element evaluation forms and the taskforce’s reflection on areas requiring improvement identified during the self study in relation to areas requiring improvement identified at the last full site visit and the intervening period. The components of the self-study report are described in detail later in this guide.

The medical school self-study report (in Word format), the ISA, the most recent graduation questionnaires and the completed DCI (in Word format) with its appendices need to be submitted to the visiting team three months prior to the visit as described on the CACMS website.

C.  THE ACCREDITATION VISIT AND PREPARATION OF THE VISIT REPORT

A full visit typically begins on Sunday evening with an entrance conference with the dean and concludes early Wednesday afternoon at exit conferences with the dean and with the university president or delegate. Schools with distributed campuses may have an additional day added to the visit. If, during the visit, the dean has concerns regarding the conduct of the visit, he or she should contact the CACMS Secretariat immediately. Prior to the visit,

the visiting team will review the materials submitted by the school in detail. At the time of the visit, the school will provide copies of these documents, as well as the individual self-study subcommittee reports, to the visiting team in print and electronic formats.

During the visit, the team will develop a list of its findings that relate to specific elements. These summary findings will be reported orally to the dean and the university official on the final day of the visit and a written copy of the team findings related to the elements will be provided to the dean. These initial findings may be revised during the process of review of the visit report. The visiting team makes neither recommendations nor decisions regarding the medical school’s accreditation status; the determination of accreditation status is the purview of the CACMS and the LCME.

After the visit, a draft report is prepared by the visiting team according to the format specified in the Site Visit Report Guide. The report evaluates the information in the DCI, the self-study report, the independent student analysis, as well as information obtained by the team during the visit, and presents the team's findings from the visit. The visit report will include the team’s judgment about findings related to elements, which will be categorized as: 1) satisfactory, 2) satisfactory with a need for monitoring, and 3)unsatisfactory.

A draft version of the report is sent by the team secretary to the CACMS Secretariat office for a preliminary review to verify that the report is complete and adequately documents the team’s findings. It is then returned to the team secretary who, in consultation with the team, will produce the draft that will be forwarded to the dean for review. The dean has 10 business days to respond to the draft report in writing (in hard copy and/or electronic format) with areas he or she believes contains errors of fact or concerns about the “tone” of the report. Information provided as part of the dean’s response must be referenced to information contained in the Data Collection Instrument or provided to the team during the visit and must refer to the time of the visit. Events occurring or actions taken by the school after the visit will not be considered in mitigation of the findings of unsatisfactory or satisfactory with a need for monitoring identified in the visit report. The dean’s comments about the site visit report are sent to the team secretary with a copy to the CACMS Secretariat. The team secretary will submit the final report to the CACMS Secretariat with a copy to the dean. Following receipt of the final report, if the dean has remaining concerns about the process of the visit, errors of fact or the tone of the report, he or she may write a letter to the CACMS Secretariat detailing these concerns within 10 business days. The information referenced must have been contained in the Data Collection Instrument or provided to the visiting team at the time of the visit. No new information, regarding events or actions taken by the school after the visit may be provided in the dean’s letter to the CACMS Secretariat and no attachments to the letter will be accepted. The dean’s letter will be provided to the CACMS when the visit report is reviewed by the committee.