Program Review Advisory Committee (PRAC)

Handbook (living document)

Last updated May 2016

Mission

The mission of PRAC is to provide assistance to programs developing their Program Review self-study report and MOU, to provide PR training for faculty in programs scheduled for PR, and to liaise with the administration in PR-related issues to ensure that programs are treated fairly.

Committee Composition

The PRAC consists of a 3 – 4 member executive committee, chaired by a UHH Faculty Congress member, and a full committee comprised of faculty in programs scheduled for PR in the next two years, and of faculty in programs that completed PR in the previous year.

Continuity of Efforts

The PRAC current chair should have received from the previous chair a document or spreadsheet identifying programs recently passing through PR, and those for the next year or two, along with contacts/representatives serving on the PRAC full committee.

Annual Tasks that the PRAC Executive Committee Needs to Fulfill

Fall Semester

  1. Contact programs scheduled for PR in the upcoming two years to:
  2. Recruit faculty members to sit on the PRAC full committee. The intent is for these faculty members to gain experience in the PR process by servingon the committee. If the continuity of members holds true, only programs coming up for review two years in the future will need to be contacted. The intent is that faculty serve three years on PRAC, two prior to their review, and the year subsequent to their review, to give upcoming programs the benefit of their experience.
  3. Remind them that their PR is coming soon. For programs scheduled for PR next year, this should be a reminder, as they were already contacted the previous year. The schedule of program reviews is available on the VCAA website at this is the responsibility of the VCAA, but this is a PRAC service to the faculty.
  4. Fall or Spring: Set up introductory advisory meetings with programs scheduled for review next year. Invite the PRAC full committee to attend, as this is where they first learn what to expect.
  1. Inform the IR as to which programs will need their quantitative data this year. This data is due from the IR to the program no later than June of the review year (the year prior to the report being due). Ideally, this information is provided to every program, not just those scheduled for review. Note: the IR needs the program's B-budget to provide the quantitative data.
  2. Send out a reminder to all programs to submit annual updates on their progress following their MOUs (optional). Programs submit these reports to the VCAA, through PRAC, to gauge progress on their MOU and to identify new issues that need addressing. If there are any major concerns included in the report, PRAC should offer its services as faculty congress representation for the program in their negotiations with the administration. In this manner, small programs in particular are not left fighting their own battles, but can rely on the full weight of the Congress to help on their behalf. These annual reports will be posted on the VCAA PR website.

These annual reports are a critical PRAC concern, as this may be the only way the MOUs will remain current throughout the period leading up to the next PR. Further, these short reports can be used to make the full PR report easier to compile. This only applies to programs that have completed PR under the current guidelines. Eventually, this will include all programs.

SpringSemester

  1. Organize PRAC meetings with programs currently completing the PR process, subsequent to their external reviewer submitting their report, to discuss the MOU prior to submitting it to the VCAA. The reasons are two-fold. First, to assist the program with the MOU, to ensure it addresses the program’s concerns and that none of the items included is inappropriate based on PRAC experience in past requests. Second, to provide training and exposure of the MOU process to the PRAC full committee members. This is an opportunity for the committee to review and discuss the external reviewer’s report, and to discuss the MOU development process.
  2. Add to the compilation that should have been provided by the previous chair, the names and institutions of all external reviewers, including how they were identified. This will be useful for future programs attempting to identify an external reviewer.

Advising Programs scheduled for Program Review

One of the primary responsibilities of PRAC is to assist departments and programs with their PR process. Historically, Program Review is not something that programs engage in on a regular basis and the large amount of work involved tends to make faculty nervous. PR is the program’s opportunity to tell others what it is they do and how well they do it, and to reflect on how best they may engage in continuous improvement. It is PRAC’s responsibility to convey this to the programs. As PR moves to a rigidly adhered to 5-year schedule, the importance of this PRAC responsibility will undoubtedly diminish.

In the Fall or early Spring of the year immediately preceding the scheduled review, PRAC executive committee members should meet with those responsible for the upcoming program review, and invite the full committee membership as well, particularly those who have not previously attended such a meeting. Materials should be provided in one form or another to the program to assist them in locating necessary materials, and discuss the following items.

  1. A link to the PR guidelines, which includes all instructions and templates, can be found at the top of the VCAA’s website, and the actual document at this point in time is stored at the following url:
  2. Emphasize that the first couple of pages contain an outline of the sections that should be included. Templates and helpful suggestions are then included in the Appendices.
  3. A schedule of when certain tasks need to be completed. Such a schedule is located within the PR document mentioned in the previous bullet. Programs should pay particular attention to AY 4, the year immediately preceding their review. It is important to note that programs should begin compiling their review no later than Spring of the year preceding their review.
  4. Inform programs thatIRhas been notified and is required to provide them their quantitative data no later than June.
  5. Examples of program review documents from recent previous reviews, including examples of the MOU. These are also available on the VCAA PR website.
  6. A discussion on how to use the quantitative data provided by the IR office. The data is used to identify how resources are being used (SSH and FTE), and should support claims in the report on how the program meets its mission. The data also indicates the numbers of majors graduating, percent of adjunct faculty to full time faculty, and trends.
  7. A discussion on what is expected regarding student learning (i.e. assessment). Appendix C outlines assessment and refers to Appendices F – I for more examples and templates to assist assessment. Departments should have been using some of this material prior to the year preceding their review. If not, it’s a bit late. PRAC Executive committee members should refer to this material when it formulates the PRAC full membership committee.
  8. External Reviewer purpose and process. In August the program chair submits to the dean prospective name(s) of external reviewers.Inform them that they should work with a prospective reviewer to gauge their interest and availability, but that the official invitation letter comes directly from the VCAA. When the program was initially notified by the VCAA of their pending review they should also have received funding for the external reviewer.

Many programs have no experience in identifying an external reviewer. Some programs have national organizations that supply names of prospective reviewers. As time goes on PRAC should compile the methods by which programs identify their reviewers. For programs that seem to be lost in the dark, get them started early. Search for similar programs from peer institutions in Hawaii or more likely, the West Coast. Suggest that faculty contact similar West Coast programs (e.g. Cal State Universities, Oregon State, and Washington State).

  1. External Reviewer on-site schedule: Usually the external reviewer will want to meet with various people and organizations during their visit. The following are provided as examples and are intended to help design the onsite visit schedule.
  2. Program Chair
  3. Program Faculty
  4. Students – either a student club or a group of advanced students that can provide them insight into their experiences. Recent alumni as well if possible.
  5. A classroom lecture is sometimes requested, to see faculty in action.
  6. Intern or service learning experiences. Labs, farm (for Ag), Four Winds (for MARE), etc.
  7. The ALO or the PRAC executive committee (to give them an idea of the program review process at UHH)
  8. Division Chairs and Administration (e.g. Dean and VCAA)
  9. Department Chairs of departments that support the instruction for the programs.
  10. Community organizations, if any, that have close ties to the program.
  11. Others of their choosing.

The following fact sheet is also useful for providing to each program during the initial meeting.

Program Review Fact Sheet

The schedule identifying when your program is up for review is on the VCAA PR web site

In the Academic Year prior to the Formal Program Review:

Spring Semester:Dean and/or Chair and the Faculty begin compiling the Program Review January 15: VCAA formally notifies the Dean of the College and/or the Chair of the Department of upcoming Program Review (to include appropriation of resources for Program Review).

March 1:IR provides comprehensive institutional data to Department Chair and faculty for review.

*August 31:The Dean or Department Chair submits prospective names and schedule for external review.

In the Academic Year of the scheduled Program Review:

Fall Semester:Dean or Chair and Faculty finalizes compiling the Program ReviewExternal Reviewer visits and evaluates the Department or Program (Alternately, the External Reviewer visits and evaluates in early Spring, which compresses the subsequent timeline below. Programs are encouraged to schedule the External Reviewer visit for the Fall.)

Spring Semester:External Review and the Drafting of the MOU

January15: External Reviewer will have submitted findings to the College, Department, or Program as well as to the Academic Program Review Advisory Committee and the Office of the Vice Chancellor for Academic Affairs. The College, Department, or Program will distribute this report to their faculty for input. The Academic Program Review Advisory Committee may also provide recommendations and suggestions for further consideration by the faculty in the unit undergoing Program Review.

March1: The Department or Program collectively responds to the reports from the external reviewer and Academic Program Review Advisory Committee, either for inclusion in the final report or for use during discussions with the administration regarding the MOU.

April15:The VCAA, Dean, Chair, and faculty in the Department begin drafting an Action Plan resulting in an MOU.

May15:MOU is signed and the action plan starts the next 5-year cycle for improvement.

Resources you may wish to peruse and/or utilize:

  • The PR guidelines document that describes the sections you need to include, and the appendices that include templates and suggestions can be found at
  • Example of a recent Program Review Report that was submitted by the Math Department for its 2013 review can be found at Note: The new PR document was not in place when Math submitted its report, but the format is very close to the new format. Others are also available on the VCAA PR website, and more will become available as the years proceed.
  • Assessment: Assessing student learning plays a much larger role in program review than in the past, per WASC requirements. Appendix D, along with its multiple appendices may be very useful in developing the Assessment portion of your report, as well as providing templates for continuing assessment efforts at the program level.
  • External Reviewer: Choosing the right external reviewer is critical to the success of your review. Some national academic organizations have lists of experienced external reviewers from which you can identify a possible prospect. If not, you will need to begin searching out someone with experience that fits well with what they will find here. Remember, it is often the case that suggestions from the External Reviewer will come from the perspective of their home institution. Looking for schools with similar size and mission is a good bet. The VCAA sets a funding limit of about $1500, although under special circumstances that limit can be raised. What this means is that most likely you will look for an external reviewer on the West Coast, which decreases travel costs and time. The university has a list of “peer” schools, but the list makes no sense. As of the time of this writing the Chancellor’s executive council was looking into the matter. It is a safe bet that you are on your own in this regard. PRAC can help.

Examples of techniques programs have used to identify an external reviewer:

-Some programs have national organizations that keep lists of reviewers.

-Call the department secretary at a smaller UC campus such as Monterey Bay, San Marcos, Channel Islands and ask if they know of any good reviewers. Some departments used this technique, looking for similar sized universities in Washington or Oregon.

-One department used HPU (not sure this is a great idea, as our missions are quite different)

-Check with contacts known within the department, see if they know of any good reviewers.

-One department found online reviews from reviewers who posted their reviews, contacted them, and chose one that seemed to best fit. This worked well.

  • Letter to the External Reviewer: Ideally you will contact the external reviewer with an inquiry as to their interest and willingness to perform the task. It is also suggested that you discuss a time frame. Ideally external reviews are done in the Fall, so early November is a reasonable cutoff date. If that cannot be achieved, then as early in Spring as possible is necessary in order that other deadlines be met (e.g. developing and signing the MOU). The official invitation letter is generated by your division or college secretary (they can contact the VCAA secretary if they need help), then sent to the VCAA, who then sends the letter directly to the reviewer.
  • Memorandum of Understanding (MOU): Subsequent to the external reviewer’s visit and report (they usually have a short timeline to submit the report, no more than 2 weeks), the Dean, working with the VCAA, compiles a list of items that are important to the program, including recognizing strengths and setting goals for addressing weaknesses or areas that require more resources to expand. The program negotiates through a back and forth process what they think should be included in the MOU. All program faculty, the Dean, and VCAA then sign the MOU. Be careful, you are the only ones who know your program well. It is often the case that the external reviewer makes suggestions based on their experiences elsewhere, and these suggestions are not always appropriate for UHH or for your program’s mission. Yet, the VCAA and Dean may include these suggestions in the MOU. PRAC is here to help with the MOU process. It is suggested that you request PRAC to review the proposed MOU to help identify items you may have missed, or items that may not be appropriate.

Examples of MOUs can be found at

All program MOUs are there, so you can choose a program close to your own.

  • Annual Progress Report: In the past the MOUs ended up on the back shelves, never to be reviewed again. UHH Faculty Congress is addressing this issue by offering each program the opportunity to submit short annual progress reports that identify how the issues in the MOU are being addressed, and to allow programs the opportunity to address changes or problems that may have occurred since the last program review. Annual assessment reports can also be included in these reports, thereby again decreasing the work load of the full report. These annual reports are optional, but if you decide to submit one, please forward it to the VCAA, through PRAC. Feel free to consult with PRAC prior to submission if you desire.

This is a new opportunity, and thus far only a few programs have gone through the new process. You can see an example of Math’s annual progress report for 2013-14 close to the bottom of the page, just above Communications MOU at

As you can see, you should begin gathering your information as soon as possible. Ideally this task is divided amongst the program’s faculty members so that everyone has a stake in the review. The PR process is designed to give the program the opportunity to reflect on what it does and how well it achieves its goals, and as such it should involve all faculty members. The WASC orientation is for PR to provide the opportunity to identify weaknesses and to develop instruments to address those weaknesses. PRAC orientation adds to this the opportunity to highlight successes as well.