Performance-Based Assessment (PBA)
Materials
Table of Contents
Student Materials Pages
Performance-Based Assessment -- Description of Expectations and Testing Procedures**Information shared with students prior to testing / 2-3
Evaluation of Student Encounter by the Senior Volunteer (SV) Feedback Form*
*Copy shared with students prior to testing / 4
Communication Questionnaire*
*Pre-assessment information collected from students prior to testing. Not a critical component of the curriculum. The information was collected to gain student self-report information regarding their levels of communication apprehension and competence as well as their thoughts regarding communication instruction inclusion within the medical school curriculum. / 5-9
Evaluation of Encounter by the Student -- Feedback Form*
*Self-evaluation and processing information completed by students directly after completing the performance-based assessment / 10-13
Post-Assessment Student Feedback Information*
*Summary information provided to all students regarding class performance / 14
Volunteer Materials Pages
Performance-Based Assessment -- Description of Testing and Volunteer Requirements**Information shared with potential volunteers to inform them of the program and recruit participants / 15
Volunteer Training Agenda / 16
Checklist:
Evaluation of Student Encounter by the Senior Volunteer (SV) Feedback Form*
*Copy shared with volunteers at training and used as the checklist during the PBA / 17-18
Checklist Training Guide -- Evaluation of Student Encounter by the JKV Volunteer
Feedback Form*
*Detailed guide to the checklist providing example behavioral descriptors for the ratings for each item. / 19-22
Sample Checklist – Evaluation of Student Encounter by the JKV Volunteer
Feedback Form*
*Sample completed checklist – During the training session volunteers view a videotape student interview conducting an interview. Volunteers practice rating the student using the checklist. Once the process has been discussed, a sample completed checklist (similar to the one provided in this packet) should be distributed to discuss examples of how to provide feedback, how to complete the form, and how to consistently evaluate student performance. / 23-24
Performance-Based Assessment
Description of Expectations and Testing Procedures
The goal of the Performance Based Assessment (PBA) is to evaluate medical students' ability to prepare and execute an abbreviated life history interview. In the execution, we expect students to present competent communication skills, elicit meaningful responses, ask questions in a logical manner, and create a constructive communication environment. It is important that students have a basic mastery of these skills as they begin working more closely with patients. Your task is to conduct an interview with a senior volunteer. The senior volunteer will provide feedback to us about your communication behavior. The evaluation of student performance by senior volunteers will help us identify communication needs.
The testing will be conducted at John Knox Village. The date for testing will be November 16th. The testing time commitment will be approximately 1 1/2 hours. (Be sure to factor in travel time before and after the test when scheduling other activities.) In the time period you will:
1. Check in for the testing
2. Receive a brief orientation
3. Conduct your interview
4. Process the interview
You should come prepared with interview questions crafted in such a way as to conduct a logical interview targeting a specific theme or themes in 20 minutes. It is important to keep in mind that the interview is an abbreviated life history. The topic options that you could focus on range from family to work to hobbies and interests. While you may ask medical questions, the interview will/should not be a medical history. Medical questions should tie back to a specific life history theme. There is no right or wrong topic and no magic number of topics that should be addressed. What will be assessed is your ability to direct the interview in a logical manner resulting in life history information surrounding a theme or connected themes.
The evaluation focuses on three components: (1) Preparation, (2) Execution, and (3) Process. You should come prepared with a set of interview questions (also known as an interview schedule -- see sample below). Come to the testing with 1 typed copy of your interview schedule. You will turn the copy in at the Check-in Station. It is important to know your questions and practice prior to the test. You are not allowed to bring copies of your questions into the test. While you will have prepared questions, you can deviate from the questions during the interview if it seems natural and appropriate. During the interview, your performance will be evaluated. A copy of the evaluation items is provided in the syllabus. If you would like an additional copy, please let me know. The senior volunteers will provide feedback using these items. The last part of the assessment will involve your analysis of the interview. During the last component of the testing you will critique your performance, explain insights you gained, share themes that emerged during the interview regarding your senior volunteer's life history, modifications you would make to the interview schedule, and areas you would like to improve on.
If you would like to share your interview schedule prior to testing, practice interviewing, or discuss any part of this assessment prior to the testing, please don't hesitate to the Office of Medical Education and Research.
______
Sample Interview Schedule
Abbreviated Life History
Topic: Career
Introduction (Think before the interview about how you plan to start the interview -- leading into your questions.)
1. I am interested in how people choose their career path. What was your occupation or career?
2. How did you pick that option/career?
3. What training did you need or receive to be successful?
-Where did you receive that training?
-What type of training would one need today?
4. What were the rewards of being a member of that occupation?
5. What were the stresses?
6. Would you pick another occupation today?
-Why or why not?
7. What advice would you give to someone just starting out in his/her career?
Conclusion (You should logically conclude the interview -- not simply end. Think about summarizing the information the interviewee provided, sharing what you are taking away from the interview, and thanking the interviewee.)
Evaluation of Student Encounter by the Senior Volunteer (SV)
Feedback Form
Student Name: ______Student #: ______
SV Name: ______Date: ______
(Check one -- *SW = Somewhat)
Yes2 / SW*
1 / No
0 / During the Interview:
1. Student introduced him or herself.
2. Student asked or used my name.
3. Student explained the purpose and goals for the encounter.
4. Student began with appropriate questions to learn about my life history.
5. Student displayed appropriate listening behaviors (i.e., no interrupting, not focused on notes, not focused on watch).
6. Student maintained eye contact.
7. Student displayed appropriate facial expressions.
8. Student asked clarifying questions.
9. Student expressed understanding (e.g., rephrased answers, nods, smiles).
10. Student had a logical flow to the questions asked in the interview.
11. Student did not belittle or talk down to me.
12. Student showed interest in me as a person.
13. Student acted bored.
14. Student presented him or herself in a professional manner.
15. I felt comfortable talking with this student.
How did the student end the interview? (Check all that apply)
_____ Student just ended abruptly. _____ Others: ______
_____ Student summarized interview. ______
_____ Student asked a final question. ______
_____ Student thanked me for my time. ______
Rate the student's ability to obtain information during the life history:
Poor Excellent
1 2 3 4 5 6 7 8 9 10
Rate the student's interpersonal communication skills:
Poor Excellent
1 2 3 4 5 6 7 8 9 10
Comments
Strengths:
Areas for Improvement:
Student Name - Print: ______Sign: ______
Student Number: ______
Age: ______Sex: M F Ethnicity: ______
(Circle One)
Communication Questionnaire
The purpose of this questionnaire is to determine the general communication abilities, needs, and perspective of medical school students as we incorporate communication training and assessment into the curriculum.
While the purpose of this questionnaire is for curriculum development and assessment, are you willing to allow the Office of Medical Education and Research to use this information for research purposes as well? All information will be kept confidential and reported in aggregate form. Your participation is greatly appreciated as it will contribute to academic understanding of medical education issues. Please indicate below your willingness to participate by checking one of the following options:
_____ Yes, you can use this information for research that may result in a presentation or
publication.
_____ No you cannot use this information for research, you can only use it for curriculum
development.
PART A
Directions: Please provide your opinion regarding the incorporation of communication training into the medical school curriculum. Your insights are critical to the development of a quality program. Record your answer to the question in the spaces provided or circle the number that represents your opinion.
8. Have you taken a communication course as part of your undergraduate coursework?
_____ No _____ Yes If so, what type of course? ______
9. Should a communication course be required as part of the undergraduate coursework for medical students?
Very Important 1 2 3 4 5 Not Very Important
to require a course to require a course
10. How important is general communication training offered as part of the medical school curriculum?
Very Important 1 2 3 4 5 Not Very Important
11. How important is communication training when tailored for the practice of medicine?
Very Important 1 2 3 4 5 Not Very Important
12. How important is communication training in the area of interacting with difficult patients?
Very Important 1 2 3 4 5 Not Very Important
13. How important is communication training in the area of breaking bad news?
Very Important 1 2 3 4 5 Not Very Important
14. How important is communication training in the area of interviewing?
Very Important 1 2 3 4 5 Not Very Important
15. How important is communication training in the area of conflict resolution?
Very Important 1 2 3 4 5 Not Very Important
16. How important is communication training in the area of interpersonal relationships?
Very Important 1 2 3 4 5 Not Very Important
17. In what areas do you feel you need communication training? Please list: ______
______
______
18. In what communication areas do you feel already proficient? Please list: ______
______
______
19. How effective is your clinic experience in providing communication training?
Very Effective 1 2 3 4 5 Not Very Effective
20. How important is combining structured communication training with clinical communication experiences?
Very Important 1 2 3 4 5 Not Very Important
21. Given the introduction of performance based assessment into the USMLE, how often do you feel your communication and clinical skills should be reviewed using a performance based assessment format at UMKC in order to adequately prepare you for passing a performance based assessment component of the USMLE?
Ideal number of assessments: ______
22. Are there any other comments or ideas regarding communication training you would like to share? Please share your ideas here or on the back of this sheet: ______
______
______
PART B
Directions: Please honestly and accurately answer the following questions regarding your communication behavior. Circle the number that best describes your perception of your communication behavior.
Strongly Agree Agree Undecided or Neutral Disagree Strongly Disagree
1 2 3 4 5
SA SD
1. I find it easy to get along with others. 1 2 3 4 5
2. I can adapt to changing communication situations. 1 2 3 4 5
3. I treat people as individuals. 1 2 3 4 5
4. I interrupt others too much. 1 2 3 4 5
5. I am "rewarding" to talk to. 1 2 3 4 5
6. I can deal with others effectively. 1 2 3 4 5
7. I am a good listener. 1 2 3 4 5
8. My personal relationships are cold and distant. 1 2 3 4 5
9. I am easy to talk to. 1 2 3 4 5
10. I will argue with someone just to prove I'm right. 1 2 3 4 5
11. My conversations tend to be awkward. 1 2 3 4 5
12. I ignore other people's feelings. 1 2 3 4 5
13. I generally know how others feel. 1 2 3 4 5
14. I let others know that I understand them. 1 2 3 4 5
15. I understand other people. 1 2 3 4 5
16. I am relaxed and comfortable when speaking. 1 2 3 4 5
17. I listen to what people say to me. 1 2 3 4 5
18. I like to be close and personal with people. 1 2 3 4 5
19. I generally know what type of behavior is appropriate in
any given communication situation. 1 2 3 4 5
20. I usually don't make unusual demands on my friends. 1 2 3 4 5
21. I am an effective conversationalist. 1 2 3 4 5
22. I am supportive of others. 1 2 3 4 5
23. I do not mind meeting strangers. 1 2 3 4 5
24. I can easily put myself in another person's shoes. 1 2 3 4 5
25. I pay attention during conversations. 1 2 3 4 5
SA SD
26. I am generally relaxed when conversing with a new
acquaintance. 1 2 3 4 5
27. I am interested in what others have to say. 1 2 3 4 5
28. I don't follow conversations very well. 1 2 3 4 5
29. I enjoy social gatherings where I can meet new people. 1 2 3 4 5
30. I am a likeable person. 1 2 3 4 5
31. I am flexible. 1 2 3 4 5
32. I am not afraid to speak with people in authority. 1 2 3 4 5
33. People feel comfortable coming to me with their problems. 1 2 3 4 5
34. I generally say the right thing at the right time. 1 2 3 4 5
35. I like to use my voice and body expressively. 1 2 3 4 5
36. I am sensitive to other's needs of the moment. 1 2 3 4 5
37. I think I communicate effectively in one-on-one situations. 1 2 3 4 5
38. I dislike participating in group discussions. 1 2 3 4 5
39. While giving a speech I get so nervous, I forget facts I really
know. 1 2 3 4 5
40. Ordinarily I am very tense and nervous in conversations. 1 2 3 4 5