User Performance Evaluation Script
using a Think-aloud Protocol
Opening Statement: / (Meet the participant and escort into the room)“Before we begin I would like to thank you for agreeing to help. My name is ______and I will be facilitating this session. (Be sure to introduce everyone in the room and explain their role.) Your participation in this study is very important. Your comments and insights will help ______develop a more usable maintenance manual. Before we begin I would like you to fill out a consent form. (Allow participants to keep a copy of the consent form if requested) Also, can you please fill out this background questionnaire? This information will help us better understand who you are.”
Handout / 1. Consent form
2. Background Questionnaire
Directions for testing session / “Before we begin, I would like to remind you of your right to withdraw from participation at any time. Should you choose to exercise this right; no repercussions or questions will be asked.”
“Your participation is also confidential. Any answers or discussions from this interview will not associated with your name. We are collecting data from several participants and will evaluate the results collectively.”
“During this session you will be given a maintenance task to complete. I will not be able to answer any questions regarding task completion; however I can clarify wording. I would like to emphasize this is not a test – we are evaluating the procedure, not you. Any questions so far?”
“Pretend you are at work and try to forget I/we am/are here. I would like you to discuss the scenario/situation and potential courses of action as you move through the task.
(Instructions for Think-aloud Protocol) “We want to know what you expect to happen when you are going through the procedure, so we would like for you to talk as you are going through it. For example, we want to know what surprises, confuses, or frustrates you and why. When you share with us what you’re thinking as you go along, we get a better understanding of how the process works for you.”
“Do you have any questions before we begin?”
Hand out the task / Step #1
Step #2 / Insert steps here
Step #3
Step #4
Step #5 / Number of steps depends on project
(modify as necessary)
General questions / (When all steps are completed, ask the participant a set of prepared questions…)
· So, what did you think?
· How easy or difficult was the procedure to use?
· What was the easiest step? Most difficult?
· Was the procedure easier or more difficult than the most of the tasks you perform?
· Did you find terminology within the steps easy to understand?
· If you had to pick one thing you liked about the procedure what would it be?
· If you had to pick one thing you disliked about the procedure what would it be?
· If you were on the technical publication staff, what would you like to see changed in the maintenance procedures?
· I noticed step number ____ was problematic. Can we return to this and discuss it further?
· I noticed _____step caused some confusion. Can we go back and review this step so I can better understand problems with the writing.
· I noticed the word ______caused some confusion, what do you think would be a more meaningful word?
· I noticed you made a comment concerning ______can you elaborate on that?
Others…?
Handout / 3. User Satisfaction Questionnaire
Closing Statement / Your comments and insights were very valuable to us. Once again I would like to thank you for your time and participation in our study. (Escort the participant out)