Home > Tools & Templates > User Performance Evaluation Script

 

User Performance Evaluation Script

Opening Statement:

(Meet the participant and escort into the room)

“Before we begin I would like to thank you for agreeing to help. My name is _________________________ and I will be facilitating this session. (Be sure to introduce everyone in the room and explain their role.)  Your participation in this study is very important. Your comments and insights will help _____________ develop a more usable maintenance manual. Before we begin I would like you to fill out a consent form. (Allow participants to keep a copy of the consent form if requested) Also, can you please fill out this background questionnaire? This information will help us better understand who you are.”

Handout

Consent form

Background Questionnaire

Directions for testing session

“Before we begin, I would like to remind you of your right to withdraw from participation at any time. Should you choose to exercise this right; no repercussions or questions will be asked.”

“Your participation is also confidential.  Any answers or discussions from this interview will not associated with your name.  We are collecting data from several participants and will evaluate the results collectively.”

“During this session you will be given a maintenance task to complete. I will not be able to answer any questions regarding task completion; however I can clarify wording. I would like to emphasize this is not a test, we are evaluating the procedure, not you. Any questions so far?”

“Pretend you are at work and try to forget I/we am/are here.  I would like you to discuss the scenario/situation and potential courses of action as you move through the task.

“Do you have any questions before we begin?”

 

Hand out the task

Step #1

Step #2

Step #3

Step #4

Step #5

General questions

 

(When all steps are completed, ask the participant a set of prepared questions.  Note:  These questions may also be included in a Satisfaction questionnaire.  Even so, you may want to probe further to make sure you understand why they responded in a certain way.)

·        So, what did you think?

·        How easy or difficult was the procedure to use?

·        What was the easiest step? Most difficult?

·        Was the procedure easier or more difficult than the most of the tasks you perform?

·        Did you find terminology within the steps easy to understand?

·        If you had to pick one thing you liked about the procedure what would it be?

·        If you had to pick one thing you disliked about the procedure what would it be?

·        If you were on the technical publication staff, what would you like to see changed in the maintenance procedures?

·        I noticed step number ____ was problematic. Can we return to this and discuss it further?

·        I noticed _____step caused some confusion. Can we go back and review this step so I can better understand problems with the writing.

·        I noticed the word __________ caused some confusion, what do you think would be a more meaningful word?

·        I noticed you made a comment concerning ________ can you elaborate on that?

Others…?

Handout

3.  User Satisfaction Questionnaire

Closing Statement

Your comments and insights were very valuable to us. Once again I would like to thank you for your time and participation in our study. (Escort the participant out)

 

 

 

 


Human Factors at NIAR | Human Factors at FAA | Human Factors Psychology at WSU

Human Factors Laboratory, National Institute for Aviation Research at Wichita State University. Research funded by the Federal Aviation Administration.  All rights reserved.
Revised: 11/05/04