Home > Evaluation Methods
> Think-Aloud Protocol
Description of the method:
A think-aloud protocol is a
method in which the participant verbalizes while they are completing the task.
The user is asked to voice their thoughts, feelings, and opinions during the
evaluation. This method in used frequently in single user performance
evaluations. It was originally applied for cognitive psychologists to obtain
data about the way in which humans cognitively process information (Ericsson &
Although some maintenance
technicians may spontaneously respond aloud to something they see or do, it is
most likely that they normally do their work silently. Getting them to “think
aloud” will probably seem unnatural, and for some uncomfortable, so it may take
some prompting and reminding during the study. It is important that the
evaluator let the technician know that they are interested in what they are
thinking about when performing the task – not just a description of the task
Examples of a
Script for User Evaluations
using a think-aloud protocol and
Think-Aloud Protocol Guidelines are in the
Tools and Templates section.
Development Lifecycle Stage:
Think-aloud is used during the
user performance evaluation method of the procedure development.
Number of Users Required:
and Co-discovery Evaluation
Type of Users:
and Co-discovery Evaluation
Evaluator Skills required to
Use the Method:
This method requires a great
deal of creativity and skill from the evaluator in that some this method may
seem unnatural to the participants; therefore, the best case would be a
specialist in user performance evaluations. Interacting with the participant
appropriately is a difficult skill to master and should not be undertaken
lightly. Even a sigh at the wrong time can influence results and render some or
all of the results useless.
Many times the evaluator will
not know exactly what areas will be problematic; therefore, the evaluator must
be alert to the technicians’ actions, expressions, as well as dialogue. This
ability will aid the evaluator in probing the maintenance technicians’ thoughts
and concerns at the appropriate point in the evaluation.
Protocol Guidelines for User Performance Evaluations is available in
the Tools and Templates
Advantages of method:
The think-aloud technique
captures preference and performance data simultaneously rather than having to
ask preference questions later.
Thinking aloud may help some
participants focus and concentrate by falling into a rhythm of working and
talking throughout the evaluation
Gives a better understanding
of the user’s mental model because you are continually getting clues about
misconceptions and confusion before they manifest as incorrect behaviors
Disadvantages of Method:
Thinking aloud slows the
thought process increasing mindfulness, which might prevent errors that might
have normally occurred in the actual work setting.
Thinking aloud may seem
unnatural and distracting to some participants since it may be very different
from their learning style.
It is exhausting to
verbalize a thought process for two to three hours while performing a
The best results require a
trained evaluator in this technique.
Level (or amount) of User and
High level of User and
Evaluator interaction since the Evaluator must monitor and often probe further
Data Recording Method(s):
It is helpful to have an audio
and/or video recording of the session to review details of the session during
the analysis period.
How to Run the Test:
You begin by providing the
user with a scenario of tasks to be completed, e.g. a new maintenance
procedure. Ask the user to perform the tasks on the aircraft using the written
documentation and explain what they are thinking about while working with the
procedure. Guidelines for using a think-aloud protocol are in the Tools and
Constructive Interaction is
identical to Think-Aloud with two people interacting simultaneously. As you
might expect, user cooperation is essential. Although the situation is more
casual, it may not be as effective due to the different strategies that users
may have to complete the task.
References / Where to Learn