Home > Evaluation Methods > Think-Aloud Protocol

Think-Aloud Protocol

Description of the method:

A think-aloud protocol is a method in which the participant verbalizes while they are completing the task.  The user is asked to voice their thoughts, feelings, and opinions during the evaluation. This method in used frequently in single user performance evaluations.  It was originally applied for cognitive psychologists to obtain data about the way in which humans cognitively process information (Ericsson & Simon, 1980). 

Although some maintenance technicians may spontaneously respond aloud to something they see or do, it is most likely that they normally do their work silently.  Getting them to “think aloud” will probably seem unnatural, and for some uncomfortable, so it may take some prompting and reminding during the study.  It is important that the evaluator let the technician know that they are interested in what they are thinking about when performing the task – not just a description of the task itself 

Examples of a Script for User Evaluations using a think-aloud protocol and Think-Aloud Protocol Guidelines are in the Tools and Templates section.

Development Lifecycle Stage: 

Think-aloud is used during the user performance evaluation method of the procedure development. 

Number of Users Required: 

See User Performance and Co-discovery Evaluation methods

Type of Users:  

See User Performance and Co-discovery Evaluation methods

Evaluator Skills required to Use the Method: 

This method requires a great deal of creativity and skill from the evaluator in that some this method may seem unnatural to the participants; therefore, the best case would be a specialist in user performance evaluations.  Interacting with the participant appropriately is a difficult skill to master and should not be undertaken lightly.  Even a sigh at the wrong time can influence results and render some or all of the results useless.

Many times the evaluator will not know exactly what areas will be problematic; therefore, the evaluator must be alert to the technicians’ actions, expressions, as well as dialogue.  This ability will aid the evaluator in probing the maintenance technicians’ thoughts and concerns at the appropriate point in the evaluation.  Think-Aloud Protocol Guidelines for User Performance Evaluations is available in the Tools and Templates section.

Advantages of method: 

• The think-aloud technique captures preference and performance data simultaneously rather than having to ask preference questions later.

• Thinking aloud may help some participants focus and concentrate by falling into a rhythm of working and talking throughout the evaluation

• Gives a better understanding of the user’s mental model because you are continually getting clues about misconceptions and confusion before they manifest as incorrect behaviors

Disadvantages of Method: 

• Thinking aloud slows the thought process increasing mindfulness, which might prevent errors that might have normally occurred in the actual work setting. 

• Thinking aloud may seem unnatural and distracting to some participants since it may be very different from their learning style.

• It is exhausting to verbalize a thought process for two to three hours while performing a maintenance procedure.

• The best results require a trained evaluator in this technique.

Level (or amount) of User and Evaluator Interaction:

High level of User and Evaluator interaction since the Evaluator must monitor and often probe further for clarification. 

Data Recording Method(s): 

It is helpful to have an audio and/or video recording of the session to review details of the session during the analysis period. 

How to Run the Test:

You begin by providing the user with a scenario of tasks to be completed, e.g. a new maintenance procedure.  Ask the user to perform the tasks on the aircraft using the written documentation and explain what they are thinking about while working with the procedure.  Guidelines for using a think-aloud protocol are in the Tools and Templates section. 

Related Tests: 

Constructive Interaction is identical to Think-Aloud with two people interacting simultaneously.  As you might expect, user cooperation is essential.  Although the situation is more casual, it may not be as effective due to the different strategies that users may have to complete the task.

References / Where to Learn More:

See References and Useful Resources.