Home > Evaluation Methods > Interviews & Focus Groups

Interviews & Focus Groups

Description of method:

Interviews and focus groups let you query users face-to-face about their experiences and preferences with the manual procedures. Interviews poise questions to one person; focus groups to a group of 6-10 persons. Both are formal, semi-structured events where you directly interact with users, asking them to voice their opinions and experiences regarding the project using a series of questions.

You begin by formulating questions about your project based on the type of information you need to know (See Interview Guidelines and Focus Group Guidelines). In interviews, as opposed to questionnaires, you are present to interact and facilitate discussion about the issues raised by your questions. With multiple users present, as with focus groups, the interaction among the users may raise additional issues, or identify common problems that many people experience. For both methods, general questions are supplemented by using prompts to guide the discussion you issues you want to cover.

Development Lifecycle Stage:  This technique can be used at any stage of development, depending on the questions that are asked. More likely, interviews and focus groups are held at very early stages of development, when the manual requirements are still being decided. Focus groups may be held to extract user requirements prior to initial development.

Number of Users Required:  For interviews, you will need approximately five participants from each different type of group you interview; for focus groups you should use at least two groups of participants.

Type of Users:  This would depend upon the information you need to obtain from the interviews/focus groups. For example, you are likely to find different information from less experienced and more experienced maintenance technicians.

Evaluator Skills required to Use the Method:  This method requires a great deal of creativity and skill from the evaluator; therefore, the best case would be an specialist in evaluative testing. Many times the evaluator will not know what direction the interview or focus group discussion will take; therefore, the evaluator must be alert to probing while gathering the relevant information necessary to meet the goals of the evaluation.

Although experience in user testing is advisable, planning and conducting interviews and focus group methods of evaluation may be accomplished with a careful review of the publications available. We have also included a Planning Template for user testing which may be adapted to this method, Interview Guidelines, and Focus Group Guidelines in the Tools & Templates section to guide your use of this evaluation method.

Number of Evaluators Required: 

One evaluator is needed for conducting interviews.  For focus group interviews, you will need to have one person as a facilitator to direct and guide the discussion and a note taker to record the session.  If the facility will allow for video taping, it is advisable to record the session.

 Advantages of method: 

• Interviews and focus groups are a flexible means of gathering user information throughout the process, although are primarily useful in the initial phase of development. 

• The likelihood of a participant misinterpreting the question is reduced as they may ask questions of the interviewer to clarify the meaning.

• Less preparation time than questionnaires; although specific goals and objectives should be established before formulating the questions to be asked.

• This method allows for further probing which gives more depth to the data collected.

• There is the potential for collecting a lot of information in a relatively short period of time.

Disadvantages of Method: 

• Interviews and focus groups are not methods to verify or assess the usability of the documentation

• There may be a "social desirability" bias to the answers given in interviews that would not be a factor in questionnaire responses.

• There may be the expectation “that something will be done” as a result of sharing information which may be problematic if change is not likely.

• There is the potential for interviews and focus groups to “off task”.

• Results are subject to the interview’s interpretation.  For critical tasks, try to include a second task analyst present during the interview perhaps as a note taker.

• The costs associated with interviews and focus groups are higher than questionnaires.

• Having the evaluators present may unrepresentatively moderate opinion or response.

Level (or amount) of User and Evaluator Interaction:

There is a high level of participant and evaluator interaction in the interview method.  The focus group is more moderate interaction due to the others involved in the discussion although the format is still interactive between evaluator and group.

Data Recording Method(s): 

A voice or video recorder may be used during interviews or focus groups if it is unobtrusive; however, in interviews, recording the session may change the dynamic of the communication.

Total Testing Time Required:

The total testing time for interviews/focus groups will be dependent upon your ability to recruit the number and type of participants you determine are required to meet your evaluation goal.  Most likely you will be able to plan, conduct, analyze, and report your results will be two to three months. 

Testing Time Per User:

For interviews, plan on 30 minutes to 1 hour maximum per person; for a focus group, you should allow 1 to 1 ½ hours per session.

Typical Output from Test:

The output from interviews is qualitative, subjective data, but is valuable in that it can be evaluated to identify areas where problems exist and areas to focus user performance evaluations. 

How to Run the Test:

Refer to Interview Guidelines and Focus Group Guidelines in the Tools & Templates section for instructions on planning, conducting, analyzing and reporting data.

Required Testing Materials:

Both interview and focus group evaluations require a prepared script, a copy of the script with space to record the answers, a copy of the procedure and/or aircraft component/illustration to be discussed, a consent form, background questionnaire, extra paper for participants to illustrate graphically, and audio/video equipment (optional).

References / Where to Learn More:

See References and Useful Resources.

Cost to Conduct Test:

The costs to conduct interviews are moderately high and focus groups are higher due to more participants and the potential need for audio-video equipment or another person for note taking. These evaluations are less costly than user performance testing but more than a questionnaire method. 

Goals of Testing:

The goal of interviews and focus groups is to probe users’ attitudes, beliefs, and desires.  They don’t give you actually information as to what the user would actually do – the data are what people say they think or do or need. 

This type of information is valuable when you need to know how the technician performs tasks and to find out what types of information are critical to their success.

Subjective or Objective Test:

These are subjective tests which collect qualitative data.

Turnaround Time:

Interviews and focus groups can be conducted within weeks depending upon the schedules of the user groups selected.  Information from these evaluations can be used along the way to probe new areas or find areas which need additional testing using other evaluation methods.

Focus of Evaluation:

The types of issues which can be evaluated are narrow due to the subjective nature of these evaluations.  However, as a tool to gain more insight, the topics can be evaluated in much more depth.