PUBLICATIONS

Transfer of Training from Virtual Reality Environments

This study evaluated the transfer of training and training efficiency of two virtual reality environments (head-mounted display and personal computer) for a complex manual assembly task. Transfer of training was measured by comparing the post-training performance of two virtual training groups, a real-world training group and a control group that received (no training. All training groups were taught to assemble a Lego™ forklift model in their respective environment. After training, participants assembled a real-world model of the forklift as well as a novel model of a racecar, which required the same parts as the forklift assembled in a different configuration. Results from this study show that virtual environments can be effective training simulators for complex assembly tasks although they are less efficient than real-world training. The results also suggest that individual differences such as general intelligence, spatial aptitude and computer user self-efficacy influence one's ability to learn in a virtual environment.

Evaluation of Pilot Performance and Workload During a "Black Tube" Approach to Land

As glass cockpits become more ubiquitous in commercial and general aviation aircraft, it is important that aircraft manufacturers consider which standby instruments should be available to the pilot if the display units (DU) should fail. This study measures pilot performance and workload while performing black tube approaches with two different standby instrument configurations. Subjective and objective measures show that: 1) pilot's overall performance declined when using standby instruments regardless of the instruments provided; 2) workload ratings were significantly lower during black tube approaches when a standby HSI was available; and 3) lateral deviations were significantly lower when during black tube approaches when a standby HSI was available. These results suggest that information provided by a standby HSI is critical to pilots for performing approaches to land during black-tube operations. The HSI provides a birds-eye perspective that is critical to maintaining the pilot's situational awareness during instrument maneuvers.

For more information or a copy of the projects listed above, contact christopher.hamblin@wichita.edu

FAA Industry Training Standards (FITS)

In 2002, the FAA, academic and industry partners established the FAA/Industry Training Standards (FITS) program, the purpose of which is to modernize pilot training. The program was motivated by several factors, including general aviation accident rates, the introduction of advanced avionics technology and an anticipated diversification of the aviation marketplace. The pilot training programs being developed by FITS attempt to address these issues by placing greater emphasis on the training of risk management, information management, aeronautical decision making and the use of computer-based training. The FITS program also proposes to change pilot training to make it more relevant to real-world flying by relying on scenario-based training.

The purpose of this research is to review the current state of research and technology needed to support the goals of the FITS program. This information can be used by the FAA to identify research areas that would aid achievement of FITS’ goals. This project will be accomplished through:

1) Review of the research findings published in the academic literature that is pertinent to goals/objectives of FITS. This will consist of investigations evaluating the effectiveness of different training techniques including computer-based training, integrated private/instrument curriculum, comparisons of skill-based testing and scenario-based training and testing, and human factors evaluations of the effectiveness of new displays including multifunction displays.

2) Review the goals of the FITS program with flight training schools, aircraft, avionics manufacturers, and individuals who have used or have been involved in training pilots to use this technology. This will include documenting what these groups believe are the major obstacles (training, regulatory and technical issues) in order to field these new technologies. This will ensure that future research initiatives address these potentially critical issues.

FITS Final report.pdf

Evaluation Toolbox for Avation Technical Writers

February 2004

This toolbox was developed as an aid for writers to evaluate the usability of aviation maintenance documentation from the initial development stage through the final pre-publication stage. The evaluation toolbox is a standardized set of "tools" that can be used by the aviation technical writer to find and correct problem areas for the user of technical documentation. Paper and electronic versions of the toolbox were developed.

Aircraft Maintenance Documentation Research

February 2004

Surveys and interviews reveal that general aviation manufacturers rely on users to identify problems in maintenance documentation with corrections typically initiated in response to user-reported problems found in the manual. This investigation compares techniques that manufacturers can use to improve the quality of the maintenance documentation developed by technical writing groups. The techniques, user performance and cognitive walkthrough, were used to identify problems in aircraft maintenance documentation. The techniques highlight probable mismatches between the intent of the technical writer and the interpretation of the maintenance procedures by aircraft maintenance technicians (AMTs). The purpose of this investigation was to determine whether these techniques elicit unique information from participants, to establish whether different groups of evaluators (engineers, technical writers, AMTs) identify different types of problems in the written documentation, and to identify common types of problems in the documentation. We also reviewed the heuristics technical writers use to develop maintenance documentation.

The results of this investigation show that user performance and cognitive walkthrough evaluations are complementary techniques for identifying problems with maintenance documentation, that the errors identified by individual participants vary in significant ways according to experience and training (engineers versus AMTs), and that procedure errors (e.g., sequencing, completeness) and language errors (e.g., clarity, meaning) are the most commonly cited errors in the maintenance documentation. Ways in which manufactures can employ the evaluated techniques in their manual development process are discussed.

A Comparison of Three Evaluative Techniques for Validating Maintenance Documentation

December 2002

This is a final report of a three-phase project. In this report, a series of recommendations are outlined that address a number of shortcomings in how aircraft manufacturers develop aircraft maintenance documentation. In Phase 1, a number of problems were documented, including the reactive rather than proactive use of user evaluations, the limited use of user input and procedure validation, the absence of systematic attempts to track errors, and the lack of standards for measuring document quality. In Phase 2, a survey of aircraft technicians revealed that, although user evaluations of the accuracy and quality of technical manuals are generally good, the manuals themselves were noted as having poor usability. In light of these deficiencies, it is recommended that (1) manufacturers and operators work to facilitate communication between the technicians submitting change requests and the technical writers making the changes to ensure prompt feedback regarding actions to be taken; (2) maintenance procedures be validated by technicians using standard human factors techniques; (3) industry cooperates in the development of a system akin to the Maintenance Steering Group (MSG) to identify maintenance procedures that should be systematically validated; and (4) manufacturers should maintain databases with a history of user-reported errors, feedback to the user, and actions taken. By tracking the history of user error reports, manufacturers can target maintenance procedures for validation that have the greatest potential impact on safety and/or economics. This information can also be used to develop more meaningful metrics of manual quality. Finally, an example is described (using the MSG-3 process) of how these recommendations may be implemented.

Final Report and Recommendations

May 2002

Until recently, little attention has been paid to the way written procedures which are used to develop and revise aircraft maintenance technical data affect the users of that data. Studies of maintenance problems have tended to focus on the actions of the mechanic, job culture, and work procedures. More recently, attempts have been made to document the source of maintenance errors and improve maintenance procedures. One of the identified contributing causes of errors is the documentation used to guide maintenance tasks. As a result, efforts have been made to establish guidelines for the design of maintenance job aids. A question that remains is how the procedures used by manufacturers to develop maintenance data may contribute to user error. This report contains the results from Phase 2 of a 3-phase research effort. This phase (1) examines the procedures used by industry to develop aircraft maintenance manuals, (2) documents the problems encountered by the users of these documents, and (3) identifies ways in which human factors principles can be used to improve the development of these documents.

This phase includes a survey and interviews of technicians responsible for maintenance of a wide variety of Federal Aviation Regulations Part 25 aircraft. Technicians were queried about technical manual usage rates, manual errors, general manual quality, potential safety impact of manual problems, and suggestions for manual improvement. In summary, the results show that the accuracy and quality of technical manuals are rated as being good but have poor usability. In light of the results of Phase 1, these results support the need for a greater degree of user involvement during the document development process.

User Evaluation of Maintenance Documents

August 2001

Until recently, little attention has been paid to the procedures used to develop and revise aircraft maintenance technical data. Studies of maintenance errors have tended to focus on the actions of the mechanic, job culture and work procedures. More recently, attempts have been made to document the source of maintenance errors and improve maintenance procedures. One of the identified contributing causes of errors is the documentation used to guide maintenance tasks. As a result, efforts have been made to establish guidelines for the design of maintenance job aids. A question that remains is how the procedures used by manufacturers to develop maintenance data may contribute to user error.

In this document, the results of Phase 1 of a three-phase research effort to (1) examine the procedures used by industry to develop maintenance manuals, (2) document the problems encountered by users of these documents, and (3) identify ways in which human factors principles can be used to improve the development of these documents are reported. Phase 1 is a survey and report of the procedures used within the aviation industry to develop maintenance technical data. A cross-section of manufacturers was surveyed regarding company policy, communication, data tracking, user feedback and error reduction efforts.