Measuring human behavior in large groups: a review of methods and techniques
H. David
Eurocontrol Experimental Centre, Bretigny-sur-Orge, France
 
The Experimental Centre of Eurocontrol (the European Organization for the Safety of Air Navigation) was built around what was then the first digital Air Traffic Control (ATC) Simulator in the world, and is certainly the most flexible. It is capable of simulating up to 40 working ATC positions simultaneously. A real-time simulation takes three to six weeks, and may involve 20 to 60 exercises, each about 90 minutes long. About 50 megabytes of data are collected by the system software for each exercise, giving exhaustive detail of the activity of the simulated aircraft, and the orders given by the controllers.
Air traffic controllers usually work in pairs on each sector. Although they employ speech communications and gestures to communicate, most of their work is mental, rather than physical. The task is not, strictly speaking, paced, although some activities are triggered by communications from aircraft or adjacent sectors. In normal operations, the controllers organize their workload to anticipate problems and maintain a continuously safe state. The ability of the controllers to cope with the workload is often the decisive factor in fixing the capacity of the system. We are therefore greatly interested in measuring the effects of carrying out air traffic control on the controllers. The methods used must be simple, reliable and labor-economic (we have, at most, five technical assistants to measure up to 40 participants). They must not be intrusive, and they must have some face validity, since controllers’ co-operation must be requested, and cannot be demanded. Fortunately, controllers are fully aware of the need to improve methods, and will put up with considerable inconvenience if they are convinced it will have useful results.
Since 1970, we have investigated the following types of measurement,
some repeatedly:
This paper will present a systematic evaluation of these techniques, classified as measures of workload (stress) or effect on the controller (strain), potential observer effects, the time -sensitivity, start-up and running costs, staff requirement, portability, risks of failure or bias, speed and cost of analysis, ethical constraints, potential for automatic data collection, and our decision on the method in our context. References to EEC reports, papers and Web sites are provided for each. This paper is a summary of a revision, currently in progress, of the existing EEC Report No.164.
Poster presented at Measuring Behavior '98, 2nd International Conference on Methods and Techniques in Behavioral Research, 18-21 August 1998, Groningen, The Netherlands
© 1998 Noldus Information Technology b.v.