Towards a research methodology for assessing command team performance

V. Mills and C. Stothard

Electronics and Surveillance Research Laboratory, Land Operations Division, Defence Science and Technology Organisation, Salisbury, Australia

Over recent years, there has been a vast growth in technological systems that aim to enhance human performance. For example, military command teams are being provided with digital support tools that automate numerous processes, and provide access to an almost unlimited amount of information. The introduction of these systems into what has previously been a manual process has the potential for significant impact. A variety of tragedies have demonstrated that one of the largest influences on system performance remains the human factor. Human error has been found to be responsible for 60% to 80% of fatal aviation accidents, and communication issues were found to be involved in more than 70% of accidents (Brannick et al., 1995). The suggestion is that a large influence on effective performance is the interaction between information technology and team processes. In addition, research has suggested that not all aspects of digitisation improve performance. For example, Bowers et al. (1998) found that automating certain tasks was associated with improved team performance on only 1 in 4 measures.

Given the increasing emphasis on digital systems, it is important that research be conducted that will assess the impact on the performance of the command team. This creates a requirement for research that investigates and establishes techniques for analysing command team tasks and processes. In particular, techniques are required to identify and delineate the teamwork and the taskwork processes. Taskwork consists of behaviours that are performed by individual team members and are critical to the execution of individual team member functions. Teamwork, in contrast, consists of behaviours that are related to team member interactions, and are necessary to establish coordination among the individual team members to achieve team goals (Gregory & Kelly, 1998).

The focus of this paper is on observational methodologies that have been developed to collect behavioural data on command team processes. The Team Behaviour Index is used to determine team performance, while the Objective Taskload Index is used to determine command team taskload and structure. The techniques are currently being used within the Australian Army to collect empirical data on the information flow among the team, the team dynamics, the task characteristics, and the overall workload. The methodology has been found to be useful at multiple headquarters levels, demonstrating a high degree of cross-situational generality. The paper will describe these indices, and will present a sample of data that can be generated using this approach.

References

  1. Bowers, C.; Thornton, C.; Braun, C.; Morgan, B.B.; Salas, E. (1998). Automation, task difficulty, and aircrew performance. Military Psychology, 10, 259-274.
  2. Brannick, M.T.; Prince, A.; Prince, C.; Salas, E. (1995). The measurement of team processes. Human Factors, 37, 641-651.
  3. Gregory, D.; Kelly, M. (1998). Impact of Digitization on Command and Staff Training: Implications for the Design of Training Simulators. DERA/CHS/MID/ CR980188/1.1.

Paper presented at Measuring Behavior 2000, 3rd International Conference on Methods and Techniques in Behavioral Research, 15-18 August 2000, Nijmegen, The Netherlands

© 2000 Noldus Information Technology b.v.