Observing human behavior

Structured observation of human behavior is a widely accepted method to gather information about how people perform a task or interact with each other. It results in time-based data suitable for quantitative analysis. Direct observation or computer-aided analysis of video recordings have become standard practice among researchers of human behavior. However, these methods fall short if one wishes to measure someone’s behavior under circumstances that are not available ‘live’ or cannot be reproduced in front of a video camera. A good example is the interaction between an operator and a machine that has yet to be built. The designer wants to deliver a machine with an optimal ‘user interface’, but how can one evaluate its ergonomics if the machine does not yet exist? A technical drawing of the machine is usually too abstract for this purpose and is not easily interpreted by the future user. However, a usability test of the completed product, followed by redesign and adaptation, is a very expensive and unattractive option. The same applies to the construction of scale models, the construction of which is very time-consuming and costly.

Virtual Reality

As an alternative for physical mockups, most designers nowadays use 3D drawing programs to visualize their concepts. Abstract 2D construction drawings are thus converted into realistic images. Although such visualizations give a good impression of what the end product or environment will look like, they don’t show if one will manage to use the product or navigate through the environment, because there is no interaction between the user and the visualization. The limitations of conventional 3D visualization tools can be overcome by Virtual Reality (VR) technology. With VR technology, abstract information about a design and possible alternatives can be turned into a virtual environment in which a person can navigate and interact with objects or other persons. Using VR simulation, virtual prototypes of designs can be built in a relatively short time. This enables one to discover design flaws and generate new ideas at an early stage. However, most VR systems have had limited use for ergonomics research because there is no record of where the user has been in the virtual world or what the user has seen and done.

Virtual Observer

To bridge the gap between VR modeling on the one hand and behavioral observation and analysis on the other hand, we have developed the "Virtual Observer". This is a real-time 3D simulation of one or multiple objects, actors and actions within a simulated 3D environment on a computer. The environment can be displayed on a standard computer screen, the display of a VR helmet, or even a room-size 3D ‘cave’. The user interacts with the virtual world using a computer mouse, joystick or ‘data glove’. While the user is navigating through this environment, all sorts of events may occur: an alarm goes off, a light goes on, the user walks in a certain direction, enters a room, presses a button, grabs an object, etc. The Virtual Observer can log all these events together with the corresponding visual scene in a format suitable for subsequent replay and quantitative analysis. During a simulation, events are automatically recorded and written to a log file. Besides this, a digital video file can be generated with user-defined ‘camera viewpoint’ for playback and review of the simulation. The exact synchronization between the events in the log file and the corresponding video frames in the video file provides a direct link to the review and analysis options present in The Observer Video-Pro. One can calculate latency, frequency and duration of behaviors, analyze the sequential structure of the behavioral process, or graph events of interest in a time-event plot. In addition, search and review functions help the user to produce a data summary or a highlight compilation of the movie file which facilitates the presentation of findings to colleagues, test subjects or clients.

Applications

Virtual Observer is especially suited for observational studies in situations that are too expensive or time-consuming to be reproduced in the real world, or in situations that are potentially harmful for the persons involved. Examples of applications are work place design, simulation of handicaps, training operators on new machines, and assessing skills of job applicants.