Empirical experiments always lead to huge piles of data, which need to be processed in order to retrieve the relevant information. The data itself may stem from different sensors, which may be of different modalities. Therefore there is a major challenge in how to combine the different data streams effectively. The aim of the Special Interest Group meeting was to discuss ways of integrating the measurement, storage, retrieval and analysis of behavioral data and physiological signals into a single work ('lab') environment. In this context, 'behavioral' includes movements, posture, gesture, facial expressions, speech, and so on, which are often recorded on video. Among the relevant physiological signals that underlie behavior are heart rate, blood pressure, body temperature, muscle tension, brain activity, etc.

In the meeting different aspects of data quality and integrity, with an emphasis on synchronization between data streams, data reduction and visualization were presented. First, four presentations introduced various aspects of problems and solutions, each followed by short discussion. The first presentation (Plant) addressed the problem of how to be certain that the timing of stimuli and complex patterns of responses is being generated and measured with sub-millisecond accuracy. Two of the following presentations emerged from the Visual Lab project (Noldus, Harlaar, Van der Pers, Hoogeboom, Geuze) on how to synchronize measurements with different sensors, including video, and how to integrate and store the data. The Visual Lab project is intended to lead to an open platform that supports existing physiological and video data acquisition, video analysis software, and synchronous visualization of physiological signals and video. Presentations address a general technique for synchronizing multi-modal data and describe and demonstrate an integrated system for gait analysis in clinical rehabilitation. The last presentation (Van Eijkern) described and demonstrated an integrated system for measurement and analysis of movement, physiological data and video as used in movement science and developmental neurology. Next, there was some opportunity for others to have a short presentation or introduce a discussion topic. Finally, the general discussion was centered around the main problems of integrated measurement and the limitations of different solutions.

Aim of the meeting

• Presentations of several projects on synchronization and visualization of multi-modal time-structured data (see below)
• Discussion and exchange of expert knowledge
• Demonstrations and/or presentations by other SIG participants (These should be submitted to the organisers of the SIG before the meeting)
• General discussion about synchronization, visualization and analysis of multi-modal time-structured data: What kinds of solutions are available? What are the advantages, disadvantages and of limitations of these solutions?

Presentation

• Benchmarking timing precision in the real world. R.R. Plant, P. Quinlan, N. Hammond & T. Whitehouse (University of York, York, United Kingdom)
• SYBAR: an application of synchronized signal- and video-recordings and analysis. J. Harlaar (Vrije Universiteit Medical Center, Amsterdam, The Netherlands) & J.N.C. van der Pers (Syntech, Hilversum, The Netherlands)
• PRN technique: results obtained synchronizing video to digital data. P.J. Hoogeboom (National Aerospace Laboratory NLR, Amsterdam, The Netherlands)
• The simultaneous acquisition of video, kinematics data and electrophysiological signals. L.A. van Eykern & B. Otten (University of Groningen, Groningen, The Netherlands)