Infrastructure to support gesture research
P. Wittenburg, H. Brugman, D. Broeder and A. Russel
Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
In the field of psycholinguistics, the Max Planck Institute was one of the first to make consequent use of computer-based audio and video material. This opened completely new types of research questions. A gesture project was established relying fully on the availability of multimedia technology at the desktop. Also, anthropologists and linguists who want to study language usage in real-life situations and to document the rich context of language acts as well are digitizing the raw data systematically to make use of the new opportunities.
The major technological problems that had to be solved were:
A first-generation multimedia program developed specifically for the Mac platform is actually in use [1]. It is flexible enough to allow the user to define an almost infinite number of annotation tiers and to describe the complex time relationships typical for independent streams of data such as speech and gesture. Nevertheless, this tool has a couple of limitations which we want to overcome with the two second generation tools called EUDICO [2] and Browsable Corpus [3]. These tools are based on modern software design methods and programming languages such as UML [4] and Java, respectively, and APIs such as Java Media Framework. EUDICO supports streaming of the relevant media fragments across the Internet and can operate in distributed environments. Browsable Corpus defines a browsable and searchable universe of meta-descriptions and offers appropriate tools such that users can easily find appropriate resources to work with. Since this technology is intended to be available in the Internet the meta-descriptions are based on XML.
The software handles Cinepak, MPEG-1 [5], and in the future certainly MPEG-2 [5] video encoding formats. For MPEG-1, streams of 3 Mbps constant bit rate are typical. A fast switching network with a Gbps Ethernet switch in the center and dedicated 100 Mbps connections to the media desktops is the basis for multimedia delivery. It was decided to rely on overcapacities in the LAN and not to apply complex technologies such as ATM offering mechanisms to reserve certain bandwidths. The media servers will be coupled with Gbps connections to be able to support several users in parallel. Separate digitization stations have been set up which now create MPEG-1 streams in real-time and in doing so making utterly time-consuming conversions from for example M-JPEG to MPEG-1 obsolete.
The success of the video setups and tools at the institute led to the fact that more and more audio/video material is digitized. Currently, the servers have to store and offer about 400 Gb video data. A new project carried out by one researcher will imply an increase of about 40 Gb per month. This leads to great organizational challenges, since the data has to be organized in maintainable structures, to be backed up, and to be made on-line available. With respect to the latter, two possibilities are given: (1) using large RAID systems or (2) using hierarchical storage management systems where only part of the data is available on disk and the larger part is migrated to a tape library. Both possibilities have been compared at the institute. The decision about which possibility to choose depends on a number of criteria such as total amount of data, robustness of hardware and software, price per Gb to be stored, and accepted latencies.
All aspects of setting up an adequate multimedia infrastructure will be discussed in detail. The focus of the talk will not be on specific tools, but discuss technological alternatives, the experiences made at the institute, and explain the choices for the coming years.
Paper presented at Measuring Behavior 2000, 3rd International Conference on Methods and Techniques in Behavioral Research, 15-18 August 2000, Nijmegen, The Netherlands