In this presentation I will defend the thesis that with newly available technologies, measuring gesture is not a technical problem anymore. Rather, it is increasingly becoming a problem of creating meaningful categories from the raw measurement data. This problem is most likely to emerge when the temporal properties of gesture are studied.

First there is the problem of 'top down' theoretical bias. Certain theoretical assumptions lead to the use of certain categories, which are then sometimes used to falsify or verify these very same assumptions. For instance, how gestures are divided up in different phases is crucial, but to a certain extent arbitrary.

Second, there is a problem with intercoder reliability (Kita et al., in press). When the human mind is used as a 'measuring instrument', the reliability and reproducibility of the results are often low. This is especially the case for spontaneous, iconic gesture, for these gestures are not conventionalized and therefore have an unpredictable shape. Although modern equipment (e.g. data gloves) have a significantly higher spatial and temporal resolution than video registration, this by itself does not reduce intercoder reliability.

Third, there is a risk of circularity. Because it is impossible to understand gestures without access to the speech, and because gestures can in turn change the meaning of the speech, one needs both to understand both because gesture and speech complement each other semantically (McNeill, 1992). This is especially noticeable when trying to establish the 'affiliate' of a gesture, which is the stretch of speech that is supposed to be semantically or pragmatically related to the gesture.

In certain limited contexts, automatic detection algorithms are possible, leading to higher reliability and well defined timing criteria. An example of automation in a limited context is De Ruiter (1998) where a fixed algorithm

is used to distinguish phases (preparatory, stroke, retraction) in very simple and predictable pointing gestures. For a more general class of gestures, such as spatial gestures, a human coder is needed to distinguish these phases, for there the meaning of the speech is needed to distinguish single complex gestures from a series of smaller gestures.

I therefore argue that, now that measurement hardware (e.g. data gloves) has reached a certain level of maturity, it is time to spend more energy on creating formal frameworks and definitions that are as precise, and as little theory-biased as possible. Measurement equipment alone will never be able to replace human interpretation completely in gesture research.

References

1. Kita, S.; van Gijn, I.; van der Hulst, H. (in press). Movement Phases in Signs and Co-Speech Gestures, and their Transcription by Human Coders. Proceedings of Bielefeld Gesture Workshop. Springer Verlag.
2. McNeill, D. (1992). Hand and Mind. Chicago, London: The Chicago University Press.
3. Ruiter, J.P.A. de (1998). The Production of Gesture and Speech. Doctoral Dissertation, University of Nijmegen.

© 2000 Noldus Information Technology b.v.