Measuring Behavior 2005: Heward et al.

FlyTracker: Real-time analysis of insect courtship

J.A. Heward, D.A. Baker, T.C. Lukins and J.D. Armstrong

School of Informatics, University of Edinburgh, Edinburgh, Scotland, UK

Insects, even in relatively restricted environments, move in 3D relative to one another thus when tracking individuals occlusion is an important concern. Since total occlusion occurs frequently, methods that erode segmented blobs until they separate do not perform very well. We present a hybrid tracking system specially developed to track multiple insects in real-time in a typical experimental observation chamber.

A vital component of any tracking application is the blob detection algorithm and its suitability to the task in hand. To work in real-time yet deal with changes in lighting conditions our system uses, pseudo-adaptive thresholding, a hybrid of means and adaptive thresholding techniques. The tracking system itself fits bounding boxes to the identified blobs. Tracking the direction of these over time allows us to handle occlusion events satisfactorily. The tracking system then returns a range of parameters about each object and its motion. Using experts to annotate a range of behaviors onto example video footage we then extracted a set of rules that we show is useful for automatically classifying the behavior of the insects. The software is highly modular with independent, but linked applications that:

Track objects.
Replay video files, produce animations from the tracking logs and allow experts to annotate either the video or the animation.
Learn classification rules from expert annotation.

The Netherlands Although developed specifically for detecting courtship behavior between a pair of insects, the algorithms and methods are very generic and we are in the process of testing them on three and more animals and for detecting interactions between other entities.

Paper presented at Measuring Behavior 2005 , 5^th International Conference on Methods and Techniques in Behavioral Research, 30 August - 2 September 2005, Wageningen, The Netherlands.