BehavioMatrix: A behaviour measuring system

Organizers: Gordon McIntyre & Roland Göcke (University of Canberra)

Description: Consider a system to measure behaviour, intended for users with average to low technical skills and capable of making use of common and ubiquitous computing resources. Among other things, the front-end component of the system must be easy to install and, ideally, not require special equipment such as a webcam or high-performance hardware and be able to operate both online and offline. Indeed, the front end to the system should ideally provide for self-service. The system should also be conducive to many types of behaviour measurement scenarios, e.g. academic studies of behaviour, adversarial computer games, measuring reactions to products by marketing groups and medical diagnosis. These are some of the criteria that have been considered in the development of BehavioMatrix.
The BehavioMatrix system is a complete and flexible end-to-end interview and behaviour analysis solution. Step 1 is to create an interview sequence which can comprise textual, audio and video prompts. This is shown below, the process of creating an interview, or “package” as it is called in the system.
Step 2 is the interview which produces a video recording of the participant while they are responding to the interview prompts, and Step 3 is the analyses of the results. Below Step 2 and 3 are illustrated, the interview with the resulting analysis.

The interview package can be created in several ways. For the less technical, an online facility is available. A catalogue of interview packages can be compiled and viewed from the interview app. Once an interview package has been selected, the data of the participant’s responses is collected. At the completion of the interview, or later if no online connection is available, the interview responses, including media, are transmitted to an online database. An analysis can then be performed providing a report on facial and audio activity.

The three essential components comprising BehavioMatrix are:
1. android app downloaded from Google Play app store which can be used on a phone or tablet device;
2. online catalogue to store and view interview packages; and
3. analyses engine to interpret videos (audio planned in future but not scheduled).

BehavioMatrix is a work in progress, initially intended for use in the medical field and has been built to extend the work by the authors on linking facial activity and expressions to depressed states [1-4]. It is easily extensible to other research and applications. At this point, although integration is not complete, all individual components are in a beta state and available for demonstration.

References:
1. McIntyre, G. & Göcke, R., et al. (2009). An Approach for Automatically Measuring Facial Activity in Depressed Subjects. Proceedings of Affective Computing and Intelligent Interaction, ACII’09, pages 223–230.

2. McIntyre, G. & Göcke, R. (2008). A composite framework for affective sensing. INTERSPEECH, 2767-2770, ISCA.

3. McIntyre, G. & Göcke, R. (2008). The Composite Sensing of Affect.. In C. Peter & R. Beale (ed.), Affect and Emotion in Human-Computer Interaction, Vol. 4868 (pp. 104-115) Springer. ISBN: 978-3-540-85098-4.

4. McIntyre, G. & Göcke, R. (2007). Towards Affective Sensing, International Conference on Human-Computer Interaction 2007, ed. Julie A. Jacko, Springer, Beijing, China, pp. 411-420.