Measuring Behavior 2005: Courteille and Karlgren

Engaged students and increased performance

O. Courteille and K. Karlgren

LIME, Karolinska Institute, Stockholm, Sweden

This study is based on an examination for all of the medical students (Term 8) in Stockholm, Sweden. About 120 students, from the four university hospitals in Stockholm, participated simultaneously in the examination. The goal was to assess their skills and ability to perform basic surgical procedures at different ‘stations’ in a limited amount of time (5/10mn at each station).

Four stations (out of sixteen) consisted of two computer-based Interactive Simulated Patients (ISP, pre-recorded videosequences of patient/actor) and two Standardized Patients (SP, real actor performing live). The common goal for the participants at these four stations was to interview a patient and then make a diagnosis. Half of the students were video-filmed when interacting with the ISP and the SP. Additional data (log files from the ISP sessions) was collected during the computer interaction.

A coding scheme has been created for observing and assessing the different behaviours of students interacting with the computer (speech, posture, gesture, gaze, think aloud, etc). Aims of this study:

To compare the flow experiences and the realism of this life-like
situation between ISP and SP.
To observe the affective impact of the simulated patient on the student’s confidence and ability to solve clinical problems.
To collect authentic emotions evoked on students for building a database.

The originality of this experiment (unique pilot study so far) has provided us interesting data about the correlation between stress and the power of video-based simulated patients to affect and move students (empathy).

Key issue in the observation: the performance of the standardized patient is not regular but varies. The actor doesn’t react and behave the same way and the same manner at each session. The simulated patient however offers a more reliable measurement tool in terms of provoking reactions and evoking emotions.

The video-analyses will be validated by the Cohen’s kappa (by means of 2 observers). Preliminary results show that the time pressure due to the examination settings affected the students’ performance on different levels (decision making, visual coordination, operational accuracy, etc).

Paper presented at Measuring Behavior 2005 , 5^th International Conference on Methods and Techniques in Behavioral Research, 30 August - 2 September 2005, Wageningen, The Netherlands.