Measuring Behavior 2005: Blignaut and McDonald

Capturing usability data on-the-go

P.J. Blignaut and T. McDonald

Department of Computer Science and Informatics (IB 65), Free State University, South Africa

A tool was developed to capture users’ responses while using software, e.g. a word processor, internet browser or customized stock control system. All actions, i.e. buttons clicked, text typed, items selected, etc. along with the user name and host name of the PC and exact date and time of every action are saved to an underlying MySQL database.

The tool fills the bottom 10% of the screen with the software application filling the remaining 90%. It is always on top of the display without obscuring any part of the application that is to be tested. Questions are displayed that prompt the user to select the answer from a mutually exclusive set of possibilities or type free text into a box. Three categories of questions can be posted into the underlying database beforehand to accommodate a researcher’s needs regarding the specific software application to be tested:

Questions to capture a user’s profile, e.g. gender, age group, computer experience, etc.
Instructions with regard to specific tasks that must be done with the software that is tested.
Questions to capture a user’s responses regarding the usability of the software to be tested. Instead of providing the user with a paper-based questionnaire that must be entered into a database or spreadsheet afterwards, the questions are posted on the screen, while the user is busy using the application, providing him/her with a space to record his/her answers directly into the database.

The tool was used successfully during research to determine the differences in websearching patterns between various user groups. More than 500 users were observed in two computer laboratories in one week. Questions were formulated to instruct the users to do specific searches on the Internet. Every URL visited and all text typed, e.g. into the search box of a search engine, were recorded. SQL queries were used afterwards to analyze the correctness of answers, obtain the average time between clicks, the average time for reading on a specific website and the average number of URLs visited per search.

Paper presented at Measuring Behavior 2005 , 5^th International Conference on Methods and Techniques in Behavioral Research, 30 August - 2 September 2005, Wageningen, The Netherlands.