Creating a web site allows millions of potential users with various goals and knowledge levels to access the information it contains. For this reason, interest in usability evaluation of web sites is rapidly increasing.

We have developed a method and an associated tool to detect usability problems in web interfaces through a remote evaluation. Our approach combines two techniques that are usually applied separately: empirical testing and model-based evaluation. The reason for this integration is that models can be useful to detect usability problems, but their use is much more effective if they can be related to the actual use of a system. Our tool is able to analyse the possible inconsistency between actual user interactions and the task model of the web site, which describes how its concrete design assumes activities should be performed. To support remote evaluation, we have developed a technique that allows user actions to be recorded during a site visit. The analysis of the logged data is based on comparing the traces of actions performed with the structure of the task model. This analysis provides evaluators with a number of results related to the tasks that users intend to perform, the web pages and their mutual relationships.

In our case, we follow a hybrid approach because our environment is able to analyse data relative to user interactions and then compare them to the task model corresponding to the design of the web site. The solution adopted to identify user intentions in WebRemUSINE is to display the high-level tasks supported by the web site and ask the user to indicate explicitly which task they want to perform. WebRemUSINE compares the logs with the task model and provides results regarding both the tasks and the web pages, thereby supporting an analysis from both viewpoints.

The method is composed of three phases: preparation, which consists of creating the task model of the web site, collecting the logged data and defining the association between logged actions and basic tasks; automatic analysis, where WebRemUSINE examines the logged data with the support of the task model and provides a number of results concerning the performed tasks, errors, loading time, etc.; and evaluation, where the information generated is analysed by the evaluators to identify usability problems and possible interface improvements.

The environment is composed mainly of three modules: the ConcurTaskTrees Editor (CTTE, publicly available at http://giove.cnuce.cnr.it/ctte.html); the logging tool, which has been implemented by a combination of Javascript and Java applet to record user interactions; and WebRemUSINE, a Java tool capable of performing an analysis of the files generated by the logging tool, using the task model created with the CTTE tool.

Paper presented at Measuring Behavior 2002 , 4^th International Conference on Methods and Techniques in Behavioral Research, 27-30 August 2002, Amsterdam, The Netherlands

© 2002 Noldus Information Technology bv