Studying usage of complex applications
J. Kort1, M.G.D. Steen1, H. de Poot2, H. ter Hofte2 and I. Mulder2
1TNO Information and Communication Technology,
Groningen, The Netherlands
2Telematica Instituut, Enschede, The Netherlands
Innovative products and services are characterized by an increased complexity
in technology
and functionality. This increased complexity is well known in product
development and
research areas like: personal networks, intelligent home environments,
ubicom, CSCW, etc.
Increased complexity poses new problems for usage studies. Current research
methods
and tools are not always suf.cient to deliver the desired insights. They
are too resource
consuming to evaluate the complete product or service, represent a limited
number of
research interests and are not applicable in complex environments like
daily life.
Furthermore, many developments directly related to usage have become
increasingly important in complex products and services (e.g. personalization,
adaptability and context awareness). To study usage in this context, existing
methods are often not sufficient because of their narrow focus on task
performance or usability.
The first steps towards developing an integrated framework for usage
analysis, addressing issues raised by complex product design, will be
made during the CHI2005 workshop Usage analysis: Combining Logging
and Qualitative Methods.
Related research is performed in the Freeband User Experience project
(FRUX), part of the research program Freeband Communication. A context
aware application is developed, which supports police officers in retrieving
the right information at the right time, based on the officers social
relations with colleagues. How to study the usage of this application
is a problem currently addressed.
In FRUX new usage analysis tools, integrated in a measurement platform,
are developed (partially based on the CHI2005 workshop results). These
tools consist of a combination of qualitative (e.g. experience sampling
(ESM), questionnaires) as well as quantitative (e.g. log files, automatic
registration of social interactions and sensor data) methods to enhance
research possibilities and insights in the usage of applications.
In this paper we present these tools, their conceptualization and design
and the preliminary
results of their application during a small scale pilot. We conclude with
suggestions for
further developments of the tools and platform.
Paper presented
at Measuring Behavior 2005
, 5th International Conference on Methods and Techniques
in Behavioral Research, 30 August - 2 September 2005, Wageningen, The
Netherlands.
© 2005 Noldus
Information Technology bv
|