Present-day technology begins to make it feasible to communicate complex information in a natural, dynamic way. Limitations in transmission bandwidth and memory are important reasons why system designers have to employ high levels of data compression. This often leads to content-dependent, temporal fluctuations in the quality of information presentation. When annoying to the end-user, such fluctuations can seriously threaten the acceptability of new media.

In this paper, continuous assessment is presented as a tool for monitoring temporal variations in subjective impressions. This method was originally developed for measuring time-varying image quality [1,2] and has recently been adopted by the International Telecommunications Union (ITU) under the name SSCQE (Single Stimulus Continuous Quality Evaluation) [3]. By this method, subjects continuously indicate the instantaneously perceived strength of a subjective impression by moving a slider along a graphical scale. As an illustration, Figure 1 shows the experimental setup for continuous assessment in case of evaluating visual information presented on a display. The observer looks at the screen and is asked to continuously adjust the position of a slider along a graphical scale according to his/her perception of a pre-specified attribute, e.g. perceived sharpness, depth or, at a more abstract level, quality or sense of presence. In general, the observer moves the slider upwards when experiencing an increase in perceived strength and downward when experiencing a decrease in perceived strength. The position of the slider is sampled at a fixed rate (e.g. 20 times per second) and the collected data are stored as files. These sampled positions can be plotted as a function of time to provide an impression of the temporal variation in the assessed attribute.

Continuous assessment has been employed to evaluate subjective impressions like time-varying quality of compressed video and speech material [2,4], perceived sharpness of blurred images [1], and perceived depth and feeling of presence in 3DTV [5]. The experimentally established relation between the results of continuous assessment and overall judgments indicate that human observers rely predominantly on the worst events in a sequence when determining their overall judgment [6].

Figure 1. Continuous assessment: experimental setup

References

1. Hamberg, R.; de Ridder, H. (1995). Continuous assessment of perceptual image quality. Journal of the Optical Society of America A, 12, 2573-2577.
2. Ridder, H. de; Hamberg, R. (1997). Continuous assessment of image quality. SMPTE Journal, 106, 123-128.
3. ITU/R Recommendation BT.500. Internet address: www.itu.ch.
4. Hansen, M.; Kollmeier, B. (1998). Continuous assessment and modeling of speech transmission quality. In: Proc. ICA/ASA '98 (Seattle), 229-230.
5. IJsselsteijn, W.A.; de Ridder, H.; Hamberg, R.; Bouwhuis, D.; Freeman, J. (1998). Perceived depth and the feeling of presence in 3DTV. Displays, 18, 207-214.
6. Hamberg, R.; de Ridder, H. (1999). Time-varying image quality: Modeling the relation between instantaneous and overall quality. SMPTE Journal, 108, 802-811.

© 2000 Noldus Information Technology b.v.