Functional inferences from experiments in cognitive neuroscience are often based on sets of methodological assumptions which cannot be always fulfilled, as in recordings of cognitive brain potentials. The effect of these violations is not always corrected through the traditional statistical analyses and may lead to inconsistent results across studies. An example comes from Event Related Potentials (ERPs) studies where electrical brain response to stimuli presented in different experimental conditions are usually estimated, separately for each subject and each condition, through an averaging procedure. Averaging assumes the brain electrical activity as composed of a constant brain response (ERP) to the same stimulus under the same experimental condition and a zero-mean random uncorrelated spontaneous EEG. Estimated ERP data are then analyzed across subjects through statistical procedures which, however, do not guarantee for the reliability of the single estimates. If the assumptions which underlay the averaging procedure are not fulfilled, as it is usually the case, the lack of reliability of the estimated ERPs may lead to statistical results which are not reliable as well. Within-subjects bootstrap analysis [1] can be used to evaluate the reliability of neurophysiological and behavioral experiments. The bootstrap technique is a general resampling procedure which estimate the probability distribution underlying the sample data. We propose to use this approach within each experimental subject to test the hypothesis regarding the reliability of the differences between ERPs to different conditions.

In this study we applied bootstrap analysis to data from an experiment aimed at investigating the relationship between ERPs and memory processes [2]. ERPs were recorded from two groups of subjects engaged in a recognition memory task. During the study phase, subjects in group A were required to make an orthographic judgment on 160 visually presented words, whereas subjects in group B were only required to pay attention to the words. During the test phase all subjects were presented with the 160 previously studied words along with 160 new words and were required to decide if the current word was "old" or "new". To assess the effect of word imagery value half of the words had a high imagery value and half a low imagery value. Anova performed on ERPs showed that an imagery-induced modulation of the old/new effect was evident only for subjects which were not engaged in the orthographic task during the study phase. This result supports the hypothesis that this modulation is due to some aspects of the recognition memory process and not to the stimulus encoding operations which occur during the recognition memory task. However, bootstrap analysis on the same data showed that the old/new effect on ERPs was not reliable for all the subjects. This result suggests that only a cautious inference can be made from these data.

References

1. Efron, B.; Tibshirani, R.J. (1993). An Introduction to the Bootstrap. New York: Chapman and Hall.
2. Rugg, M.D. (1995). ERP studies of memory. In M.D. Rugg & M.G.H. Coles (eds.). Electrophysiology of Mind, pp.132-170. Oxford: Oxford University Press.