Recordings from the sensory areas of cortex have demonstrated the existence of neurons with remarkably sharply tuned responses. Reports of visual responses selective to complex visual forms such as monkey paws in the early 1970s were dismissed by many as inconclusive, but it is much harder to ignore the increasingly detailed descriptions of neurons selective for faces in the temporal lobe [Thorpe, 1995]. Field (1994) has proposed that the goal of sensory coding is to produce sparser and sparser representation as the processing proceeds to higher cortical areas.
Evaluating the sparseness of coding in the brain is difficult: it is hard to record a set of neurons simultaneously across which sparseness could be measured. We have more information about neurons' breadth of tuning across various stimulus sets than about sparseness per se. Coding across stimuli and across cells are, however, closely related. The most immediate observation during physiological experiments is the difficulty of finding effective stimuli for neurons in most cortical areas. Each neuron appears to have specific response properties, typically being tuned to several stimulus parameters. In primary visual cortex, many neurons only respond strongly when an elongated stimulus, such as a line, edge, or grating, is presented within a small part of the visual field, and then only if other parameters, including orientation, spatial frequency (width), stereoscopic disparity, and perhaps colour or length fall within a fairly narrow range. This suggests that at any moment during the animal's life, only a small fraction of these neurons will be strongly activated by natural stimuli [Földiák and Young, 1995].
The problem of finding the preferences of cells becomes even more severe in higher visual areas, such as area V4, and especially in inferotemporal cortex. Cells' preferences in inferotemporal cortex are often difficult to account for by reference to simple stimulus features, such as orientation, motion, position, or colour. Cells here show selectivity for complex visual patterns and objects, such as faces, hands, complex geometrical shapes, and fractal patterns [Földiák and Young, 1995].
It has been argued that the brain uses distributed representations because they are tolerant to damage. However, redundancy far smaller than that in dense codes is sufficient to produce robust behaviour. By simply duplicating units with 99% reliability (assuming independent failures), reliability increases to 99.99%. Sparse representations can be even more tolerant to damage than dense ones if high accuracy is required or if the units are highly unreliable.
Arguments supporting dense coding may also be challenged by empirical evidence. Recent studies on Alzheimer's disease [Hodges et al., 1992] suggest that patients may irreversibly lose specificity, and even whole concepts, independently for individual objects. Salzman and Newsom (1994) have recorded movement-sensitive neurons in the middle temporal area of rhesus monkeys. Their data suggested a mechanism in which monkeys chose the direction encoded by the largest signal in the representation of motion direction. This means that the direction was represented by a very small proportion of all direction encoding neurons. Also, in previous studies Newsom, Britten, and Movshon (1989) found out that, when tested with the appropriate stimuli, the most sensitive neurons can rival or even exceed the performance of a psychophysical observer in the same conditions. They recorded the movement-sensitive neurons in the middle temporal area while the monkey was performing a perceptual discrimination and found that responses of some single neurons contained as much information about the stimulus as the animal's behavioural response.