# NSci 8217 Spring 2012:Perceptual Constancy

Spatial updating and the maintenance of visual constancy

E. M. Klier and D. E. Angelaki

Neuroscience  156  801-18  (2008)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=18786618

Spatial updating is the means by which we keep track of the locations of objects in space even as we move. Four decades of research have shown that humans and non-human primates can take the amplitude and direction of intervening movements into account, including saccades (both head-fixed and head-free), pursuit, whole-body rotations and translations. At the neuronal level, spatial updating is thought to be maintained by receptive field locations that shift with changes in gaze, and evidence for such shifts has been shown in several cortical areas. These regions receive information about the intervening movement from several sources including motor efference copies when a voluntary movement is made and vestibular/somatosensory signals when the body is in motion. Many of these updating signals arise from brainstem regions that monitor our ongoing movements and subsequently transmit this information to the cortex via pathways that likely include the thalamus. Several issues of debate include (1) the relative contribution of extra-retinal sensory and efference copy signals to spatial updating, (2) the source of an updating signal for real life, three-dimensional motion that cannot arise from brain areas encoding only two-dimensional commands, and (3) the reference frames used by the brain to integrate updating signals from various sources. This review highlights the relevant spatial updating studies and provides a summary of the field today. We find that spatial constancy is maintained by a highly evolved neural mechanism that keeps track of our movements, transmits this information to relevant brain regions, and then uses this information to change the way in which single neurons respond. In this way, we are able to keep track of relevant objects in the outside world and interact with them in meaningful ways.

The role of chromatic scene statistics in color constancy: spatial integration

J. Golz

J Vis  8  6.1-16  (2008)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=19146336

The human visual system has the ability to perceive approximately constant surface colors despite changes in the retinal input that are induced by changes in illumination. Based on computational analyses as well as psychophysical experiments, J. Golz and D. I. MacLeod (2002) proposed that the correlation between luminance and redness within the retinal image of a scene is used as a cue to the chromatic properties of the illuminant. However, J. J. Granzier, E. Brenner, F. W. Cornelissen, and J. B. Smeets (2005) found that the spatial extent in the field of vision that is relevant for the effect of the luminance-redness correlation on color appearance is very local and therefore questioned whether this scene statistic is used for estimating the illuminant. Here, I present evidence that the spatial extent is substantially more global than claimed by Granzier et al. and consistent with the hypothesis that this scene statistic is used for estimating the illuminant. It is further shown for two figural parameters of the stimuli that they influence the spatial extent and hence could have contributed to an underestimation of the spatial extent by Granzier et al. Finally, it is shown that the spatial extent relevant for the effect of mean surround chromaticity on color appearance is very similar to that found for the luminance-redness correlation.

Can illumination estimates provide the basis for color constancy?

J. J. M. Granzier and E. Brenner and J. B. J. Smeets

J Vis  9  18.1-11  (2009)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=19757957

Objects hardly appear to change color when the spectral distribution of the illumination changes: a phenomenon known as color constancy. Color constancy could either be achieved by relying on properties that are insensitive to changes in the illumination (such as spatial color contrast) or by compensating for the estimated chromaticity of the illuminant. We examined whether subjects can judge the illuminant's color well enough to account for their own color constancy. We found that subjects were very poor at judging the color of a lamp from the light reflected by the scene it illuminated. They were much better at judging the color of a surface within the scene. We conclude that color constancy must be achieved by relying on relationships that are insensitive to the illumination rather than by explicitly judging the color of the illumination.

Spatial constancy mechanisms in motor control

W. P. Medendorp

Philos Trans R Soc Lond B Biol Sci  366  476-91  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21242137

The success of the human species in interacting with the environment depends on the ability to maintain spatial stability despite the continuous changes in sensory and motor inputs owing to movements of eyes, head and body. In this paper, I will review recent advances in the understanding of how the brain deals with the dynamic flow of sensory and motor information in order to maintain spatial constancy of movement goals. The first part summarizes studies in the saccadic system, showing that spatial constancy is governed by a dynamic feed-forward process, by gaze-centred remapping of target representations in anticipation of and across eye movements. The subsequent sections relate to other oculomotor behaviour, such as eye-head gaze shifts, smooth pursuit and vergence eye movements, and their implications for feed-forward mechanisms for spatial constancy. Work that studied the geometric complexities in spatial constancy and saccadic guidance across head and body movements, distinguishing between self-generated and passively induced motion, indicates that both feed-forward and sensory feedback processing play a role in spatial updating of movement goals. The paper ends with a discussion of the behavioural mechanisms of spatial constancy for arm motor control and their physiological implications for the brain. Taken together, the emerging picture is that the brain computes an evolving representation of three-dimensional action space, whose internal metric is updated in a nonlinear way, by optimally integrating noisy and ambiguous afferent and efferent signals.

Dynamic sound localization during rapid eye-head gaze shifts

J. Vliegen and T. J. Van Grootel and A. J. Van Opstal

J Neurosci  24  9291-302  (2004)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=15496665

Predictive remapping of visual features precedes saccadic eye movements

D. Melcher

Nat Neurosci  10  903-7  (2007)

/net/pooh/mnt/raid0/Reprints/PMID/17589507.pdf

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=17589507

Electrophysiological correlates of presaccadic remapping in humans

N. A. Parks and P. M. Corballis

Psychophysiology  45  776-83  (2008)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=18513363

Saccadic eye movements cause rapid displacements of space, yet the visual field is perceived as stable. A mechanism that may contribute to maintaining visual stability is the process of predictive remapping, in which receptive fields shift to their future locations prior to the onset of a saccade. We investigated electrophysiological correlates of remapping in humans using event-related potentials. Subjects made horizontal saccades that caused a visual stimulus to remain within a single visual field or to cross the vertical meridian, shifting between visual hemifields. When an impending saccade would shift the stimulus between visual fields (requiring remapping between cerebral hemispheres), presaccadic potentials showed increased bilaterality, having greater amplitudes over the hemisphere ipsilateral to the grating stimulus. Results are consistent with interhemispheric remapping of visual space in anticipation of an upcoming saccade.

Evidence for the predictive remapping of visual attention

S. Mathôt and J. Theeuwes

Exp Brain Res  200  117-22  (2010)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=19882149

When attending an object in visual space, perception of the object remains stable despite frequent eye movements. It is assumed that visual stability is due to the process of remapping, in which retinotopically organized maps are updated to compensate for the retinal shifts caused by eye movements. Remapping is predictive when it starts before the actual eye movement. Until now, most evidence for predictive remapping has been obtained in single cell studies involving monkeys. Here, we report that predictive remapping affects visual attention prior to an eye movement. Immediately following a saccade, we show that attention has partly shifted with the saccade (Experiment 1). Importantly, we show that remapping is predictive and affects the locus of attention prior to saccade execution (Experiments 2 and 3): before the saccade was executed, there was attentional facilitation at the location which, after the saccade, would retinotopically match the attended location.

Auditory spatial perception dynamically realigns with changing eye position

B. Razavi and W. E. O'Neill and G. D. Paige

J Neurosci  27  10249-58  (2007)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=17881531

Audition and vision both form spatial maps of the environment in the brain, and their congruency requires alignment and calibration. Because audition is referenced to the head and vision is referenced to movable eyes, the brain must accurately account for eye position to maintain alignment between the two modalities as well as perceptual space constancy. Changes in eye position are known to variably, but inconsistently, shift sound localization, suggesting subtle shortcomings in the accuracy or use of eye position signals. We systematically and directly quantified sound localization across a broad spatial range and over time after changes in eye position. A sustained fixation task addressed the spatial (steady-state) attributes of eye position-dependent effects on sound localization. Subjects continuously fixated visual reference spots straight ahead (center), to the left (20 degrees), or to the right (20 degrees) of the midline in separate sessions while localizing auditory targets using a laser pointer guided by peripheral vision. An alternating fixation task focused on the temporal (dynamic) aspects of auditory spatial shifts after changes in eye position. Localization proceeded as in sustained fixation, except that eye position alternated between the three fixation references over multiple epochs, each lasting minutes. Auditory space shifted by approximately 40% toward the new eye position and dynamically over several minutes. We propose that this spatial shift reflects an adaptation mechanism for aligning the "straight-ahead" of perceived sensory-motor maps, particularly during early childhood when normal ocular alignment is achieved, but also resolving challenges to normal spatial perception throughout life.

Keeping the world a constant size: object constancy in human touch

M. Taylor-Clarke and P. Jacobsen and P. Haggard

Nat Neurosci  7  219-20  (2004)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=14966526

The perceived size of objects touching different regions of skin varies across the body surface by much less than is predicted from variations in tactile receptor density. Here we show that altering the visual experience of the body alters perceived tactile distances. We propose that the brain attempts to preserve tactile size constancy by rescaling the primary, distorted body-surface representation into object-centered space according to visual experience of the body.

Investigation of perceptual constancy in the temporal-envelope domain

M. Ardoint and C. Lorenzi and D. Pressnitzer and A. Gorea

J Acoust Soc Am  123  1591-601  (2008)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=18345847

The ability to discriminate complex temporal envelope patterns submitted to temporal compression or expansion was assessed in normal-hearing listeners. An XAB, matching-to-sample-procedure was used. X, the reference stimulus, is obtained by applying the sum of two, inharmonically related, sinusoids to a broadband noise carrier. A and B are obtained by multiplying the frequency of each modulation component of X by the same time expansion/compression factor, alpha (alphain[0.35-2.83]). For each trial, A or B is a time-reversed rendering of X, and the listeners' task is to choose which of the two is matched by X. Overall, the results indicate that discrimination performance degrades for increasing amounts of time expansion/compression (i.e., when alpha departs from 1), regardless of the frequency spacing of modulation components and the peak-to-trough ratio of the complex envelopes. An auditory model based on envelope extraction followed by a memory-limited, template-matching process accounted for results obtained without time scaling of stimuli, but generally underestimated discrimination ability with either time expansion or compression, especially with the longer stimulus durations. This result is consistent with partial or incomplete perceptual normalization of envelope patterns.

Comparing passive and active hearing: spectral analysis of transient sounds in bats

H. R. Goerlitz and M. Hübner and L. Wiegrebe

J Exp Biol  211  1850-8  (2008)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=18515714

In vision, colour constancy allows the evaluation of the colour of objects independent of the spectral composition of a light source. In the auditory system, comparable mechanisms have been described that allows the evaluation of the spectral shape of sounds independent of the spectral composition of ambient background sounds. For echolocating bats, the evaluation of spectral shape is vitally important both for the analysis of external sounds and the analysis of the echoes of self-generated sonar emissions. Here, we investigated how the echolocating bat Phyllostomus discolor evaluates the spectral shape of transient sounds both in passive hearing and in echolocation as a specialized mode of active hearing. Bats were trained to classify transients of different spectral shape as low- or highpass. We then assessed how the spectral shape of an ambient background noise influenced the spontaneous classification of the transients. In the passive-hearing condition, the bats spontaneously changed their classification boundary depending on the spectral shape of the background. In the echo-acoustic condition, the classification boundary did not change although the background- and spectral-shape manipulations were identical in the two conditions. These data show that auditory processing differs between passive and active hearing: echolocation represents an independent mode of active hearing with its own rules of auditory spectral analysis.

Auditory color constancy: calibration to reliable spectral properties across nonspeech context and targets

C. E. Stilp and J. M. Alexander and M. Kiefte and K. R. Kluender

Atten Percept Psychophys  72  470-80  (2010)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=20139460

Brief experience with reliable spectral characteristics of a listening context can markedly alter perception of subsequent speech sounds, and parallels have been drawn between auditory compensation for listening context and visual color constancy. In order to better evaluate such an analogy, the generality of acoustic context effects for sounds with spectral-temporal compositions distinct from speech was investigated. Listeners identified nonspeech sounds-extensively edited samples produced by a French horn and a tenor saxophone-following either resynthesized speech or a short passage of music. Preceding contexts were "colored" by spectral envelope difference filters, which were created to emphasize differences between French horn and saxophone spectra. Listeners were more likely to report hearing a saxophone when the stimulus followed a context filtered to emphasize spectral characteristics of the French horn, and vice versa. Despite clear changes in apparent acoustic source, the auditory system calibrated to relatively predictable spectral characteristics of filtered context, differentially affecting perception of subsequent target nonspeech sounds. This calibration to listening context and relative indifference to acoustic sources operates much like visual color constancy, for which reliable properties of the spectrum of illumination are factored out of perception of color.

Temporal-envelope constancy of speech in rooms and the perceptual weighting of frequency bands

A. J. Watkins and A. P. Raimond and S. J. Makin

J Acoust Soc Am  130  2777-88  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=22087906

Three experiments measured constancy in speech perception, using natural-speech messages or noise-band vocoder versions of them. The eight vocoder-bands had equally log-spaced center-frequencies and the shapes of corresponding "auditory" filters. Consequently, the bands had the temporal envelopes that arise in these auditory filters when the speech is played. The "sir" or "stir" test-words were distinguished by degrees of amplitude modulation, and played in the context; "next you'll get _ to click on." Listeners identified test-words appropriately, even in the vocoder conditions where the speech had a "noise-like" quality. Constancy was assessed by comparing the identification of test-words with low or high levels of room reflections across conditions where the context had either a low or a high level of reflections. Constancy was obtained with both the natural and the vocoded speech, indicating that the effect arises through temporal-envelope processing. Two further experiments assessed perceptual weighting of the different bands, both in the test word and in the context. The resulting weighting functions both increase monotonically with frequency, following the spectral characteristics of the test-word's [s]. It is suggested that these two weighting functions are similar because they both come about through the perceptual grouping of the test-word's bands.

Contrast constancy in natural scenes in shadow or direct light: A proposed role for contrast-normalisation (non-specific suppression) in visual cortex

J. S. Lauritzen and D. J. Tolhurst

Network  16  151-73  (2005)

/net/pooh/mnt/raid0/Reprints/PMID/16411494.pdf

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=16411494

The range of contrasts in natural scenes is generally thought to far exceed the limited dynamic ranges of individual contrast-encoding neurons in the primary visual cortex. The visual system may employ gain-control mechanisms (Ohzawa et al. 1985) to compensate for the mismatch between the range of natural contrast energies and the limited dynamic range of visual neurons; one proposed mechanism is contrast normalisation or non-specific suppression (Heeger 1992a). This paper aims to evaluate the role of contrast normalisation in human contrast perception, using a computer model of primary visual cortex. The model uses orthogonal pairs of Gabor patches to simulate simple-cell receptive-fields to calculate local, band-limited contrast in a series of 50 digitised photographs of natural scenes. The average range of contrast energies in each image was 2.29 log units, while the "lifetime range" each model simple cell would see across all images was 2.98 log units. These ranges are greater than the dynamic range of real mammalian simple cells. Contrast normalisation (dividing contrast responses by the summed responses of all nearby neurons) reduces contrast ranges, perhaps sufficiently to match them to neurons' limited dynamic ranges. Comparison of images taken under diffuse and direct lighting conditions showed that contrast normalisation can sometimes match these conditions effectively. This may lead to perceptual contrast constancy in the face of spurious changes in contrast caused by natural environmental conditions.

Perceptual distance and the constancy of size and stereoscopic depth

L. Kaufman and J. H. Kaufman and R. Noble and S. Edlund and S. Bai and T. King

Spat Vis  19  439-57  (2006)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=17131650

The relationship between distance and size perception is unclear because of conflicting results of tests investigating the size-distance invariance hypothesis (SDIH), according to which perceived size is proportional to perceived distance. We propose that response bias with regard to measures of perceived distance is at the root of the conflict. Rather than employ the usual method of magnitude estimation, the bias-free two-alternative forced choice (2AFC) method was used to determine the precision (1/sigma) of discriminating depth at different distances. The results led us to define perceptual distance as a bias free power function of physical distance, with an exponent of approximately 0.5. Similar measures involving size differences among stimuli of equal angular size yield the same power function of distance. In addition, size discrimination is noisier than depth discrimination, suggesting that distance information is processed prior to angular size. Size constancy implies that the perceived size is proportional to perceptual distance. Moreover, given a constant relative disparity, depth constancy implies that perceived depth is proportional to the square of perceptual distance. However, the function relating the uncertainties of depth and of size discrimination to distance is the same. Hence, depth and size constancy may be accounted for by the same underlying law.

Selective visual attention ensures constancy of sensory representations: testing the influence of perceptual load and spatial competition

D. Wegener and F. O. Galashan and D. N. Markowski and A. K. Kreiter

Vision Res  46  3563-74  (2006)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=16879852

We report findings from several variants of a psychophysical experiment using an acceleration detection task in which we tested predictions derived from recent neurophysiological data obtained from monkey area MT. The task was designed as a Posner paradigm and required subjects to detect the speed-up of a moving bar, cued with 75% validity. Displays varied according to number of simultaneously presented objects, spatial distance, and difficulty of the task. All data obtained under different levels of competition with multiple objects were compared to a corresponding condition, in which only a single moving bar was presented in the absence of any interfering distracter object. For attended objects, subjects did not show any difference in their ability to detect accelerations, regardless of the strength of inter-object competition or spatial distance. This finding was consistent in all of the experiments, and was even obtained when the acceleration was made hardly detectable. In contrast, increasing competitive interactions either by enhancing number of objects or spatial proximity resulted in strong reduction of performance for non-attended objects. The findings support current noise reduction models and suggest that attention adjusts neuronal processing to ensure a constant sensory representation of the attended object as if this object was the only one in the scene.

Perceptual learning depends on perceptual constancy

P. Garrigan and P. J. Kellman

Proc Natl Acad Sci U S A  105  2248-53  (2008)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=18250303

Perceptual learning refers to experience-induced improvements in the pick-up of information. Perceptual constancy describes the fact that, despite variable sensory input, perceptual representations typically correspond to stable properties of objects. Here, we show evidence of a strong link between perceptual learning and perceptual constancy: Perceptual learning depends on constancy-based perceptual representations. Perceptual learning may involve changes in early sensory analyzers, but such changes may in general be constrained by categorical distinctions among the high-level perceptual representations to which they contribute. Using established relations of perceptual constancy and sensory inputs, we tested the ability to discover regularities in tasks that dissociated perceptual and sensory invariants. We found that human subjects could learn to classify based on a perceptual invariant that depended on an underlying sensory invariant but could not learn the identical sensory invariant when it did not correlate with a perceptual invariant. These results suggest that constancy-based representations, known to be important for thought and action, also guide learning and plasticity.

Lightness constancy and illumination discounting

A. D. Logvinenko and R. Tokunaga

Atten Percept Psychophys  73  1886-902  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21688072

Contrary to the implication of the term "lightness constancy", asymmetric lightness matching has never been found to be perfect unless the scene is highly articulated (i.e., contains a number of different reflectances). Also, lightness constancy has been found to vary for different observers, and an effect of instruction (lightness vs. brightness) has been reported. The elusiveness of lightness constancy presents a great challenge to visual science; we revisit these issues in the following experiment, which involved 44 observers in total. The stimuli consisted of a large sheet of black paper with a rectangular spotlight projected onto the lower half and 40 squares of various shades of grey printed on the upper half. The luminance ratio at the edge of the spotlight was 25, while that of the squares varied from 2 to 16. Three different instructions were given to observers: They were asked to find a square in the upper half that (i) looked as if it was made of the same paper as that on which the spotlight fell (lightness match), (ii) had the same luminance contrast as the spotlight edge (contrast match), or (iii) had the same brightness as the spotlight (brightness match). Observers made 10 matches of each of the three types. Great interindividual variability was found for all three types of matches. In particular, the individual Brunswik ratios were found to vary over a broad range (from .47 to .85). That is, lightness matches were found to be far from veridical. Contrast matches were also found to be inaccurate, being on average, underestimated by a factor of 3.4. Articulation was found to essentially affect not only lightness, but contrast and brightness matches as well. No difference was found between the lightness and luminance contrast matches. While the brightness matches significantly differed from the other matches, the difference was small. Furthermore, the brightness matches were found to be subject to the same interindividual variability and the same effect of articulation. This leads to the conclusion that inexperienced observers are unable to estimate both the brightness and the luminance contrast of the light reflected from real objects lit by real lights. None of our observers perceived illumination edges purely as illumination edges: A partial Gelb effect ("partial illumination discounting") always took place. The lightness inconstancy in our experiment resulted from this partial illumination discounting. We propose an account of our results based on the two-dimensionality of achromatic colour. We argue that large interindividual variations and the effect of articulation are caused by the large ambiguity of luminance ratios in the stimulus displays used in laboratory conditions.

Differential intrinsic bias of the 3-D perceptual environment and its role in shape constancy

A. Aznar-Casanova and M. S. Keil and M. Moreno and H. Supèr

Exp Brain Res  215  35-43  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21927826

In a three-dimensional (3-D) environment, sensory information is projected on a 2-D retina with the consequence that the visual system needs space information for accurately reconstructing the visual world. However, the 3-D environment is not accurately represented in the brain; in particular, the perception of distances in depth is imprecise. It has been argued that the visual system has an intrinsic bias of visual space where targets located on the ground floor are perceived on an implicit elevated surface. We studied how such an intrinsic bias of visual space affects shape constancy. We found that the projected shape of a semicircle can be explained taking into account a differential implicit slant surface. The depth/width ratio, which is a measure for the shape of the stimulus, is overestimated for angular declination smaller than ~60$\,^{\circ}$, while it is underestimated for larger angular declinations. Our results are important for explaining shape constancy and may be important for understanding some perceptual illusions.

Perceptual constancy of texture roughness in the tactile system

T. Yoshioka and J. C. Craig and G. C. Beck and S. S. Hsiao

J Neurosci  31  17603-11  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=22131421

Our tactual perception of roughness is independent of the manner in which we touch the surface. A brick surface feels rough no matter how slowly or how rapidly we move our fingers, despite the fluctuating sensory inputs that are transmitted to the finger. Current theories of roughness perception rely solely on inputs from the cutaneous afferents, which are highly affected by scan velocity and force. The question then is: how is roughness constancy achieved? To this end, we characterized the subject's perceived roughness in six scanning conditions. These included two modes of touch: direct touch, where the finger is in contact with the surface, and indirect touch, where the surface is scanned with a hand-held probe; and three scanning modes: active (moving the hand across a stationary surface), passive (moving the surface across a stationary hand), and pseudo-passive (subject's hand is moved by the experimenter across a stationary surface). Here, we show that roughness constancy is preserved during active but not passive scanning, indicating that the hand movement is necessary for roughness constancy in both direct and indirect touch. Roughness constancy is also preserved during pseudo-passive scanning, which stresses the importance of proprioceptive input. The results show that cutaneous input provides the signals necessary for roughness perception and that proprioceptive input resulting from hand movement-rather than a motor efference copy-is necessary to achieve roughness constancy. These findings have important implications in providing realistic sensory feedback for prosthetic-hand users.

Localization of visual and auditory stimuli during smooth pursuit eye movements

K. Königs and F. Bremmer

J Vis  10  8  (2010)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=20884583

Humans move their eyes more often than their heart beats. Although these eye movements induce large retinal image shifts, we perceive our world as stable. Yet, this perceptual stability is not complete. A number of studies have shown that visual targets which are briefly presented during such eye movements are mislocalized in a characteristic manner. It is largely unknown, however, if auditory stimuli are also mislocalized, i.e. whether or not perception generalizes across senses and space is represented supramodally. In our current study subjects were asked to localize brief visual and auditory stimuli that were presented during smooth pursuit in the dark. In addition, we measured auditory and visual detection thresholds. Confirming previous studies, perceived visual positions were shifted in direction of the pursuit. This shift was stronger for the hemifield the eye was heading towards (foveopetal). Perceptual auditory space was compressed towards the pursuit target (ventriloquism effect). This perceptual error was slightly reduced during pursuit as compared to fixation and differed clearly from the mislocalization of visual targets. While we found an influence of pursuit on localization, we found no such effect on the detection of visual and auditory stimuli. Taken together, our results do not provide evidence for the hypothesis of a supramodal representation of space during active oculomotor behavior.

Visual suppression in the superior colliculus around the time of microsaccades

M. Rolfs and S. Ohl

J Neurophysiol  105  1-3  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21084681

Miniature eye movements jitter the retinal image unceasingly, raising the question of how perceptual continuity is achieved during visual fixation. Recent work discovered suppression of visual bursts in the superior colliculus around the time of microsaccades, tiny jerks of the eyes that support visual perception while gaze is fixed. This finding suggests that corollary discharge, supporting visual stability when rapid eye movements drastically shift the retinal image, may also exist for the smallest saccades.

S. Klingenhoefer and F. Bremmer

Vision Res  51  881-9  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21163288

Perceptual classification in a rapidly changing environment

C. Summerfield and T. E. Behrens and E. Koechlin

Neuron  71  725-36  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21867887

Humans and monkeys can learn to classify perceptual information in a statistically optimal fashion if the functional groupings remain stable over many hundreds of trials, but little is known about categorization when the environment changes rapidly. Here, we used a combination of computational modeling and functional neuroimaging to understand how humans classify visual stimuli drawn from categories whose mean and variance jumped unpredictably. Models based on optimal learning (Bayesian model) and a cognitive strategy (working memory model) both explained unique variance in choice, reaction time, and brain activity. However, the working memory model was the best predictor of performance in volatile environments, whereas statistically optimal performance emerged in periods of relative stability. Bayesian and working memory models predicted decision-related activity in distinct regions of the prefrontal cortex and midbrain. These findings suggest that perceptual category judgments, like value-guided choices, may be guided by multiple controllers.

H. Awater and D. Burr and M. Lappe and M. C. Morrone and M. E. Goldberg

J Neurophysiol  93  3605-14  (2005)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=15843478

Spatiotemporal distortions of visual perception at the time of saccades

P. Binda and G. M. Cicchini and D. C. Burr and M. C. Morrone

J Neurosci  29  13147-57  (2009)

/net/pooh/mnt/raid0/Reprints/PMID/19846702.pdf

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=19846702

Directional remapping in tactile inter-finger apparent motion: a motion aftereffect study

S. Kuroki and J. Watanabe and K. Mabuchi and S. Tachi and S. Nishida

Exp Brain Res      (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=22080151

Tactile motion provides critical information for perception and manipulation of objects in touch. Perceived directions of tactile motion are primarily defined in the environmental coordinate, which means they change drastically with body posture even when the same skin sensors are stimulated. Despite the ecological importance of this perceptual constancy, the sensory processing underlying tactile directional remapping remains poorly understood. The present study psychophysically investigated the mechanisms underlying directional remapping in human tactile motion processing by examining whether finger posture modulates the direction of the tactile motion aftereffect (MAE) induced by inter-finger apparent motions. We introduced conflicts in the adaptation direction between somatotopic and environmental spaces by having participants change their finger posture between adaptation and test phases. In a critical condition, they touched stimulators with crossed index and middle fingers during adaptation but with uncrossed fingers during tests. Since the adaptation effect was incongruent between the somatotopic and environmental spaces, the direction of the MAE reflects the coordinate of tactile motion processing. The results demonstrated that the tactile MAE was induced in accordance with the motion direction determined by the environmental rather than the somatotopic space. In addition, it was found that though the physical adaptation of the test fingers was not changed, the tactile MAE disappeared when the adaptation stimuli were vertically aligned or when subjective motion perception was suppressed during adaptation. We also found that the tactile MAE, measured with our procedure, did not transfer across different hands, which implies that the observed MAEs mainly reflect neural adaptations occurring within sensor-specific, tactile-specific processing. The present findings provide a novel behavioral method to analyze the neural representation for directional remapping of tactile motion within tactile sensory processing in the human brain.

Dynamic, object-based remapping of visual features in trans-saccadic perception

D. Melcher

J Vis  8  2.1-17  (2008)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=19146303

Saccadic eye movements can dramatically change the location in which an object is projected onto the retina. One mechanism that might potentially underlie the perception of stable objects, despite the occurrence of saccades, is the "remapping" of receptive fields around the time of saccadic eye movements. Here we examined two possible models of trans-saccadic remapping of visual features: (1) spatiotopic coordinates that remain constant across saccades or (2) an object-based remapping in retinal coordinates. We used form adaptation to test "object" and "space" based predictions for an adapter that changed spatial and/or retinal location due to eye movements, object motion or manual displacement using a computer mouse. The predictability and speed of the object motion was also manipulated. The main finding was that maximum transfer of the form aftereffect in retinal coordinates occurred when there was a saccade and when the object motion was attended and predictable. A small transfer was also found when observers moved the object across the screen using a computer mouse. The overall pattern of results is consistent with the theory of object-based remapping for salient stimuli. Thus, the active updating of the location and features of attended objects may play a role in perceptual stability.

Perceptual evidence for saccadic updating of color stimuli

M. Wittenberg and F. Bremmer and T. Wachtler

J Vis  8  9.1-9  (2008)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=19146310

Perceptual stability during dramatic changes in olfactory bulb activation maps and dramatic declines in activation amplitudes

R. Homma and L. B. Cohen and E. K. Kosmidis and S. L. Youngentob

Eur J Neurosci  29  1027-34  (2009)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=19291227

We compared the concentration dependence of the ability of rats to identify odorants with the calcium signals in the nerve terminals of the olfactory receptor neurons. Although identification performance decreased with concentrations both above and below the training stimuli it remained well above random at all concentrations tested (between 0.0006% and 35% of saturated vapor). In contrast, the calcium signals in the same awake animals were much smaller than their maximum values at odorant concentrations <1% of saturated vapor. In addition, maps of activated glomeruli changed dramatically as odorant concentration was reduced. Thus perceptual stability exists in the face of dramatic changes in both the amplitude and the maps of the input to the olfactory bulb. The data for the concentration dependence of the response of the most sensitive glomeruli for each of five odorants was fitted with a Michaelis-Menten (Hill) equation. The fitted curves were extrapolated to odorant concentrations several orders of magnitude lower the smallest observed signals and suggest that the calcium response at low odorant concentrations is > 1000 times smaller than the response at saturating odorant concentrations. We speculate that only a few spikes in olfactory sensory neurons may be sufficient for correct odorant identification.

The relationship between saccadic suppression and perceptual stability

T. L. Watson and B. Krekelberg

Curr Biol  19  1040-3  (2009)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=19481454

How actions alter sensory processing: reafference in the vestibular system

K. E. Cullen and J. X. Brooks and S. G. Sadeghi

Ann N Y Acad Sci  1164  29-36  (2009)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=19645877

Our vestibular organs are simultaneously activated by our own actions as well as by stimulation from the external world. The ability to distinguish sensory inputs that are a consequence of our own actions (vestibular reafference) from those that result from changes in the external world (vestibular exafference) is essential for perceptual stability and accurate motor control. Recent work in our laboratory has focused on understanding how the brain distinguishes between vestibular reafference and exafference. Single-unit recordings were made in alert rhesus monkeys during passive and voluntary (i.e., active) head movements. We found that neurons in the first central stage of vestibular processing (vestibular nuclei), but not the primary vestibular afferents, can distinguish between active and passive movements. In order to better understand how neurons differentiate active from passive head motion, we systematically tested neuronal responses to different combinations of passive and active motion resulting from rotation of the head-on-body and/or head-and-body in space. We found that during active movements, a cancellation signal was generated when the activation of proprioceptors matched the motor-generated expectation.

The geometry of perisaccadic visual perception

A. Richard and J. Churan and D. E. Guitton and C. C. Pack

J Neurosci  29  10160-70  (2009)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=19675250

Our ability to explore our surroundings requires a combination of high-resolution vision and frequent rotations of the visual axis toward objects of interest. Such gaze shifts are themselves a source of powerful retinal stimulation, and so the visual system appears to have evolved mechanisms to maintain perceptual stability during movements of the eyes in space. The mechanisms underlying this perceptual stability can be probed in the laboratory by briefly presenting a stimulus around the time of a saccadic eye movement and asking subjects to report its position. Under such conditions, there is a systematic misperception of the probes toward the saccade end point. This perisaccadic compression of visual space has been the subject of much research, but few studies have attempted to relate it to specific brain mechanisms. Here, we show that the magnitude of perceptual compression for a wide variety of probe stimuli and saccade amplitudes is quantitatively predicted by a simple heuristic model based on the geometry of retinotopic representations in the primate brain. Specifically, we propose that perisaccadic compression is determined by the distance between the probe and saccade end point on a map that has a logarithmic representation of visual space, similar to those found in numerous cortical and subcortical visual structures. Under this assumption, the psychophysical data on perisaccadic compression can be appreciated intuitively by imagining that, around the time of a saccade, the brain confounds nearby oculomotor and sensory signals while attempting to localize the position of objects in visual space.

F. Bremmer and M. Kubischik and K.-P. Hoffmann and B. Krekelberg

J Neurosci  29  12374-83  (2009)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=19812313

We make fast, ballistic eye movements called saccades more often than our heart beats. Although every saccade causes a large movement of the image of the environment on our retina, we never perceive this motion. This aspect of perceptual stability is often referred to as saccadic suppression: a reduction of visual sensitivity around the time of saccades. Here, we investigated the neural basis of this perceptual phenomenon with extracellular recordings from awake, behaving monkeys in the middle temporal, medial superior temporal, ventral intraparietal, and lateral intraparietal areas. We found that, in each of these areas, the neural response to a visual stimulus changes around an eye movement. The perisaccadic response changes are qualitatively different in each of these areas, suggesting that they do not arise from a change in a common input area. Importantly, our data show that the suppression in the dorsal stream starts well before the eye movement. This clearly shows that the suppression is not just a consequence of the changes in visual input during the eye movement but rather must involve a process that actively modulates neural activity just before a saccade.

Human thalamus contributes to perceptual stability across eye movements

F. Ostendorf and D. Liebermann and C. J. Ploner

Proc Natl Acad Sci U S A  107  1229-34  (2010)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=20080657

We continuously move our eyes when we inspect a visual scene. Although this leads to a rapid succession of discontinuous and fragmented retinal snapshots, we perceive the world as stable and coherent. Neural mechanisms underlying visual stability may depend on internal monitoring of planned or ongoing eye movements. In the macaque brain, a pathway for the transmission of such signals has been identified that is relayed by central thalamic nuclei. Here, we studied a possible role of this pathway for perceptual stability in a patient with a selective lesion affecting homologous regions of the human thalamus. Compared with controls, the patient exhibited a unilateral deficit in monitoring his eye movements. This deficit was manifest by a systematic inaccuracy both in successive eye movements and in judging the locations of visual stimuli. In addition, perceptual consequences of oculomotor targeting errors were erroneously attributed to external stimulus changes. These findings show that the human brain draws on transthalamic monitoring signals to bridge the perceptual discontinuities generated by our eye movements.

Microsaccadic suppression of visual bursts in the primate superior colliculus

Z. M. Hafed and R. J. Krauzlis

J Neurosci  30  9542-7  (2010)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=20631182

Summation of visual motion across eye movements reflects a nonspatial decision mechanism

A. P. Morris and C. C. Liu and S. J. Cropper and J. D. Forte and B. Krekelberg and J. B. Mattingley

J Neurosci  30  9821-30  (2010)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=20660264

Human vision remains perceptually stable even though retinal inputs change rapidly with each eye movement. Although the neural basis of visual stability remains unknown, a recent psychophysical study pointed to the existence of visual feature-representations anchored in environmental rather than retinal coordinates (e.g., "spatiotopic" receptive fields; Melcher and Morrone, 2003). In that study, sensitivity to a moving stimulus presented after a saccadic eye movement was enhanced when preceded by another moving stimulus at the same spatial location before the saccade. The finding is consistent with spatiotopic sensory integration, but it could also have arisen from a probabilistic improvement in performance due to the presence of more than one motion signal for the perceptual decision. Here we show that this statistical advantage accounts completely for summation effects in this task. We first demonstrate that measurements of summation are confounded by noise related to an observer's uncertainty about motion onset times. When this uncertainty is minimized, comparable summation is observed regardless of whether two motion signals occupy the same or different locations in space, and whether they contain the same or opposite directions of motion. These results are incompatible with the tuning properties of motion-sensitive sensory neurons and provide no evidence for a spatiotopic representation of visual motion. Instead, summation in this context reflects a decision mechanism that uses abstract representations of sensory events to optimize choice behavior.

Stability of the visual world during eye drift

M. Poletti and C. Listorti and M. Rucci

J Neurosci  30  11143-50  (2010)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=20720121

We are normally not aware of the microscopic eye movements that keep the retinal image in motion during visual fixation. In principle, perceptual cancellation of the displacements of the retinal stimulus caused by fixational eye movements could be achieved either by means of motor/proprioceptive information or by inferring eye movements directly from the retinal stimulus. In this study, we examined the mechanisms underlying visual stability during ocular drift, the primary source of retinal image motion during fixation on a stationary scene. By using an accurate system for gaze-contingent display control, we decoupled the eye movements of human observers from the changes in visual input that they normally cause. We show that the visual system relies on the spatiotemporal stimulus on the retina, rather than on extraretinal information, to discard the motion signals resulting from ocular drift. These results have important implications for the establishment of stable visual representations in the brain and argue that failure to visually determine eye drift contributes to well known motion illusions such as autokinesis and induced movement.

Electrophysiological correlates of inter- and intrahemispheric saccade-related updating of visual space

J. Peterburs and K. Gajda and K.-P. Hoffmann and I. Daum and C. Bellebaum

Behav Brain Res  216  496-504  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=20797412

Computational models of spatial updating in peri-saccadic perception

F. H. Hamker and M. Zirnsak and A. Ziesche and M. Lappe

Philos Trans R Soc Lond B Biol Sci  366  554-71  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21242143

Perceptual phenomena that occur around the time of a saccade, such as peri-saccadic mislocalization or saccadic suppression of displacement, have often been linked to mechanisms of spatial stability. These phenomena are usually regarded as errors in processes of trans-saccadic spatial transformations and they provide important tools to study these processes. However, a true understanding of the underlying brain processes that participate in the preparation for a saccade and in the transfer of information across it requires a closer, more quantitative approach that links different perceptual phenomena with each other and with the functional requirements of ensuring spatial stability. We review a number of computational models of peri-saccadic spatial perception that provide steps in that direction. Although most models are concerned with only specific phenomena, some generalization and interconnection between them can be obtained from a comparison. Our analysis shows how different perceptual effects can coherently be brought together and linked back to neuronal mechanisms on the way to explaining vision across saccades.

Spatiotopic coding and remapping in humans

D. C. Burr and M. C. Morrone

Philos Trans R Soc Lond B Biol Sci  366  504-15  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21242139

How our perceptual experience of the world remains stable and continuous in the face of continuous rapid eye movements still remains a mystery. This review discusses some recent progress towards understanding the neural and psychophysical processes that accompany these eye movements. We firstly report recent evidence from imaging studies in humans showing that many brain regions are tuned in spatiotopic coordinates, but only for items that are actively attended. We then describe a series of experiments measuring the spatial and temporal phenomena that occur around the time of saccades, and discuss how these could be related to visual stability. Finally, we introduce the concept of the spatio-temporal receptive field to describe the local spatiotopicity exhibited by many neurons when the eyes move.

Neuronal mechanisms for visual stability: progress and problems

R. H. Wurtz and W. M. Joiner and R. A. Berman

Philos Trans R Soc Lond B Biol Sci  366  492-503  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21242138

How our vision remains stable in spite of the interruptions produced by saccadic eye movements has been a repeatedly revisited perceptual puzzle. The major hypothesis is that a corollary discharge (CD) or efference copy signal provides information that the eye has moved, and this information is used to compensate for the motion. There has been progress in the search for neuronal correlates of such a CD in the monkey brain, the best animal model of the human visual system. In this article, we briefly summarize the evidence for a CD pathway to frontal cortex, and then consider four questions on the relation of neuronal mechanisms in the monkey brain to stable visual perception. First, how can we determine whether the neuronal activity is related to stable visual perception? Second, is the activity a possible neuronal correlate of the proposed transsaccadic memory hypothesis of visual stability? Third, are the neuronal mechanisms modified by visual attention and does our perceived visual stability actually result from neuronal mechanisms related primarily to the central visual field? Fourth, does the pathway from superior colliculus through the pulvinar nucleus to visual cortex contribute to visual stability through suppression of the visual blur produced by saccades?

Internal models of self-motion: computations that suppress vestibular reafference in early vestibular processing

K. E. Cullen and J. X. Brooks and M. Jamali and J. Carriot and C. Massot

Exp Brain Res  210  377-88  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21286693

In everyday life, vestibular sensors are activated by both self-generated and externally applied head movements. The ability to distinguish inputs that are a consequence of our own actions (i.e., active motion) from those that result from changes in the external world (i.e., passive or unexpected motion) is essential for perceptual stability and accurate motor control. Recent work has made progress toward understanding how the brain distinguishes between these two kinds of sensory inputs. We have performed a series of experiments in which single-unit recordings were made from vestibular afferents and central neurons in alert macaque monkeys during rotation and translation. Vestibular afferents showed no differences in firing variability or sensitivity during active movements when compared to passive movements. In contrast, the analyses of neuronal firing rates revealed that neurons at the first central stage of vestibular processing (i.e., in the vestibular nuclei) were effectively less sensitive to active motion. Notably, however, this ability to distinguish between active and passive motion was not a general feature of early central processing, but rather was a characteristic of a distinct group of neurons known to contribute to postural control and spatial orientation. Our most recent studies have addressed how vestibular and proprioceptive inputs are integrated in the vestibular cerebellum, a region likely to be involved in generating an internal model of self-motion. We propose that this multimodal integration within the vestibular cerebellum is required for eliminating self-generated vestibular information from the subsequent computation of orientation and posture control at the first central stage of processing.

M. Zirnsak and R. G. K. Gerhards and R. Kiani and M. Lappe and F. H. Hamker

J Neurosci  31  17887-91  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=22159103

The spatial distribution of receptive field changes in a model of peri-saccadic perception: predictive remapping and shifts towards the saccade target

M. Zirnsak and M. Lappe and F. H. Hamker

Vision Res  50  1328-37  (2010)

/net/pooh/mnt/raid0/Reprints/PMID/20152853.pdf

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=20152853

Visual stability based on remapping of attention pointers

P. Cavanagh and A. R. Hunt and A. Afraz and M. Rolfs

Trends Cogn Sci  14  147-53  (2010)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=20189870

When we move our eyes, we easily keep track of where relevant things are in the world. Recent proposals link this stability to the shifting of receptive fields of neurons in eye movement and attention control areas. Reports of 'spatiotopic' visual aftereffects have also been claimed to support this shifting connectivity even at an early level, but these results have been challenged. Here, the process of updating visual location is described as predictive shifts of location 'pointers' to attended targets, analogous to predictive activation seen cross-modally. We argue that these location pointers, the core operators of spatial attention, are linked to identity information and that such a link is necessary to establish a workable visual architecture and to explain frequently reported positive spatiotopic biases.

Robustness of the retinotopic attentional trace after eye movements

J. D. Golomb and V. Z. Pulido and A. R. Albrecht and M. M. Chun and J. A. Mazer

J Vis  10  19.1-12  (2010)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=20377296

With each eye movement, the image received by the visual system changes drastically. To maintain stable spatiotopic (world-centered) representations, the relevant retinotopic (eye-centered) coordinates must be continually updated. Although updating or remapping of visual scene representations can occur very rapidly, J. D. Golomb, M. M. Chun, and J. A. Mazer (2008) demonstrated that representations of sustained attention update more slowly than the remapping literature would predict; attentional benefits at previously attended retinotopic locations linger after completion of the saccade, even when this location is no longer behaviorally relevant. The present study explores the robustness of this "retinotopic attentional trace." We report significant retinotopic facilitation despite attempts to eliminate or reduce it by enhancing spatiotopic reference frames with permanent visual cues in the stimulus display and by introducing a different task where the attended location is the saccade target itself. Our results support and extend our earlier model of native retinotopically organized salience maps that must be dynamically updated to reflect the task-relevant spatiotopic location with each saccade. Consistent with the idea that attentional facilitation arises from persistent, recurrent neural activity, it takes measurable time for this facilitation to decay, leaving behind a retinotopic attentional trace after the saccade has been executed, regardless of conflicting task demands.

Spatial updating in monkey superior colliculus in the absence of the forebrain commissures: dissociation between superficial and intermediate layers

C. A. Dunn and N. J. Hall and C. L. Colby

J Neurophysiol  104  1267-85  (2010)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=20610793

In previous studies, we demonstrated that the forebrain commissures are the primary pathway for remapping from one hemifield to the other. Nonetheless, remapping in lateral intraparietal cortex (LIP) across hemifield is still present in split brain monkeys. This finding indicates that a subcortical structure must contribute to remapping. The primary goal of the current study was to characterize remapping activity in the superior colliculus in intact and split brain monkeys. We recorded neurons in both the superficial and intermediate layers of the SC. We found that across-hemifield remapping was reduced in magnitude and delayed compared with within-hemifield remapping in the intermediate layers of the SC in split brain monkeys. These results mirror our previous findings in area LIP. In contrast, we found no difference in the magnitude or latency for within- compared with across-hemifield remapping in the superficial layers. At the behavioral level, we compared the performance of the monkeys on two conditions of a double-step task. When the second target remained within a single hemifield, performance remained accurate. When the second target had to be updated across hemifields, the split brain monkeys' performance was impaired. Remapping activity in the intermediate layers was correlated with the accuracy and latency of the second saccade during the across-hemifield trials. Remapping in the superficial layers was correlated with latency of the second saccade during the within- and across-hemifield trials. The differences between the layers suggest that different circuits underlie remapping in the superficial and intermediate layers of the superior colliculus.

Saccadic foveation of a moving visual target in the rhesus monkey

J. Fleuriet and S. Hugues and L. Perrinet and L. Goffart

J Neurophysiol  105  883-95  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21160007

Predictive remapping of attention across eye movements

M. Rolfs and D. Jonikaitis and H. Deubel and P. Cavanagh

Nat Neurosci  14  252-6  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21186360

Many cells in retinotopic brain areas increase their activity when saccades (rapid eye movements) are about to bring stimuli into their receptive fields. Although previous work has attempted to look at the functional correlates of such predictive remapping, no study has explicitly tested for better attentional performance at the future retinal locations of attended targets. We found that, briefly before the eyes start moving, attention drawn to the targets of upcoming saccades also shifted to those retinal locations that the targets would cover once the eyes had moved, facilitating future movements. This suggests that presaccadic visual attention shifts serve to both improve presaccadic perceptual processing at the target locations and speed subsequent eye movements to their new postsaccadic locations. Predictive remapping of attention provides a sparse, efficient mechanism for keeping track of relevant parts of the scene when frequent rapid eye movements provoke retinal smear and temporal masking.

Remapping attention in multiple object tracking

P. D. L. Howe and T. Drew and Y. Pinto and T. S. Horowitz

Vision Res  51  489-95  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21236290

Which coordinate system do we use to track moving objects? In a previous study using smooth pursuit eye movements, we argued that targets are tracked in both retinal (retinotopic) and scene-centered (allocentric) coordinates (Howe, Pinto, & Horowitz, 2010). However, multiple object tracking typically also elicits saccadic eye movements, which may change how object locations are represented. Observers fixated a cross while tracking three targets out of six identical disks confined to move within an imaginary square. The fixation cross alternated between two locations, requiring observers to make repeated saccades. By moving (or not moving) the imaginary square in sync with the fixation cross, we could disrupt either (or both) coordinate systems. Surprisingly, tracking performance was much worse when the objects moved with the fixation cross, although this manipulation preserved the retinal image across saccades, thereby avoiding the visual disruptions normally associated with saccades. Instead, tracking performance was best when the allocentric coordinate system was preserved, suggesting that targets locations are maintained in that coordinate system across saccades. This is consistent with a theoretical framework in which the positions of a small set of attentional pointers are predictively updated in advance of a saccade.

Visual stability and the motion aftereffect: a psychophysical study revealing spatial updating

U. Biber and U. J. Ilg

PLoS One  6  e16265  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21298104

Eye movements create an ever-changing image of the world on the retina. In particular, frequent saccades call for a compensatory mechanism to transform the changing visual information into a stable percept. To this end, the brain presumably uses internal copies of motor commands. Electrophysiological recordings of visual neurons in the primate lateral intraparietal cortex, the frontal eye fields, and the superior colliculus suggest that the receptive fields (RFs) of special neurons shift towards their post-saccadic positions before the onset of a saccade. However, the perceptual consequences of these shifts remain controversial. We wanted to test in humans whether a remapping of motion adaptation occurs in visual perception.The motion aftereffect (MAE) occurs after viewing of a moving stimulus as an apparent movement to the opposite direction. We designed a saccade paradigm suitable for revealing pre-saccadic remapping of the MAE. Indeed, a transfer of motion adaptation from pre-saccadic to post-saccadic position could be observed when subjects prepared saccades. In the remapping condition, the strength of the MAE was comparable to the effect measured in a control condition (33$\pm$7% vs. 27$\pm$4%). Contrary, after a saccade or without saccade planning, the MAE was weak or absent when adaptation and test stimulus were located at different retinal locations, i.e. the effect was clearly retinotopic. Regarding visual cognition, our study reveals for the first time predictive remapping of the MAE but no spatiotopic transfer across saccades. Since the cortical sites involved in motion adaptation in primates are most likely the primary visual cortex and the middle temporal area (MT/V5) corresponding to human MT, our results suggest that pre-saccadic remapping extends to these areas, which have been associated with strict retinotopy and therefore with classical RF organization. The pre-saccadic transfer of visual features demonstrated here may be a crucial determinant for a stable percept despite saccades.

Context dependence of receptive field remapping in superior colliculus

J. Churan and D. Guitton and C. C. Pack

J Neurophysiol  106  1862-74  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21753030

Our perception of the positions of objects in our surroundings is surprisingly unaffected by movements of the eyes, head, and body. This suggests that the brain has a mechanism for maintaining perceptual stability, based either on the spatial relationships among visible objects or internal copies of its own motor commands. Strong evidence for the latter mechanism comes from the remapping of visual receptive fields that occurs around the time of a saccade. Remapping occurs when a single neuron responds to visual stimuli placed presaccadically in the spatial location that will be occupied by its receptive field after the completion of a saccade. Although evidence for remapping has been found in many brain areas, relatively little is known about how it interacts with sensory context. This interaction is important for understanding perceptual stability more generally, as the brain may rely on extraretinal signals or visual signals to different degrees in different contexts. Here, we have studied the interaction between visual stimulation and remapping by recording from single neurons in the superior colliculus of the macaque monkey, using several different visual stimulus conditions. We find that remapping responses are highly sensitive to low-level visual signals, with the overall luminance of the visual background exerting a particularly powerful influence. Specifically, although remapping was fairly common in complete darkness, such responses were usually decreased or abolished in the presence of modest background illumination. Thus the brain might make use of a strategy that emphasizes visual landmarks over extraretinal signals whenever the former are available.

A lack of anticipatory remapping of retinotopic receptive fields in the middle temporal area

W. S. Ong and J. W. Bisley

J Neurosci  31  10432-6  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21775588

The middle temporal (MT) area has traditionally been thought to be a retinotopic area. However, recent functional magnetic resonance imaging and psychophysical evidence have suggested that human MT may have some spatiotopic processing. To gain an understanding of the neural mechanisms underlying this process, we recorded neurons from area MT in awake behaving animals performing a simple saccade task in which a spatially stable moving dot stimulus was presented for 500 ms in one of two locations: the presaccadic receptive field or the postsaccadic receptive field. MT neurons responded as if their receptive fields were purely retinotopic. When the stimulus was placed in the presaccadic receptive field, the response was elevated until the saccade took the stimulus out of the receptive field. When the stimulus was placed in the postsaccadic receptive field, the neuron only began its response after the end of the saccade. No evidence of presaccadic or anticipatory remapping was found. We conclude that gain fields are most likely to be responsible for the spatiotopic signal seen in area MT.

Receptive field positions in area MT during slow eye movements

T. S. Hartmann and F. Bremmer and T. D. Albright and B. Krekelberg

J Neurosci  31  10437-44  (2011)

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=AbstractPlus&list_uids=21775589

Perceptual stability requires the integration of information across eye movements. We first tested the hypothesis that motion signals are integrated by neurons whose receptive fields (RFs) do not move with the eye but stay fixed in the world. Specifically, we measured the RF properties of neurons in the middle temporal area (MT) of macaques (Macaca mulatta) during the slow phase of optokinetic nystagmus. Using a novel method to estimate RF locations for both spikes and local field potentials, we found that the location on the retina that changed spike rates or local field potentials did not change with eye position; RFs moved with the eye. Second, we tested the hypothesis that neurons link information across eye positions by remapping the retinal location of their RFs to future locations. To test this, we compared RF locations during leftward and rightward slow phases of optokinetic nystagmus. We found no evidence for remapping during slow eye movements; the RF location was not affected by eye-movement direction. Together, our results show that RFs of MT neurons and the aggregate activity reflected in local field potentials are yoked to the eye during slow eye movements. This implies that individual MT neurons do not integrate sensory information from a single position in the world across eye movements. Future research will have to determine whether such integration, and the construction of perceptual stability, takes place in the form of a distributed population code in eye-centered visual cortex or is deferred to downstream areas.