Behavior-relevant top-down cross-modal predictions in mouse neocortex

Behavior-relevant top-down cross-modal predictions in mouse neocortex

Animals adapt to a constantly changing world by predicting their environment and the consequences of their actions. The predictive coding hypothesis proposes that the brain generates predictions and continuously compares them with sensory inputs to guide behavior. However, how the brain reconciles conflicting top-down predictions and bottom-up sensory information remains unclear. To address this question, we simultaneously imaged neuronal populations in the mouse somatosensory barrel cortex and posterior parietal cortex during an auditory-cued texture discrimination task. In mice that had learned the task with fixed tone-texture matching, the presentation of mismatched pairing induced conflicts between tone-based texture predictions and actual texture inputs. When decisions were based on the predicted rather than the actual texture, top-down information flow was dominant and texture representations in both areas were modified, whereas dominant bottom-up information flow led to correct representations and behavioral choice. Our findings provide evidence for hierarchical predictive coding in the mouse neocortex.

Predictive processing has long been an attractive theory of the mind. This theory states that the brain is organized hierarchically, with predictions generated in high-level areas passed down to lower areas, and mismatched sensory inputs that do not fit the predictions creating bottom-up flow that represents prediction errors. Despite the computational attractiveness of this model, its implementation in the brain remains elusive. While reward prediction has been studied extensively, sensory prediction in the neocortex is less understood. It often originates from prior experience, typically through learned associations with other sensory cues, and occurs across many sensory modalities. Such prediction can increase the encoding speed and reduce the neural response to expected stimuli in primary sensory areas, facilitating decisions and behavioral output. Strong sensory predictions can also modify perception, in extreme cases causing hallucination.

One challenge in studying sensory prediction is to simultaneously observe bottom-up and top-down information. Studies targeting long-range projection axons as a proxy for top-down inputs to local populations have demonstrated that such pathways can indeed modulate sensory perception and decision-making. However, studies focusing on how neuronal populations along the brain hierarchy represent and transform information, as well as how they communicate with each other, began only recently. These studies discovered, for example, that top-down and bottom-up information is channeled through separate activity subspaces, and that the communication channels are shaped by experience or learning, especially the top-down subspace. Despite these insights, it is still unknown how top-down predictions and bottom-up sensory inputs interact during behavior and affect behavioral outputs, particularly when they are in conflict.

A key area for routing primary sensory information during active behaviors is the posterior parietal cortex. PPC is densely interconnected with primary sensory areas such as visual, somatosensory and auditory cortex, as well as frontal areas such as the orbitofrontal cortex and the anterior cingulate cortex, and the associative subdivision of thalamus. PPC subserves a wide range of functions including multisensory integration, decision-making, working memory and navigation. In particular, PPC integrates tactile, visual and auditory information in rodents and routes relevant sensory information to frontal areas during active behaviors. As a key area for multisensory integration, PPC is a candidate for generating cross-modal sensory predictions from previously learned associations. Of particular relevance here is that different subdivisions of PPC engage differentially in processing distinct stimulus modalities-the rostrolateral area is activated together with somatosensory barrel cortex during texture discrimination, whereas the anterior area activates with auditory cortex areas in auditory discrimination task. These PPC areas are critical for generating sensory associations and transforming sensory information into decisions, making them potential key high-level areas for generating predictions.

Here we aim to better understand how cortical areas along the cortical hierarchy interact when sequential stimuli from two modalities (auditory and tactile) provide task-relevant information. In this case, repeatedly matching specific pairs of auditory-tactile stimuli allows the animal to form predictions about the second stimulus. It is then especially interesting to reveal how regional neural representations and cross-areal interactions are affected when conflicts between predictions and sensory inputs are imposed. Specifically, we focused on the somatosensory barrel field and PPC subdivisions as representative areas along the hierarchy. We designed a behavioral task with cross-modal sensory predictions (by training mice on matched tone-texture sequence pairs) and then introduced prediction conflicts by occasionally presenting mismatched tone-texture pairs. We used the behavioral choices of mice as a proxy for their perceptual representations. We found that during mismatch, when mice made decisions according to predictions, texture representation in both somatosensory and PPC was modified. Moreover, these changes only occurred when top-down information flow from the relevant subdivision of PPC was dominant, whereas strong bottom-up information flow from somatosensory led to both correct texture encoding and corresponding behavioral outcomes. These results demonstrate the impact of predictions on sensory encoding and suggest that the dynamic interaction between top-down and bottom-up information shapes sensory encoding and affects perceptual choice.

Results

An auditory-cued texture discrimination task

To study sensory prediction, we developed an auditory-cued texture discrimination task for mice. Mice were trained to discriminate two textures (rough versus smooth), each associated with a distinct preceding tone (ten kilohertz versus eighteen kilohertz). Each tone-texture sequence entailed a reward from one of the two lick ports. During learning, the tone-texture pairing remained fixed, allowing mice to develop specific tone-texture association (matched trials). Then, in expert mice, we randomly presented ten to thirty percent tone-texture mismatches (mismatched trials) to introduce prediction conflicts. In these trials, a reward was given according to the tone, to encourage mice to engage in active predictions. For tone-texture mismatches, mice could make the following two types of choices: (one) choose the lick port according to the tone (mismatch-choose-tone or MM>tone trials), indicating that the tone-based prediction, rather than the actual texture stimulus, dominated the decision; (two) choose according to the texture (mismatch-choose-texture or MM>texture trials), indicating that the decision was made based on the actual texture rather than the tone-based prediction.

We trained sixteen mice expressing GCaMP6f in L two/three neurons, all of which could successfully learn the task. Compared to naïve condition, expert mice showed suppressed licking during tone presentation and delayed decision time during texture presentation, indicating that mice associated the tone-texture sequence rather than tone alone with reward. At the end of the experiment, we presented the mice with only tone or only texture, while maintaining the same task structure. With single-modality stimuli, mice still performed above chance level,

albeit with a lower success rate, indicating that mice integrated both sensory modalities to make decisions. Furthermore, mice performed better for only texture compared to only tone presentation. The latter condition resulted in a higher task disengagement rate (miss rate), suggesting that mice regard the missing texture as an incomplete task structure. Finally, testing under texture-only conditions with whiskers removed diminished task performance to chance level. Together, these results indicate that mice integrate tone and texture to perform the task, with texture being the most relevant stimulus, presumably due to its closer temporal link to the trial outcome.

In mismatched trials, when an unexpected texture followed the tone, mice decided less likely according to the texture identity than in matched trials. In both these trial types, decisions were according to texture identity. Similarly, the tone biased the choice of mice to a degree not explainable by mere mistakes. In both these trial types, decisions were opposite to texture identity. When mice chose according to texture under mismatched conditions, the lick probability during the texture and decision windows was slightly reduced, indicating lower decision confidence. When mice chose according to tone, mice licked more decisively, responded faster in general, and were more likely to lick before texture onset. It is worth noting that in most trials, for all trial types, mice started licking after texture onset, indicating that the texture was the most relevant stimulus for the task. These observations were not due to the reward rule in mismatched conditions-the observations were similar when we rewarded mismatched trials according to texture. We also analyzed the movements and pupil diameter of mice. While face and body movements were not substantially different across trial types, the pupil diameter was overall higher in mismatch-choose-texture trials, indicating a higher arousal state that could contribute to mice paying more attention to the texture type. Overall, despite the prominence of the texture stimulus, mice do associate the auditory tone with the following texture stimulus, and the preceding auditory tone does bias the behavior and choice of mice.