Annual Review of Neuroscience Planning in the Brain: It's Not What You Think It Is

Annual Review of Neuroscience Planning in the Brain: It's Not What You Think It Is

Abstract

The neuroscience of planning has long been analogized to search algorithms in artificial intelligence, which simulate future actions to guide immediate choices. We argue that advances in both neuroscience and artificial intelligence suggest that planning is better understood to encompass a broader class of computations where mental simulation supports learning, often well before a decision is needed. We review three neurocomputational mechanisms that illustrate this shift. First, hippocampal replay resembles search but also often occurs prospectively or offline, likely training downstream circuits rather than directly guiding choice. Second, temporally abstract representations, such as grid cells, can enable planning without iterative search. Third, metalearning may shape how prefrontal dynamics implement task-specific planning strategies, echoing how artificial intelligence systems learn to adapt across contexts. This view recasts the brain's planning machinery as a family of learning processes that leverage simulations to build representations and strategies, with forward search as one special case.

One. INTRODUCTION

In the brain sciences, the study of planning-indeed, even the idea that planning is a discrete function to be studied-has long been driven by analogy to a class of algorithms from artificial intelligence known as tree search. Most tree search algorithms in artificial intelligence, from the earliest chess and checkers players to modern refinements with superhuman performance, center around forward search through the "tree" of future board positions. Psychologists have long envisioned that a similar simulation-based search mechanism might underlie biological decision-making in many behavioral domains beyond board games, endowing humans and even animals with the capability to flexibly discover novel courses of action appropriate to whatever circumstance they find themselves in. These psychological theories rested on two key ideas drawn from a stylized version of their artificial intelligence counterpart: that the brain learns a world model (or cognitive map) to mentally simulate actions, and that this simulation unfolds as a forward search from the current situation at the time of the decision. While this framework aligned well with gross behavioral capabilities, the details of how such an algorithm would run in the brain remained unclear, largely because such hypothetical search occurs covertly and is difficult to infer from behavior alone. This limitation motivated a more recent turn to neural measurements in an attempt to more directly observe such model-based simulation and search.

Here, we revisit this evidence, exploring the neuroscience of planning in light of recent progress in artificial intelligence. In doing so, we argue that the classical view of planning as forward search to guide the next choice is, at best, too narrow: a rather unrepresentative special case of a more general family of computations. Instead, we argue that planning is better understood as any type of mental simulation that enables learning, often to improve behavior further in the future. In turn, the mechanisms of planning are themselves fundamentally sculpted by learning in ways that go well beyond the stereotyped search algorithms derived from a simple analogy with artificial intelligence. This argument is grounded in recent developments in both artificial intelligence and neuroscience.

The current explosion of artificial intelligence dramatically demonstrates that the fundamental engine of intelligent behavior is learning from data. In light of this, the usefulness of simulation models lies not only in helping to select the next action but also in their ability to generate limitless, synthetic experience from which one can learn to improve many future choices. Such learning need not, and typically does not, happen at choice times: On the contrary, having a model frees the learner from entrainment to real-world events and enables learning that is faster, safer, and more scalable than learning from real-world interaction. Indeed, in modern artificial intelligence-most obviously in the case of large language models, but even in apparently purer examples of game tree search such as AlphaGo-the benefits of model-generated data disproportionately accrue through precomputation and learning during pretraining, rather than through situation-specific search when a decision is faced.

In psychology and neuroscience, a similar theme is emerging in planning research, emphasizing the critical role of precomputation, often via learning from model-generated data. In what follows, we review this theme in light of three (not mutually exclusive) neurocomputational mechanisms. First, we consider planning via iterative search, most often associated with neural replay. This is the case closest to the classical picture, but both behavioral and neural experiments indicate that the same mechanisms are often engaged prospectively to compute appropriate responses well before a choice is needed.

Second, we consider planning without search. Converging lines of work suggest that world models can also be built to support long-run predictions, enabling flexible planning without iterative simulation. Neurally, these theories intersect with ideas about the function of the grid cells of the entorhinal cortex, which represent physical space (and other metric feature spaces) with periodic functions that capture distant relationships. Such planning-specialized representations must, of course, be built for each task, though the emerging picture of this process looks rather different than the classical picture of learning a one-step world model for search.

Finally, such learning is one example of the critical role of metalearning in planning. In machine learning, this idea formalizes how an agent's strategy for within-task learning can be adapted (metalearned) by outer-loop training across an ensemble of tasks. This process is again most dramatically exemplified by large language models, whose powerful learning in context is enabled by pretraining of the model weights. Neurally, this idea has been used to suggest how activity dynamics in the prefrontal cortex can be shaped to enable bespoke computations for particular tasks, including planning. Coming full circle, this puts a modern spin on venerable psychological ideas of classic planning being supported by working memory in prefrontal cortex, though again, the more computational view emphasizes that this type of planning is not generic but instead shaped by task-specific adaptations.