2026 High-fidelity intravital imaging of biological dynamics with latent-space-enhanced digital adaptive optics

High-fidelity intravital imaging of biological dynamics with latent-space-enhanced digital adaptive optics

Intravital fluorescence microscopy is hampered by optical aberrations arising from heterogeneous distributions of the refractive index. Adaptive optics methods are either costly and slow, requiring additional hardware, or inaccurate due to lack of wavefront information in multiple angular directions. Here we present a latent-space-enhanced digital adaptive optics method that uses wave-optics priors embedded in high-dimensional spatial-angular data and semantically disentangles their representations in the latent space. Latent-space-enhanced digital adaptive optics achieves more than sixfold higher aberration estimation accuracy than the existing approach (coordinate-based neural representations for computational adaptive optics). It also exhibits strong robustness across different system configurations and imaging conditions, achieving almost an order of magnitude higher accuracy than iterative digital adaptive optics under extreme conditions such as a low signal-to-noise ratio of three point four decibels. We experimentally demonstrate that latent-space-enhanced digital adaptive optics improves diverse biological observations in vivo, such as large-scale tracking of T cells across an entire lymph node, multiregional neural recording in mouse cortex and long-term monitoring of neutrophil activation, extravasation and clearing through mouse intact skull after traumatic brain injury.

Fluorescence optical microscopy has remained a mainstream technique for probing the intricate and versatile world of cells and tissues, making tremendous contributions to immunology, neuroscience, and cell biology. However, optical aberration induced by the non-uniform distribution of the refractive index across deep tissues or manufacturing flaws in the optical system leads to the distortion of the light wavefront and the subsequent degradation of imaging performance in fluorescence microscopy. Without proper estimation and correction of the aberration, observations will be hindered and many downstream tasks, such as tracking and spike inference, cannot be accurately performed. To correct the distorted wavefront caused by thick tissues, multiple types of adaptive optics methods have been proposed, commonly categorized into hardware adaptive optics, computational adaptive optics and the combination of both. Hardware adaptive optics uses phase-modulating elements such as deformable mirrors to compensate for aberrations, but this increases the complexity of the optical system, and the deformable mirrors can perform only discrete compensation based on ray optics, leading to decreased accuracy. Furthermore, during wavefront estimation, direct sensing requires the formation of a 'guide star', which may not always be feasible, and indirect sensing without a wavefront sensor grows more time-consuming if dealing with higher correction modes and more complicated looping strategies. In contrast to hardware adaptive optics, deep learning computational adaptive optics approaches preserve imaging throughput by deducing the wavefront directly from acquired images and then applying digital compensation during post-processing. Although they require no additional budget or time, these approaches can only deal with small aberrations, typically less than one l, due to a lack of spatial-angular information, which is crucial for decoding wavefront distortion. Recent years have witnessed the emergence of a framework called digital adaptive optics scanning light-field mutual iterative tomography, which synergistically combines hardware-based adaptive optics with computational adaptive optics methodologies. Digital adaptive optics makes use of light-field microscopy spatial-angular data that physically decouple multi-angular information at the hardware level; it algorithmically enables iterative computation of disparities between angular dimensions for aberration estimation. However, it works under simplified ray-optics approximations while neglecting full wave-optics physical models, resulting in limited adaptive optics accuracy. A more effective tool is needed to use embedded wave-optics priors in high-dimensional microscopy measurements. Neural networks have been proven to be capable feature extractors and have the potential to characterize physics-aware aberration representations from spatial-angular measurements. Probing the features extracted by deep learning methods can be beneficial for a wide range of tasks, from detecting hidden variables in dynamical systems to identifying semantic disentanglement in single inferotemporal face patch neurons.

Here we present latent-space-enhanced digital adaptive optics, a latent-space-enhanced digital adaptive optics method in which an autoencoder is employed to utilize physical priors embedded in light-field microscopy spatial-angular measurements and represent them as high-dimensional features in the latent space while preserving wave-optics properties. A customized latent loss is then applied to encourage the disentangling of latent features encoding structural information and aberration wavefronts, the latter of which can be individually decoded by a well-designed estimator. After the joint optimization of the autoencoder and estimator by iterative training, our method is able to accurately estimate aberrations encoded in spatial-angular measurements. Comprehensive simulations confirmed that our method achieved accuracy at least six times higher than a representative deep-learning-based approach, coordinate-based neural representations for computational adaptive optics, in cases with a large aberration magnitude from one l to five l. Another advantage of latent-space-enhanced digital adaptive optics is its strong robustness in various challenging conditions. In the presence of Gaussian noise and photon shot noise, latent-space-enhanced digital adaptive optics can tolerate a fluorescence photon count three times lower than digital adaptive optics, with uncompromised aberration estimation accuracy until the signal-to-noise ratio drops below three point four decibels. The performance of latent-space-enhanced digital adaptive optics is stable with different spatial sampling rates and across five different system modalities, covering almost all light-field-based systems. Latent-space-enhanced digital adaptive optics also shows stability to different angle numbers of spatial-angular measurements, with more than one order of magnitude fewer fluctuations than digital adaptive optics. Experimental results further demonstrated latent-space-enhanced digital adaptive optics' capability to restore high-quality images from aberration contamination. Latent-space-enhanced digital adaptive optics enabled high-fidelity recording and tracking of around five thousand T cells simultaneously across an entire mouse lymph node. After the correction of spatially non-uniform aberrations by latent-space-enhanced digital adaptive optics, mesoscale analysis of multiregional neural activities was achieved with consistent accuracy at the cortex-wide level. Moreover, we applied latent-space-enhanced digital adaptive optics on through-intact-skull imaging, which allows for non-invasive visualization of the mouse brain without introducing additional, uncontrolled damage that disrupts the experimental design. After correction, we observed long-term, sophisticated processes of neutrophil activation in bone marrow, extravasation outside blood vessels and large-scale influx into the major venous sinus through the intact skull of mice challenged with traumatic brain injury, which was impossible by previous means due to severe aberrations.

Results Principle of latent-space-enhanced digital adaptive optics

Ideally, with a spatially uniform refractive index, fluorescence emitted from a point source forms a spherical wavefront. However, in the presence of refractive index inhomogeneities, light rays emitted at different angles undergo deflections, which can be partially corrected by hardware adaptive optics or light-field microscopy with digital adaptive optics, using a Shack-Hartmann sensor or microlens array to collect light and estimate the wavefront based on a simplified geometric ray-optics model. The estimated wavefront at the objective pupil plane is essentially treated as an approximation of the discrete angular components based on pupil segmentation. However, this model neglects the diffraction effect in light propagation, which becomes dominant at the micrometer scale. Wave-optics theory should be taken into consideration to accurately reconstruct a continuous aberration wavefront.

A straightforward approach is to train a neural network that maps raw spatial-angular measurements directly to the aberration wavefront, implicitly encoding the wave-optics prior in a large training set. However, this vanilla regressor yields inaccurate aberration estimations and is unstable to changes in sample structure because it fails to explicitly represent the physical priors required for reliable correction and discard unwanted information that impedes accurate estimation. To better utilize wave-optics features, the proposed LEAO encodes spatial-angular measurements into a high-dimensional latent space using a well-designed encoder. We crafted an encoder-decoder architecture to project physical priors into the latent space, where features that represent the aberration wavefront and the sample structure can potentially be disentangled, reducing the variance of the estimated aberration when the sample structure changes. An estimator operating on the refined latent space then outputs an accurate aberration wavefront, further aided by a dedicated latent loss computed on data triplets that pulls together similar features and pushes apart dissimilar ones during the iterative training. Collectively, these modules constitute the full network architecture of LEAO. Therefore, in the latent manifold encoding aberrations, features corresponding to the same aberration form a distinct cluster; in the manifold encoding sample structures, features associated with the same structural patterns form another. The recovered wavefront is finally applied at the objective pupil to regenerate wave-optics point spread functions, which are incorporated into the three-dimensional reconstruction, thereby generating high-resolution volumes with improved fidelity.

We experimentally evaluated the performance achieved by methods without AO, with DAO and with LEAO on spatial-angular measurements of diverse biological structures. Results without AO exhibited severe structural distortion and blurring due to uncorrected aberrations. DAO only partially recovered structural details due to limited aberration estimation accuracy. In contrast, LEAO produced the reconstructions with high fidelity and contrast, comparable to the ground truth. Statistical analysis further confirmed that LEAO achieved an aberration estimation accuracy at least two times higher than DAO, along with superior reconstruction performance across multiple biological structures.