Deep self-learning enables volumetric microscopy with 3D isotropic resolution

Volumetric fluorescence microscopy is an indispensable tool for comprehensive studies of cells and organs. Since the specimens are inherently three-dimensional (3D), the optimal imaging system should possess high spatial resolution in all directions. However, limited by the operation principle, most microscopic imaging modalities suffer from an anisotropic point spread function (PSF), i.e., the axial resolution is worse than the lateral resolution by 2-3 times, which severely hinders the accurate visualization as well as dissection and analysis of complex volumetric structures inside the biological samples. Hardware solutions to this problem are often very complex, which ultimately restricts the applicability of such methods. Hence, it is an urgent need for current life science research to develop simple, effective, and widely-applicable 3D resolution isotropic restoration methods, so that 3D imaging data can be observed with consistent high resolution from different perspectives.

In a new paper published in Light Science & Application, a team of scientists, led by Professors Hui Gong and Jing Yuan from Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, China, and co-workers have developed a general-purpose two-stage deep self-learning approach termed Self-Net to improve the resolution isotropy for volumetric fluorescence microscopy with fast training and inference speed as well as high reconstruction fidelity. This method employs the self-learning strategy that improves the axial resolution of the anisotropic raw data by using the high-resolution lateral images in the same dataset as rational targets (Fig. 1). The unique feature of this strategy is that it fully exploits the 3D distribution characteristics of the system PSF and eliminates the need for acquiring registered training data or physically modeling the image formation process. Based on this method, it is easy to realize isotropic 3D imaging across different microscopy platforms and is promising to encourage discoveries of new biological insights.

Distinct from the previous approaches, Self-Net employs unsupervised learning for realistic anisotropic degradation, thus allowing the construction of supervised training to impose a strong constraint on the isotropic restoration results, to ensure high-fidelity reconstruction. This two-stage learning framework effectively mitigates the hallucination problems of previously reported methods with substantially enhanced image quality. The authors have verified this method on a variety of commonly used imaging systems and biological samples. The results showed that it cannot only effectively improve the 3D resolution isotropy of wide-field, confocal, two-photon, light-sheet microscopy, but also be applied to stimulated emission depletion (STED) microscopy and super-resolution-structured illumination microscopy (SR-SIM), achieving isotropic 3D super resolution beyond the resolution limit of the original system, as shown in Figure 2.

The authors further demonstrated the effectiveness of Self-Net on imaging data of densely-packed neurites. As shown in Figure 3, as one of the most distinctive interneurons, the axo-axonic cells (AACs) contain extremely dense axon arbors, and the morphological reconstruction of these cells has been limited by the low axial resolution of volumetric imaging. With the help of Self-Net, this problem will no longer be a headache. It can be seen that the fine intermingled axon fibers of the AACs were indistinguishable in the raw axial views, but easily resolved after Self-Net restoration. The isotropic ratio of the 3D data has increased from 0.51 to 0.94, which means that the image quality is generally consistent in different observation angles. The isotropic 3D resolution provided by Self-Net allows the clear visualization and accurate analysis of the complex 3D data of the AACs. The neuron reconstruction speed is nearly four times higher than before, greatly promoting the analysis efficiency.

Furthermore, the high effectiveness and applicability of Self-Net enable a time- and data-efficient pipeline to achieve isotropic whole-brain imaging (DeepIsoBrain) at a voxel resolution of 0.2 × 0.2 × 0.2 μm³, which significantly improves the reconstruction efficiency and accuracy of complex single-neuron morphology and can be a promising approach to facilitate morphology-based neuroscience research.

Overall, the Self-Net deep learning framework is an effective and reliable approach to achieving resolution isotropy of volumetric microscopy. It can be anticipated that Self-Net will be useful in a variety of fields to improve not only existing 3D imaging datasets but also new data acquisitions. For biological organs and tissues with inherent 3D structure distribution, the isotropic 3D resolution provided by this method can effectively promote the accurate 3D reconstruction and analysis of samples, which is expected to promote new discoveries and new progress in life sciences and other fields.

###

References

DOI

10.1038/s41377-023-01230-2

Original Source URL

https://doi.org/10.1038/s41377-023-01230-2

Funding information

This work was supported by the National Science and Technology Innovation 2030 Grant (2021ZD0201001), the National Natural Science Foundation of China (Grant No. 81827901, T2122015), the CAMS Innovation Fund for Medical Sciences (2019-I2M-5-014), and the Fundamental Research Funds for the Central Universities (HUST: 2019kfyXMBZ011). We also thank the Optical Bioimaging Core Facility of WNLO-HUST for the support in data acquisition, and the director fund of the WNLO.

About Light: Science & Applications

The Light: Science & Applications will primarily publish new research results in cutting-edge and emerging topics in optics and photonics, as well as covering traditional topics in optical engineering. The journal will publish original articles and reviews that are of high quality, high interest and far-reaching consequence.