Scientists Develop Novel Deep Neural Networks For Image Super-Resolution In Optical Microscopy

LI Dong's group from Institute of Biophysics of the Chinese Academy of Sciences, collaborating with Bioland Laboratory, Guangzhou, and DAI Qionghai's group from Tsinghua University, developed the deep Fourier channel attention network (DFCAN) and its derivative that was trained with generative adversarial network (GAN) strategy, termed as DFGAN.

DFCAN and DFGAN are able to outperform the conventional super-resolution (SR) microscopy under most of routine live-cell SR imaging conditions, thus have great potential for democratizing SR imaging with commonly used diffraction-limited microscope. The study was published in Nature Methods.

In this work, scientists made a home-built multi-modality structured illumination microscope (Multi-SIM) that first integrated their previously developed SIM techniques, including TIRF-SIM, Nonlinear SIM (Science), and Grazing Incidence (GI-SIM) (Cell). Multi-SIM allowed acquiring high quality experimental dataset of well-registered pairs of diffraction-limited wide-field and ground truth SIM images. This dataset named as "BioSR" is now open to all researchers.

The dataset provides a benchmark to systematically assess the fidelity and quantifiability of various deep learning super-resolution (DLSR) models in terms of the complexity of observed biological structures, the signal-to-noise ratio (SNR) of input low-resolution images, and the desired upscaling-factors. However, it turns out that current DLSR models hardly gain as high-fidelity SR information as the conventional hardware SR microscopy under the commonly used live-cell imaging conditions, which potentially impedes their applications onto practical experiments.

To improve the DLSR imaging performance, scientists devised DFCAN and DFGAN that utilized the power-spectrum-coverage (PSC) of individual feature maps to adaptively rescale their weightings when propagating through the network.

Because PSC difference in Fourier domain is more prominent than the difference of detailed structures in spatial domain between diffraction-limited input images and ground truth SR images, the discriminative learning ability of DFCAN/DFGAN is significantly enhanced. Therefore, DFCAN/DFGAN is able to outperform the conventional SR microscopy under most of routine live-cell SR imaging conditions, suggesting that it has great potential to democratize super-resolution imaging with commonly used conventional microscope.

To demonstrate the capability of DFCAN and DFGAN for SR live-cell imaging, scientists applied DFCAN/DFGAN to the study of fragile biological processes that are challenging for conventional SR microscopy and other DLSR models.

"We believe that the assessment platform and the concept of DFCAN/DFGAN developed in this work fill an unmet need for evaluating the performance of deep learning imaging methods, and shed new light on future development of SR microscopy," said Prof. LI.