Abstract
3D microscopy is a useful tool to visualize the detailed structures and mechanisms of biomedical specimens. In particular, biophysical phenomena such as neural activity require fast 3D volumetric imaging because fluorescence signals degrade quickly. A light-field microscope (LFM) has recently attracted attention as a high-speed volumetric imaging technique by recording 3D information in a single-snapshot. This review highlighted recent progress in LFM techniques for 3D biomedical applications. In detail, various image reconstruction algorithms according to LFM configurations are explained, and several biomedical applications such as neuron activity localization, live-cell imaging, locomotion analysis, and single-molecule visualization are introduced. We also discuss deep learning-based LFMs to enhance image resolution and reduce reconstruction artifacts.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
3D volumetric microscopy technology is evolving to analyze cell behaviors and functions for biomedical applications. In particular, 3D biological information such as neural activity in vivo is a crucial parameter for brain or biological research [1, 2]. Techniques for 3D microscopic imaging can be classified into point-scanning, structured illumination, and light-sheet imaging, which have pros and cons, respectively (Fig. 1a–c) [3,4,5,6]. First, scanning microscopy captures only light emitted from a focal plane by removing out-focused fluorescence through a pinhole, and obtains 3D images through the raster scanning of excitation and detection [4, 7, 8]. However, the overall processing time of scanning microscopy is relatively slow due to the scanning of an entire field-of-view (FOV), and the poor axial resolution is an unresolved limitation [9]. The structured illumination microscopy (SIM) uses patterned illumination to improve spatial resolution, whereas the illumination has limitations in increased shot noise and reduced signal when the frequency of illumination is close to the cutoff frequency of a detector [10,11,12]. Light-sheet microscopy is a technique of collecting scattered light with an objective lens by projecting a light sheet onto a sample using an illuminating objective after aligning the imaging lens and the illuminating lens vertically [6, 13, 14]. The light-sheet approach has also limitations in that the FOV is limited by the depth of field (DOF) and the image quality can be reduced by scattering. 3D tomographic microscopy is also an emerging method to investigate biological cells without fluorescent labeling [15,16,17]. The tomographic approach uses the reconstruction of the 3D refractive index (RI) distribution by capturing holograms with various incident angles of illuminations. Tomographic microscopy can not only acquire the morphology, protein concentration, and dry mass of a target but also evade photo-bleaching and photo-toxicity. However, the frame rate of 3D imaging is relatively low due to mechanical scanning of illuminations, and the tomographic system has a limitation in observing electrical signals such as neural activity [18].
Light-field (LF) imaging is a strategy that captures plenoptic functions representing the intensity and directions of light rays in a 3D space [19, 20]. In particular, a handheld LF camera has emerged for recording 3D imaging without an external light source or array cameras [21,22,23]. The LF camera usually uses microlens arrays (MLAs) placed in the intermediate image plane of optical configuration to collect spatial and directional information about light [24,25,26]. Such a simple optical configuration allows 3D-depth estimation, sub-aperture imaging, and depth refocusing after single exposure capturing [27,28,29,30]. Also, the optical configuration of the LF camera extends the depth of field by reconstructing sub-aperture images [31, 32].
A light-field microscope (LFM) was first designed by Levoy et al. and the 3D information of objects was acquired by placing a microlens array (MLA) on an image plane [33]. Unlike conventional 3D microscopies such as confocal or tomographic microscopy that require mechanical scanning or multiple frames, the LFM captures spatial and angular information of an object without delay by collecting partial images through each channel of the MLA in a single frame (Fig. 1d). The 3D volumetric imaging of LFM realizes a faster frame rate compared to the other 3D microscopy because the LFM can acquire 3D imaging through a single-snapshot [34,35,36]. The LFM is available for various biomedical applications such as the imaging of neuronal activity [37, 38], single-molecule [39], and live-cell [40] owing to the fast 3D imaging of LFM. However, the limitations of LFM are spatial resolution and a low signal-to-noise ratio (SNR) due to the superimposition of spatial information through the MLA [41]. In addition, the issues of photo-bleaching and photo-toxicity still remain because the labeling process is required to observe biological structures or signals. Recent studies are being progressed to solve the resolution issues through novel LFM configurations and deep learning algorithms.
In this review, we will introduce microscopes using light-field technology that can acquire 3D volumetric information in a single-shot (Fig. 1e). This review aims to mainly discuss image acquisition methods and various biomedical applications via the LFM. This article starts by introducing the descriptions of an LFM principle, optical configurations, and image processing methods. The subsequent sections cover biomedical applications such as the imaging of neurons, live-cell, worm locomotion, and single-molecule through light-field microscopic imaging (Table 1). Also, the review includes recent studies of deep learning-based light-field microscopic imaging to enhance image quality. Finally, the outlook and summary conclude the review.
2 LFM Principle
2.1 LF Imaging
A conventional wide-field imaging method captures an image from an image plane formed by a single objective lens, which has a limitation in acquiring the depth information of an object. On the other hand, light-field imaging can record 3D information in a 2D image sensor by dividing spatial and directional data through an objective lens and MLA [44,45,46]. The initial type of LF imaging device consists of an objective lens, MLA, and an image sensor, and the MLA is placed at the image plane of the objective lens [20]. This concept is called a plenoptic 1.0 system, and the MLA separates the converging ray to acquire the spatial and angular information of light. A plenoptic 2.0 system transformed the position of MLA to acquire a high-resolution LF image, and the system can be divided by a Galilean LF scheme and a Keplerian LF scheme [47]. The Galilean LF scheme means that the MLA is placed in front of an image plane position, and the Keplerian LF scheme uses the MLA placed behind an image plane. The LF system has a trade-off relationship between spatial and angular resolution because the system acquires spatial and angular information with a single image sensor. The LF configurations of Galilean and Keplerian have the advantage of high-spatial resolution compared with the plenoptic 1.0 system due to the trade-off relationship. The Keplerian LF scheme captures a real image by recording the image plane in front of the MLA, whereas the Galilean LF scheme collects an inverted virtual image of image plane formed behind the MLA.
2.2 LFM Configurations
The basic configuration of LFM consists of an objective lens, a tube lens, MLA, and an image sensor. The methods of image acquisition and reconstruction can depend on the location of MLA and additional lenses (Fig. 2). The conventional LFM comprises the MLA positioned on the native image plane of the microscope system [45, 48], like the plenoptic 1.0 system. Each MLA perceives the in-focus image of an objective lens and divides the images as micro-images. A Galilean-scheme LFM is also similar to the Galilean LF scheme with the MLA in front of the native plane [49]. The Galilean-scheme LFM can reduce the overall thickness of microscope system by diminishing the optical path length, but needs the MLA with a relatively long focal length. The LFM that the MLA positioned behind the native image plane is called a Keplerian-scheme LFM [40]. The Keplerian-scheme LFM with a relatively long optical path has a high-spatial resolution at the cost of angular resolution degradation, whereas the Keplerian LFM has the disadvantage of a relatively large optical configuration. A Fourier LFM uses the MLA placed on a Fourier (pupil) plane to acquire uniform resolution between each MLA and increase spatial resolution and depth of field [50,51,52]. In addition, the Fourier LFM configuration can increase an image volume and perform efficient computational costs by reducing the overlap of images formed by the MLA. However, the Fourier system still has a trade-off between spatial and angular sampling issues. Various optical parameters such as magnification and focal length can be controlled by additionally integrating relay lenses of various magnifications. Additional optical components such as the relay lens can efficiently improve the optical performance of LFM when applying a commercial MLA with limitations in parameter selection [53, 54]. The confocal LFM technique utilizes a mask to selectively and efficiently detect signals in a focus volume [55]. The confocal LFM has the advantage of a high signal-to-noise ratio (SNR) and low reconstruction artifacts through the removal of background noise.
2.3 LF Image Processing
Each microlens captures micro-images containing spatial and angular information by dividing image volume and recording the partial images in a 2D image sensor. The micro-images captured by the LF imaging are converted into 3D images through computational reconstruction. The pixel signal information recorded in the image sensor can be converted into sub-aperture images as obtained from different angles because the pixel information represents spatial and angular information. Depth information of an object can be analyzed through stereo-matching, which uses visual disparity of each sub-aperture image [28, 56]. Focal stacking or slope estimation of sub-aperture images also allows 3D depth imaging [57, 58]. More efficient and effective algorithms are evolving in LFM applications compared with conventional LF imaging to observe the location of neurons and demix fast signals of neurons [53, 59, 60]. However, conventional LFM algorithms still have issues due to ringing effects, block-wise artifacts, and depth cross talk. The drawbacks are generated by the process of storing 3D information in a 2D image sensor plane, and various studies have been introduced to solve the artifacts by using novel LFM algorithms. A method for reducing artifacts was developed through depth-dependent sampling patterns [61]. Different filter shapes and sizes were registered depending on depths, and artifact-free 3D reconstruction was efficiently performed through additional anti-aliasing filters. A novel deconvolution algorithm using a phase-space deconvolution has also been reported to solve optical aberrations and background noises during image reconstruction [62]. The phase-space approach offers a more uniform 3D point-spread-function (PSF) than the beam propagation model as well as high image contrast and fast convergence speed imaging.
3 Applications of LFM
3.1 Neuron Imaging
One of the critical techniques for understanding neuronal activations in neuroscience is the visualization of neural signals. The analysis of neural activity requires high-speed and high-resolution 3D imaging in entire brain areas because of weak and rapid neural signals [63,64,65]. Although neural imaging has been developed in various methods such as functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) to visualize neuron activity, optical neuron activity imaging provides higher spatial resolution than other methods [40]. A calcium indicator is a representative factor that can observe neural activity using fluorescence changes [66, 67]. The indicator indirectly monitors nerve membrane potential but has the advantage of relatively simple observation due to slow transients and high-signal strength. Calcium imaging is also a useful method for measuring thousands of neuron activity at the same time. Optical imaging of neuronal activity is often observed through signals from genetically encoded organic fluorescence dyes for high contrast imaging [68, 69]. Neuron communication by electrical stimulation lasts about 1 ms and repeats at about hundreds of Hz [35]. Therefore, observation devices with high-spatial and temporal resolution are required for precise analysis, and the 3D imaging of neural imaging offers a broad understanding of neural activity in vivo. Various animals such as C. elegans, zebrafish, and mice are usually used for neural activity imaging, and each animal has its own characteristics [70]. C. elegans and zebrafish are representative objects as animal models in the field of neuroscience due to their transparency and small size [71, 72]. In addition, the objects are relatively easy to identify because the density of neurons is sparse compared to a mammalian brain. For example, the brain of zebrafish contains approximately 100,000 neurons, and the overall brain size is approximately 700 × 500 × 250 μm [40, 73]. The neural activity of objects in a freely behaving state requires a faster frame rate compared to a static state because the object movements cause motion artifacts. Therefore, approximately 50 Hz of LFM imaging is required to observe Ca2+ signals in C. elegans in the freely behaving state.
A compressive LFM was introduced for the high-resolution and high-speed 3D imaging of the zebrafish brain [74]. Conventional LFMs had a problem with image degradation due to scattering in deep brain tissue. To solve this problem, the compressive LFM utilized high-accuracy 3D neuron localization by applying a wave-optical multi-slice model. The position and fluorescence data of neurons were quickly collected by skipping 3D image reconstruction steps. As a result, 3D neural structures of the zebrafish brain were obtained at a sampling rate of 100 Hz. A dictionary LFM technique was also introduced for observing the brain and blood vessels of zebrafish (Fig. 3a and b) [75]. The system reduces image noise and artifacts, which are chronic issues in conventional LFMs due to low laser power, by using the dictionary information trained from general biological samples. The dictionary LFM also demonstrates high contrast and artifacts-free Zebrafish calcium imaging by reducing ambiguity in blood cell counting and providing clarity of nerve observation.
Drosophila brain is opaque and has a dense neural structure, thus the observation of neural activity in the Drosophila is more challenging compared to C. elegans or zebrafish due to light scattering. The removal brain cuticle layer is also required to perform the calcium imaging of the opaque brain. A technique that sparse decomposition LFM combined with light-sheet microscopy was reported for the high-resolution and wide-volume imaging of the Drosophila brain (Fig. 3c) [76]. Two microscopic images were acquired by shifting in the light-sheet mode and the wide-field mode, and the images were merged for clear high-resolution imaging. In addition, the sparse decomposition LFM acquired the image of neuronal cell bodies at a depth of 300 µm through a GCaMP6 injection that efficiently improves neural activity signals. The intensity of neuronal activities was expressed as signal traces by dividing spatial information.
The visualization of neural activity in a mammalian model such as a mouse is a challenging technique due to huge scattering. Recently, a quantitative LFM using the incoherent multiple-scattering method was introduced for the imaging of mouse brain [42]. The quantitative LFM considered various factors such as system aberrations and non-uniform resolution along axial planes to improve image resolution and contrast. The improved fluorescence signal of calcium was also acquired by labeling GCaMP6s and using a high NA water-immersion objective lens. The quantitative LFM offers clear images with up to a 300 μm depth in the awakened mouse brain (Fig. 3d). Conventional LFMs had a limitation that an object is required to be fixed in a position near an objective lens due to the bulky size of LFM systems, except for tiny models such as C. elegans. The method of observing neural activity through head fixation has also limitations in acquiring information according to movements; thus, a head-mounted mini-scope is usually used to observe the neural activity of freely moving animals. A head-mounted miniaturized LFM was developed to observe the brains of freely behaving mice [77]. The LFM comprises miniaturized components such as grin objective lens, MLA, ball lens, and tube lens, resulting in achieving the weight of 4 g. The volumetric Ca2+ imaging was demonstrated by fixing the miniaturized LFM on the head of freely behaving mice. The compact LFM system provides volumetric imaging in the hippocampus area with a 16-Hz frame rate and an imaging area of 700 × 600 × 360 µm.
3.2 Live-Cell Imaging
The observation of structures and mechanisms inside living cells is a crucial technique in biomedical fields such as pharmacology and diagnosis [78, 79]. High-resolution 3D imaging is required to visualize the anatomy and functions of cellular components such as mitochondria or cell membranes. An LFM through the compressed sampling of spatial and angular data was introduced for high-resolution live-cell imaging (Fig. 4a) [40]. The dense sampling was optimized through MLA and an image sensor, and the image was reconstructed by using wave-optics with an inverse-problem deconvolution framework. A spatial resolution of the LFM achieves 300–700 nm, and the reconstruction time of 3D volume is less than milliseconds. The LFM with the compressed sampling provides not only the mitochondria imaging of mouse embryo fibroblasts and HeLa cells but also the membrane imaging of COS-7 with 3D volume rendering. An LFM using mirror-enhanced scanning was developed for high-speed and high-resolution 3D cellular imaging (Fig. 4b) [80]. The conventional LFM had a limitation in the low axial resolution due to a missing-cone issue, i.e., limited spatial frequency components caused by the projection angle of an objective lens [81, 82]. A scanning LFM with a tilted mirror was devised to simultaneously capture a target image and a reflected image by a mirror. A sample was placed on the tilted mirror, and the captured target and mirror images were reconstructed through a phase-space deconvolution algorithm. The approach provides a clear image of the cell edge in an X–Z direction and shows the lateral resolution of 0.4 μm and the axial resolution of 1.5 μm. The 3D volumetric images of mitochondria and membranes within NRK cells were acquired through the scanning LFM, which has a 2-Hz volume rate and an FOV of 90 × 70 × 70 µm. The conventional 3D microscopic systems had difficulty observing Dictyostelium discoideum due to light sensitivity and quick movements of the object. The scanning LFM using the tilted mirror also achieves the 3D imaging of contractile vacuoles and membranes in the Dictyostelium discoideum. A 3D cellular imaging method was also introduced through a Fourier LFM with a customized MLA by optimizing aperture division and a wave optics framework (Fig. 4c) [83]. The aperture division and wave optics framework improve image quality for subcellular imaging, and hybrid PSFs combining experimental and numerical considerations reduce computational artifacts. Imaging performance was analyzed through fluorescent beads before in vitro cellular imaging, and the full width half maximum (FWHM) in 3 µm observation ranges shows 0.3–0.7 µm in a lateral dimension and 0.5–1.5 µm in an axial dimension. The Fourier LFM achieves imaging of immune-stained mitochondria and GFP-stained peroxisomes in COS-7 cells.
3.3 Locomotion Analysis
C. elegans is a useful model not only for neural activity imaging but also for analyzing genetic mutations and external stimuli through behavioral changes. Conventional 2D imaging had limitations in observing the postures and movements of the worm on agar gel; thus, fast 3D microscopic imaging in a wide range is required for accurate analysis. The 3D motion imaging of C. elegans was introduced through a computational depth imaging-based LFM (Fig. 5a) [43]. The posture and movement speed of C. elegans were analyzed by comparing different phenotypes of cuticle collagen mutants. The experimental results show that the movement speed in the two mutants is similar, but that of non-planar deviation and curving rate are clearly distinguished. A study analyzing the locomotion of C. elegans through an FM embedded with a deep learning algorithm was also reported for fast and artifact-free 3D imaging [84]. The system classified behaviors as irregular crawling, forward, or backward as well as analyzed movement speeds and curvatures. The approach also observed calcium signal patterns changes according to the motion behaviors.
3.4 Single-Molecule Imaging
Single-molecule localization microscopy (SMLM) is an imaging technique that efficiently detects molecules in biological structures with a high-spatial resolution. The device allows the observation of subcellular compartments such as neuronal synapses, lysosomes, and nuclear proteins that perform significant roles in cellular functions. The conventional 3D SMLM had some issues in that an axial resolution is reduced due to low photon throughput and extended PSFs. An LFM integrated with an SMLM was developed to improve the axial resolution (Fig. 5b) [39]. The system included the configuration of Fourier light-field microscopy, and utilized algorithms and software optimized for the conventional 2D SMLM. The analyzed results through intervals between beads show that the near-isotropic precision of the system achieves 20 nm over a DOF of 6 µm. The LFM combined with the SMLM also demonstrates sufficient resolution to observe DNA origami nanorulers and the microvilli of Jurkat T cells.
4 Deep Learning Enhanced LFM
Deep learning is a powerful technique that utilizes artificial neural networks for automatically performing feature detections and classifying data. Recently, this approach has been implemented in various biomedical applications such as microscopy [86,87,88,89], MRI [90], and ultrasound imaging [91]. In particular, microscopic imaging can efficiently enhance image quality through deep learning algorithms because the gap in image performances is obvious according to system configurations. For example, high-resolution microscopic imaging was performed through a compact portable microscope system implemented with a deep learning algorithm [92, 93]. This method improves image resolution and corrects color aberrations through the deep learning algorithm trained by high-resolution images from a benchtop microscope and low-resolution images from a portable microscope. The deep neural network was iteratively trained to reduce the gap in image quality between the conventional microscope and the portable microscope. This deep learning-based microscopic imaging is being extended to LFM applications to solve conventional LFM issues (Fig. 6).
The LFM has limitations in low and non-uniform resolution as well as reconstruction artifacts and requires high throughput computational processing to recover complex pixel data. An LFM using a view-channel-depth (VCD) neural network was introduced to overcome these limitations [84]. Synthetic light-field images were produced by matching 3D high-resolution images already acquired through confocal microscopy and input data. The VCD network converted 2D light-field raw data into 3D depth information, and the reliability of the VCD network was improved by iteratively comparing transformed 3D depth images with ground-truth images. Iterations of signal inference continued until the resolution became uniform across the imaging depth. The LFM with the VCD network demonstrates the calcium imaging of C. elegans moving in a microfluidic channel, and represents an acquisition rate of 100 Hz and a processing speed of 13 volumes/s. Various factors such as the number of output channels, filter sizes, and stride were modified to efficiently observe the heart of zebrafish. As a result, the direction of blood flow movement was successfully predicted, and cardiac dynamics were investigated at an acquisition rate of 200 Hz across a 250 × 250 × 150 µm chamber. However, this approach has a limitation in the verification of deep-learning algorithm network for interpreting raw light-field images because ground truths rely on images acquired by conventional microscope systems.
A framework-based hybrid LFM technique was developed to overcome dependence on conventional systems and perform the fast and high-fidelity reconstruction [94]. A new algorithm, called HyLFM-Net, offers high-resolution images by simultaneously performing verification and learning light-field images. This method also combined selective illumination microscopy (SIM) setup with an LFM setup for continuously scanning high-speed volume and contributing to image learning. The LFM and SIM images were simultaneously acquired through a 30/70 beam-splitter, and each acquired image was registered in the same reference volume. A convolutional neural networks (CNN) architecture was designed for image learning, and the LFM image was reconstructed by rearranging individual pixels. The reconstructed 2D convolution layer was converted into a 3D image using axial network filters, and the 3D image was further processed through 3D residual blocks to improve fidelity. The hybrid LFM technique not only achieves a field of view of 350 × 300 × 150 µm and a volume rate of 40–100 Hz but also shows the high-resolution imaging of medaka fish heart. Also, the performance improvement was demonstrated through the multi-scale-structural similarity index measure (MS-SSIM) and peak signal-to-noise ratio (PSNR) of images.
A deep learning LFM technique using a learned iterative shrinkage and thresholding algorithm (LISTA) was also introduced to reduce reconstruction artifacts and simplify a system model [95]. This method can efficiently perform a forward model calibration with labeled data by compressing the number of light-field views. In addition, an unsupervised technique with Wasserstein generative adversarial networks (WGANs) was developed to perform image reconstruction in a no-labeled dataset. The background noise reduction by the unsupervised technique was demonstrated by the imaging of genetically encoded mouse brain slices.
5 Conclusions and Outlooks
In this review, the principles of LFM, image processing methods, and biomedical applications for exploring living organisms have been presented. The LFM is evolving into various LFM configurations through the arrangement of optical components such as an objective lens, MLA, and a relay lens. In addition, various image reconstruction algorithms have been reported to increase image resolution and reduce artifacts. The LFM has been demonstrated through various biomedical applications such as neuron activity visualization, live-cell monitoring, locomotion analysis, and single-molecule imaging. Various LFM approaches were introduced to achieve optimal performances in each application. Also, the deep learning-based LFM successfully provides images with improved spatial resolution and without artifacts. Despite the current progress of LFM, continuous advanced studies are required to realize superior performance compared to other 3D microscope imaging techniques. Improved image resolution and deep penetration performances are required in LFM imaging. The resolution of LFM is inevitably low because the MLA divides spatial information, which reduces resolution compared to other super-resolution microscopes. Sub-cellular imaging requires a high-resolution performance for a deeper understanding of mechanisms in vivo. In addition, the penetration depth of LFM has a restriction due to the scattering of tissue, and image resolution is degraded according to the depth. Overcoming these challenges requires new approaches that diversify optical arrangement, illumination, or image processing algorithms to improve image resolution and penetration depth comparable to that of advanced microscopes. Also, improvements in image-processing speed are also required. One of the LFM advantages is a fast 3D volume image acquisition speed compared to other microscopes, but the image processing time occupies the most time of volumetric imaging. The physical acquisition time of 3D information is relatively fast compared to scanning methods because the LFM acquired 3D information about an object through a single-shot. However, computational processes with time-consuming are required for relocating the mixed information. Advanced techniques for real-time 3D volumetric imaging with deep learning algorithms may continue to reduce the time. The combination of LFM with a microfluidic chip has the advantage of fixing a target model within the observation range. 3D light-field imaging of microfluidic systems such as organ-on-a chips can help to understand the functions of the body. Especially, the LFM can efficiently acquire the in vitro calcium imaging of 3D neural environments with the advantage of fast volumetric imaging. Studies for the miniaturization of LFM are also expected to be developed for expanding various applications such as endoscopy, and point-of-care testing devices. The LFM will also help to efficiently acquire various biological information in diverse animal models with fast volumetric imaging.
References
Vogt, N.: Monitoring 3D neural activity at large scale. Nat. Methods 13, 195–195 (2016)
Marquet, P., Depeursinge, C., Magistretti, P.J.: Review of quantitative phase-digital holographic microscopy: promising novel imaging technique to resolve neuronal network activity and identify cellular biomarkers of psychiatric disorders. Neurophotonics 1, 020901 (2014)
Schermelleh, L., et al.: Super-resolution microscopy demystified. Nat. Cell Biol. 21, 72–84 (2019)
Jonkman, J., Brown, C.M., Wright, G.D., Anderson, K.I., North, A.J.: Tutorial: guidance for quantitative confocal microscopy. Nat. Protoc. 15, 1585–1611 (2020)
Wu, Y., Shroff, H.: Faster, sharper, and deeper: structured illumination microscopy for biological imaging. Nat. Methods 15, 1011–1019 (2018)
Santi, P.A.: Light sheet fluorescence microscopy: a review. J. Histochem. Cytochem. 59, 129–138 (2011)
Pawley, J.: Handbook of biological confocal microscopy, vol. 236. Springer, US (2006)
Yoo, H.-K., et al.: Confocal scanning microscopy: a high-resolution nondestructive surface profiler. Int. J. Precis. Eng. Manuf. 7, 3–7 (2006)
Fischer, R.S., Wu, Y., Kanchanawong, P., Shroff, H., Waterman, C.M.: Microscopy in 3D: a biologist’s toolbox. Trends Cell Biol. 21, 682–691 (2011)
Fu, H.L., et al.: Optimization of a wide field structured illumination microscope for non-destructive assessment and quantification of nuclear features in tumor margins of a primary mouse model of sarcoma. PLoS ONE 8, e68868 (2013)
Schermelleh, L., et al.: Subdiffraction multicolor imaging of the nuclear periphery with 3D structured illumination microscopy. Science 320, 1332–1336 (2008)
Gustafsson, M.G.: Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy. J. Microsc. 198, 82–87 (2000)
Hillman, E.M., Voleti, V., Li, W., Yu, H.: Light-sheet microscopy in neuroscience. Annu. Rev. Neurosci. 42, 295–313 (2019)
Olarte, O.E., Andilla, J., Gualda, E.J., Loza-Alvarez, P.: Light-sheet microscopy: a tutorial. Adv. Opt. Photonics 10, 111–179 (2018)
Kim, K., Kim, K.S., Park, H., Ye, J.C., Park, Y.: Real-time visualization of 3-D dynamic microscopic objects using optical diffraction tomography. Opt. Express 21, 32269–32278 (2013)
Lee, M., Kim, K., Oh, J., Park, Y.: Isotropically resolved label-free tomographic imaging based on tomographic moulds for optical trapping. Light: Sci. Appl. 10, 1–9 (2021)
Kleiber, A., Kraus, D., Henkel, T., Fritzsche, W.: Tomographic imaging flow cytometry. Lab Chip 21, 3655–3666 (2021)
Lee, A.J., Hugonnet, H., Park, W., Park, Y.: Three-dimensional label-free imaging and quantification of migrating cells during wound healing. Biomed. Opt. Express 11, 6812–6824 (2020)
Wilburn, B., et al.: High performance imaging using large camera arrays. ACM Trans. Graph. 24(3), 765–776 (2005)
Ng, R., et al.: Light Field Photography with a Hand-held Plenoptic Camera. Stanford Tech Report, pp. 1–11 (2005)
Lin, R.J., et al.: Achromatic metalens array for full-color light-field imaging. Nat. Nanotechnol. 14, 227–231 (2019)
Shehzad, K., Xu, Y.: Graphene light-field camera. Nat. Photonics 14, 134–136 (2020)
Fan, Q., et al.: Trilobite-inspired neural nanophotonic light-field camera with extreme depth-of-field. Nat. Commun. 13, 1–10 (2022)
Bae, S.I., Kim, K., Jang, K.W., Kim, H.K., Jeong, K.H.: High contrast ultrathin light-field camera using inverted microlens arrays with metal–insulator–metal optical absorber. Adv. Opt. Mater. 9, 2001657 (2021)
Kim, K., Jang, K.W., Ryu, J.K., Jeong, K.H.: Biologically inspired ultrathin arrayed camera for high-contrast and high-resolution imaging. Light Sci. Appl. 9, 28 (2020)
Kim, K., et al.: Ultrathin arrayed camera for high-contrast near-infrared imaging. Opt. Express 29, 1333–1339 (2021)
Martínez-Corral, M., Javidi, B.: Fundamentals of 3D imaging and displays: a tutorial on integral imaging, light-field, and plenoptic systems. Adv. Opt. Photonics 10, 512–566 (2018)
Jeon, H.-G., et al.: Accurate depth map estimation from a lenslet light field camera. in Proceedings of the IEEE conference on computer vision and pattern recognition (2015)
Hahne, C., Aggoun, A., Velisavljevic, V., Fiebig, S., Pesch, M.: Refocusing distance of a standard plenoptic camera. Opt. Express 24, 21521–21540 (2016)
Kim, H.M., Kim, M.S., Lee, G.J., Jang, H.J., Song, Y.M.: Miniaturized 3D depth sensing-based smartphone light field camera. Sensors 20, 2129 (2020)
Bishop, T.E., Favaro, P.: The light field camera: extended depth of field, aliasing, and super resolution. IEEE Trans. Pattern Anal. Mach. Intell. 34, 972–986 (2011)
Kim, H.M., et al.: Vari-focal light field camera for extended depth of field. Micromachines 12, 1453 (2021)
Levoy, M., Ng, R., Adams, A., Footer, M., Horowitz, M.: Light field microscopy. ACM Trans. Graph. 25(4), 924–934 (2006)
Bimber, O., Schedl, D.C.: Light-field microscopy: a review. J. Neurol. Neuromed. 4, 1–6 (2019)
Song, P., Verinaz-Jadan, H., Howe, C.L., Foust, A.J., Dragotti, P.L.: Light-field microscopy for the optical imaging of neuronal activity: when model-based methods meet data-driven approaches. IEEE Signal Process. Mag. 39, 58–72 (2022)
Wang, D., Roy, S., Rudzite, A.M., Field, G.D., Gong, Y.: High-resolution light-field microscopy with patterned illumination. Biomed. Opt. Express 12, 3887–3901 (2021)
Wang, D., et al.: Hybrid light-sheet and light-field microscope for high resolution and large volume neuroimaging. Biomed. Opt. Express 10, 6595–6610 (2019)
Wang, D., Zhu, Z., Xu, Z., Zhang, D.: Neuroimaging with light field microscopy: a mini review of imaging systems. Eur. Phys. J. Spec. Top. 231, 749–761 (2022)
Sims, R.R., et al.: Single molecule light field microscopy. Optica 7, 1065–1072 (2020)
Li, H., et al.: Fast, volumetric live-cell imaging using high-resolution light-field microscopy. Biomed. Opt. Express 10, 29–49 (2019)
Broxton, M., et al.: Wave optics theory and 3-D deconvolution for the light field microscope. Opt. Express 21, 25418–25439 (2013)
Zhang, Y., et al.: Computational optical sectioning with an incoherent multiscale scattering model for light-field microscopy. Nat. Commun. 12, 1–11 (2021)
Shaw, M., et al.: Three-dimensional behavioural phenotyping of freely moving C. elegans using quantitative light field microscopy. Plos one 13, e0200108 (2018)
Lin, Z., Shum, H.-Y.: A geometric analysis of light field rendering. Int. J. Comput. Vis. 58, 121–138 (2004)
Levoy, M., Zhang, Z., McDowall, I.: Recording and controlling the 4D light field in a microscope using microlens arrays. J. Microsc. 235, 144–162 (2009)
Ng, R.: Digital light field photography. Stanford University (2006)
Lumsdaine, A. and T. Georgiev. The focused plenoptic camera. In 2009 IEEE International Conference on Computational Photography (ICCP). IEEE (2009)
Kim, J., Jung, J.-H., Jeong, Y., Hong, K., Lee, B.: Real-time integral imaging system for light field microscopy. Opt. Express 22, 10210–10220 (2014)
Chen, Y., et al.: Design of a high-resolution light field miniscope for volumetric imaging in scattering tissue. Biomed. Opt. Express 11, 1662–1678 (2020)
Guo, C., Liu, W., Hua, X., Li, H., Jia, S.: Fourier light-field microscopy. Opt. Express 27, 25573–25594 (2019)
Sánchez-Ortiga, E., G. Scrofani, M. Martinez-Corral, and G. Saavedra. Fourier-domain lightfield microscopy: a new paradigm in 3D microscopy. In Biomedical Imaging and Sensing Conference 2020. International Society for Optics and Photonics (2020)
Liu, F.L., Kuo, G., Antipa, N., Yanny, K., Waller, L.: Fourier diffuser scope: single-shot 3D Fourier light field microscopy with a diffuser. Opt. Express 28, 28969–28986 (2020)
Prevedel, R., et al.: Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy. Nat. Methods 11, 727–730 (2014)
Quicke, P., et al.: Subcellular resolution three-dimensional light-field imaging with genetically encoded voltage indicators. Neurophotonics 7, 035006 (2020)
Zhang, Z., et al.: Imaging volumetric dynamics at high speed in mouse and zebrafish brain with confocal light field microscopy. Nat. Biotechnol. 39, 74–83 (2021)
Rogge, S., Schiopu, I., Munteanu, A.: Depth estimation for light-field images using stereo matching and convolutional neural networks. Sensors 20, 6188 (2020)
Zhang, C., J. Bastian, C. Shen, A. Van Den Hengel, and T. Shen. Extended depth-of-field via focus stacking and graph cuts. In 2013 IEEE International Conference on Image Processing. IEEE (2013)
Zhang, Y., et al.: Light-field depth estimation via epipolar plane image analysis and locally linear embedding. IEEE Trans. Circuits Syst. Video Technol. 27, 739–747 (2016)
Perez, C.C., et al.: Calcium neuroimaging in behaving zebrafish larvae using a turnkey light field camera. J. Biomed. Opt. 20, 096009 (2015)
Taylor, M.A., Nöbauer, T., Pernia-Andrade, A., Schlumm, F., Vaziri, A.: Brain-wide 3D light-field imaging of neuronal activity with speckle-enhanced resolution. Optica 5, 345–353 (2018)
Stefanoiu, A., Page, J., Symvoulidis, P., Westmeyer, G.G., Lasser, T.: Artifact-free deconvolution in light field microscopy. Opt. Express 27, 31644–31666 (2019)
Lu, Z., et al.: Phase-space deconvolution for light field microscopy. Opt. Express 27, 18131–18145 (2019)
Lin, A., et al.: Imaging whole-brain activity to understand behavior. Nat. Rev. Phys. 4(5), 292–305 (2022)
Yang, W., Yuste, R.: In vivo imaging of neural activity. Nat. Methods 14, 349–359 (2017)
Ji, N., Freeman, J., Smith, S.L.: Technologies for imaging neural activity in large volumes. Nat. Neurosci. 19, 1154–1164 (2016)
Akerboom, J., et al.: Optimization of a GCaMP calcium indicator for neural activity imaging. J. Neurosci. 32, 13819–13840 (2012)
Tian, L., et al.: Imaging neural activity in worms, flies and mice with improved GCaMP calcium indicators. Nat. Methods 6, 875–881 (2009)
Hires, S.A., Tian, L., Looger, L.L.: Reporting neural activity with genetically encoded calcium indicators. Brain Cell Biol. 36, 69–86 (2008)
Lin, M.Z., Schnitzer, M.J.: Genetically encoded indicators of neuronal activity. Nat. Neurosci. 19, 1142–1153 (2016)
Bansal, P., Abraham, A., Garg, J., Jung, E.E.: Neuroscience research using small animals on a chip: from nematodes to zebrafish larvae. BioChip J. 15, 42–51 (2021)
Andalman, A.S., et al.: Neuronal dynamics regulating brain and behavioral state transitions. Cell 177, 970-985.e20 (2019)
Chronis, N., Zimmer, M., Bargmann, C.I.: Microfluidics for in vivo imaging of neuronal and behavioral activity in Caenorhabditis elegans. Nat. Methods 4, 727–731 (2007)
Sumbre, G., De Polavieja, G.G.: The world according to zebrafish: how neural circuits generate behavior. Front. Neural Circuits 8, 91 (2014)
Pégard, N.C., et al.: Compressive light-field microscopy for 3D neural activity recording. Optica 3, 517–524 (2016)
Zhang, Y., et al.: DiLFM: an artifact-suppressed and noise-robust light-field microscopy through dictionary learning. Light: Sci. Appl. 10, 1–12 (2021)
Yoon, Y.-G., et al.: Sparse decomposition light-field microscopy for high speed imaging of neuronal activity. Optica 7, 1457–1468 (2020)
Skocek, O., et al.: High-speed volumetric imaging of neuronal activity in freely moving rodents. Nat. Methods 15, 429–432 (2018)
Yun, H., Kim, K., Lee, W.G.: Cell manipulation in microfluidics. Biofabrication 5, 022001 (2013)
Kim, K., Lee, W.G.: Electroporation for nanomedicine: a review. J. Mater. Chem. B 5, 2726–2738 (2017)
Xiong, B., et al.: Mirror-enhanced scanning light-field microscopy for long-term high-speed 3D imaging with isotropic resolution. Light: Sci. Appl. 10, 1–11 (2021)
Gustafsson, M.G., et al.: Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination. Biophys. J. 94, 4957–4970 (2008)
Campagnola, P.J.: High-speed 3D mapping of nonlinear structures. Nat. Photonics 14, 531–532 (2020)
Hua, X., Liu, W., Jia, S.: High-resolution Fourier light-field microscopy for volumetric multi-color live-cell imaging. Optica 8, 614–620 (2021)
Wang, Z., et al.: Real-time volumetric reconstruction of biological dynamics with light-field microscopy and deep learning. Nat. Methods 18, 551–556 (2021)
Nöbauer, T., et al.: Video rate volumetric Ca2+ imaging across cortex using seeded iterative demixing (SID) microscopy. Nat. Methods 14, 811–818 (2017)
Rivenson, Y., et al.: Deep learning microscopy. Optica 4, 1437–1443 (2017)
Rivenson, Y., Wu, Y., Ozcan, A.: Deep learning in holography and coherent imaging. Light: Sci. Appl. 8, 1–8 (2019)
de Haan, K., et al.: Automated screening of sickle cells using a smartphone-based microscope and deep learning. NPJ Digit. Med 3, 1–9 (2020)
Chen, X., et al.: Deep-learning on-chip light-sheet microscopy enabling video-rate volumetric imaging of dynamic biological specimens. Lab Chip 21, 3420–3428 (2021)
Lundervold, A.S., Lundervold, A.: An overview of deep learning in medical imaging focusing on MRI. Z. Med. Phys. 29, 102–127 (2019)
Liu, S., et al.: Deep learning in medical ultrasound analysis: a review. Engineering 5, 261–275 (2019)
Rivenson, Y., et al.: Deep learning enhanced mobile-phone microscopy. ACS Photonics 5, 2354–2364 (2018)
Huang, X., et al.: Smartphone-based analytical biosensors. Analyst 143, 5339–5351 (2018)
Wagner, N., et al.: Deep learning-enhanced light-field imaging with continuous validation. Nat. Methods 18, 557–563 (2021)
Verinaz-Jadan, H., et al.: Deep Learning for Light Field Microscopy Using Physics-Based Models. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). IEEE (2021)
Acknowledgements
This research was financially supported by a grant of the National Research Foundation of Korea (NRF) (No. 2021R1F1A1048603), the Ministry of SMEs and Startups (No. S3103859), and the Ministry of Trade, Industry and Energy(No. 20020866).
Funding
Korea Technology and Information Promotion Agency for SMEs, No. S3103859, Kisoo Kim, National Research Foundation of Korea (NRF), No. 2021R1F1A1048603, Kisoo Kim, Ministry of Trade, Industry and Energy, No. 20020866, Kisoo Kim.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing financial interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Kim, K. Single-Shot Light-Field Microscopy: An Emerging Tool for 3D Biomedical Imaging. BioChip J 16, 397–408 (2022). https://doi.org/10.1007/s13206-022-00077-w
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13206-022-00077-w