Introduction

Fluorescence microscopy has become an essential tool for visualizing cellular structures and properties of biological specimens by using fluorescence as a means of identifying cellular components. For example, using fluorescent live/dead stains as well as fluorescently-labeled particles or molecules (such as antibodies), researchers are able to characterize the effect of a treatment on cell population. While many techniques allow for a quantitative assessment of cell phenotype, in the case of low cell numbers and when studying structure and/or viability of adherent cells, fluorescence microscopy remains the tool of choice. Features of the biological sample and staining procedures may affect the quality of the acquired image. Deconvolution fluorescence microscopy provides a way of overcoming the inherent limitations of the optical microscopy system by making use of the intrinsic properties of the system to increase image contrast and resolution after image acquisition. This allows for improved visualization at subresolution scales.

A number of deconvolution methods have been previously introduced and include nearest neighbor methods1, Wiener filtering2, iterative constrained regularization methods2,3,4,5,6 and Bayesian-based methods7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23. Bayesian-based deconvolution methods are statistical methods that model images, point spread functions (PSF) and imaging noise as probability distributions and can be separated into two classes: maximum likelihood (ML) methods and maximum a posteriori (MAP) methods. While current Bayesian-based deconvolution methods account for inherent image noise characteristics, they are sensitive to low signal-to-noise ratio (SNR) imaging conditions where prior model parameter estimation is very challenging due to the presence of noise, thus leading to reduced deconvolved image quality. A lot of focus on deconvolution fluorescence microscopy in dealing with the presence of noise in low SNR imaging conditions has revolved around the introduction of spatial regularization priors17,20 to impose spatial smoothness for suppressing noise characteristics. Other strategies, such as ForWarD24, have been proposed to deal with the presence of noise by first performing deconvolution followed by noise suppression25,26,27,28. In this study, we investigate and attempt to mitigate this issue from a different perspective by introducing an MAP method that performs deconvolution on fluorescence microscopy images using dynamically updated nonstationary expectation estimates, which does not make explicit use of spatial regularization.

Here, we model the deconvolved fluorescence microscopy image (f) and the measured fluorescence microscopy image (g) as probability distributions. In the proposed MAP method, the goal is to determine the most probable deconvolved image given the measured image g, based on prior knowledge related to f. The MAP problem can be formulated as:

where p(f | g) is the conditional probability of f given g. We model g as Poisson distributed and model f as a nonstationary process with a nonstationary expectation E(fs) and a variance τ2, where s denotes the pixel site location. In previous MAP methods10, the measured fluorescence microscopy image g was used as an estimate of E(fs), based on the assumption that the measured image, having undergone the influence of the PSF, is similar to a nonstationary local average and thus representative of a nonstationary expectation E(fs). However, this assumption may not be reliable in high noise situations under low SNR imaging conditions, as well as result in high-varying estimates depending on the inherent PSF. One can obtain a more reliable estimate of E(fs) by computing local windowed averages of g; however, this method is also highly sensitive to the presence of high noise levels. To tackle this issue, we introduce a dynamically updated kernel-based estimate (based on the work by28,29) of E(fs) that updates with each iteration. Hence, we derive the iterative solution of the proposed MAP method (see Methods for full derivation), yielding:

where denotes the deconvolved image at iteration j, denotes convolution operator, H denotes the point spread function (PSF), H−s denotes the adjoint of H, λ and β denote relaxation parameters and Wi denotes a window centered at location i.

Eq. (2) denotes an update step of the iteration solution. The importance of introducing a dynamically updated kernel-based estimate of that updates with each iteration is that it allows for a reliable estimate of under the influence of image noise. Hence, the proposed MAP method results in a solution that remains stable under low SNR imaging conditions, leading to improved image quality of the deconvolved image f. The proposed solution, to the best of our knowledge, is the first to consider to incorporate dynamically updated nonstationary expectation estimates within a Bayesian theoretical framework for the purpose of deconvolution fluorescence microscopy, which allows for more stable solutions without the explicit use of spatial regularization.

Results

To evaluate the proposed MAP method with dynamically updated nonstationary expectation estimates (which we will refer to as MAP-D in figures), we first simulated a fluorescence microscopy data set with fluorescence-stained cell populations using a modified version of SIMCEP30, a computational framework for simulating fluorescence microscopy images of fluorescence-stained cell populations (see Fig. 1b and Methods). For comparison purposes, three other Bayesian-based methods were tested: Lucy-Richardson (LR) deconvolution7,8, Markov-Chain Monte-Carlo Wiener-Hunt (MCMC-WH) deconvolution12 and Hunt MAP (MAP-Hunt) deconvolution method with Poisson likelihood and Gaussian image prior10. Note that LR, MAP-Hunt and the proposed MAP-D methods are performed without the explicit use of spatial regularization to test the hypothesis that MAP-D can achieve improved fluorescence microscopy image quality in the presence of noise, without explicit use of spatial regularization. The proposed MAP-D method (Fig. 1f) was able to achieve a significant increase in contrast and resolution when compared to the original fluorescence microscopy acquisition, as well as a noticeable SNR increase (Fig. 1f). This is most evident by the fact that all 4 subcellular structures inside the cytoplasm of each cell are visible in the deconvolved image produced using the proposed MAP-D method when compared to the original fluorescence microscopy acquisition. Compared to MAP-Hunt deconvolution10 (Fig. 1e), the proposed MAP-D method achieved greater increases in contrast and resolution and noticeable SNR improvements. This is particular relevant when studying bacteria binding to cells as the increase in contrast and resolution enables to accurately identify individual particles. The proposed MAP-D method also provides added benefits when studying cell processes such as microvesicles and cell interactions with nanoparticles.

Figure 1
figure 1

Application to simulated fluorescence microscopy data of fluorescence-stained cell population.

(a) ground-truth simulated fluorescence microscopy imaging data and (b) simulated measured fluorescence microscopy imaging data. (c)-(f) deconvolution results for b (c): LR, (d): MCMC-WH, (e): MAP-Hunt, (f): MAP-D. Contrast is significantly improved in MAP-D when compared to b, along with noticeable SNR increases.

Full size image

The proposed MAP-D method was then evaluated on fluorescence microscopy acquisitions of cells obtained from an ocular surface wash following sodium fluorescein instillation (see Methods). Fig. 2a shows an acquisition of cell aggregates, some staining with fluorescein (green) while others stain red with the dead stain (ethidium bromide). Fig. 2f shows a live/dead (calcein blue/ethidium bromide-red) acquisition of two corneal epithelial cells. (Fig. 2a,f and Methods). The proposed MAP-D method (Fig. 2e,j) was able to achieve a significant increase in contrast and resolution when compared to the original fluorescence microscopy acquisitions (Fig. 2). Compared to MAP-Hunt deconvolution10 (Fig. 2), the proposed MAP-D method achieved greater increases in contrast and resolution and moderate SNR improvements. Based on a cross-sectional profile of an area of interest (Fig. 2k) and a background area (Fig. 2l) from Fig. 2a, we can observed that the proposed method improve contrast and resolution without introducing undesirable artifacts. Additional deconvolution results from microscopy acquisitions in Fig. 3 further reinforce these observations. In the original fluorescence microscopy acquisitions (Fig. 3a,f,k,p), due to low contrast levels, it was difficult to determine the nature of the cells present as the cell walls could not be clearly identified. Deconvolution results achieved using LR, MAP-Hunt and MCMC-WH resulted in a cell wall that is easier to detect but also introduced noticeable noise artifacts and thus reduced the overall image quality (Fig. 3b-d, g-i, l-n, q-s). With the proposed MAP-D method, improved cell wall identification as well as improved contrast were obtained without increasing noise artifacts (Fig. 3f, k, o, t).

Figure 2
figure 2

Application to fluorescence microscopy data of biological specimens, using fluorescence microscopy imaging of ocular surface wash.

(a) Cell aggregates can be observed, some staining with fluorescein (green) while others stain red with the dead stain (ethidium bromide). (f) live/dead (calcein blue/ethidium bromide-red) fluorescent image of two corneal epithelial cells. The left and right dotted white lines found in a mark the plots shown in k and l, respectively. The top row corresponds to deconvolution results for a (b): LR, (c): MCMC-WH, (d): MAP-Hunt, (e): MAP-D. Contrast and resolution is significantly improved in MAP-D when compared to a, along with noticeable SNR increases when compared to b-d. The bottom row corresponds to deconvolution results for f (g): LR, (h): MCMC-WH, (i): MAP-Hunt, (j): MAP-D. Similarly, contrast and resolution is significantly improved in MAP-D when compared to f, along with noticeable SNR increases when compared to g-j. (k) Intensity-normalized line plot (red channel) through an area of interest in a. Contrast is significantly enhanced in MAP-D when compared to the original acquisition. (l) Intensity-normalized line plot (blue channel) through an background area in a. SNR is noticeably improved in MAP-D when compared the other tested methods. The PSFs of the microscope for (m) blue, (n) green and (o) red channels, respectively.

Full size image
Figure 3
figure 3

Application to fluorescence microscopy data of biological specimens.

(a) fluorescence microscopy imaging data of an ocular surface wash following sodium fluorescein instillation. Debris and dead corneal epithelial cells can be observed. The deconvolution results for a (b): LR, (c): MCMC-WH, (d): MAP-Hunt, (e): MAP-D). Contrast and resolution is significantly improved in MAP-D when compared to a, along with noticeable SNR increases when compared to b-d. (f) and (k) fluorescence microscopy imaging data of an ocular surface wash following sodium fluorescein instillation. A potential fluorescein-stained cell (green staining) surrounded by dead corneal epithelial cells are somewhat visible. The deconvolution results for f,k (g,l): LR, (h,m): MCMC-WH, (i,n): MAP-Hunt, (j,o): MAP-D). Contrast and resolution is significantly improved in MAP-D when compared to f and k, along with noticeable SNR increases when compared to g-i and l-n, respectively. (p) fluorescence microscopy imaging data of an ocular surface wash following sodium fluorescein instillation: three potential fluorescein-stained cell (green intra-cellular staining) and each nucleus stained with ethiudium bromide (red) indicating cell death. The deconvolution results for p (q): LR, (r): MCMC-WH, (s): MAP-Hunt, (t): MAP-D). The contrast of the cell wall is improved in MAP-D compared to p with SNR increases compared to q-s. (u) imaging apparatus for obtaining fluorescence microscopy acquisitions of cells obtained from an ocular surface wash following sodium fluorescein instillation.

Full size image

Discussion

In this study, we introduce a Bayesian-based method for performing deconvolution using dynamically updated nonstationary expectation estimates that can improve the fluorescence microscopy image quality. The key factor to the improved performance of the proposed method compared to the other tested Bayesian-based deconvolution methods is the introduction of dynamically updated expectation estimates into a Bayesian-based deconvolution framework, which greatly increases the stability of the solution in the presence of noise, without explicit use of spatial regularization. Furthermore, by dynamically updating the expectation estimate at each iteration, the estimate of E(fs) becomes increasingly more reliable as the deconvoluted image estimate becomes more accurate. The only free parameters of the proposed MAP method are λ, β, W and the number of iterations for deconvolution, which can be adjusted by the user to find a tradeoff between image quality and computational costs.

Analysis of ex vivo collected human cells present different challenges. In the context of our study, due to the nature of the collection (gentle eye wash), the presence of debris can make cell identification difficult (as illustrated in Figs. 2a and 3a). Corneal epithelial cells collected from the ocular surface can also uptake various stains during the procedures; green from instillation of sodium fluorescein on the ocular surface, blue from calcein blue if the cell is alive and red from ethidium bromide if the cell is dead. Ghost cells, cells without a nucleus, may also be present in the ocular wash. Current research aims at determining how corneal epithelial cells uptake fluorescein and how treatment, such as different combinations of lens cleaning solutions and contact lenses, can affect the ocular surface and cell shedding (the cells collected during the gentle eye wash). Fluorescence microscopy allows to characterize these cells, however poor image quality can prevent efficient counting and investigation of the population of cells present in the ocular wash. While confocal microscopy may be seen as an alternative, cost, cell transport, acquisition time and operator experience make confocal microscopy ill-suited for such investigations. Other techniques have been used to collect and study these cells31,32; however, the image quality of fluorescein-stained cells was also a challenge, highlighting the need of a back-end approach to help resolve such issues as the cell collection technique is not able to provide significant improvements in the image quality of the cell samples. This is likely due to the complexity of ex vivo cell collection and the fragility of the cells studied which restricts processing protocols and times. The ability to increase fluorescence image quality using the proposed MAP-D method so that cellular membranes and staining patterns can be better identified can further enhance the use and benefits of fluorescence microscopy. Furthermore, increasing fluorescence image quality using the proposed MAP-D method could lead to improvements in tasks used for data analysis such as cell counting33 and data alignment34,35. As such, the proposed method can be valuable for improving fluorescence microscopy image quality to facilitate for the study of biological specimens, more specifically ex vivo collected samples where the presence of debris or other collection artifacts may exist and where significant cell processing is not an option due to low cell numbers and poor cell viability. Future work involves the investigation of alternative expectation estimation approaches, such as Monte Carlo estimation strategies36,37,38,39, to further improve fluorescence microscopy image quality in the presence of noise.

Methods

Derivations

A full derivation of the iterative solution of the proposed MAP method can be described as follows. Let S be a set of pixel locations into a discrete lattice and be a pixel location in . Let and be random fields on S, with Fs and Gs taking on values representing the deconvolved fluorescence microscopy image and measured fluorescence microscopy image at pixel s respectively. Let and be realizations of F and G respectively. Let H denote the point-spread function. The goal is to estimate f given g, based on prior knowledge related to f.

this is equivalent to:

where p(g | f) is the likelihood and p(f) is the prior. In the case of fluorescence microscopy, image noise is primarily related to quantum photon emission and as such can express p(g | f) as follows:

where denotes the convolution operator. As a prior model p(f), we employ a Gaussian prior model with a nonstationary expectation E(fs) and a variance τ2:

Therefore, Eq. (4) can be reformulated as:

This is equivalent to minimizing the negative logarithm of p(g | f) p (f), giving us:

Eq. (8) is equivalent to the application of Tikhonov L2 regularization to Kullback-Leibler divergence of f from g. To find the solution that minimizes Eq. (8), we take the derivative and set it to zero:

where λ is a relaxation parameter and H−s is the adjoint of H. Simplifying further:

Eq. (11) is the direct result of the regularization used in Eq. (8). Other choices of regularization in Eq. (8) could lead to alternative strategies16.

To establish the iterative solution, let us assume that at convergence, where denotes the deconvolved image at iteration j. The assumption that at convergence is a commonly-made assumption behind many iterative Bayesian-based deconvolution methods7,8,9,10,11. The rationale behind this assumption is that convergence to the solution is reached when the estimate no longer changes from one iteration to the next (i.e., the estimate at iteration j + 1 (i.e., ) is equal to the estimate at iteration j (i.e., ). Therefore, at convergence, , which is equal to . Therefore,

Since E(fs) is unknown, we instead use an estimate , giving us:

Contrary to existing methods10, which employ the measured fluorescence microscopy image g as an estimate of , we introduce a dynamically updated estimate of , based on the concept of kernel regression, that updates with each iteration for a more reliable estimate of under the influence of noise, particularly low-SNR scenarios, which can be expressed by:

where Ws denotes a window centered at location s and K is a kernel function. Therefore, Eq. (15) is in essence a weighted mean estimate, where the kernel function K acts as a weighting function such that more relevant samples (as determined using K) contribute more to the estimate of . Plugging Eq. (15) into Eq. (14), with the following Gaussian kernel function:

where β is a relaxation parameter, gives us the final iterative solution of the proposed MAP method, yielding:

Imaging apparatus

For this study, the Nikon fluorescence microscope Eclipse, TE2000-S, with a triple band fluorescence filter module (DAPI/FITC/TRITC) was used (Fig. 3k) to obtain fluorescence microscopy acquisitions of cells obtained from an ocular surface wash following sodium fluorescein instillation. The microscope was equipped with a motorized XY stage and a microcontroller from Applied Scientific Instrument. A DS-Fi1 digital camera and its controller as well as the NIS-Elements software for multi-dimensional acquisition were purchased from Nikon. Image acquisition was performed using a 20x objective (resulting in a 200x magnification when accounting for magnification of the eye piece). Before deconvolution on fluorescence microscopy imaging data can take place with the proposed MAP method, it is first necessary to estimate the PSF of the microscope. In this study, the Nikon fluorescence microscope Eclipse TE2000-S was characterized based on a number of image acquisitions of point source targets to estimate the PSF of the microscope. Note that the PSF is characterized at each of the three different wavelengths used in the microscope to account for differences in PSF at different wavelengths. The PSFs of the microscope for blue, green and red channels are shown in Fig. 2m,n,o, respectively. In the situation where the PSF of the microscope is not known, PSFs obtained using simulations or estimated using other means may also be used in the proposed MAP deconvolution method.

Simulation of fluorescence microscopy data sets

To evaluate the proposed MAP method, we simulated a ground-truth fluorescence microscopy data set with fluorescence-stained cell populations (Fig. 1a) and a convolved and noisy fluorescence microscopy data set from the same cell population (Fig. 1b) using a modified version of SIMCEP30, a computational framework for simulating fluorescence microscopy images of fluorescence-stained cell populations. The only modification to SIMCEP is the introduction of an additional Poisson process to generate pixel intensities in the SIMCEP framework. We configure SIMCEP to generate cell populations with visible cytoplasm, nuclei and a total of 4 subcellular structures inside the cytoplasm of each cell and allowing for overlapping cells, with energy of autofluorescence and variance of CCD detector noise at 0.05 and 0.001, respectively, resulting in an SNR of ∼20.7dB. The PSF used in the simulations is a Gaussian function with a standard deviation of 4 pixels.

Sample preparation

Sample preparation was performed for this study in the following manner. Following sodium fluorescein instillation, cells from the ocular surface were collected by a gentle eye wash using the Ocular Surface Cell Collection Apparatus40. Cells were centrifuged and then stained using the live/dead stain (calcein blue/ethidium bromide). Following a 45 minute incubation in the dark at room temperature, cells were imaged immediately using a Nikon Eclipse fluorescence microscope. Some cells uptook sodium fluorescein and appeared green; live cells stained with Calcein blue and their nucleus appeared blue while dead cells stained with ethidium bromide and were characterized by a red nucleus.

Implementation details

The proposed MAP method (referred to here as MAP-D) is implemented in MATLAB (The MathWorks, Inc.), with the kernel-based estimate of implemented in C + + and compiled a dynamically linked MATLAB Executable (MEX) for to improve computational speed. The only free parameters of the proposed MAP-D method are λ, β, W and the number of iterations for deconvolution, which can be adjusted by the user to find a tradeoff between image quality and computational costs. For the experiments using fluorescence microscopy acquisitions, the parameters λ, β, W and the number of iterations for deconvolution for MAP-D are set to 0.2, 625, a 81-sample window and 50, respectively, as they were found via empirical testing to provide strong results. Note that there is a tradeoff between estimation reliability and computational complexity when selecting W. A W that is too small would not provide reliable enough statistical information to produce a reliable estimate, while a W that is too large would be computationally costly with limited gain in estimation reliability. For Lucy-Richardson (LR) deconvolution7,8, the only free parameter was the number of iterations and that was set to 50. For the Hunt MAP (MAP-Hunt) deconvolution method with Poisson likelihood and Gaussian image prior10, the free parameters are λ and the number of iterations and these were set to 0.2 and 50, respectively. All parameters are chosen to be consistent across the LR, MAP-Hunt and MAP-D methods (e.g., all shared parameters are the same) and provide strong image contrast and resolution improvements across tested acquisitions. Finally, for Markov-Chain Monte-Carlo Wiener-Hunt (MCMC-WH) deconvolution12, the code was provided in MATLAB by the MCMC-WH authors12 and the free parameters are the number of iterations for burn-in and the maximum number of iterations and these were set to 30 and 150, respectively, based on author-provided information. The discontinuities along the image boundaries are tapered using the PSF of the microscope for border protection. The PSF of the microscope obtained as discussed above is used as an input parameter for all tested deconvolution methods. For these configurations, the implementations of MAP-D, LR, MAP-Hunt and MCMC-WH can process 800 × 800 three-channel fluorescence microscopy acquisitions in ∼45 s, ∼39 s, ∼40 s and ∼79 s, respectively, on an Intel(R) Core(TM) i5-3317U CPU at 1.70 GHz CPU with 4 Gb RAM.

Additional Information

How to cite this article: Wong, A. et al. Bayesian-based deconvolution fluorescence microscopy using dynamically updated nonstationary expectation estimates. Sci. Rep. 5, 10849; doi: 10.1038/srep10849 (2015).