Abstract
Multiphoton microscopes are hampered by limited dynamic range, preventing weak sample features from being detected in the presence of strong features, or preventing the capture of unpredictable bursts in sample strength. We present a digital electronic add-on technique that vastly improves the dynamic range of a multiphoton microscope while limiting potential photodamage. The add-on provides real-time negative feedback to regulate the laser power delivered to the sample, and a log representation of the sample strength to accommodate ultrahigh dynamic range without loss of information. No microscope hardware modifications are required, making the technique readily compatible with commercial instruments. Benefits are shown in both structural and in-vivo functional mouse brain imaging applications.
Similar content being viewed by others
Introduction
Multiphoton microscopy has become the most common and effective method for high-resolution functional brain imaging because of its remarkable depth penetration in thick tissue1. In standard configurations, such microscopy involves scanning a femtosecond laser focus in 3D throughout a sample. The laser power is fixed during the scan and image information is contained in the time dependence of the detected fluorescence signal (Fig. 1A). Several problems can occur with this technique. First, in common cases where the sample contains extreme variations in brightness, for example between large somas and much finer dendritic processes, it is often impossible to capture the full range of signals without either saturating the detector when scanning over bright regions, or losing signal when scanning over dim regions. Second, when imaging time-varying signals from functional reporters such as GCaMP2, large brightness variations occur that cannot be predicted in advance, forcing the user to operate with low illumination power to minimize the possibility of detector saturation, thus compromising SNR. Third, when performing volumetric scans through an extended range of depths, a single laser power becomes either too weak at large depths or too strong at shallow depths.
A simple solution to all these problems involves actively regulating the laser power pixel by pixel using negative feedback electronics3, 4. However, to date, implementations of this solution have been based on custom analog electronics that have been difficult to build and calibrate, leading to image reconstruction that was unreliable. We present here a modified technique based on simultaneous detection of the signal and illumination powers that is user-friendly, calibration-free and can be assembled from readily available off-the-shelf components. Our solution is a self-contained unit that can be attached to any multiphoton microscope, commercial or otherwise, with no hardware modifications whatsoever. Our only assumptions are that the microscope is based on standard (i.e. non-resonant) galvanometric scanning and that it is equipped with a method to rapidly control laser power (for example, most commercial vendors provide power control with an electro-optic modulator (EOM)), and a simple photodiode to probe the laser power. Because our unit is based on digital field-programmable gate array (FPGA) electronics and dual detection of signal and illumination powers, its operational parameters are well-defined and image reconstruction is robust and accurate.
Technique
Our strategy, which we refer to as active illumination, is illustrated in Fig. 1B. We define the fluorescent sample strength X to be a variable that includes all factors contributing to the local fluorescent emissivity, including concentration, cross-section, quantum yield, etc., such that the detected multi-photon excited fluorescence is S = XP α, where α is the excitation order (2 for two-photon microscopy). Negative feedback is used to hold the detected fluorescence to a constant S 0 by controlling the input illumination power, up to a user-defined maximum power P max . When sufficient power is available to hold S 0, the system is in feedback active mode. When more than P max would be needed to reach S 0, the power is automatically set to P max and the system switches to power-limited mode. In either case, the desired quantity of interest is the sample strength X = S/P α. This is evaluated directly in the electronic unit and supplied as an output. In other words, the user need not be concerned whether the system is in feedback-active or power-limited mode – the output remains the desired X regardless. Moreover, even if the feedback fails to some degree and neither S nor P attains its targeted value, the ratio X remains correct, which is all the user cares about.
While the strategy outlined above is useful for limiting the exposure of the sample to unnecessary illumination, and thus limiting the possibility of photobleaching or phototoxicity (a similar strategy involves on/off illumination control5,6,7), it does nothing to improve the dynamic range of the acquired images. For this, an additional feature to our strategy is required. The problem is made clear with an example. Let us consider the standard case where microscope signal acquisition is performed with a 12-bit A/D converter. In this case, even if the optical detector (typically a PMT) is noiseless, the best dynamic range one can hope to achieve is 4096:1. And even if one were to switch to a converter with larger bit depth, one would then run into dynamic range limitations imposed by the detector itself, which for a PMT is typically 104–105. These ranges are often short of the brightness ranges occasioned in fluorescently labeled brain tissue, sometimes by orders of magnitude.
When examining the problem more closely, one realizes one could do much better, even with a standard 12-bit converter. The problem is that the converter is not efficiently utilized. As the optical signal increases, so too does the shot noise associated with the signal, meaning that the fine sampling provided by the 12-bit converter becomes wasted at high signal levels where all it does is oversample noise. A much better strategy is to redistribute the sampling so that fine sampling is provided at low signal levels where it is needed, while coarser sampling is relegated to larger signal levels. Such a redistribution of sampling can be achieved by applying a nonlinear (more precisely, sub-linear) operation to the signal before it is sampled by the linear A/D converter. The operation we choose for our system is the logarithm operator (see Fig. 2). To be clear, we apply the logarithm operator to the sample strength X. But in turn, this is derived from the logarithms of the fluorescence signal S and the laser power P by the simple relation \({\mathrm{log}}_{2}X={\mathrm{log}}_{2}S-\alpha {\mathrm{log}}_{2}P\). We emphasize that such a redistribution of gray levels entails no additional loss of information, since, up to sample strengths much larger than X sat , loss of information remains dominated by uncertainties due to shot noise rather than inaccuracies due to sampling (see Fig. 3).
A theoretical evaluation of the dynamic range provided by our system is provided in Supplementary Information, and illustrated in Fig. 3 for the case when S 0 is set to half the detector saturation level. The resulting maximum gain in dynamic range compared to standard multi-photon microscopy is given by ×2(αB−1), where B is the bit depth of the microscope acquisition electronics. For two-photon microscopy with typical bit depth of 12, this corresponds to a maximum gain in dynamic range of almost 107. For three-photon microscopy8, the increase would be greater than 1010.
Results
We demonstrate some benefits of active illumination when applied to neuronal imaging with commonly used genetic indicators. During structural imaging of neuronal anatomy, different cellular compartments often lead to a range of signal intensities greatly exceeding what can be captured by current multiphoton microscopes. For example, dendritic spines are a widely used readout of neural plasticity9, but these small and dim structures are often difficult to resolve without saturating signals from larger adjacent compartments such as parent dendrites or the cell body. This problem is remedied by active illumination, as illustrated in Fig. 4, greatly improving the SNR of spine imaging while preventing clipping of signals from larger dendritic branches. Indeed, the sample strengths in Fig. 4B span a range \(\sim \,{10}^{8}\mathrm{:1}\). The display of such a large dynamic range is facilitated with our log representation, and a conventional linear representation can be readily recovered post-acquisition by applying the antilog operator to X. Importantly, the overlay of linear histograms in Fig. 4 confirms that the fluorescence levels acquired with and without AI are reliably consistent over the full span of brightness ranges. That is, AI allows both bright and dim cellular compartments in mouse brain tissue to be simultaneously and accurately quantified without the need for repeated exposures.
Active illumination also offers substantial benefits for imaging neural activity with genetically encoded Ca2+ indicators. Activity levels often vary dramatically across different neurons within a circuit10, posing challenges for capturing both large fluorescence increases within the most highly active cells without sacrificing signals from weakly active neurons that still encode meaningful information. This problem is compounded by the increased dynamic range of the latest generation of Ca2+ indicators such as GCaMP62. To illustrate this situation, we imaged sensory responses to odors in the olfactory bulb of Thy1-GCaMP3 mice11, where neurons exhibit both increases and decreases in neural activity with diverse amplitudes and temporal dynamics (Fig. 5A,B). Active illumination enabled the use of sufficient laser power to reveal weakly activated cells while at the same time preventing the clipping of more strongly responsive cells. Active illumination also provided improved resolution of Ca2+ changes in smaller dendritic compartments (Fig. 5C), which play important roles in integrating input from other neurons12 and can even act as independent input-output sites13, 14. Overall, active illumination decreased the noise levels in resting signals from individual cells, while providing a corresponding increase in the SNR of sensory responses (Fig. 5D–I), illustrating an increase in the amount of information about circuit dynamics that can be extracted with Ca2+ indicators.
Discussion
The pixel sampling rates achieved by our system are as high as 1 MHz (see Supplementary Information and Supplementary Video 9, acquired at 1.2 μs pixel time), limited by the bandwidth of our EOM. Such rates are easily on par with what is generally employed with non-resonant-galvanometer-based microscopes. Faster rates still, such as video rate, could be accommodated by our FPGA electronics and are attainable in principle, but would require an EOM of bandwidth approaching 10 MHz and a detector of shorter response latency than our PMT.
One might wonder how it is possible to improve dynamic range beyond the intrinsic range of the detector acquisition electronics. The key here is that two detectors are used, not just one. Moreover, the signal of interest, namely the sample strength X, depends on the simultaneous measurements provided by both of these detectors, not just one. In the case of multiphoton microscopy the dependence on one of these measurements is nonlinear, further increasing the sensitivity to this measurement.
We did not attain the full potential gain in dynamic range predicted by theory (see Fig. 3) for a variety of reasons. First, we gave ourselves some wiggle room (X t < X sat ) to allow for the possibility of errors in the feedback, such as feedback overshoot. Second, it was necessary to introduce a constant offset to log2 X because our microscope acquisition electronics did not allow negative input levels (see Materials and Methods). Third, a roll-off in SNR occurred at X r , caused by our inability to precisely control and detect the illumination power P at levels when it is very small (owing in part to digitization inaccuracies, noise in the EOM control and photodiode electronics, and a weak quadratic dependence of the EOM response at that level). Nevertheless, the gains in dynamic range we achieved are substantial. For example, a gain of ×214 is illustrated in Fig. 4B, above and beyond the range of 212 provided by standard microscope acquisition electronics.
Compared to multi-exposure15, multi-signal-attenuation16, or post-detection17 strategies for achieving high dynamic range, our technique provides the added benefits of limiting photobleaching or phototoxicity in the sample (P max is set to no higher than the illumination power P 0 normally used in standard imaging – see Supplementary Information), and reducing potential of damage to the detector itself caused by sudden signal transients (S is prevented from being larger than S 0). It should be noted that active illumination can also be applied to single photon scanning microscopy (e.g. confocal), however with reduced benefits to dynamic range and photodamage limitation.
The purpose of our active illumination add-on is to enable scanning microscopes, particularly multiphoton microscopes, to see more, in many cases much more, in a reliable, quantitative manner and without significant sacrifice in performance. To maximize impact, we have focused on making our add-on easy to assemble from only a few off-the-shelf components, and to implement with no requirement of microscope hardware modifications.
Materials and Methods
Active illumination
A layout of our system is presented in Supplementary Information. We use a Red Pitaya development kit featuring a Xilinx Zynq 7010 system on chip (CPU & FPGA) and integrated 125 MS/s 14 bit ADCs (2×) and DACs (2×). The fluorescence signal S obtained from the microscope PMT (Hamamatsu H7422PA-40 with Prairie Ultima pre-amp) is normally routed to the microscope acquisition electronics. Instead, we route it here to one of the ADC input of the Red Pitaya board, through a variable voltage divider (5 kΩ) adjusted so that the overall ADC gain G S is nominally adjusted to fill the converter bit depth (minus one). That is, G S = \({2}^{{B}_{S}}/{S}_{sat}\), where B S = 13 and S sat is the saturation signal obtained from the PMT. The error difference between the digitized signal \(\hat{S}={G}_{S}S\) and a programmed set point \({\hat{S}}_{0}\) is then fed into a PID controller to compute the targeted digitized laser power \(\hat{P}\) (see Supplementary Information for details18). This is sent to the EOM (Conoptics 350-80LA with 302RM driver) via one of the Red Pitaya DAC outputs such that the nominal overall DAC gain is G P = P max /\({2}^{{B}_{P}}\), where B P = 14 and P max is the EOM driver voltage corresponding to the maximum allowed laser power. This is adjusted using a variable voltage diver (5 kΩ) between the Red Pitaya output and the EOM driver input (alternatively, it can be loaded into the Red Pitaya FPGA through its webpage interface).
In the event that the target \(\hat{P}\) is less than the maximum allowed \({\hat{P}}_{max}\) the board is in feedback-active mode. Otherwise the board automatically switches to power-limited mode and \(\hat{P}\) is held fixed at \({\hat{P}}_{max}\). To turn active illumination off we bypass the feedback and apply a user-defined constant voltage to EOM.
Log-encoding and reconstruction of sample strength
Regardless of whether the board is in feedback-active or power-limited mode, the sample strength X is computed from a knowledge of \(\hat{S}\) (see above) and \(\hat{P}\) (obtained by probing the laser power with a \(\sim 2\)% beamsplitter, neutral density filter, and a Thorlabs PDA36A amplified photodiode, and adjusted with a variable voltage divider to match the photodiode output range to the 1 V input range of the second Red Pitaya ADC input). Once computed and encoded by a logarithm operation, X is routed to the microscope acquisition electronics via the second Red Pitaya DAC output. For the logarithm operation, both \(\hat{P}\) and \(\hat{S}\) are converted to \({\mathrm{log}}_{2}\hat{P}\) and \({\mathrm{log}}_{2}\hat{S}\) using a lookup table containing B L -bit entries pre-calculated by MATLAB and stored in onboard ROM. These entries are scaled by G L = \({2}^{{B}_{L}}\)/B S , and processed to obtain \({\hat{L}}_{X}\equiv {G}_{L}{\mathrm{log}}_{2}\hat{X}+C={G}_{L}({\mathrm{log}}_{2}\hat{S}-\alpha {\mathrm{log}}_{2}\hat{P})+C\), where C is an additive constant chosen to avoid the possibility of negativities in \({\hat{L}}_{X}\) in the event the microscope acquisition electronics does not allow negative input levels (we used here B L = 13 and C = \({2}^{{B}_{L}}\)). After \({\hat{L}}_{X}\) is output by our active illumination unit D/A converter it is then re-digitized by the microscope acquisition electronics. The link between \({\hat{\hat{L}}}_{X}\) after re-digitization and \({\hat{L}}_{X}\) is given by \({\hat{\hat{L}}}_{X}={G}_{M}{\hat{L}}_{X}\), where ideally G M is adjusted to fill the bit depth B of the microscope acquisition electronics when \({\hat{L}}_{X}\) is maximum. What is displayed on the computer screen, which, as far as the microscope is concerned corresponds to a standard image, is then \({\hat{\hat{L}}}_{X}\). If desired, the numerical reconstruction of X, to within an unimportant scaling factor, can be performed post-hoc using the simple relation \(X\propto {2}^{({\hat{\hat{L}}}_{X}/{G}_{L}{G}_{M})}\). Note that an accurate knowledge of G M is critical here to ensure that the reconstruction of X is properly linear.
Code availability
The Red Pitaya software and operation manual for the particular active illumination implementation described here will be made freely available on our web site: http://biomicroscopy.bu.edu.
Mouse brain imaging
All samples were imaged with a Prairie Ultima two-photon microscope using an Olympus 20× NA 1.0 objective. Structural imaging of fluorescently labeled cortical pyramidal neurons was performed in vitro in acute, 300 μm thick brain slices prepared from YFP-M mice using standard techniques1. Functional Ca2+ imaging was performed in vivo in the olfactory bulb of transgenic mice with widespread expression of the genetically encoded indicator GCaMP32. Animals were anesthetized with isoflurane, and a craniotomy was performed over the dorsal olfactory bulb. Sensory responses to odorants were measured in mitral cells at a depth of 300–350 μm from the dorsal surface. Odorants were amyl acetate and 2-methyl butyraldehyde, diluted in mineral oil to a concentration of 2%, and added to the main airstream using a custom olfactometer for a final concentration of 1%. All animal procedures were approved by the Boston University Institutional Animal Care and Use Committee and carried out in accordance with NIH standards.
References
Helmchen, F. & Denk, W. Deep tissue two-photon microscopy. Nat. Methods 2, 932–940 (2005).
Chen, T. W. et al. Ultrasensitive fluorescent proteins for imaging neuronal activity. Nature 499, 295–300 (2013).
Chu, K. K., Lim, D. & Mertz, J. Enhanced weak-signal sensitivity in two-photon microscopy by adaptive illumination. Opt. Lett. 32, 2846–2848 (2007).
Chu, K. K., Lim, D. & Mertz, J. Practical implementation of log-scale active illumination microscopy. Biomed. Opt. Express 1, 236–245 (2010).
Hoebe, R. A. et al. Controlled light-exposure microscopy reduces photobleaching and phototoxicity in fluorescence live-cell imaging. Nat. Biotechnol. 25, 249–253 (2007).
Hoebe, R. A. V. D., Voort, H. T. M., Stap, J., Van Noorden, C. J. F. & Manders, E. M. M. Quantitative determination of the reduction of phototoxicity and photobleaching by controlled light exposure microscopy. J. Microsc. 231, 9–20 (2008).
Caarls, W., Rieger, B., De Vries, A. H. B., Arndt-Jovin, D. J. & Jovin, T. M. Minimizing light exposure with the programmable array microscope. J. Microsc. 241, 101–110 (2010).
Horton, N. et al. In vivo three-photon microscopy of subcortical structures within an intact mouse brain. Nat. Photon 7, 205–209 (2013).
Wilbrecht, L., Holtmaat, A., Wright, N., Fox, K. & Svoboda, K. Structural plasticity underlies experience-dependent functional plasticity of cortical circuits. J. Neurosci. 30, 4927–4932 (2010).
O’Connor, D. H., Peron, S. P., Huber, D. & Svoboda, K. Neural activity in barrel cortex underlying vibrissa-based object localization in mice. Neuron 67, 1048–1061 (2010).
Chen, Q. et al. Imaging neural activity using Thy1-GCaMP transgenic mice. Neuron 76, 297–308 (2012).
Magee, J. C. Dendritic integration of excitatory synaptic input. Nat. Rev. Neurosci. 1, 181–190 (2000).
Urban, N. N. & Arevian, A. C. Computing with dendrodendritic synapses in the olfactory bulb. Ann. New York Acad. of Sci. 1170, 264–269 (2009).
Euler, T., Detwiler, P. B. & Denk, W. Directionally selective calcium signals in dendrites of starburst amacrine cells. Nature 418, 845–852 (2002).
Burt, P. J. & Kolczynski, R. J. Enhanced image capture through fusion. Proc. ICCV 4, 173–182 (1993).
Vinegoni, C. et al. Real-time high dynamic range laser scanning microscopy. Nat. Commun. 7, 11077, doi:10.1038/ncomms11077 (2016).
Kissick, D. J., Muir, R. D., Sullivan, S. Z., Oglesbee, R. A. & Simpson, G. J. Real-time dynamic range and signal to noise enhancement in beam-scanning microscopy by integration of sensor characteristics, data acquisition hardware, and statistical methods. Proc. SPIE-IS&T Elec. Imag. 8657, 86570E (2013).
Lima, J., Menotti, R., Cardoso, J. M. & Marques, E. A. Methodology to design FPGA-based PID controllers. IEEE Int. Conf. Systems, Man and Cybernetics 3, 2577–2583 (2006).
Acknowledgements
This work was funded by the National Institute of Health (R21EY027549).
Author information
Authors and Affiliations
Contributions
R.Y. designed, built, and programmed the active illumination unit, helped with experiments, performed data analysis and contributed to the manuscript; T.W. helped with experiments, performed data analysis and contributed to the manuscript; E.W. helped with imaging experiments; I.D. conducted the imaging experiments, performed data analysis and contributed to the manuscript; J.M. supervised the project, performed data analysis and wrote the manuscript.
Corresponding author
Ethics declarations
Competing Interests
The authors declare that they have no competing interests.
Additional information
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Yang, R., Weber, T.D., Witkowski, E.D. et al. Neuronal imaging with ultrahigh dynamic range multiphoton microscopy. Sci Rep 7, 5817 (2017). https://doi.org/10.1038/s41598-017-06065-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-017-06065-7
- Springer Nature Limited
This article is cited by
-
Pixel-wise programmability enables dynamic high-SNR cameras for high-speed microscopy
Nature Communications (2024)
-
An adaptive excitation source for high-speed multiphoton microscopy
Nature Methods (2020)