Key words

1 Introduction

The advent of optogenetics, pioneered already more than a decade ago [1, 2], has revolutionized our understanding of the contribution of individual, genetically defined neurons to circuit function. For the first time, we could start to unravel the causal relation of a distinct genetically defined compartment of the network to whole-brain circuit dynamics and ultimately behavior. Optogenetics impinged upon all fields of neuroscience. For each of those fields, individual roadblocks needed to be overcome based on the principle of optogenetics, i.e., the illumination of a defined brain volume with rather high light intensity exceeding 1 mW mm−2. In the field of single-cell electrophysiology, the problem of the Becquerel effect had to be solved by the design of specific electrodes, in the field of behavior the problem of flexible and non-tethered light delivery had to be solved. These obstacles could be removed already in the first years upon the advent of optogenetics [3]. Yet, there is one field of neuroscience, which maybe poses the ultimate challenge for the integration of optogenetics: optical functional imaging. The field of optical imaging, particularly of individual neuron function in the intact tissue, relies on detecting subtle changes in light intensity [4]. The implementation of two-photon microscopy in combination with calcium indicators in the early 2000 allowed for the first time a functional readout of neuronal circuits comprising of several hundred neurons in the rodent cortex [4,5,6]. This leads to a tremendous advance in our understanding on how complex circuit dysfunction arises, particularly also in the early stages of neurological disorders [7,8,9,10,11]. The ability to identify a rather small fraction of dysregulated neurons in the circuitry makes the true difference in comparison to single-cell or population readouts. And of course, also here, the implementation of optogenetics holds tremendous potential, as a causal manipulation of, e.g., single dysregulated neurons and the simultaneous readout of circuit function would truly advance our understanding on cause and effect in network disorders. And indeed, in 2012 [12, 13] the efficient two-photon excitation of opsins became a reality, followed by combined two-photon imaging and two-photon optogenetics [14,15,16,17]. In these early proof of concept studies, it already became apparent, that all-optical approaches pose unique challenges, both in terms of system integration, experimental design, and not the least, analysis. First, conventional line-scanning excitation schemes have proven to be rather less efficient. Second, only a few opsins seem to be two-photon excitable, and there is yet no molecular or structural predictor, which could guide molecular engineering of opsins to modify their two-photon cross section. Thirdly, as mentioned above, the light intensity used for two-photon optogenetics ranges orders of magnitude higher compared to the light intensities used for the excitation of fluorophores, creating problems in terms of indicator bleaching and tissue heating. Lastly, also the analysis poses problems in terms of artifact removal and synchronization. All of this prevented up to now the broad implementation of all-optical approaches, despite their promises. Here we will focus on the entire workflow for an all-optical experiment in circuit neuroscience, reporting on recent advances, and giving guidance for the unique requirements of all-optical physiology.

2 Experimental Framework

Circuit neuroimaging historically originated in invertebrate species, and, alongside with the technique maturation and development, has been introduced in the application of various model organisms, such as Zebrafish, allowing, e.g., for the whole-body imaging of transparent zebrafish larvae [18]. In recent years, optical imaging of local circuits has been predominantly applied in mouse models. This is on the one hand due to the availability of mouse models of human disorders, and also due to the rather preserved cortical cytoarchitecture in comparison to humans. The limitation of penetration depth and of course the invasiveness of the method currently seems not to allow the implementation in human preclinical research. Here, we focus on the implementation of all-optical approaches in mouse cortex. Cortical networks are critically involved in fundamental tasks of the brain, starting from sensory processing [19,20,21], to decision making [22,23,24] and mechanisms involved in consciousness [25, 26]. Optical neuroimaging using genetically encoded calcium indicators [27] can resolve the activity of local neuronal population even with single-action potential (AP) resolution of sparsely firing neurons. Recent advances [28, 29] even allow for the detection of cortex-wide activity of thousands of neurons in real time. Certainly, despite these advances, genetically encoded calcium indicators are inherently slow, so single APs can only be resolved if the inter-AP interval is sufficiently long, preventing the resolution of individual spikes, e.g., of fast-spiking interneurons, such as PV interneurons [27]. What is more, the frame rate of current commercially available two-photon microscopes ranges at 30 Hz for full-field imaging, at least for the majority of microscopes equipped with resonant scanners [30]. Using random-access scanning, the temporal resolution can be increased, but at the expense of a limited ability to do post-hoc operations for movement correction. While inertia-free AOM systems allow for a much higher framerate [31] (see also Chap. 3), the current standard in the field is single-plane full-field imaging at 30 Hz, a good trade-off between speed, signal-to-noise ratio (SNR), and field of view size. This has important implications for the integration of optogenetics in an optical microcircuit imaging framework. With an effective temporal resolution of 30 Hz, the window of coincidence which can be resolved equals 33.3 ms. It is therefore imperative to design a framework which avoids the loss of even a single imaging frame. Here, with these limitations in mind, we put forward concepts focusing on artifact avoidance or removal, to retain the utmost temporal resolution. Even more so, for the relation of a per se unspecific signal, i.e., the increase of fluorescence intensity to an underlying AP, specific criteria need to be met in terms of the temporal dynamics of an AP-related event. The entire workflow of an optical functional imaging approach has to be tailored, starting from the design of the hardware, to the experiment itself, and the subsequent analysis. This chapter does not strive for giving an overview of all possible solutions of the integration of optogenetics for all-optical experiments, but rather focuses on a few tried-and-tested pipelines.

3 Technical Framework for Functional Neuronal Circuit Mapping

An optimal system integration needs to be tailored toward the specific requirements for addressing the neurophysiological research question. Investigating action-potential-related neurophysiological events mandates a precise temporal synchronization of all acquired signals and interventions, in our case opsin excitation, to ultimately gain evidence of cause and effect in the investigated neural circuit.

Multichannel-data-acquisition (DAQ) interfaces can be employed to collect signals from all involved subsystems. We would strongly suggest to design a hierarchical configuration, in which all subsystems, including the microscope, report the critical signals, such as the frame trigger to this unifying and synchronizing device. Commonly used manufacturers are National Instruments (Austin, Texas, USA) or Cambridge Electronic Design (Cambridge, UK) [32,33,34]. Integrating multichannel-data-acquisition hardware in the microscope itself is also a viable solution offered by many major microscope companies. Connecting this device with a suitable integration software generates a master file for each experiment, containing individual traces for all relevant signals, such as breathing rate, temperature, frame trigger, and certainly optogenetic or sensory stimulation pulses. Referring to the mentioned manufacturers, LABView is used for National Instruments interfaces, whereas DAQ boards from Cambridge Electronic Design use the software package Spike2. Particular care has to be exerted in terms of gathering all relevant information important for the current experiment in this master file. For example, the frame trigger signal of a microscope can give insights at which absolute time point images were acquired. For tactile stimulation, a TTL-pulse can be generated by the DAQ boards itself, avoiding varying time delays as occurring with using PC-based USB control. Thereby, the experimenter gains full control and a complete report of all relevant systems parameters without temporal jitter.

4 Somatic Calcium Influx as Correlate of Suprathreshold Neuronal Activity

4.1 Two-photon Raster Scanning

Functional calcium imaging utilizes the action-potential-related calcium influx into a cell as a correlate of neuronal activity [35,36,37]. A genetically encoded calcium indicator (GECI) is undergoing conformational changes upon binding of calcium ions, causing a change in its fluorescence properties [38]. Currently, the most commonly employed GECI is GCaMP6. It has to be mentioned that there are different subtypes of GCaMP6: GCaMP6s (the “s” stands for “slow”), exhibits a high SNR but slow off kinetics, GCaMP6m (“intermediate”) exhibits medium, and GCaMP6f (“fast”) fast kinetics [27] (see Note 1). We recommend using GCaMP6f whenever possible, particularly for event-related analyses with the aim of extracting action-potential-related calcium transient with utmost temporal resolution. The excitation spectrum of GCaMP6f has its maximum at 497 nm [27] under a one-photon excitation regime. However, the penetration depth using the principle of one-photon excitation is limited, inter alia caused by light scattering in the brain which requires a pinhole to avoid detection of scattered light that does not derive from the focal point [39, 40]. Furthermore, the deposited energy in the brain by photons of this wavelength can lead to phototoxicity and bleaching of the indicator [41]. The development of microscopes using the principle of two-photon excitation combined with raster scanning approaches [42] overcame these limitations and therefore revolutionized functional neural circuit imaging [5]. Furthermore, pinhole solutions became dispensable since all emitted photons necessarily originated from the fluorophores excited at the focal point.

Fluorescence excitation by two-photon excitation requires two photons being absorbed by the fluorophore within a time window of less than 10−16 s [40], resulting in a tremendously low probability for a two-photon excitation. Even though two-photon excitation can be achieved by continuous-wave lasers [43, 44], the development of mode-locked femtosecond pulsed lasers, like the Titanium-Sapphire (TiSa) laser (Fig. 1a), enabled focusing light at high-intensity spots and thus efficient two-photon excitation at the focal point at moderate laser power in vivo [42]. Two-photon excited fluorescence images can be acquired by employing raster scanning of the focused laser beam (Fig. 1a), where the beam is deflected by a galvo-resonant mirror system to the specimen covering the field of view (FOV) [5]. The dimensions of the FOV depend on the chosen objective and its numerical aperture (NA). The duration that the laser beam remains focused on a pixel is defined as the pixel dwell time. For fluorescence excitation, a pixel dwell time of 1 μs is commonly applied, resulting in frame rates of 30 Hz for galvo-resonant systems [9]. Note that this temporal resolution is critical for the subsequent identification of AP-related calcium transients based on their temporal dynamics.

Fig. 1
Two block diagrams a) 2 photon microscope and b) 1 photon miniature microscope depicts the source, delivery, image generation, and detection method used for calcium imaging.

Principles of functional calcium imaging. (a) Scheme of two-photon-based readout using a femtosecond laser and raster scanning principle. The laser is focused by the objective and the focal point is guided by a galvo-resonant scanhead over the field of view. Emitted fluorescence is collected and detected by a PMT. (b) One-photon Miniature Microscopes used for freely moving animals require a miniaturization of light sources, achieved by bandpass filtered LEDs. A full-field illumination via a GRIN lens is carried out and a CCD-Chip collects emitted fluorescence

Full size image

The emitted fluorescence light is collected by the objective and is deflected to a photo-multiplier tube (PMT). In front of the PMT, a spectral bandpass filter ensures that only light in the bandwidth of the fluorophore’s emission band is detected.

4.2 One-photon Miniature Microscope Full-Field Imaging via GRIN Lens

In the field of in vivo calcium imaging, a recently emerging method are miniature microscopes, mounted on an implanted baseplate on the head of the animal. In this chapter, we solely refer to one-photon miniature microscopes, whereas Chap. 7 gives a detailed introduction to two-photon miniature microscopes. Recently, the first three-photon miniature microscope was proposed by the group of Jason Kerr [45]. Miniature microscopes paved the way for the combination of functional imaging with behavioral assays in which the animal moves freely. What is more, brain regions not accessible by two-photon microscopy due to its limited penetration depth can be targeted. First developed by the group of Mark Schnitzer [46], nowadays several manufacturers offer ready-to-use systems (INSCOPIX, Palo Alto, CA, USA [47]) but also open-source systems are being provided by the community, e.g., the UCLA Miniscope [48] or the FinchScope [49].

Retaining the image information also in deeper brain regions is afforded by a gradient-index optic (GRIN lens, Fig. 1b). GRIN lenses have a cylindrical shape with a radially decreasing refraction index, and planar surfaces for optimizing optic interfacing. These two factors, a gradient of refractory index and the planar surface, retain, at least to some extent, the image information throughout the passage of light through the GRIN lens. It is important to note that there is a deterioration of image quality with increasing the length of the GRIN lens. As a light source, a LED in combination with spectral bandpass filters is used [47], and the image is typically recorded by a CCD chip. The sampling rates for functional calcium imaging differ depending on the chosen model, ranging from 15 Hz with a field of view of 1440 by 1080 pixels (nVoke, Inscopix, Palo Alto, California) up to 60 Hz at a field of view of 752 by 480 pixels (Miniscope-v4, UCLA, Los Angeles, California). The following issue needs to be considered: the reliable recording of neuronal layers depends on the location of the GRIN lenses tip, neurons located 100–300 μm below the tip can be resolved by electronically adjusting a focusing lens [47]. However, changing the x-y-position of the recorded field of view is not possible and is predefined by the implantation site of the GRIN lens.

5 The Principles of Optogenetic Manipulation Methods in a Nutshell

There is a vivid research in the field of molecular biophysics on the development of new opsin classes paralleled by the improvement of existing opsins by molecular engineering. The researcher needs to decide which opsin is best suited for the question at hand, i.e., should a specific component of the network be silenced or stimulated. Importantly, at this time, there are no a priori predictions on the suitability of a given opsin for two-photon excitation. Here, we cannot give a comprehensive guide on the specific requirements for any specific opsin. We suggest caution on the use of newly developed opsins for non-experts in the field of optogenetics: Achieving strong and stable expression of an opsin requires dedicated and careful titration steps [33, 50, 51]. Also, it so happened more than once, that a new opsin, while promising at first, showed rather negative cytotoxic effects, requiring the development of new versions [52]. Consequently, if there are no critical requirements such as kinetics or specific ion conductance, we would strongly suggest to use tried-and-tested opsins, even if they might not be tailored to the specific neuronal class. For example, if a neuronal subpopulation is probed which exhibits an intrinsic firing rate exceeding the optimal kinetics of a given opsin, it might still be advantageous to choose this opsin. Remember, the opening time of an opsin is not determined by the duration of the excitation pulse, but by the opsins’ intrinsic dynamics, described by the parameter τoff, i.e., the time at which the probability of an open state is reduced to 1/e [50]. While this can result in not achieving to optogenetically mimic the intrinsic firing frequency, it might increase the overall probability of a successful all-optical experimental approach. Critically evaluate, which experimental aims are mandatory, and which parameters of the experimental design can be adapted. This is of particular importance for an all-optical experiment, requiring the optimal expression of indicator and opsin, and all the other technical requirements stated above and below. Spending too much time on individual components of this chain might reduce the chance of completing the overall aims of the given study in the inherently limited time frame. It is already quite complex to set up an all-optical experimental framework, and we suggest to limit the complexity whenever possible.

5.1 One-photon Raster Scan Opsin Excitation

For one-photon optogenetic interrogation, typically a solid-state laser at the optimal excitation wavelength for the respective opsin should be employed [50, 53]. Opsins successfully used for the manipulation of neurons in a one-photon regime include the veteran ChR2 (134) for AP initiation [2, 54] and ArchT for inhibition [2, 51]. For these tried-and-tested opsin pairs, two solid-state lasers can be employed, with 488 nm for ChR2 and 552 nm for ArchT, provided that the light intensity at the excitation site is ≥1 mW mm−2 [55]. The one-photon light source is coupled to the microscopes beam path via optic fibers. The region of interest is raster-scanned typically by a galvo-galvo scanning head (Fig. 2a). To avoid the detection of the laser pulse for opsin excitation, a notch filter is blocking the excitation wavelength from entering the PMTs. Note that there might be a problem in terms of the drastic increase of autofluorescence during the one-photon excitation pulse, in the wavelength bandwidth of the signal of interest, here of GCaMP, leading to a stimulation artifact. Alternatively, if necessary, the PMTs can be blanked by a shutter during opsin stimulation, which results in a sacrifice of signal detection during stimulation. This might pose problems for subsequent trace analysis, as particularly detecting the stereotypical onset dynamics of an AP-related calcium transients is vital for ensuring specificity. Apart from focusing on a single neuron, this approach can also be used to illuminate a section of or the whole field of view to excite a larger subset of opsin-expressing neurons. However, the downside of this approach is that not only the layer recorded in the field of view gets excited, but also neurons located above or below the recorded layer can get excited as long as they are exposed to a light intensity that exceeds 1 mW mm−2, as depicted in Fig. 3a.

Fig. 2
A block diagram depicts the delivery and pattern generation of the microscope. It has a) 2-photon microscope and b)1-photon miniature microscope for opsin excitation.

Schematic overview of system combinations for opsin excitation. (a) Two-photon functional calcium imaging can be combined with one-photon or two-photon raster scan methods to target opsin-expressing neurons. The method of Computer Generated Holography is a scanless approach and enables the experimenter to excite several neurons at the same time under a two-photon regime using spatial light modulators (SLMs). (b) Miniature microscopes equipped with a second LED for opsin excitation are based on one-photon full-field illumination delivered via an implanted GRIN lens

Full size image
Fig. 3
A multipart diagram shows the mechanism for opsin excitation. A. one-photon scanning excitation. B. one-photon miniature microscope fulfilled excitation. C. two-photon scanning excitation. D. two-photon scanless excitation.

Mechanisms for opsin excitation. (a) One-photon excitation by a focused beam is carried out by raster scanning the target neurons. Neurons located above or below the imaging plane are exposed to the excitation beam cone as well. (b) One-photon full-field paradigms are used in miniature microscopes and do not allow for neuron-specific excitation but illuminate the entire FOV at once, eventually exciting neurons below the recorded layers. (c) Two-photon raster scanning approaches overcome this disadvantage due to the physical principle of two-photon excitation, reaching the necessary photon density only in the focal spot. (d) The scanless paradigm also enables experimenters to simultaneously excite several target neurons under a two-photon regime

Full size image

5.2 Two-photon Raster Scan Opsin Excitation

Utilizing the same physical mechanism like in functional two-photon calcium imaging, two photons of roughly the double wavelength of the optimal one-photon excitation can evoke an activation, i.e., conformational change of an opsin. It has to be noted that there cannot be an a priori assessment on the suitability of a given opsin for two-photon excitation regimes (see Note 2). One of the first opsins probed for two-photon excitation represents C1V1, typically excited at around 1080 nm [12, 13]. Yet, excitation beyond the standard range between 1030 and 1100 nm of Ytterbium lasers [56] is possible. We explored wavelengths of up to 1250 nm using an Optical Parametric Oscillator (OPO) as light source, being able to decrease the spectral cross-talk [34]. There is a wide range of possible light sources for two-photon optogenetic raster scan-based excitation: A tunable Ti:Sa laser is well suited for wavelengths up to 1080 nm. However, the femtosecond pulse dynamics of a two-photon imaging laser is not as critical for opsin activation as it is critical for imaging, since it is not necessary to obtain the shortest pixel dwell time possible. Therefore, fixed wavelength Ytterbium lasers with lower pulse repetition rates became the most widely applied light source [57]. Since opsin activation is based on conformational changes of the protein initiated by an isomerization from all-trans to 13-cis retinal upon photon absorption, the physical principle differs from functional calcium imaging with fluorophores, i.e., GECIs. Therefore, the pixel dwell time is, besides the chosen objectives point-spread function (PSF) and laser intensity, a crucial factor. Studies suggest a pixel dwell time of at least 3.2 μs for efficient excitation [13, 33, 58].

The raster scan method enables the experimenter to define several target neurons that are scanned successively in the same plane (Figs. 2a and 3c). Depending on the chosen line spacing, this can lead to rather long excitation times of up to 100 ms. Please note that there is an ideal line spacing value, to cover the entire neuronal membrane. The minimal excitation volume of a microscope depends on the theoretically reachable optical resolution, defined by the full width at half maximum (FWHM) of the PSF, and can be estimated by the magnification of the objective and the chosen wavelength. The reachable optical resolution can be calculated with \( \Delta r\approx \frac{\lambda }{2\bullet \sqrt{2}\bullet NA} \) [59]. With a given wavelength of λ = 1100 nm and a NA = 1.0, an average neuron surface of 20 × 20 μm and a pixel dwell time of 6 μs, it takes 16 ms to cover the entire neuron. For spiral excitation patterns, a similar, yet slightly shorter stimulation time is needed, ranging at 12.5 ms per neuron. If the experimenter aims to excite successively 6 neurons, the excitation process itself takes 96 ms even without consideration of the time required to move from target to target. These technical roadblocks limit the effective use of a raster scanning-based approach, at least if the entire membrane of the neuron needs to be excited for AP-inducing depolarization or AP-suppressing hyperpolarization, respectively. The development of opsins with either a higher ion conductance or more efficient two-photon excitability might reduce the membrane area which needs to be excited, and consequently decrease the stimulation time.

5.3 Two-photon Scanless Opsin Excitation via CGH

As described in Subsection 5.2, if more than two neurons need to be efficiently activated within the time window of a typical imaging frame, a two-photon raster scanning approach is reaching its limits. Parallel excitation methods in principle allow for an infinite number of simultaneously activated regions of interest (ROIs), limited only by the laser power and the spatial resolution afforded by the spatial light modulator used [59]. Parallel methods most commonly use Computer Generated Holography (CGH) for the generation of the excitation pattern [60] (Fig. 2a). In CGH, liquid crystal spatial light modulators (LC-SLMs) are integrated into the beam path to modulate the phase of the electric field of the laser beam, and typically a low-repetition rate, high-energy Ytterbium laser is used as a light source [57]. A precise control of the spatial specificity of the calculated phase hologram is achieved by applying a Fourier transform-based iterative algorithm on previously acquired fluorescence images of the region of interest [61]. A detailed description of the technical aspects of this method can be found in Chaps. 4 and 11.

With this approach, an effective, simultaneous excitation of several neurons is amenable. The illumination targets do not need to be located in the same z plane, neurons above or below the imaging plane can be interrogated as well, provided that the location of all targets is known (Fig. 3d). Depending on the respective microscope and illumination concept, neurons can be excited which are located several hundreds of μm distant from the current z plane. Yet, upgrading an existing microscope setup with this technique can be cumbersome and expensive, as major changes in software and hardware need to be done. However, it might be an interesting option for scientists experienced in two-photon microscopy to upgrade a microscope with this technology. A further interesting option represent hybrid solutions, combining CGH and scanning [12]. These hybrid solutions guide the illumination patterns generated by the SLM on a galvanometric mirror. These patterns comprise multiple typically rather small focal points, but with high light intensity. This cloud of focal points is then scanned by the galvanometric mirror, resulting in each focal point to cover an entire given neuron either using line or spiral scanning approaches. Thereby, a limitation of CGH-only approaches in terms of the ever-decreasing light intensity when increasing the number of neuron-sized patterns is circumvented. For a constant laser power, the number of neurons which can be effectively activated by a two-photon regime, as discussed in the previous section, and in [12] is consequently higher with hybrid solutions (see also Chap. 11).

5.4 One-photon Miniature Microscopes with GRIN Lenses

Recent one-photon miniature microscopes are capable of opsin excitation via an implanted GRIN lens. Typically, for optogenetics, a second LED light source equipped with corresponding bandpass filters is used in these systems. The excitation paradigm is based on one-photon excitation (Fig. 2b).

A clear advantage of using GRIN-lens-based solutions is the ability to specifically target deep structure for optogenetic interrogation, without undesired activation, e.g., of passing opsin-expressing neurons or neurites dorsal of the target region. The ventral regions below the GRIN lens are illuminated by the light emitted by the lens, in the geometrical shape of a frustum (Fig. 3b). An estimation of the effective penetration depth, i.e., exceeding the threshold for opsin excitation, can be achieved using the Kubelka-Munk model of light transmission through diffuse scattering media [55, 62, 63]. The light intensity is decreasing by the distance to the lens surface, estimated by:

$$ I(z)={I}_0T(z)G(z), $$

where I0 is the light intensity at the tip of the lens, T(z) the transmission factor according to the Kubelka-Munk model for diffuse scattering media, and G(z) is a geometry factor. The transmission factor is defined as:

$$ T(z)=\frac{1}{Sz+1}, $$

with S = 11.2 mm−1, the damping constant in mice [55]. As light is spreading in a conical shape from the tip of the lens, the angle of divergence is calculated by \( {\theta}_{\textrm{div}}={\sin}^{-1}\left(\frac{NA}{n}\right) \), where NA is the numerical aperture of the GRIN lens and n = 1.36, the refraction index of gray matter in the brain [64]. Using the definition \( \rho =r\sqrt{{\left(\frac{n}{NA}\right)}^2-1} \), the geometric component, the conical light spread can be calculated as:

$$ G(z)=\frac{\rho^2}{{\left(z+\rho \right)}^2}. $$

Therefore, the intensity depending on the depth is defined as:

$$ I(z)=\frac{I_0{\rho}^2}{\left( Sz+1\right){\left(z+\rho \right)}^2}. $$

For a GRIN lens with a diameter of 200 μm, a numerical aperture (NA) of 0.5, a light intensity of 35 mW mm−2 at the tip of the lens, and an excitation wavelength of 630 nm, the effective penetration depth for opsin excitation assuming a minimal threshold for excitation of 1 mW mm−2 is 650 μm with a geometric loss of 0.08, resulting in a total volume of 0.118 mm3 with the Kubelka-Munk model. Note that the lateral spread close to the tip of the fiber may be underestimated by the Kubelka-Munk model [54, 62], at least for highly scattering shorter wavelengths.

6 Everything You Always Wanted to Know About All-Optical Data Processing But Were Afraid to Ask

Independently of the chosen technical implementation, the goal of any pipeline for data processing is to extract a subset of features from the raw data, either based on a priori hypotheses, or by data-driven unsupervised methods. Here, we present a pipeline based on the idea of an event-related analysis: We put forward the notion that any functional circuit imaging of a neuronal ensemble at the end only serves as a tool to extract the underlying action potential train of each individual neuron. Consequently, we provide a step-by-step guide to extract putatively AP-related calcium transients from the raw images. Once this optically derived matrix of AP-related events is generated, e.g., by transforming an extracted intensity trace to a binarized train of zeros and ones, already well-established correlation or interaction concepts can be employed on these dimensionality-reduced datasets.

All-optical experiments require a tailored pipeline for each technical implementation and mandate the avoidance of false-positive identification of putatively AP-related calcium transients.

6.1 Roadmap for Processing All-Optical Data

6.1.1 System Integration

Prior to image data acquisition, the synchronicity of all necessary subsystems needs to be guaranteed (Fig. 4a). This can be achieved by copying all relevant trigger and stimulation pulses to a multidata acquisition interface. These signals include the frame trigger of the microscope, the logic level controlling the light source for opsin excitation, and biomonitoring such as the breathing rate and body temperature. Upon digitalization, the signals are being centrally displayed by a control software, and a master file for each experiment is generated. Reading out the master files allows for a post hoc exact assignment of each raw image to a given time and the status of, e.g., the stimulation regime.

Fig. 4
A pipeline flow with 6 data. It follows system integration of imaging and stimulation hardware, image acquisition, segmentation, trace extraction, artifact removal, binarization, and connectivity.

Workflow of a pipeline for an event-related analysis of all-optical data. Raw data is segmented into ROIs and intensity traces are extracted. A binarization of transients is carried out and a temporal and spatial classification of the recorded microcircuit is applied

Full size image

6.1.2 Image Data Acquisition

The initial step for analyzing the data is to reduce the impact of movement artifacts (Fig. 4b). We differentiate between two types of artifacts: displacements induced by movement in the x-y plane, and in z plane. While changes in the x-y plane can be corrected retrospectively by algorithms based on Hidden-Markov-Model [65] or discrete Fourier transformation-based image alignment [66], provided that the field of view is sufficiently large, movements in the z plane are more complicated to correct. Note that the possibility for x-y movement correction represents the main reason why random-access scanning is not recommended, and the field is currently mainly conducting single-plane resonant imaging. Simply put, if a soma of a given neuron is displaced below the current x-y imaging plane, this information cannot be regained. A potential solution for this problem is to record a 3D volume. However, this results inevitably in a reduction of temporal resolution for any given neuron recorded. If working with currently most used resonant systems with a frame rate of 30 Hz, we would strongly discourage recording 3D volumes, as the temporal resolution is critical for subsequent identification of putatively AP-related calcium transients. This is a prime example on how analysis needs should inform the previous technical implementation steps: Take all efforts to minimize movement by devising mechanical stability, this cannot be stressed enough (see Note 3).

6.1.3 Segmentation

The raw images comprising typically 512 × 512 pixels contain several biological compartments: neuronal somata, axons, dendrites, blood vessels, and certainly a multitude of non-neuronal cells, such as astrocytes. While the one critical advantage of microscopy using fluorescent indicators represents the reduction of image complexity, i.e., ideally only the features of interest express the fluorophore, still, only a fraction of the image contains the signal of interest. What is more, in the context of neuronal microcircuit imaging, it is of advantage to integrate several pixels which reflect the same functional compartment: when the experimenter aims for identification of the suprathreshold activity of individual neurons in the microcircuit, each neuron can be identified as functional unit. Consequently, each neuronal soma is defined and segmented as ROI (Fig. 4c). In recent years, the range of applications that support the experimenter in ROI segmentation grew rapidly: mathematical models to perform automatic segmentation based on deep learning algorithms drastically shorten the time-consuming step to manually identify neurons in the recorded images [67]. While there are deep learning-based methods that process the average of all images, other techniques integrate the temporal information of neuronal activity by using subsets of all images for the segmentation of active ROIs [67, 68]. The approach proposed by Soltanian-Zadeh et al. is based on a 3D technique: subsets of the acquired images are created and used to predict 2D probability maps for active neurons utilizing a neural network. Upon applying a threshold to exclude low-probability regions from each probability map, individual somata are extracted from high-probability regions [67]. Ultimately, all somata positions from each image from the image sequence are combined to acquire a final output of active somata areas. However, manual segmentation of functional calcium imaging data by marking neuronal somata with polygon shapes still prevails due to the applicability on datasets of varying quality, e.g., datasets of a particularly low signal-to-noise ratio, when deep learning algorithms reach their limits. Depending on the strength of GCaMP6 expression and the overall SNR ratio, if targeting somatic changes in calcium concentrations, a decontamination of the ROIs containing the neuronal soma from neuropil signal might be useful. The decontamination is carried out by expanding a given ROI in cardinal and diagonal directions, beyond the region of the soma, which are then separated into several neuropil regions surrounding the initial ROI. The size of each neuropil subregion should equal the size of the initial ROI. Under the assumption that the intensity trace of a given ROI is generated by a mixture of different underlying signal components, but certainly one of the components exhibits the somatic signal of interest, non-negative matrix factorization or independent component analysis are employed to perform a blind source separation, resulting in a separation into the underlying signal components. By weighting the presence of each component in the central ROI, under the assumption that the somatic signal is the prevailing component contributing to the ROIs intensity trace, the strongest signal component is defined as the somatic signal [69].

6.1.4 Trace Extraction

After the locations of somata constituting a ROI have been identified within the image sequence, the signal over time of each ROI is extracted (Fig. 4d). The intensity values of all pixels in a given ROI are averaged for every image of the temporal sequence, resulting in an intensity trace for every ROI, with a temporal resolution determined by the frame rate. It has to be noted that the absolute level of intensity is based on multiple factors such as autofluorescence, the expression levels of the GECI, or stray light entering the objective. Therefore, it is recommended to calculate the relative change of fluorescence, as these dynamic changes, depending on their temporal dynamics, are most likely due to changes in intracellular calcium levels, and can therefore be termed calcium transients. Note that there is inevitably a drift of the baseline levels due to bleaching. These drifts can be compensated for, as long as linear operations are being used (see Note 4). There is no convention on how to perform baseline correction. An option used by us and others represents the definition of a sufficiently long period of quiescence, i.e., stable baseline non-interrupted by any transients, and defines this period as baseline (F0), separately for each neuron [7, 9]. The relative change of fluorescence is then calculated by relating the intensity of each time point (F) to the baseline fluorescence \( \Delta F=\frac{F-{F}_0}{F_0} \) [70].

6.1.5 Artifact Removal

Prior to binarizing the extracted intensity traces, potential photostimulation artifacts superimposing the signal need to be identified and, if possible, corrected (Fig. 4e). Characteristic properties of photostimulation artifacts can serve as a basis for reliable identification and subsequent correction. Firstly, looking only at individual responses to an optogenetic stimulus might lead to the notion of a physiological signal. Yet maybe the most decisive difference between an artifact and a physiological response is the inherent variability of the physiological signal. Artifacts, with rare exceptions, are rather consistent. Overlaying and averaging the individual responses therefore gives important cues on the probability of a physiological origin. For that, the temporal section of a trace upon the photostimulation is assessed by a n∗m matrix M, with n = stimulus intervall, m = total frames/n. Figure 5b shows an overlay of the intensity traces of all photostimulation periods of Fig. 5a (gray lines).

Fig. 5
A three-part diagram shows the evaluation of photostimulation artifacts. A. Long simultaneous spikes. B. One large spike generated in the middle. C. Short randomly generated spikes throughout.

Evaluation of photostimulation artifacts. (a) Artifacts caused by photostimulation do not represent a physiological signal. Onsets of photostimulation are indicated by black triangles. (b) By overlaying (gray) and averaging (red) periods of photostimulation, an estimation on the impact of the artifact can be made. Here, 100 sample points prior to and 200 sample points after the photostimulation onset, indicated by the black triangle, are considered. The averaged signal waveform can be used in an algorithm to minimize the photostimulation artifact. (c) Subtracting the averaged waveform depicted in (b) at each stimulation onset can reduce the intensity of the photostimulation artifact. Note that signal components containing the physiological signal of interest can be altered by the algorithm as well and, in the worst-case scenario, will be completely eliminated

Full size image

Photostimulation artifacts exhibit several typical features: Firstly, a photostimulation artifact will show both a sharp onset, and a sharp offset. Functional calcium transients of physiological origin might also display a rather sharp onset, due to the high affinity of calcium to the indicator represented by a small dissociation constant KD and the high temporal gradient of calcium influx. But it will be characterized by a slow decay, an inevitable consequence of the high affinity of the indicator to calcium. The off kinetics does not mirror the true time course of the decrease of somatic calcium concentration. If a researcher would be interested in assessing the re-uptake of calcium, an indicator with a high KD/low affinity would be advantageous, but these indicators typically display a lower SNR [71]. Any trace deflection with a sharp decay when using a low KD indicator is therefore almost certainly not associated with an AP-related somatic calcium response. Secondly, if the duration of the intensity deflection equals the duration of light administration, a physiological origin is highly unlikely. As depicted in Fig. 5b, the intensity traces during photostimulation exhibit both of the mentioned features.

What is more, but this very much depends on the imaging setup used, the latency between the onset of the stimulation pulse and the putative response is critical: If the latency ranges <3 ms, it cannot be considered as a physiological response, due to the time needed for the AP initiation and the influx of calcium into the cytosol. Please note that this criterion can only be used if the sampling rate of the GCaMP emission channel is high enough to resolve durations less than 3 ms. Lastly, a physiological response might be subject to adaptation, and often times does not occur upon every stimulus, e.g., due to changes in local inhibition [54]. Consequently, while a signal deflection might surely be the result of a physiological response if it occurs upon each stimulus, nonetheless, if a response is drastically changing its amplitude, or sometimes is not present, it is likely to be of physiological origin. To test for that, it might be useful to modulate the light intensity of the optogenetic stimulus: if a decrease of stimulation leads to the sudden disappearance of a transient, it is again likely that an AP-related origin can be assumed, as a technical artifact would simply scale with the light intensity, even though also non-linear correlations can occur. Certainly, a control experiment comprising of indicator-only expressing cells should be conducted in any case.

The subsequent correction of photostimulation artifacts in the intensity trace is a delicate task that needs to be conducted carefully. A possible approach is to employ non-negative matrix factorization [72] to identify the background noise of a given ROI containing the stimulation artifact. In a second step, the identified noise component is subtracted from the ROIs signal [73]. Here, for illustration purposes we obtained the averaged intensity trace of all time periods of photostimulation (Fig. 5b) and subtracted the given value from the raw intensity trace (Fig. 5a). This results in an intensity trace devoid of at least the majority of the non-physiological signal sources (Fig. 5c). Please note that the processed intensity trace has to be treated with caution: we can still observe fluctuations in the signal that could be misinterpreted as functional calcium transients. Furthermore, using methods based on complex mathematical models often act as a black-box, making it difficult to grasp the underlying methods for non-experts. We strongly emphasize to design the experimental paradigm in a manner to avoid any stimulation artifact during the measurement and to minimize the need for post-processing of raw data as much as possible. Characteristic temporal dynamics and latencies of the used GECI, as put forward before, can serve as a sanity check, and, following up with an event-based identification of AP-related calcium transients is mandatory.

6.1.6 Event-Related Binarization

Action potential-related calcium transients exhibit typical signal dynamics and waveforms. Methodologically, the rather stereotypical dynamics of the somatic calcium influx and re-uptake respectively efflux is convolved with the binding dynamics of the fluorescent calcium indicator [74]. Therefore, each binarization approach critically has to be tailored to the calcium indicator used. The resulting AP-related calcium transient can be described based on parameters such as rise time, decay, and duration, or, alternatively, an idealized transient is used for performing deconvolution approaches [75,76,77]. A rather straightforward but yet effective method, with rather low false positives [7, 9, 34] represents a threshold-based algorithm. For that, first, the standard deviation of the noise band of the trace is determined, ideally in a section of the trace not containing the signal of interest. In the case of an all-optical experiment, time periods outside of stimulation events and with low spontaneous activity should be used. Once the signal exceeds this threshold, and if additional criteria such as minimal duration are met, a putative event is detected. Furthermore, as the decay of an AP-related calcium transient typically follows a roughly exponential decay, at least for commonly used indicators such as GCaMP6 and OGB-1, a regression of the trace with an exponential function can be conducted. The goodness of fit for this regression gives additional evidence for the physiological nature of the given transient. By identifying onset, peak, and offset for each detected transient, we can represent the fluorescence signal as a binarized event array (see Notes 5 and 6). Note that also deconvolution-based methods provide such a binarized array. All following analyses are carried out on these reduced datasets (Fig. 4f).

6.1.7 Connectivity Analysis

The coherent activity of neurons of a given microcircuit or ensemble in the field of view is calculated and visualized based on two variables for each neuron: first, the window of co-incidence, in which two events are considered to occur simultaneously, has to be defined (Fig. 4g).

Certainly, for achieving information on Granger causality [78], this coincidence window should be as short as possible, ranging at only a few milliseconds. Yet, in the field of imaging, the minimal coincidence window cannot fall below the imaging frame rate, for resonant scanning approach this ranges at 33.3 ms, and it might even be necessary to further lengthen this co-incidence window, particularly in scan-based imaging methods. Otherwise, coincidence would be biased by the individual location of a given neuron in the scan field. Within this given coincidence interval, at least one other neuron needs to exhibit an event. Second, the average number of simultaneously active neurons is determined. The binarized calcium transients of each neuron i are used to identify the number of active neurons per frame by introducing a counter ci, registering the coincident active time points, and an array Ai, storing the number of coincident active neurons for each time point. Now, we ask for each active time point j if the sum of active neurons is nact, i > 1. If this is the case, the counter ci is incremented by 1 and Ai is appended by nact, i − 1. Afterward, ci is divided by the total number of active time points defined by Xi, representing the relative number of coincident active time points of each neuron i and the mean of Ai gives insights into the average number of coincident active neurons.

The number of simultaneously active time points ci relative to the number of all active frames per neuron ci/Xi are plotted against the overall number of active frames of the respective neuron Xi. Additionally, a linear regression analysis is applied to the data.

The average number of simultaneously active neurons can be visualized using box plots. The distribution of the average number of coherent active neurons are compared by a statistical test, Students t-test, or Wilcoxon rank sum test.

To gain insights into the connectivity pattern of the given microcircuit, the information about the position of a ROI, its activity, and the correlation between each pair of ROIs, a network graph can be constructed. Each ROI is represented by one vertex. The position of each vertex is given by the ROIs x and y position. The color is based on the ROIs activity. In the example given in Fig. 4, the color scale is constructed as a non-linear gradient between green, yellow, and red in order to show single ROIs as hypoactive (green), normal (yellow), or hyperactive (red). The pairwise Pearson’s correlation between each pair of ROIs is represented by the width of the edges.

Together, such a connectivity analysis takes advantage of the spatial information obtained by an imaging experiment, which at least partially overcomes the inherent drastic limitations in terms of temporal resolution compared to methods such as single-cell patch-clamp recordings. Only both methods together, high temporal resolution of electrophysiological methods, directly acquiring the signal of interest, and functional imaging obtaining the local functional architecture, can provide a holistic understanding of neuronal network (dys)function.

6.2 Toward Cross-Talk-Free Experimental Designs

6.2.1 Assessing the Impact of Continuous Illumination for Calcium Imaging on Opsin Excitation

While the strong light intensities needed for an effective excitation of an opsin pose the aforementioned problems for simultaneous functional calcium imaging, we need to also consider a putative impact on the constant excitation of the calcium indicator in terms of unwanted activation of opsins (see also Chap. 2). The activation of an opsin requires a distinct quantal energy; in case of the ChR2-based opsins or the cis-trans isomerization of retinal, this threshold ranges from 0.1 to 1 mW mm−2. All-optical one-photon experiments, even using the same wavelength for imaging and opsin activation, is therefore possible, as long as the excitation intensity ranges below this threshold [32, 47, 54]. However, for two-photon line scanning conditions, the effective light intensity per pixel can well exceed this barrier. Nevertheless, fortunately, there is also a temporal barrier. It so seems that any pixel dwell times below 3.2 μs may not suffice for efficient opsin excitation at least for opsins such as C1V1 [12]. Pixel dwell times of regular resonant scanning range well below that number. However, novel generations of opsins designed for more efficient two-photon excitation might allow shorter pixel dwell times.

6.2.2 Increasing the Spectral Separation Between Opsin and Indicator to Minimize Optogenetic Stimulation Artifacts on the Imaging Data

Avoiding cross-talk in the GECI emission channel is an important criterion in the design of all-optical experiments. Both for one-photon and two-photon regimes, the excitation light can be rather easily prevented from entering the PMTs, e.g., by notch filters. But nonetheless, the strong light pulse might lead to a broadband increase in autofluorescence, also in the emission band of the fluorescence calcium indicator, leading to a decrease in SNR. While in principle, the utmost spectral separation of opsin and indicator excitation wavelengths is advisable, there might be good reasons to choose an opsin indicator pair with similar or even identical excitation wavelengths, if, e.g., both indicator and opsin are best suited for the scientific question at hand, as mentioned above. Post-hoc identification of photostimulation artifacts is manifold and should be conducted with utmost care. Common approaches compare the dynamics of the signal’s intensity during photostimulation to the expected dynamics of calcium transients [17] to make an assumption of the presence of a physiological response (see also Subheading 6.1.5). As a consequence, the specificity of the analysis toward the detection of AP-related calcium transients is reduced. Other approaches sacrifice pixels that are considered to contain signal components that represent an artifact [79, 80] at the cost of a reduction of the field of view. Another option, if a spectral separation of excitation and emission wavelengths cannot be implemented, is to perform gating of light sources, inevitably leading to a loss of data [81]. Yet, it is possible to avoid any stimulation artifact altogether, which has to be tailored for each opsin/GECI pair (see Note 7). For the pair GCaMP6/C1V1, artifact-free all-optical physiology experiments are possible if using excitation wavelengths at or exceeding 1100 nm while simultaneously imaging GCaMP6 fluorescence at 920 nm [34]. Employing a blue-shifted opsin and a red-shifted calcium indicator resents another viable option [58].

7 Outlook

All-optical physiology in neuroscience, i.e., simultaneously recording and manipulating individual functional components of a given microcircuit, opens up truly new experimental designs [80]. Particularly in the field of research on preclinical rodent models of neurological disorders, these “dream” experiments can provide evidence for the impact of the (dys)function of individual neurons, and neuronal ensemble activity to network performance and behavior. Yet, all-optical physiology did not at all exploit its full potential yet. The reasons are manifold: First, the ideal opsin for efficient induction of action potentials, mimicking the temporal dynamics of physiological inputs, while tremendous progress has been made, is not available yet. While efficient two-photon excitation seems to have almost achieved this goal, particularly the efficient two-photon-based inhibition is still in its infancy [82]. Nevertheless, the opsin and the stimulation method is only a part of the entire workflow. We need to move beyond rather synthetic, non-physiological stimulation patterns, which often evoke non-physiological responses, e.g., due to hypersynchronization of the network caused by the simultaneous activation of multiple neurons. For that, we not only need to record the neuronal activity but also analyze the activity pattern in real time. Only then, we will be able to re-play and meaningfully modify the endogenous ensemble activity, closing the loop. In this chapter, we covered key aspects of system integration and the basic concept of analysis of all-optical functional data. Indeed, in our view, the entire workflow, from the acquisition of raw images to the identification and binarization of events, and the design of the stimulation pattern, has to be transformed toward real-time closed-loop performance [83], as put forward in the context of global functional brain state changes [84]. This requires first of all dedicated fast hardware. Each component of the system: microscope, optogenetic pattern generator, signal acquisition, and signal computation, has to operate on real time. This is in reach for the microscope, optogenetic pattern generator, and signal acquisition, as put forward in this chapter, but, is still far from achievable for an event-based signal processing and signal analysis in real time. Achieving real-time capability of the critical analysis step requires the close collaboration between neuroscientists, mathematicians, and (bio-)informaticians. First, the neurophysiological event of interest has to be identified and described in painstaking detail. For instance, in our own research interest involving state transitions of local populations, these events needed to be discerned from the large parameter space of neurophysiological events, and described in terms of their spatio-temporal dynamics and its variance. Only then, the mathematicians can either take advantage of statistical methods, or of new unsupervised machine learning algorithms. Indeed, the advent of artificial intelligence may enable to accelerate at least key aspects of analysis routines which traditionally take weeks and months, to seconds or even milliseconds [67, 85, 86].

The fast-evolving field of all-optical physiology of neuronal microcircuits can only thrive in a multi-disciplinary environment and is critically dependent on each component to be optimized and ideally integrated. Yet, now is the time, now all individual advances can come together to make real-time closed-loop all-optical physiology a reality.

8 Notes

  1. 1.

    Choice of indicators: Adapt the kinetics of your indicator to the respective firing properties of the neuronal populations of interest: i.e., in rather fast-spiking neurons, indicators with a slow off kinetics such as GCaMP6s would not be advisable, but, in cases of deeper structures with low SNR, slow indicators might be the better choice.

  2. 2.

    Testing expression levels for novel experiments: We suggest testing the functionality of opsins in in vivo experiments by initially exciting the microcircuit by one-photon stimulation. Since the success rate of opsin excitation is generally higher under one-photon regimes, conclusions regarding the level of the expression can be drawn easily before a more error-prone approach based on two-photon excitation is used. If no activation upon opsin excitation is detected, a sanity check if the cell is expressing the opsin can be carried out by combining the opsin with a fluorescent marker with different spectral properties than the used GECI. For example, mCherry or tDtomato can be used in combination with an opsin while functional calcium imaging is carried out with GCamP6. If a sanity check using optical methods is not possible, electrophysiological recordings can be used as an alternative.

  3. 3.

    Mechanical Stability: Since movement-induced distortions in z-direction cannot be corrected post hoc, it is of utmost importance to ensure mechanical stability of the integrated and mechanically connected unit comprising of the brain as target structure, the cranium, and the holder. The head holder provides the mechanical connection to the stage and represents the most critical component. Here, one-point fixations on only one hemisphere of the skull, often designed as a metal bar connected to the skull by dental cement or UV-glue, are not providing sufficient stability in our view, at least a two-point fixation is considered as mandatory. We strongly suggest to maximize the adhesive surface between the fixation device and skull and to attach it to both cranial hemispheres.

  4. 4.

    Avoid non-linear bleaching: Typically, in the first few seconds of data acquisition, the intensity trace displays a rather non-linear, exponential decrease which cannot be compensated for. Any non-linear manipulations are highly problematic, as they affect the dynamics of the calcium transients and therefore strictly speaking manipulate the data. However, linear bleaching artifacts can be addressed by baseline correction [9, 34, 87].

  5. 5.

    Implement event-related analysis to identify AP-related transients: Temporal classifications of noisy transients, caused by the physical principle of the recording as well as the source of the physiological signal (burst firing), are often carried out by correlation approaches which are suitable for highly fluctuating signals. However, we propose the application of event-related analysis pipelines whenever possible to ensure the specificity of neuronal responses with respect to the underlying signal of interest, i.e., the neuronal action potential.

  6. 6.

    Perform manual inspections: Every algorithm has its strengths and weaknesses, and, due to the highly variable quality of all-optical experiments, e.g., in terms of SNR, might lead to false-positive results. Therefore, no matter which method is used to identify AP-related calcium transients in an intensity trace, a manual inspection of the binarized event array needs to be conducted to eliminate all potential false-positive results.

  7. 7.

    Avoid cross-talk at any time: Capturing the onset of a calcium transient can be impossible due to cross-talk between emission/excitation wavelengths: Artifacts can be ruled out by blanking the PMTs at the cost of sacrificing information about fluorescence during opsin excitation, while on the other hand applying algorithms to minimize photostimulation artifacts may result in a loss of the physiological signals of interest. We strongly suggest to design experimental paradigms in all-optical physiology to avoid cross-talk at any time by choosing spectral independent opsins and calcium indicators.