Introduction

Two-photon intravital microscopy (2P-IVM) has become a tool of choice to longitudinally explore dynamic physiological events within living tissues with minimal photobleaching and phototoxicity. This technology provides not only the ability to track cell movements [1], [2], but also to observe morphological changes in small cellular structures [3], [4], and even monitor cell proliferation and death [5]. 2P-IVM has already revolutionized all areas of biology [6], and yet we are only at the beginning of a new era in dynamic biological imaging.

One major limitation of 2P-IVM that has yet to be addressed is that image sequences are commonly plagued with motion artifacts caused by cardiac and respiratory cycles and muscular and vascular tone. These movements are especially present in body parts that cannot be completely restrained, even under deep anesthesia. To make things worst, animals anesthetized with isoflurane, a safe anesthetic that provides rapid induction and recovery, often do not breathe regularly, which impairs the possibility of triggering the acquisition following a constant frequency. Other methods that reduce the source of movements, such as endotracheal intubation for mechanical ventilation [7], [8], cardiopulmonary bypass [9], and injection of myorelaxants [10] or muscle paralyzing drugs [11], can physiologically affect the experiment and lead to death if not carefully monitored. Recently, Laffray and co-workers have developed an ingenious stabilization device, featuring an off-axis laser and a piezo nanopositioner, that constantly maintains the focus [12]. However, such a system is relatively costly and requires a custom installation. Here, we introduce a free public-domain software package that allows for automated processing and filtering of image sequences generated using 2P-IVM.

Materials and Methods

Results

To test our macro, named Intravital_Microscopy_Toolbox, we performed 2P-IVM on various tissues of the nervous system (brain, spinal cord, sciatic nerve and myenteric plexus) both in naïve and injury/disease conditions. For that purpose, we used transgenic mice that express fluorescent proteins in a cell-lineage-specific manner (see Materials and Methods). We began by implementing new restraining techniques or ameliorated existing ones to minimize motion artifacts caused by animal breathing and other physiological movements (Figure S1). Although we achieved a major improvement in terms of stabilization, the images acquired were still prone to different kinds of motion artifacts (i.e. movements in the x–y, and/or z direction; see Videos S1–S2). Importantly, even the implantation of a spinal cord chamber, as recently developed by Farrar et al. [14], was associated with the presence of artifacts during 2P-IVM imaging (Video S2, right panel). Although the amplitude of movement in the z direction was significantly reduced with this method, it was sufficient to affect our ability to visualize fine movements of small cellular structures (e.g. motility of microglial processes). Thus, the animal models examined in this study were showing distinct motion artifacts, which we used to our advantage to develop and test the efficiency of the program on multichannel image sequences acquired using 2P-IVM.

The macro was designed to automatically generate reference frames, compute dissimilarity scores and remove distorted, translated, and out-of-focus images. To measure the presence of artifacts within the image sequence, we used a dissimilarity score calculated with a square difference algorithm against a reference image. In brief, the function slides through images, compares all the frames from the image sequence against a reference frame and stores dissimilarity scores in an array for subsequent gating using various filters (Figure 1). We intentionally avoided designs requiring command line inputs and opted to develop the program as a “one-click” macro for ImageJ, the most widely used public domain image processing software, usable by all. The macro was designed to handle 3 fluorescent imaging channels in simultaneous acquisition, and to sort images based on pixel-by-pixel comparisons using multiple reference frames. Because blood vessels are rarely subjected to rapid morphological changes, we used them as the reference channel to sort out images and remove artifactual frames.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Macro flow chart for the “Intravital_Microscopy_Toolbox” program.

Starting from the file selection, the image dissimilarity score is computed for each frame in the video. Main processing options are identified with grey shading.

https://doi.org/10.1371/journal.pone.0053942.g001

To optimize the macro efficiency, we manually analyzed the original videos and qualitatively ranked each frame as either good, with minor artifacts or with major artifacts. We then tested two different approaches to calculate dissimilarity scores; one relying on the manual selection of a single reference frame (Figs. S2, S3, S4, S5 and Video S3), and the other on the automated generation of groups of reference frames along the video sequence (Figure S5 and Videos S3–S4), from now on referred to as automatic reference frame generation (RFG). For RFG, the macro was programmed to regroup images in series of n adjacent frames (e.g. series of 5–30 frames) centered on the image to be scored, and an average intensity projection calculated for each series, hence generating a reference frame. Each frame was then scored relative to its own reference frame. Our testing revealed that, although the single reference frame method allows to detect motion artifacts efficiently in short videos, the position of the reference frame within the image sequence had a profound effect on the dissimilarity scores and thus the filtering efficiency (Figs. S2 and S5), a problem that became even more noticeable in long videos (Video S3). As shown in Table S1, the presence of artifactual images and false positive and false negative rates were lower when using the automatic RFG compared with the single reference frame strategy. Furthermore, the baseline of the dissimilarity scores was more linear when the automatic RFG approach was used (Figure S5). For these reasons, all subsequent analyses were performed using the RFG method.

It thus became interesting to reject the highest dissimilarity scores using a percentile filter. With a cutoff percentile of 60%, we were able in most cases to reject nearly 100% of the artifacts, which made the rate of artifactual images to drop from 39.8% to 2.4% in the provided example (Figure 2 and Video S3 and Table S1). It is however advantageous to compare groups of n neighboring images and filter out the image with the highest dissimilarity score when the percentile gating is set low, as it would otherwise result in removing too many good frames. Because the relative maximum filtering option does not allow the elimination of two consecutive distorted images, filtering the image sequence twice with the same filter (double filtering) can help remove most of the local maxima (Figure S4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Calculation of dissimilarity scores using the automatic reference frame generation (RFG) option, and subsequent removal of artifactual frames from the video sequence.

(a) Graph plotting of the total number of frames in a 1000-frame video against the dissimilarity score, identified as ‘square difference’ score on the y-axis. (b) Relative frequency distribution of the number of frames per dissimilarity score. (c) Graph showing the frame-by-frame analysis of a 1000-frame video sequence. Artifactual frames were eliminated (identified by red crosses) using a cutoff percentile of 60% (defined by the horizontal blue line). Major artifacts (shown as black dots) are typically artifacts resulting in a full field distortion of the image, while minor artifacts (shown as green dots) are small glitches in frames that do not impair the image interpretation. (d) Images corresponding to local maxima (upper panels) and minima (lower panels) in the first 200 frames of the video sequence. Note that frames corresponding to a local maximum are associated with heavy artifacts, while frames corresponding to a local minimum are typically very similar to the reference frame. All images are derived from Video S3.

https://doi.org/10.1371/journal.pone.0053942.g002

Convenient extra tools were added to the program such as x-y alignment of the image sequence, automatic channel subtraction, time stamp and scale bars overlays. In brief, the possible alignments include: 1) translation; translates the source image, 2) rigid body; translates and rotates, 3) scaled rotation; translates, rotates and enlarges or reduces, and 4) affine; translates, rotates, shears, skews, and enlarges or reduces. The translation feature was particularly useful to correct for the x–y motion caused by breathing in brain and spinal cord videos, while the affine transformation was more efficient in tissues prone to peristaltism (e.g. myenteric plexus) because of the observed changes in shape, volume, and position (see Figure 3 and Video S5). It should be emphasized that the rigid body, scale rotation and affine transformation options rely on changing pixel positions. The translation transformation, which is the most frequently used of all transformations, has an effect on the image as a whole, without being able to alter a separate part of the image specifically. Since the alignment proceeds by propagation from the previous frame, the x–y alignment efficiency was dramatically improved by the removal of artifactual frames. The channel subtraction option was implemented to remove cross-contamination of dyes in other channels, due to the simultaneous excitation of fluorophores (e.g. CFP in the YFP channel). Finally, the macro is able to display an accurate timestamp on the video, even when frames are removed.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Comparison of x–y transformations efficiency to correct for motion artifacts induced by peristaltism in the murine myenteric plexus.

(a) Artifactual displacement from frame to frame for an object either located in the center (object identified by the white cross in Video S5) or periphery (white circle in Video S5) of the field of view. The ‘Sum’ indicates the total artifactual displacement of an object in the video sequence. (b) Plots illustrating the relative motion of selected objects within the field of view. ***p<0.001, **p<0.01, and *p<0.05, one-way ANOVA with Bonferroni post-test.

https://doi.org/10.1371/journal.pone.0053942.g003

In addition, various image-processing options were incorporated in the macro, such as maximum intensity projection and average intensity projection (Videos S6–S7). These options are useful to improve the signal-to-noise ratio and frame rate (frequency). We also implemented a Kalman filter that allows the removal of small artifacts within frames, such as warping and blinking when the scanning speed is too slow. Although this feature is inefficient in the presence of major artifacts and fast moving objects, it becomes extremely useful following artifact removal by the macro because it removes the noise and enhances small details. Since this filter improves the image quality by using previous frames as templates for calculation, it keeps the current video frame rate, which is a major benefit compared to a maximum intensity projection or an average intensity projection. As shown in Videos S8, S9, S10, S11, S12, this filtering option was essential for imaging subtle movements of microglial processes and morphological details of neurites.

Discussion

To summarize, we report here the creation of a new ImageJ macro program for automated removal of frames contaminated with movements artifacts caused by cardiac and respiratory cycles and muscular and vascular tone during in vivo imaging studies. The “Intravital_Microscopy_Toolbox” program has been extensively tested and proven effective in solving problems related to tissue movements in series of images acquired from the brain, spinal cord, sciatic nerve and intestine (myenteric plexus) in living mice. Compared with manual processing of data, the automatic processing conferred by the program not only save time and effort, but also reduces the risks of errors and biases introduced by the experimenter. In addition, several key features were incorporated in the program such as filtering options, extended field of view with maximum or average projections, image-to-image alignment, channel subtraction, and automatic time stamp and scale bar overlay. The “Intravital_Microscopy_Toolbox” ImageJ plug-in, its User’s Guide and high resolution Videos S1, S2, S3, S4, S5, S6, S7, S8, S9, S10, S11, S12 have been made freely available to the public and can be downloaded from the (Toolbox S1 & Users Guide S1) or at the following websites: http://stevelacroix.crchuq.ca and http://denissoulet.crchuq.ca.

Image distortion caused by movements such as breathing is a major impediment to creating movies out of image sequences captured by 2P-IVM. If not filtered and removed from the image sequence, shifted and distorted frames will alter the quality of any time-lapse movie. Although strategies were developed to circumvent problems related to intrinsic animal movement, such as animal intubation [7], [8], [18], cardiopulmonary bypass [9], induction of muscle paralysis [11], and deep anesthesia using powerful anesthetic cocktails [10], we feel that none of these strategies are perfectly compatible with the long-term goal of repeatedly imaging the same animal. Endotracheal intubation, for example, has been used in the setting of mechanical ventilation under general anesthesia to regularize breathing and allow for steady-state conditions during which in vivo imaging can be triggered [7], [8], [18]. However, like cardiopulmonary bypass, this technique is a complicated and invasive surgical procedure in small laboratory rodents. The use of injectable anesthetic agents combined with muscular relaxants, such as the ketamine-xylazine-acepromazine anesthetic cocktail, was recently shown to be useful for minimizing movement during 2P-IVM, as it not only induces deep anesthesia but also reduces heart rhythm [10]. However, because these anesthetic agents are often associated with heart arrhythmias, low blood pressure, and respiratory depression, overdoses are common with these drugs and often fatal. Repeated injections will therefore multiply the chances of the animals dying from an overdose, which is a major limitation when one has to deal with costly and rare animals on which were performed complicated surgeries. In this paper, we present a new and simple post-processing method that controls for intrinsic motion artifacts without the need to perform invasive surgeries or administrate drugs that may interfere with physiological properties of the tissue.

The Intravital_Microscopy_Toolbox program presents a broad range of advantages. What perhaps distinguishes it most from other existing programs is that it is available on the free, multi-platform and widely used image analysis software ImageJ, meaning that it has the potential to be used on any intravital microscopy video recordings. The Intravital_Microscopy_Toolbox thus offers an easy and reliable filtering method for laboratories specialized in in vivo microscopy without the need to buy costly external devices. The removal of motion artifacts is accurate, rapid and fully automated, and therefore not subjected to errors and biases that could be introduced by the operator. On a more practical scale, the removal of distorted and out-of-focus frames allows a better understanding of biological processes and phenomena occurring within tissue of living animals. Along the same line, the complementary features of the Intravital_Microscopy_Toolbox, such as projections, channel subtraction and transformations, increase the quality of videos and facilitate data interpretation and analysis. Additionally, the program allows to efficiently exploit plugins (e.g. StackReg, JavaSIFT) that were initially developed to realign and/or transform series of images obtained from fixed-tissue sections or using in vivo methods that are less prone to motion artifact than 2P-IVM, such as positron emission tomography and functional magnetic resonance imaging [19], [20]. Without first filtering image sequences from 2P-IVM-generated movies to remove artifactual frames, we found that the above-mentioned plugins almost always create erroneous alignments.

It should be noted that at least one other study has developed an ImageJ-based macro (implemented in the UnwarpJ plugin) to correct motion artifacts in time-lapse videos [21]. The methodology was found efficient in resolving single cell movement relative to global tissue distortion using data sets obtained from slice cultures. However, we found that the unwarping method is inefficient when the distortion is not uniform, as it is almost always the case in tissue prone to a high the degree of movement, such as the living spinal cord, sciatic nerve or intestine myenteric plexus. The same limitations are likely to apply as well to the Matlab-based software recently developed by Greenberg and Kerr [22]. Although we did not test this program yet because it is not available in the public domain, the authors themselves state that their program is time-consuming and specifically designed for correction of movements of limited amplitude (head-fixed rats). It should be pointed out that the later methodology has the distinct advantage of not requiring the collection of a reference frame or channel. An additional drawback of the above two mentioned programs is that they rely on local changes of pixel positions within the image, a manipulation that is incompatible with publication guidelines of many journals. Importantly, the “Intravital_Microscopy_Toolbox” allows to process images as a whole rather than in parts, except when making use of the rigid body, scaled rotation, and affine options to rotate images, enlarge or reduce image size and/or transform images afflicted by shears and skews.

More recently, an ingenious and effective technology designed specifically for live animal imaging was introduced by Laffray et al. [12]. The technology consists of an optical stabilization sensor, fully compatible with 2P-IVM, that monitors the animal position to refocus the objective onto the plane of interest dynamically. The main disadvantages of this movement compensation device are its cost and the fact that it has yet to be implemented on a commercial system. In contrast, the computer-assisted motion filtering algorithm presented in this paper is free and compatible with both commercial and custom-built multiphoton microscopes. This was made possible by the implementation in the macro of a module that loads any RGB 24-bit image sequences, thus ensuring full compatibility with other systems.

As with any other programs, the “Intravital_Microscopy_Toolbox” also has some limitations, which gives place to future improvements. Perhaps the main limitation of the macro is that it does not allow the generation of more good frames in a video sequence, much like any other post-processing methods, but rather focuses on eliminating bad ones. Thus, the program requires that only a subset of frames are affected by distortions, otherwise too many would be thrown out. Our experience has been that this limitation becomes much less noticeable with accelerated image acquisition, as this increases the good-to-bad frame ratio. If the scan speed is set to slow (i.e. less than a couple of frames per second), fast moving objects, such as circulating cells in blood vessels, will appear as lines in images and the filtration will be impaired (with or without projections or Kalman filter). In order to work to its full potential, the macro requires a reference signal (structure) not subjected to plasticity under conditions of stress or injury for proper alignment of the frames. In the present study, we chose blood vessels as a reference structure. One needs to keep in mind, however, that blood vessels have been shown to grow or regress in response to factors released during the tissue repair process and during various pathological conditions such as cancer and neurological disorders [23], [24]. However, this phenomenon usually occurs over a protracted period of time and is not a major problem for short imaging sessions (2–3 h). The macro is also unable to process more than three fluorescent channels at the same time since the macro is designed to split RGB images in 3 single channels. Finally, the Intravital_Microscopy_Toolbox has not been conceived for 4D movies. However, we are currently working on a way to support a fourth channel and implement the processing of 4D movies with assignment of pseudo colors to channels in a 2.0 version that we intend to make freely available to the user as an update of the program.

Acknowledgments

We acknowledge P. Thévenaz, U.E. Ruttimann and M. Unser (École Polytechnique Fédérale de Lausanne, Switzerland) for the use of their StackReg and TurboReg plug-ins as scripts that are called from the macro to perform the x–y realignment. We also acknowledge Stephan Saalfeld and Christopher Philip Mauer for the use of the JAVASIFT and Kalman stack filter plug-ins, respectively. Finally, we acknowledge Tiago Ferreira for the modification of his distribution plot code in the macro. We thank Nadia Fortin and Nicolas Vallières for their invaluable technical assistance. We are grateful to Éric Lebel (Olympus Canada Inc.) for providing advices on the configuration of the two-photon system. J.C. is a trainee from École Nationale Supérieure de physique de Strasbourg (Télécom Physique Strasbourg), France.

Supporting Information