Abstract
In vivo confocal microscopy is a technique that allows to acquire images of the corneal layers in a rapid and noninvasive way. Analysis of sub-basal nerve allows obtaining important clinical information regarding the eye and the human body’s health. To obtain that information, it is necessary to correctly identify and trace the nerve fibers. Manual analysis is time-consuming and subjective. Numerous automatic algorithms have been proposed to overcome these problems, but none have been included in clinical practice yet. In this work, we take advantage of deep learning techniques. We present an analysis of the performances obtained through UNet (baseline) to which various architectural solutions have been added to boost performance. The variation of the tracing results is also analyzed according to the use of different loss functions, one of which is introduced here: It considers a tolerance margin (Dice with tolerance). The investigated configurations have been shown to be capable of improving the tracing of corneal nerve fibers. The model with attention modules and atrous-spatial pyramid pooling modules showed the greatest improvement compared to the baseline, increasing in the evaluation score from 86.51 to 90.21%. Furthermore, the proposed loss function further increases the results (achieving 92.44%).
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
The introduction of in vivo confocal microscopy (IVCM), as a technique for acquiring corneal images, has revolutionized the study and analysis of the structures of this tissue [1, 2]. In particular, thanks to this tool, it is possible to quickly and noninvasively acquire images of the sub-basal plexus (specific layer within the epithelium), allowing the visualization of the corneal nerve fibers.
As the cornea is the most innervated tissue in the human body [3], the analysis by confocal microscopy has made it possible to greatly expand the knowledge related to corneal structures and has allowed us to deepen the relationship between the characteristics of the corneal nerves and some pathologies, both ocular (dry eye, keratoconus, herpes keratitis) and systemic (diabetes, etc.) [4,5,6,7,8,9].
Moreover, several investigators have examined these structures and verified that they provide important clinical information related to age, prolonged use of contact lenses, surgery (such as LASIK or PRK), and transplantation [9,10,11,12].
However, quantitative analysis of corneal nerve fibers remains complex for daily clinical practice due to the execution time required and the difficulty of manual or semi-automatic analysis.
Additionally, manual analysis varies according to the clinician who performs the analysis and according to his or her experience. This subjectivity in manual analysis means that the clinical parameters, derived from the tracing, are also subjective and prone to errors. In the literature, some studies have been presented for the automatic analysis of images of the sub-basal plexus and, in particular, for the tracing of the nerve fibers. To the best of our knowledge, none of them have been included in the diagnostic process yet.
Scarpa et al. [13] proposed a method based on improving contrast through Gabor filters, followed by a threshold and clustering according to the fuzzy c-means technique. Subsequently, Dabbah et al. [14] presented an extension of some previous enhancement techniques, promoting a multi-scale approach, followed by an artificial neural network to classify each pixel as a foreground or background. On the basis of the last, Chen et al. [15] figured out the best configuration to obtain nerve segmentation. Poletti and Ruggeri [16] proposed a new approach based on a sparse tracking scheme. Annunziata et al. [17], presented a hybrid segmentation method, which combines an appearance model, based on a scale and curvature-invariant ridge detector (SCIRD), with a context model, including multi-range learned context filters. Al-Fahdawi et al. [18] proposed a three-step segmentation model. These steps included anisotropic diffusion filtering (to enhance nerves and remove noise), morphological operations (to remove unwanted objects, such as epithelial cells and small nerve segments), and edge detection (to detect all nerves in the input image). Afterward, Guimarães et al. [19] proposed another automatic method that processed the images with a filter bank (which includes the log-Gabor filter). Hysteresis thresholding was used to isolate candidate nerve pixels, and each pixel was then classified through a support vector machine.
One of the major problems related to the methods presented is their execution time (up to hundreds of seconds for a single image), followed by a not always excellent correspondence with manual analysis. In Dehghani et al. [20], the ability of automatic, semi-automatic, and manual methods was analyzed to detect the decrease in the length of the corneal nerves in patients with diabetes. They demonstrated that all three methods could distinguish healthy subjects from diabetic ones, but the manual method was found to distinguish subjects better, especially in low-contrast images.
Thanks to the new deep learning techniques, it is possible to significantly reduce analysis times and improve the correspondence with manual analysis, compared to previous algorithms. In our previous work [21], this technique was used for the first time for the analysis of corneal nerves, implementing a convolutional neural network (CNN), based on a U-shaped architecture. Zhang et al. [22] also used a UNet to segment nerve fibers, but they first implemented pre-processing steps to obtain image normalization. Mehrgardt et al. [23] presented a new multi-step approach, called UNet segmented adjacent angle detection (USAAD), for nerve fiber segmentation (also based on UNet) and automatic tortuosity estimation.
In this work, we continued our previous investigation (presented at the MICCAI 2018 conference [21]) on the corneal nerve fibers segmentation, based on a convolutional neural network. In the related article, as mentioned above, we mainly focused on the classical UNet encoder-decoder architecture [24] and demonstrate the ability of that network to individuate and trace the corneal nerves in IVCM images. In this work, we investigate whether some architectural convolutional neural network blocks added to the simple UNet could improve the performance. Moreover, we investigate how the use of different loss functions will influence the results, and we introduce a new loss function, which aims to consider a margin of tolerance in the tracing of nerve fibers.
2 Materials and methods
2.1 Materials
In this investigation, we used the same dataset presented by Colonna et al. [21] for training all networks. It consists of 8909 confocal images of the sub-basal nerve plexus from healthy and pathological (with type 1 or 2 diabetes) subjects. Each image covers a field of 400 \(\times \) 400 \(\upmu \)m (384 \(\times \) 384 pixels) and was acquired using the Heidelberg retina tomograph (HRT-II) with the Rostock cornea module (Heidelberg Engineering GmbH, Heidelberg, Germany). The acquisition was carried out at different clinical centers, and all data have been anonymized. Due to the difficulty and execution time required for manual analysis of such a large dataset, we decided to obtain the labeled images for the training by using the algorithm proposed by Guimarães et al. [25]
For the testing phase, we used a dataset composed of 90 images, acquired with the same tool, from healthy and pathological subjects. Each image was manually analyzed using the NeuronJ [26] tracing plug-in for ImageJ (the software is available in the public domain at http://imagescience.org/meijering/software/neuronj/).
2.2 Convolutional neural network modules
As a baseline for this work, we chose the UNet [24], which has proven to be an optimal framework for semantic segmentation of biomedical images in general and IVCM images in particular. It is composed of the encoding and the corresponding decoding units, with a four-layer depth. The architecture has been designed in such a way as to have the input and output size equal to the original image dimensions (384 \(\times \) 384 pixels).
A brief description of the coding and decoding blocks is now reported. The encoding path is made up of four-layer depth, each, in turn, made up of 2 convolutional blocks (consists of two convolutional layers to learn feature maps, followed by batch normalization to increase the CNN stability, and a ReLU as a non-linear activation function). Each convolutional block is followed by a max-pooling operation, obtained using a generic 2 \(\times \) 2 window, allowing to down-sample the feature maps by half of their original xy size.
After the encoding path, there is a block called the ‘bridge block,’ composed of two convolutional blocks and followed by the start of the decoding part. The decoding path is made up of four-layer groups. The transposed convolution allows us to double the xy dimensions of the features maps received in input (therefore, acts as the opposite operation of the max-pooling in the encoding part); the output of the transposed convolution is concatenated with the corresponding features maps in the encoding path. The concatenated features are subjected to two convolutional blocks (as the ones described above in the encoding part) and then again to a transposed convolution. The structure is repeated in the same way until it reaches xy dimensions equal to the input dimensions. Finally, the output of the decoding path is subjected to a 1 \(\times \) 1 convolution with sigmoid as activation function, obtaining a probability of ‘how likely it is, for each pixel, to be part of a nerve fiber.’
In addition to the architecture of the UNet described above, we investigated the ability of some architectures, which use the same UNet as a baseline, to increase performance. We modify the UNet by adding residual connections, the atruos-spatial pyramid pooling (ASPP) module, and the attention modules.
2.2.1 Residual connections
Several studies have added residual connections to classic CNN architectures [27,28,29,30]. This insertion has been shown to fight the vanishing of the gradient and the degradation of accuracy [27], also increasing the performance of the network [28,29,30]. As shown in Fig. 1, the residual connection was introduced between the input and the output of each convolutional block (both in the encoding and decoding path). Since in each convolutional block the number of features changes (respectively, doubles in the encoding part and halves in the decoding phase), in each residual connection a 1 \(\times \) 1 convolution has been added to adjust the number of features.
2.2.2 Atrous-spatial pyramid pooling module
The atrous-spatial pyramid pooling (ASPP) module is used to obtain information on the multi-scale context, through the use of parallel convolutions with different dilation rates. As proposed by Chen et al [31], we decided to implement this module as shown in Fig. 2. Four parallel branches are created: in the first three branches, atrous convolutions are carried out with a different dilation rate, while, in the last branch, an image-level feature is extracted, through a global average pooling. After applying all the operations in parallel, the outputs are concatenated, and a 1 \(\times \) 1 convolution is applied to obtain the overall output of the module.
2.2.3 Attention module
The first time, the attention module was proposed for the classification task [32]. In particular, the purpose was to perform some operations that would allow analyzing with greater attention those regions of the image that most affect the prediction of the model.
Subsequently, this idea was adapted and generalized to improve performance on the segmentation task [33, 34], amplifying relevant spatial information and reducing the weight of background features.
This module, schematized in Fig. 3, was developed as described by Schlemper et al [34]. The attention module is inserted within each of the skipped connections deriving from the encoding path. The module takes as input the skipped connection and the output of the corresponding transposed convolution of the decoding branch, while the output is concatenated with the up-sampled data.
2.3 Loss function
The learning process of a deep learning algorithm is strongly reliant on the loss/objective function chosen during the design of the architecture. For quick and accurately, the loss function must be able to mathematically represent the target, even in its borderline cases. In this work, we choose to compare the performance of four widely used loss functions in semantic segmentation. We also proposed a new tolerance Dice loss function and showcased its efficiency.
2.3.1 Balanced binary cross-entropy loss
Balanced binary cross-entropy is a variant of the well-known cross-entropy, which is defined as the difference between two probability distributions for a given random variable [35]. Cross-entropy is widely used in the field of classification and semantic segmentation via deep learning, but its balanced variant proves to be more effective when it comes to working with unbalanced data (as in the case of corneal nerves where pixels belonging to the structures are very lower than the ones corresponding to the background). The balanced binary cross-entropy allows giving weight to both positive (\(\beta \)) and negative (1-\(\beta \)) examples, and is defined as follows:
where y represents the true value, and \(\hat{y}\) the predicted value. In this work, \(\beta \) is derived from the frequency of the True value in the image. The limit of the cross-entropy (and therefore also of its variants) is to calculate the loss as the average of the loss per pixel, without considering the adjacent pixels and therefore without considering the continuity of the object to segment.
2.3.2 Dice loss
Dice loss is another widely used loss function in the semantic segmentation task [36, 37]. It is based on the coefficient of the same name [38], which calculates the similarity between two images considering the overlap between the two samples. The Dice coefficient is defined as
where \(\Vert Y\Vert \) (respectively, \(\Vert \hat{Y}\Vert \)) represents the number of pixels of object (nerve fibers in our case) in the True set Y (respectively, predicted set \(\hat{Y}\)), and \(\Vert Y \cap \ \hat{Y}\Vert \) represents the common pixels between the two datasets. We also report an equivalent definition of the coefficient: In this case, it is highlighted how false positives (FP) and false negatives (FN) are weighed equally, and consequently precision and recall are also weighed in the same way. The Dice loss function is derived from this coefficient and it is calculated as
2.3.3 Tversky loss
The Tversky loss function [39] is derived from the homonymous coefficient [40]. The Tversky coefficient is considered as a generalization of the Dice coefficient descripted above, and allows balancing FP and FN. The coefficient is defined as follows:
The tuning of \(\alpha \) and \(\beta \) can put more emphasis on FPs or FNs, respectively (e.g., increasing the \(\beta \) value, increases the emphasis associated with FNs and leads to trying to decrease them during the training process, thus increasing recall). As for the Dice loss, also in this casa the Tversky loss function is derived from the coefficient as
2.3.4 Focal Tversky loss
The focal Tversky Loss [41] is a variant of the Tversky loss described above: in this case, a focal parameter \(\gamma \) is applied.This parameter attempts to learn hard examples (i.e., examples with small regions of interest). The focal Tversky loss is defined as
A value of \(\gamma <1\) increases the focus on learning hard examples, while a value of \(\gamma =1\) simplifies the loss function to the Tversky Loss.
2.3.5 Dice with tolerance loss
In the semantic segmentation of corneal nerves, interest is mainly related to the recognition of the central line that identifies the nerve fiber. In tracing this line, it is correct to set a tolerance margin within which a nerve fiber can be recognized (i.e., a pixel recognized to be a nerve will be accepted as TP if it was within the margin of tolerance from the reference nerve). Since the purpose of the study of the corneal nerves is to recognize and segment the nerve fibers present in the images, trying to maximize both precision and sensitivity, we decided to adapt the Dice loss to our task: To obtain the Dice with tolerance coefficient, we calculated TP, FP and FN considering the tolerance margin. The loss is calculated as follows:
where \(\widetilde{TP}\), \(\widetilde{FP}\), and \(\widetilde{FN}\) indicate the corresponding indices calculated with tolerance.
3 Experiments
As mentioned in previous sections, we conducted experiments on several models applying several loss functions. The main aspect of our work was to investigate the performance of each architecture and the loss function, in the tracing of corneal nerve fibers. Starting from the modules described above (in paragraph ‘Convolutional neural network modules’) and using UNet as the baseline for each architecture, we decided to investigate the following convolutional neural networks:
-
Simple UNet (the one used as baseline).
-
UNet with attention modules (AM-UNet).
-
UNet with residual connections and attention modules (AM-ResUnet).
-
UNet with the ASPP module (ASPP-UNet).
-
UNet with attention modules and the ASPP module (AM-ASPP-UNet).
We trained each model for 60 epochs, considering each time one of the 5 loss functions presented above.
4 Results
As at the beginning of the previous section, models were trained on 8909 images, which were labeled using the algorithm proposed by Guimarães et al. [25], and tested on 90 images analyzed manually.
For each model trained, we have analyzed the goodness of its performance taking into account two indices:
-
The true positive rate - TPR, (or recall) which corresponds to the proportion of nerves correctly identified by the proposed algorithm (the higher, the better) and which is calculated as: \(\widetilde{TPR}=\frac{\ \widetilde{TP}}{\ \widetilde{TP}+\widetilde{FN}}\)
-
The false discovery rate - FDR (or 1-precision) which corresponds to the proportion of nerves mistakenly identified as such (the lower, the better), calculated as: \(\widetilde{FDR}=\frac{\widetilde{FP}}{\ \widetilde{TP}+\widetilde{FP}}\)
Both indices were calculated considering a tolerance margin of 3 pixels (i.e., a pixel classified by the algorithm as a nerve, will be considered a TP if it is found within an area of 3 pixels thick with respect to the reference nerve). The obtained values of TPR and FDR for each model and each loss function tested are shown in Tables 1 and 2.
5 Discussion
Taking the Unet (trained with binary cross-entropy loss function) as a baseline for comparison, it can be seen how the performances improve both by modifying the structure and by using some different losses.
In particular, it should be noted that the introduction of a module (or combinations of them) brings an improvement in the evaluation indices. Only in one case (ASPP-UNet), the TPR decreases, but it is important to observe the significant decrease also in the FDR. Moreover, after training with cost functions other than binary cross-entropy, we observe (especially for Dice, Tversky, and focal Tversky) a decrease in the TPR values and, at the same time, a significant decrease in the FDR index. In the case of the proposed loss function, the TPR values obtained are very similar to the values obtained with the binary cross-entropy, while the FDR values are considerably reduced (up to 50% in the case of the UNet).
Since it is not easy to assess whether simultaneous variation in TPR and FDR leads to a better result, it was decided to introduce an additional performance evaluation index. Starting from the two indices, the value of the dice with tolerance index is obtained as:
This index tends to 1 if \(\widetilde{TPR}\) is 1 and \(\widetilde{FDR}\) is zero (case of perfect correspondence between manual and automatic analysis); it gradually decreases with the decrease of \(\widetilde{TPR}\) and the increase of \(\widetilde{FDR}\) up to 0 (case in which manual and automatic analysis are complementary).
The values of this index for each network tested are shown in Table 3. It can be observed that the decrease in TPR mentioned previously is actually justified by an evident decrease in the FDR values, as it leads to a higher \(DICE_{tol}\) value. To further compare the results obtained, boxplots have been created. From the latter, it is possible to observe the variability of the indices between single images: These results are shown in Figs. 4 and 5. From the box plots representing the TPR values, it can be observed that the loss function proposed presents, for almost every model, a reduced variability compared to the other losses tested. Furthermore, the TPR values of single images never drop below 60%. Regarding the FDR values, it is evident that in the case of training through Tversky loss and focal Tversky loss, there is a decrease in both the values and the variability with respect to the baseline, this is linked to the choice of values of \(\alpha \) \(\beta \) and \(\gamma \). By observing the values relating to the loss proposed, it can be seen that the inclusion of the ASPP module considerably reduces the variability of the index.
The reduction of FDR is linked to the reduction of FP, and this is evident in the more complex images where the nerve fibers are very thin, and with relatively low contrast, the Unet, considered as a baseline, struggles to give continuity in the segmentation. With the introduction of the ASPP module, better results are obtained: the pixels classified as FP are reduced. Figure 6 shows the result obtained through 2 analyzed models: the baseline and the one chosen as the best model (higher Dice with tolerance score). The image shows thin nerve fibers, poor contrast in some areas, and the presence of dendritic cells. The baseline appears to mistake dendritic cells for nerve fibers and, at the same time, makes it difficult to trace nerve fibers that appear thinner and with low contrast. On the other hand, the UNet with Attention modules and Atrous-Spatial Pyramid Pooling module recognizes the low-contrast fibers and discards the luminous patterns that belong to the dendritic cells. Moreover, it seems to recognize some fibers that have not been traced manually, such as the one in the lower part of the image (traced in red since it corresponds to an FP in the analysis).
6 Conclusion
In vivo confocal microscopy is a technique that allows the acquisition of images of the corneal layers rapidly and noninvasively. The acquisition of images of the corneal sub-basal plexus allows the analysis of the nerve fibers present in it, which are strictly correlated to the presence of ocular or systemic diseases. It is important to note that all clinical parameters (useful for the diagnostic process) depend on identifying and tracing the nerve fibers.
The analysis of IVCM images in clinical practice is complicated: The manual tracing phase is very time-consuming, and, to the best of our knowledge, there is still no universally accepted automatic technique for performing it.
In recent years, thanks to new deep learning techniques, better results have been obtained in the field of tracing structures present in images: With an adequate dataset and appropriate training process, it is possible to obtain excellent results. In this paper, we presented an extension and improvement of our previous work on tracing the corneal nerve fibers in IVCM images of the sub-basal plexus. We investigated the ability to improve tracing through architectural improvements to the baseline model (UNet). We improved the architecture by adding residual connections, Atrous-Spatial Pyramid Pooling (ASPP) module, and attention modules (AM).
To boost the prediction performance, we also investigated four different loss functions and proposed a new tolerance Dice loss function. We trained all the architecture (with all five loss functions) using the automatic nerve tracings obtained by Guimarães et al. [25], which may present errors (missing or misclassifying nerves). To evaluate the performance of all the models and loss functions under investigation, the true positive rate (TPR) and false discovery rate (FDR) with a tolerance margin were calculated. Furthermore, to better compare the performances, starting from these last two indices, a third was obtained: the Dice with tolerance index (\(DICE_{tol}\)). In almost all loss functions cases, the introduction of new structures had outperformed the UNet (baseline architecture): This is clear looking at the \(DICE_{tol}\) index table.
Looking at the same architecture and examining the results obtained using a different loss function during training, the proposed loss function presents a higher Dice with tolerance score in almost all cases. Even the TPR index is almost always the best, while the FDR only improves with respect to the binary cross-entropy case. Furthermore, the use of the proposed loss function makes the continuity of the nerve fibers even more evident, giving a better result.
For future work, it will be interesting to analyze the data used during the training phase, keeping only those images that present results that are acceptable from a clinical point of view (reducing the error in the ground truth dataset, the performance will improve). Selecting images already analyzed automatically or improving an automatic analysis is less time-consuming than completely manual analysis.
Data availability
Data and code are not available.
References
Villani, E., Baudouin, C., Efron, N., Hamrah, P., Kojima, T., Patel, S.V., Pflugfelder, S.C., Zhivov, A., Dogru, M.: In vivo confocal microscopy of the ocular surface: from bench to bedside. Curr. Eye Res. 39, 213 (2014). https://doi.org/10.3109/02713683.2013.842592
Patel, D.V., McGhee, C.N.: Quantitative analysis of in vivo confocal microscopy images: a review. Surv. Ophthalmol. 58, 466–475 (2013). https://doi.org/10.1016/J.SURVOPHTHAL.2012.12.003
Marfurt, C.F., Cox, J., Deek, S., Dvorscak, L.: Anatomy of the human corneal innervation. Exp. Eye Res. 90, 478–492 (2010). https://doi.org/10.1016/J.EXER.2009.12.010
Flockerzi, E., Daas, L., Seitz, B.: Structural changes in the corneal subbasal nerve plexus in keratoconus. Acta Ophthalmol. 98(8), 928–932 (2020). https://doi.org/10.1111/aos.14432
Deák, E.A., Szalai, E., Tóth, N., Malik, R.A., Berta, A., Csutak, A.: Longitudinal changes in corneal cell and nerve fiber morphology in young patients with type 1 diabetes with and without diabetic retinopathy: A 2-year follow-up study. Investig. Ophthalmol. Vis. Sci. 60, 830–837 (2019). https://doi.org/10.1167/IOVS.18-24516
Cillà, S.D., Ranno, S., Carini, E., Fogagnolo, P., Ceresara, G., Orzalesi, N., Rossetti, L.M.: Corneal subbasal nerves changes in patients with diabetic retinopathy: an in vivo confocal study. Investig. Ophthalmol. Vis. Sci. 50, 5155–5158 (2009)
Badian, R.A., Allgeier, S., Scarpa, F., Andréasson, M., Bartschat, A., Mikut, R., Colonna, A., Bellisario, M., Utheim, T.P., Köhler, B., Svenningsson, P., Lagali, N.: Wide-field mosaics of the corneal subbasal nerve plexus in parkinson’s disease using in vivo confocal microscopy. Sci. Data 8, 1–10 (2021). https://doi.org/10.1038/s41597-021-01087-3
Andréasson, M., Lagali, N., Badian, R.A., Utheim, T.P., Scarpa, F., Colonna, A., Allgeier, S., Bartschat, A., Köhler, B., Mikut, R., Reichert, K.M., Solders, G., Samuelsson, K., Zetterberg, H., Blennow, K., Svenningsson, P.: Parkinson’s disease with restless legs syndrome-an in vivo corneal confocal microscopy study. npj Parkinson’s Dis. 7, 1–8 (2021). https://doi.org/10.1038/s41531-020-00148-5
Cruzat, A., Qazi, Y., Hamrah, P.: In vivo confocal microscopy of corneal nerves in health and disease hhs public access. Ocul. Surf. 15, 15–47 (2017). https://doi.org/10.1016/j.jtos.2016.09.004
Giannaccare, G., Bernabei, F., Pellegrini, M., Guaraldi, F., Turchi, F., Torrazza, C., Senni, C., Scotto, R., Sindaco, D., Cello, L.D., Versura, P., Scorcia, V., Traverso, C.E., Vagge, A.: Bilateral morphometric analysis of corneal sub-basal nerve plexus in patients undergoing unilateral cataract surgery: a preliminary in vivo confocal microscopy study. Brit. J. Ophthalmol. (2020). https://doi.org/10.1136/bjophthalmol-2019-315449
Bitirgen, G., Ozkagnici, A., Malik, R.A., Oltulu, R.: Evaluation of contact lens-induced changes in keratoconic corneas using in vivo confocal microscopy. Investig. Ophthalmol. Vis. Sci. 54, 5385–5391 (2013). https://doi.org/10.1167/IOVS.13-12437
Theophanous, C., Jacobs, D.S., Hamrah, P.: Corneal neuralgia after lasik. Optom. Vis. Sci. 92, 233–240 (2015). https://doi.org/10.1097/OPX.0000000000000652
Scarpa, F., Grisan, E., Ruggeri, A.: Automatic recognition of corneal nerve structures in images from confocal microscopy. Investig. Ophthalmol. Vis. Sci. 49, 4801–4807 (2008). https://doi.org/10.1167/IOVS.08-2061
Dabbah, M.A., Graham, J., Petropoulos, I.N., Tavakoli, M., Malik, R.A.: Automatic analysis of diabetic peripheral neuropathy using multi-scale quantitative morphology of nerve fibres in corneal confocal microscopy imaging. Med. Image Anal. 15, 738–747 (2011). https://doi.org/10.1016/J.MEDIA.2011.05.016
Chen, X., Graham, J., Dabbah, M.A., Petropoulos, I.N., Tavakoli, M., Malik, R.A.: An automatic tool for quantification of nerve fibers in corneal confocal microscopy images. IEEE Trans. Biomed. Eng. 64, 786–794 (2017). https://doi.org/10.1109/TBME.2016.2573642
Poletti, E., Ruggeri, A.: Automatic nerve tracking in confocal images of corneal subbasal epithelium. In: Proceedings of CBMS 2013—26th IEEE International Symposium on Computer-Based Medical Systems, pp. 119–124 (2013). https://doi.org/10.1109/CBMS.2013.6627775
Annunziata, R., Kheirkhah, A., Aggarwal, S., Hamrah, P., Trucco, E.: A fully automated tortuosity quantification system with application to corneal nerve fibres in confocal microscopy images. Med. Image Anal. 32, 216–232 (2016). https://doi.org/10.1016/j.media.2016.04.006
Al-Fahdawi, S., Qahwaji, R., Al-Waisy, A.S., Ipson, S., Malik, R.A., Brahma, A., Chen, X.: A fully automatic nerve segmentation and morphometric parameter quantification system for early diagnosis of diabetic neuropathy in corneal images. Comput. Methods Programs Biomed. 135, 151–166 (2016). https://doi.org/10.1016/J.CMPB.2016.07.032
Guimarães, P., Wigdahl, J., Ruggeri, A.: Automatic estimation of corneal nerves focused tortuosities. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2016-October, pp. 1332–1335 (2016). https://doi.org/10.1109/EMBC.2016.7590953
Dehghani, C., Pritchard, N., Edwards, K., Russell, A.W., Malik, R.A., Efron, N.: Fully automated, semiautomated, and manual morphometric analysis of corneal subbasal nerve plexus in individuals with and without diabetes. Cornea 33, 696–702 (2014). https://doi.org/10.1097/ICO.0000000000000152
Colonna, A., Scarpa, F., Ruggeri, A.: Segmentation of corneal nerves using a u-net-based convolutional neural network. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 11039 LNCS, pp. 185–192 (2018). https://doi.org/10.1007/978-3-030-00949-6_22
Zhang, S., Yan, H., Teng, J., Sheng, D.: A mathematical model of tortuosity in soil considering particle arrangement. Vadose Zone J. 19, 20004 (2020). https://doi.org/10.1002/VZJ2.20004
Mehrgardt, P., Zandavi, S.M., Poon, S.K., Kim, J., Markoulli, M., Khushi, M.: U-net segmented adjacent angle detection (usaad) for automatic analysis of corneal nerve structures. Data (2020). https://doi.org/10.3390/data5020037
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9351, pp. 234–241 (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Guimarães, P., Wigdahl, J., Ruggeri, A.: A fast and efficient technique for the automatic tracing of corneal nerves in confocal microscopy. Transl. Vis. Sci. Technol. (2016). https://doi.org/10.1167/tvst.5.5.7
Meijering, E., Jacob, M., Sarria, J.C.F., Steiner, P., Hirling, H., Unser, M.: Design and validation of a tool for neurite tracing and analysis in fluorescence microscopy images. Cytom. A 58, 167–176 (2004). https://doi.org/10.1002/CYTO.A.20022
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Xiao, X., Lian, S., Luo, Z., Li, S.: Weighted res-unet for high-quality retina vessel segmentation; weighted res-unet for high-quality retina vessel segmentation. In: 2018 9th International Conference on Information Technology in Medicine and Education (ITME) (2018). https://doi.org/10.1109/ITME.2018.00080
Chu, Z., Tian, T., Feng, R., Wang, L.: Sea-land segmentation with res-unet and fully connected crf; sea-land segmentation with res-unet and fully connected crf. In: IGARSS 2019–2019 IEEE International Geoscience and Remote Sensing Symposium (2019)
Wei, S., Shi, F., Wang, Y., Chou, Y., Li, X.: A deep learning model for automated sub-basal corneal nerve segmentation and evaluation using in vivo confocal microscopy. Transl. Vis. Sci. Technol. 9, 32–32 (2020). https://doi.org/10.1167/tvst.9.2.32
Chen, L.-C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. Computer Vision and Pattern Recognition (2017)
Jetley, S., Lord, N.A., Lee, N., Torr, P.H.S.: Learn to pay attention. In: International Conference on Learning Representations (ICLR) (2018)
Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., Mcdonagh, S., Hammerla, N.Y., Kainz, B., Glocker, B., Rueckert, D.: Attention u-net: learning where to look for the pancreas. Medical Imaging with Deep Learning (2018)
Schlemper, J., Oktay, O., Schaap, M., Heinrich, M., Kainz, B., Glocker, B., Rueckert, D.: Attention gated networks: learning to leverage salient regions in medical images. Med. Image Anal. 53, 197–207 (2019). https://doi.org/10.1016/J.MEDIA.2019.01.012
Jadon, S.: A survey of loss functions for semantic segmentation. In: 2020 IEEE Conference on Computational Intelligence and Bioinformatics and Computational Biology (CIBCB), pp. 1–7 (2020). https://doi.org/10.1109/CIBCB48159.2020.9277638
Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Cardoso, M.J.: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 10553 LNCS, pp. 240–248 (2017). https://doi.org/10.1007/978-3-319-67558-9_28
Soomro, T.A., Afifi, A.J., Gao, J., Hellwich, O., Paul, M., Zheng, L.: Strided u-net model: Retinal vessels segmentation using dice loss; strided u-net model: Retinal vessels segmentation using dice loss. In: 2018 Digital Image Computing: Techniques and Applications (DICTA) (2018)
Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26, 297–302 (1945). https://doi.org/10.2307/1932409
Sadegh, S., Salehi, M., Erdogmus, D., Gholipour, A., Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. Machine Learning in Medical Imaging. MLMI 2017. Lecture Notes in Computer Science 10541 LNCS, pp. 379–387 (2017). https://doi.org/10.1007/978-3-319-67389-9_44
Tversky, A.: Features of similarity. Psychol. Rev. 84, 327–352 (1977). https://doi.org/10.1037/0033-295X.84.4.327
Abraham, N., Khan, N.M.: A novel focal tversky loss function with improved attention u-net for lesion segmentation. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 683–687 (2019). https://doi.org/10.1109/ISBI.2019.8759329
Funding
Open access funding provided by Universitá degli Studi di Padova. No funding was received for this work.
Author information
Authors and Affiliations
Contributions
AC designed and performed the study. FS supervised the work. AC wrote the main manuscript text and prepared all the figures. All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
There are not Conflict of interest/Conflict of interest to disclose
Ethical approval
As the acquisition of these images was approved by the respective local ethical review committees, occurred with informed consent, and followed the tenets of the Declaration of Helsinki, no specific further ethical approval was sought for the analysis of the resulting compilation of images.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Colonna, A., Scarpa, F. Improving corneal nerve segmentation using tolerance Dice loss function. SIViP 18, 1069–1077 (2024). https://doi.org/10.1007/s11760-023-02790-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-023-02790-x