Abstract
This chapter describes the annotation of the medical image data that were used in the VISCERAL project. Annotation of regions in the 3D images is non-trivial, and tools need to be chosen to limit the manual work and have semi-automated annotation available. For this, several tools that were available free of charge or with limited costs were tested and compared. The GeoS tool was finally chosen for the annotation based on the detailed analysis, allowing for efficient and effective annotations. 3D slice was chosen for smaller structures with low contrast to complement the annotations. A detailed quality control was also installed, including an automatic tool that attributes organs to annotate and volumes to specific annotators, and then compares results. This allowed to judge the confidence in specific annotators and also to iteratively refine the annotation instructions to limit the subjectivity of the task as much as possible. For several structures, some subjectivity remains and this was measured via double annotations of the structure. This allows the judgement of the quality of automatic segmentations.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Bitter I, Van Uitert R, Wolf I, Ibanez L, Kuhnigk JM (2007) Comparison of four freely available frameworks for image processing and visualization that use ITK. IEEE Trans Vis Comput Graph 13(3):483–493
Caban JJ, Joshi A, Nagy P (2007) Rapid development of medical imaging tools with open-source libraries. J Digit Imaging 20(1):83–93
Chapman BE, Wong M, Farcas C, Reynolds P (2012) Annio: a web-based tool for annotating medical images with ontologies. In: 2012 IEEE second international conference on healthcare informatics, imaging and systems biology. IEEE, New Jersey, p 147
Criminisi A, Sharp T, Blake A (2008) GeoS: geodesic image segmentation. In: Forsyth D, Torr P, Zisserman A (eds) ECCV 2008. LNCS, vol 5302. Springer, Heidelberg, pp 99–112. doi:10.1007/978-3-540-88682-2_9
Dice LR (1945) Measures of the amount of ecologic association between species. Ecology 26(3):297–302
Doi K (2005) Current status and future potential of computer-aided diagnosis in medical imaging. Br J Radiol 78:3–19
Engle RL (1992) Attempts to use computers as diagnostic aids in medical decision making: a thirty-year experience. Perspect Biol Med 35(2):207–219
Hanbury A, Müller H, Langs G, Weber MA, Menze BH, Fernandez TS (2012) Bringing the algorithms to the data: cloud-based benchmarking for medical image analysis. In: Catarci T, Forner P, Hiemstra D, Peñas A, Santucci G (eds) CLEF 2012. LNCS, vol 7488. Springer, Heidelberg, pp 24–29. doi:10.1007/978-3-642-33247-0_3
Langs G, Hanbury A, Menze B, Müller H (2013) VISCERAL: towards large data in medical imaging — challenges and directions. In: Greenspan H, Müller H, Syeda-Mahmood T (eds) MCBR-CDS 2012. LNCS, vol 7723. Springer, Heidelberg, pp 92–98. doi:10.1007/978-3-642-36678-9_9
Mata C, Oliver A, Torrent A, Marti J (2012) Mammoapplet: an interactive Java applet tool for manual annotation in medical imaging. In: 2012 IEEE 12th international conference on bioinformatics and bioengineering (BIBE). IEEE, New Jersey, pp 34–39
Pieper S, Halle M, Kikinis R (2004) 3D slicer. In: IEEE international symposium on biomedical imaging: nano to macro. IEEE, New Jersey, pp 632–635
Sharma N, Aggarwal LM (2010) Automated medical image segmentation techniques. J Med Phys Assoc Med Phys India 35(1):3
Jiménez-del Toro OA, Müller H, Krenn M, Gruenberg K, Taha AA, Winterstein M, Eggel I, Foncubierta-Rodríguez A, Goksel O, Jakab A, Kontokotsios G, Langs G, Menze B, Salas Fernandez T, Schaer R, Walleyo A, Weber MA, Dicente Cid Y, Gass T, Heinrich M, Jia F, Kahl F, Kechichian R, Mai D, Spanier AB, Vincent G, Wang C, Wyeth D, Hanbury A (2016) Cloud-based evaluation of anatomical structure segmentation and landmark detection algorithms: visceral anatomy benchmarks. IEEE Trans Med Imaging 35(11):2459–2475
Yushkevich PA, Piven J, Hazlett HC, Smith RG, Ho S, Gee JC, Gerig G (2006) User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 31(3):1116–1128
Zou KH, Warfield SK, Bharatha A, Tempany CM, Kaus MR, Haker SJ, Wells WM III, Jolesz FA, Kikinis R (2004) Statistical validation of image segmentation quality based on a spatial overlap index\(^1\): scientific reports. Acad Radiol 11(2):178–189
Acknowledgements
The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007–2013) under grant agreement 318068 (VISCERAL).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is distributed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (http://creativecommons.org/licenses/by-nc/2.5/), which permits any noncommercial use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. The images or other third party material in this book are included in the work’s Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work’s Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.
Copyright information
© 2017 The Author(s)
About this chapter
Cite this chapter
Grünberg, K. et al. (2017). Annotating Medical Image Data. In: Hanbury, A., Müller, H., Langs, G. (eds) Cloud-Based Benchmarking of Medical Image Analysis. Springer, Cham. https://doi.org/10.1007/978-3-319-49644-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-49644-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-49642-9
Online ISBN: 978-3-319-49644-3
eBook Packages: Computer ScienceComputer Science (R0)