Abstract
This chapter describes the metrics for the evaluation of information retrieval and natural language processing systems, the annotation techniques and evaluation metrics and the concepts of training, development and evaluations sets for information retrieval systems.
Chapter PDF
References
Artstein, R., & Poesio, M. (2008). Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4), 555–596.
Cleverdon, C. (1967). The Cranfield tests on index language devices. In Aslib Proceedings (pp. 173–194). MCB UP Ltd.
Hripcsak, G., & Rothschild, A. S. (2005). Agreement, the F-measure, and reliability in information retrievas. Journal of the American Medical Informatics Association, 12(3), 296–298.
Japkowicz, N., & Shah, M. (2011). Evaluating Learning Algorithms: A Classification Perspective. Cambridge: Cambridge University Press.
Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In International Joint Conference on Artificial Intelligence (IJCAI) (pp. 1137–1145).
Neves, M., & Leser, U. (2012). A survey on annotation tools for the biomedical literature. Briefings in Bioinformatics, 15(2), 327–340.
Pustejovsky, J., & Stubbs, A. (2012). Natural Language Annotation for Machine Learning. O’Reilly Media, Inc. Beijing.
Stenetorp, P., Pyysalo, S., Topić, G., Ohta, T., Ananiadou, S., & Tsujii, J. (2012). BRAT: A web-based tool for NLP-assisted text annotation. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics (pp. 102–107). Association for Computational Linguistics.
Van Rijsbergen, C. J. (1979). Information Retrieval. Butterworth & Co. http://www.dcs.glasgow.ac.uk/Keith/Preface.html. Accessed 11 Jan 2018.
Voorhees, E. M. (2001). The philosophy of information retrieval evaluation. In Evaluation of Cross-Language Information Retrieval Systems (pp. 355–370). Berlin: Springer.
Author information
Authors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. The images or other third party material in this book are included in the book's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2018 The Author(s)
About this chapter
Cite this chapter
Dalianis, H. (2018). Evaluation Metrics and Evaluation. In: Clinical Text Mining. Springer, Cham. https://doi.org/10.1007/978-3-319-78503-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-78503-5_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-78502-8
Online ISBN: 978-3-319-78503-5
eBook Packages: Computer ScienceComputer Science (R0)