Abstract
Quantum software—like classic software—needs to be designed, specified, developed, and, most importantly, tested by developers. Writing tests is a complex, error-prone, and time-consuming task. Due to the particular properties of quantum physics (e.g., superposition), quantum software is inherently more complex to develop and effectively test than classical software. Nevertheless, some preliminary works have tried to bring commonly used classical testing practices for quantum computing to assess and improve the quality of quantum programs. In this chapter, we first gather 16 quantum software testing techniques that have been proposed for the IBM quantum framework, Qiskit. Then, whenever possible, we illustrate the usage of each technique (through the proposed tool that implements it, if available) on a given running example. We showcase that although several works have been proposed to ease the burn of testing quantum software, we are still in the early stages of testing in the quantum world. Researchers should focus on delivering artifacts that are usable without much hindrance to the rest of the community, and the development of quantum benchmarks should be a priority to facilitate reproducibility, replicability, and comparison between different testing techniques.
Chapter PDF
References
Russell, J.: IBM Quantum Update: Q System One Launch, New Collaborators, and QC Center Plans. HPC Wire
Collins, H., Nay, C.: IBM Unveils 400 Qubit-Plus Quantum Processor and Next-Generation IBM Quantum. IBM Newsroom
Ferreira, F.: An Exploratory Study on the Usage of Quantum Programming Languages. Available at http://hdl.handle.net/10451/56751
Hevia, J.L., Peterssen, G., Ebert, C., Piattini, M.: Quantum computing. IEEE Software 38(5), 7–15 (2021). https://doi.org/10.1109/MS.2021.3087755
Barrera, A., Guzmán, I., Polo, M., Piattini, M.: Quantum software testing: state of the art. J. Software Evol. Process 35(4), 2419 (2023). https://doi.org/10.1002/smr.2419
Weder, B., Barzen, J., Leymann, F., Salm, M., Vietz, D.: The Quantum Software Lifecycle. In: Proceedings of the 1st ACM SIGSOFT International Workshop on Architectures and Paradigms for Engineering Quantum Software. APEQS 2020, pp. 2–9. Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3412451.3428497
Arias, D., García Rodríguez de Guzmán, I., Rodríguez, M., Terres, E.B., Sanz, B., Gaviria de la Puerta, J., Pastor, I., Zubillaga, A., García Bringas, P.: Let’s do it right the first time: Survey on security concerns in the way to quantum software engineering. Neurocomputing 538, 126199 (2023). https://doi.org/10.1016/j.neucom.2023.03.060
Tao Yue, P.A., Ali, S.: Quantum Software Testing: Challenges, Early Achievements, and Opportunities. ERCIM News
Ying, M.: Floyd–Hoare logic for quantum programs. ACM Trans. Program. Lang. Syst. 33(6) (2012). https://doi.org/10.1145/2049706.2049708
Zhou, L., Yu, N., Ying, M.: An Applied Quantum Hoare Logic. In: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. PLDI 2019, pp. 1149–1162. Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3314221.3314584
Honarvar, S., Mousavi, M.R., Nagarajan, R.: Property-Based Testing of Quantum Programs in q#. In: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops, pp. 430–435 (2020)
Zhao, P., Wu, X., Li, Z., Zhao, J.: QChecker: Detecting Bugs in Quantum Programs via Static Analysis (2023)
Yu, N., Palsberg, J.: Quantum Abstract Interpretation. In: Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, pp. 542–558. ACM, Virtual Canada (2021). https://doi.org/10.1145/3453483.3454061. https://dl.acm.org/doi/10.1145/3453483.3454061
Xia, S., Zhao, J.: Static Entanglement Analysis of Quantum Programs (2023). https://doi.org/10.48550/arXiv.2304.05049. arXiv:2304.05049 [quant-ph]
Kaul, M., Küchler, A., Banse, C.: A Uniform Representation of Classical and Quantum Source Code for Static Code Analysis (2023). https://doi.org/10.48550/arXiv.2308.06113. arXiv:2308.06113 [cs]
Wang, J., Gao, M., Jiang, Y., Lou, J., Gao, Y., Zhang, D., Sun, J.: QuanFuzz: Fuzz Testing of Quantum Program (2018). arXiv:1810.10310 [cs]
Wang, J., Zhang, Q., Xu, G.H., Kim, M.: QDiff: Differential Testing of Quantum Software Stacks. In: 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 692–704 (2021). https://doi.org/10.1109/ASE51524.2021.9678792
Wang, X., Yu, T., Arcaini, P., Yue, T., Ali, S.: Mutation-Based Test Generation for Quantum Programs with Multi-Objective Search. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1345–1353. ACM, Boston Massachusetts (2022). https://doi.org/10.1145/3512290.3528869. https://dl.acm.org/doi/10.1145/3512290.3528869
Abreu, R., Fernandes, J.P., Llana, L., Tavares, G.: Metamorphic Testing of Oracle Quantum Programs. In: Proceedings of the 3rd International Workshop on Quantum Software Engineering, pp. 16–23. ACM, Pittsburgh Pennsylvania (2022). https://doi.org/10.1145/3528230.3529189. https://dl.acm.org/doi/10.1145/3528230.3529189
Paltenghi, M., Pradel, M.: MorphQ: Metamorphic Testing of Quantum Computing Platforms (2022). https://doi.org/10.48550/arXiv.2206.01111. arXiv:2206.01111 [cs]
Wang, X., Arcaini, P., Yue, T., Ali, S.: Application of Combinatorial Testing to Quantum Programs. In: 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS), pp. 179–188 (2021). https://doi.org/10.1109/QRS54544.2021.00029
Wang, X., Arcaini, P., Yue, T., Ali, S.: QuCAT: A Combinatorial Testing Tool for Quantum Software (2023). https://arxiv.org/abs/2309.00119v1
Wang, X., Arcaini, P., Yue, T., Ali, S.: Generating Failing Test Suites for Quantum Programs With Search. In: O’Reilly, U.-M., Devroey, X. (eds.) Search-Based Software Engineering. Lecture Notes in Computer Science, pp. 9–25. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88106-1_2
Wang, X., Arcaini, P., Yue, T., Ali, S.: QuSBT: Search-Based Testing of Quantum Programs. In: Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings, pp. 173–177 (2022)
Ali, S., Arcaini, P., Wang, X., Yue, T.: Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs. In: 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST), pp. 13–23 (2021). https://doi.org/10.1109/ICST49551.2021.00014
Wang, X., Arcaini, P., Yue, T., Ali, S.: Quito: A Coverage-Guided Test Generator for Quantum Programs. In: 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 1237–1241 (2021). https://doi.org/10.1109/ASE51524.2021.9678798
Rajak, A., Suzuki, S., Dutta, A., Chakrabarti, B.K.: Quantum annealing: an overview. Phil. Trans. Roy. Soc. A Math. Phys. Eng. Sci. 381(2241), 20210417 (2023). https://doi.org/10.1098/rsta.2021.0417. https://royalsocietypublishing.org/doi/pdf/10.1098/rsta.2021.0417
Aleksandrowicz, G., Alexander, T., Barkoutsos, P., Bello, L., Ben-Haim, Y., Bucher, D., Cabrera-Hernández, F.J., Carballo-Franquis, J., Chen, A., Chen, C.-F., Chow, J.M., Córcoles-Gonzales, A.D., Cross, A.J., Cross, A., Cruz-Benito, J., Culver, C., González, S.D.L.P., Torre, E.D.L., Ding, D., Dumitrescu, E., Duran, I., Eendebak, P., Everitt, M., Sertage, I.F., Frisch, A., Fuhrer, A., Gambetta, J., Gago, B.G., Gomez-Mosquera, J., Greenberg, D., Hamamura, I., Havlicek, V., Hellmers, J., Herok, Horii, H., Hu, S., Imamichi, T., Itoko, T., Javadi-Abhari, A., Kanazawa, N., Karazeev, A., Krsulich, K., Liu, P., Luh, Y., Maeng, Y., Marques, M., Martín-Fernández, F.J., McClure, D.T., McKay, D., Meesala, S., Mezzacapo, A., Moll, N., Rodríguez, D.M., Nannicini, G., Nation, P., Ollitrault, P., O’Riordan, L.J., Paik, H., Pérez, J., Phan, A., Pistoia, M., Prutyanov, V., Reuter, M., Rice, J., Davila, A.R., Rudy, R.H.P., Ryu, M., Sathaye, N., Schnabel, C., Schoute, E., Setia, K., Shi, Y., Silva, A., Siraichi, Y., Sivarajah, S., Smolin, J.A., Soeken, M., Takahashi, H., Tavernelli, I., Taylor, C., Taylour, P., Trabing, K., Treinish, M., Turner, W., Vogt-Lee, D., Vuillot, C., Wildstrom, J.A., Wilson, J., Winston, E., Wood, C., Wood, S., Wörner, S., Akhalwaya, I.Y., Zoufal, C.: Qiskit: An Open-source Framework for Quantum Computing. Zenodo (2019). https://doi.org/10.5281/zenodo.2562111
Nielsen, M.A., Chuang, I.L.: Quantum Computation and Quantum Information: 10th Anniversary Edition. Cambridge University Press, Cambridge (2010). https://doi.org/10.1017/CBO9780511976667
Alaqail, H., Ahmed, S.: Overview of software testing standard iso/iec/ieee 29119. Int. J. Comput. Sci. Network Secur. (IJCSNS) 18(2), 112–116 (2018)
Miranskyy, A., Zhang, L.: On Testing Quantum Programs. In: 2019 IEEE/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), pp. 57–60 (2019). https://doi.org/10.1109/ICSE-NIER.2019.00023. http://arxiv.org/abs/1812.09261
De Stefano, M., Pecorelli, F., Di Nucci, D., Palomba, F., De Lucia, A.: Software engineering for quantum programming: How far are we? J. Syst. Software 190, 111326 (2022). https://doi.org/10.1016/j.jss.2022.111326
Paltenghi, M., Pradel, M.: LintQ: A Static Analysis Framework for Qiskit Quantum Programs (2023). arXiv:2310.00718 [cs]
Fortunato, D., Campos, J., Abreu, R.: Mutation testing of quantum programs: a case study with Qiskit. IEEE Trans. Quant. Eng. 3, 1–17 (2022). https://doi.org/10.1109/TQE.2022.3195061
Fortunato, D., Campos, J., Abreu, R.: Mutation Testing of Quantum Programs Written in QISKit. In: 2022 IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pp. 358–359 (2022). https://doi.org/10.1145/3510454.3528649
Fortunato, D., Campos, J., Abreu, R.: QMutPy: A Mutation Testing tool for Quantum algorithms and Applications in Qiskit. In: Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 797–800. ACM, Virtual South Korea (2022). https://doi.org/10.1145/3533767.3543296 . https://dl.acm.org/doi/10.1145/3533767.3543296
Mendiluze, E., Ali, S., Arcaini, P., Yue, T.: Muskit: A Mutation Analysis Tool for Quantum Software Testing. In: 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 1266–1270 (2021). https://doi.org/10.1109/ASE51524.2021.9678563
Li, G., Zhou, L., Yu, N., Ding, Y., Ying, M., Xie, Y.: Projection-based runtime assertions for testing and debugging quantum programs (2020). Accepted: 2021-03-14T22:46:19Z
Muqeet, A., Yue, T., Ali, S., Arcaini, P.: Noise-Aware Quantum Software Testing (2023)
Zhang, L., Radnejad, M., Miranskyy, A.: Identifying Flakiness in Quantum Programs. Preprint (2023). arXiv:2302.03256
Long, P., Zhao, J.: Testing multi-subroutine quantum programs: From unit testing to integration testing (2023). arXiv:2306.17407 [quant-ph]
Long, P., Zhao, J.: Testing quantum programs with multiple subroutines (2023). arXiv:2208.09206 [cs]
Hoare, C.A.R.: An axiomatic basis for computer programming. Commun. ACM 12(10), 576–580 (1969). https://doi.org/10.1145/363235.363259
Yamaguchi, F., Golde, N., Arp, D., Rieck, K.: Modeling and Discovering Vulnerabilities with Code Property Graphs. In: 2014 IEEE Symposium on Security and Privacy, pp. 590–604 (2014). https://doi.org/10.1109/SP.2014.44
Barrera, A.G., Guzmán, I.G.-R., Polo, M., Cruz-Lemus, J.A.: In: Serrano, M.A., Pérez-Castillo, R., Piattini, M. (eds.) Quantum Software Testing: Current Trends and Emerging Proposals, pp. 167–191. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-05324-5_9
Sych, D., Leuchs, G.: A complete basis of generalized bell states. New J. Phys. 11(1), 013006 (2009). https://doi.org/10.1088/1367-2630/11/1/013006
Harrow, A.W., Hassidim, A., Lloyd, S.: Quantum algorithm for linear systems of equations. Phys. Rev. Lett. 103(15), (2009). https://doi.org/10.1103/physrevlett.103.150502
Lloyd, S., Mohseni, M., Rebentrost, P.: Quantum principal component analysis. Nature Phys. 10(9), 631–633 (2014) https://doi.org/10.1038/nphys3029
Zhao, P., Zhao, J., Ma, L.: Identifying Bug Patterns in Quantum Programs. In: 2021 IEEE/ACM 2nd International Workshop on Quantum Software Engineering (Q-SE), pp. 16–21. IEEE, Madrid, Spain (2021). https://doi.org/10.1109/Q-SE52541.2021.00011. https://ieeexplore.ieee.org/document/9474564/
Zhao, P., Zhao, J., Miao, Z., Lan, S.: Bugs4Q: A Benchmark of Real Bugs for Quantum Programs. In: 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 1373–1376 (2021). https://doi.org/10.1109/ASE51524.2021.9678908
Chen, Q., Câmara, R., Campos, J., Souto, A., Ahmed, I.: The Smelly Eight: An Empirical Study on the Prevalence of Code Smells in Quantum Computing. In: 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pp. 358–370 (2023). https://doi.org/10.1109/ICSE48619.2023.00041
Cross, A.W., Bishop, L.S., Smolin, J.A., Gambetta, J.M.: Open Quantum Assembly Language (2017)
Liang, H., Pei, X., Jia, X., Shen, W., Zhang, J.: Fuzzing: state of the art. IEEE Trans. Reliab. 67(3), 1199–1218 (2018). https://doi.org/10.1109/TR.2018.2834476
Zhu, X., Wen, S., Camtepe, S., Xiang, Y.: Fuzzing: A survey for roadmap. ACM Comput. Surv. 54(11s), (2022). https://doi.org/10.1145/3512345
Manès, V.J.M., Han, H., Han, C., Cha, S.K., Egele, M., Schwartz, E.J., Woo, M.: The art, science, and engineering of fuzzing: A survey. IEEE Trans. Software Eng. 47(11), 2312–2331 (2021). https://doi.org/10.1109/TSE.2019.2946563
Li, J., Zhao, B., Zhang, C.: Fuzzing: a survey. Cybersecurity 1(1), 1–13 (2018)
Godefroid, P.: Fuzzing: Hack, art, and science. Commun. ACM 63(2), 70–76 (2020)
Wang, Y., Jia, P., Liu, L., Huang, C., Liu, Z.: A systematic review of fuzzing based on machine learning techniques. PLOS ONE 15(8), 1–37 (2020). https://doi.org/10.1371/journal.pone.0237749
Offutt, A.J., Pan, J.: Automatically detecting equivalent mutants and infeasible paths. Software Test. Verif. Reliab. 7(3), 165–192 (1997). https://doi.org/10.1002/(SICI)1099-1689(199709)7:3%3C165::AID-STVR143%3E3.0.CO;2-U
Just, R., Kapfhammer, G.M., Schweiggert, F.: Do Redundant Mutants Affect the Effectiveness and Efficiency of Mutation Analysis? In: 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, pp. 720–725 (2012). https://doi.org/10.1109/ICST.2012.162
Madeyski, L., Orzeszyna, W., Torkar, R., Józala, M.: Overcoming the equivalent mutant problem: a systematic literature review and a comparative experiment of second order mutation. IEEE Trans. Software Eng. 40(1), 23–42 (2014). https://doi.org/10.1109/TSE.2013.44
Just, R., Kapfhammer, G.M., Schweiggert, F.: Using Non-redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and Scalable Mutation Analysis. In: 2012 IEEE 23rd International Symposium on Software Reliability Engineering, pp. 11–20 (2012). https://doi.org/10.1109/ISSRE.2012.31
Just, R., Schweiggert, F.: Higher accuracy and lower run time: efficient mutation analysis using non-redundant mutation operators. Software Test. Verif. Reliab. 25(5-7), 490–507 (2015). https://doi.org/10.1002/stvr.1561
Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002). https://doi.org/10.1109/4235.996017
Barr, E.T., Harman, M., McMinn, P., Shahbaz, M., Yoo, S.: The Oracle problem in software testing: a survey. IEEE Trans. Software Eng. 41(5), 507–525 (2015). https://doi.org/10.1109/TSE.2014.2372785
Sicilia, M.-A., Sánchez-Alonso, S., Mora-Cantallops, M., García-Barriocanal, E.: On the Source Code Structure of Quantum Code: Insights from Q# and QDK. In: Shepperd, M., Abreu, F., Silva, A., Pérez-Castillo, R. (eds.) Quality of Information and Communications Technology. Communications in Computer and Information Science, pp. 292–299. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58793-2_24
Yamashita, K., Huang, C., Nagappan, M., Kamei, Y., Mockus, A., Hassan, A.E., Ubayashi, N.: Thresholds for Size and Complexity Metrics: A Case Study from the Perspective of Defect Density. In: 2016 IEEE International Conference on Software Quality, Reliability and Security (QRS), pp. 191–201 (2016). https://doi.org/10.1109/QRS.2016.31
Kumar, A.: Formalization of structural test cases coverage criteria for quantum software testing. Int. J. Theor. Phys. 62, (2023). https://doi.org/10.1007/s10773-022-05271-y
Just, R., Jalali, D., Inozemtseva, L., Ernst, M.D., Holmes, R., Fraser, G.: Are Mutants a Valid Substitute for Real Faults in Software Testing? In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 654–665. ACM, Hong Kong China (2014). https://doi.org/10.1145/2635868.2635929. https://dl.acm.org/doi/10.1145/2635868.2635929
Just, R., Jalali, D., Ernst, M.D.: Defects4j: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs. In: Proceedings of the 2014 International Symposium on Software Testing and Analysis, pp. 437–440. ACM, San Jose, CA, USA (2014). https://doi.org/10.1145/2610384.2628055. https://dl.acm.org/doi/10.1145/2610384.2628055
Gyimesi, P., Vancsics, B., Stocco, A., Mazinanian, D., Beszédes, A., Ferenc, R., Mesbah, A.: BugsJS: A Benchmark of JavaScript Bugs. In: 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST), pp. 90–101 (2019). https://doi.org/10.1109/ICST.2019.00019
Widyasari, R., Sim, S.Q., Lok, C., Qi, H., Phan, J., Tay, Q., Tan, C., Wee, F., Tan, J.E., Yieh, Y., Goh, B., Thung, F., Kang, H.J., Hoang, T., Lo, D., Ouh, E.L.: BugsInPy: A Database of Existing Bugs in Python Programs to Enable Controlled Testing and Debugging Studies. In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ESEC/FSE 2020, pp. 1556–1560. Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3368089.3417943
Shamshiri, S., Just, R., Rojas, J.M., Fraser, G., McMinn, P., Arcuri, A.: Do Automatically Generated Unit Tests Find Real Faults? An Empirical Study of Effectiveness and Challenges. In: 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 201–211 (2015). https://doi.org/10.1109/ASE.2015.86
Lukasczyk, S., Kroiß, F., Fraser, G.: An empirical study of automated unit test generation for Python. Empirical Software Eng. 28(2), 36 (2023)
Paterson, D., Campos, J., Abreu, R., Kapfhammer, G.M., Fraser, G., McMinn, P.: An Empirical Study on the Use of Defect Prediction for Test Case Prioritization. In: 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST), pp. 346–357 (2019). https://doi.org/10.1109/ICST.2019.00041
Miranda, B., Cruciani, E., Verdecchia, R., Bertolino, A.: FAST Approaches to Scalable Similarity-Based Test Case Prioritization. In: Proceedings of the 40th International Conference on Software Engineering. ICSE ’18, pp. 222–232. Association for Computing Machinery, New York, NY, USA (2018). https://doi.org/10.1145/3180155.3180210
Pearson, S., Campos, J., Just, R., Fraser, G., Abreu, R., Ernst, M.D., Pang, D., Keller, B.: Evaluating and Improving Fault Localization. In: 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE), pp. 609–620 (2017). https://doi.org/10.1109/ICSE.2017.62
Li, X., Li, W., Zhang, Y., Zhang, L.: DeepFL: Integrating Multiple Fault Diagnosis Dimensions for Deep Fault Localization. In: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. ISSTA 2019, pp. 169–180. Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3293882.3330574
Zou, D., Liang, J., Xiong, Y., Ernst, M.D., Zhang, L.: An empirical study of fault localization families and their combinations. IEEE Trans. Software Eng. 47(2), 332–347 (2021). https://doi.org/10.1109/TSE.2019.2892102
Sarhan, Q.I., Beszédes, A.: A survey of challenges in spectrum-based software fault localization. IEEE Access 10, 10618–10639 (2022). https://doi.org/10.1109/ACCESS.2022.3144079
Widyasari, R., Prana, G.A.A., Haryono, S.A., Wang, S., Lo, D.: Real world projects, real faults: evaluating spectrum based fault localization techniques on Python projects. Empirical Software Eng. 27(6), 147 (2022)
Durieux, T., Madeiral, F., Martinez, M., Abreu, R.: Empirical Review of Java Program Repair Tools: A Large-Scale Experiment on 2,141 Bugs and 23,551 Repair Attempts. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ESEC/FSE 2019, pp. 302–313. Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3338906.3338911
Campos, J., Souto, A.: QBugs: A Collection of Reproducible Bugs in Quantum Algorithms and a Supporting Infrastructure to Enable Controlled Quantum Software Testing and Debugging Experiments (2021)
Paltenghi, M., Pradel, M.: Bugs in quantum computing platforms: an empirical study. Proc. ACM Program. Lang. 6(OOPSLA1), (2022). https://doi.org/10.1145/3527330
Lewis, C., Lin, Z., Sadowski, C., Zhu, X., Ou, R., Whitehead Jr., E.J.: Does Bug Prediction Support Human Developers? Findings from a Google Case Study. In: Proceedings of the 2013 International Conference on Software Engineering. ICSE ’13, pp. 372–381. IEEE Press, San Francisco, CA, USA (2013)
Freitas, P.A.F.: Software repository mining analytics to estimate software component reliability (2015)
D’Ambros, M., Lanza, M., Robbes, R.: Evaluating defect prediction approaches: a benchmark and an extensive comparison. Empirical Software Eng. 17(4–5), 531–577 (2012). https://doi.org/10.1007/s10664-011-9173-9
Catal, C., Diri, B.: A systematic review of software fault prediction studies. Expert Syst. Appl. 36(4), 7346–7354 (2009). https://doi.org/10.1016/j.eswa.2008.10.027
Ostrand, T.J., Weyuker, E.J., Bell, R.M.: Predicting the location and number of faults in large software systems. IEEE Trans. Software Eng. 31(4), 340–355 (2005). https://doi.org/10.1109/TSE.2005.49
Arcuri, A., Campos, J., Fraser, G.: Unit Test Generation During Software Development: EvoSuite Plugins for Maven, IntelliJ and Jenkins. In: 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST), pp. 401–408 (2016). https://doi.org/10.1109/ICST.2016.44
Campos, J., Riboira, A., Perez, A., Abreu, R.: Gzoltar: An Eclipse Plug-in for Testing and Debugging. In: Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering. ASE ’12, pp. 378–381. Association for Computing Machinery, New York, NY, USA (2012). https://doi.org/10.1145/2351676.2351752
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2024 The Author(s)
About this chapter
Cite this chapter
Fortunato, D., Jiménez-Navajas, L., Campos, J., Abreu, R. (2024). Verification and Validation of Quantum Software. In: Exman, I., Pérez-Castillo, R., Piattini, M., Felderer, M. (eds) Quantum Software. Springer, Cham. https://doi.org/10.1007/978-3-031-64136-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-64136-7_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-64135-0
Online ISBN: 978-3-031-64136-7
eBook Packages: Computer ScienceComputer Science (R0)