Avoid common mistakes on your manuscript.
Dear Editor,
Vreman et al. [1] regard a probabilistic deterministic sensitivity analysis (DSA) as “the most appropriate method for providing insight on the effect of uncertainty in individual parameters on the estimate of cost effectiveness”. Furthermore, they “advocate to at least implement distributional DSA in all cases”. Distributional DSA aims at demonstrating outcomes for a number of intermediate steps between the base case and the minimum and maximum values that are based on percentiles of the probability density function of the parameter. The authors present the results of the distributional and probabilistic DSA in a tornado diagram.
In my opinion, the approach suggested by the authors is intuitively appealing if a probability distribution is known. Nevertheless, the authors implicitly assume that assigning probabilities is always preferable to not assigning probabilities. This assumption, however, is contentious when decision-makers must decide under complete ignorance, that is, when they do not have information about probabilities [2]. In this situation, assigning equal probabilities (based on the Laplace rule) can lead to preference reversals [3]. Hence, when deciding under ignorance it is not rational to apply a distributional or probabilistic DSA (based on an equiprobability assumption). Instead, under ignorance, it is rational to make decisions only based on the extreme values, without assigning probabilities [3]. This approach aligns well with the original purpose of a tornado diagram, which has been described as a graphical illustration of the pessimistic and optimistic values for each uncertain variable [4]. That is, the classic tornado diagram does not intend to make a statement about the probability of the values that occur in-between the extreme values.
The authors do caution against assigning equal probabilities. Instead of dropping probabilities, however, they suggest using (true) probability distributions. Yet, under ignorance it would be preferable not to assign any probabilities and, instead, make decisions based on the extreme values analyzed in univariate sensitivity analyses and illustrated by a classical tornado diagram. While it is true that correlations between variables need to be considered, their formal consideration needs to rely on probabilities. Instead, under ignorance, decision-makers may incorporate them implicitly in the optimistic and pessimistic values.
Finally, I would like to provide examples of variables for which information on probability distributions is lacking. Examples are long-term effectiveness, side effects, or downstream costs of new technologies [5]. Furthermore, some authors have advocated that cost-effectiveness analysis should go beyond the consideration of present real-world conditions and incorporate life-cycle or dynamic aspects. Life-cycle aspects include entry of biosimilars in an orphan drug designation and the arrival of technological advancements in the future [6]. While it may be appropriate to assign a specific probability distribution for the former event based on experience, for the latter it may not be. Instead, it seems more appropriate to use a pessimistic–optimistic range, that is, to model 'yes' and 'no' scenarios. In more general terms, all variables to which uniform probability distributions would be applied otherwise due to a lack of information on probabilities appear to be candidates for the classic tornado diagram. Another, very recent example of decision making under ignorance is the coronavirus crisis. Current uncertainties revolve around the ability of the vaccine(s) to prevent transmission of new variant strains, the harms and benefits in children (e.g., in terms of preventing long COVID), and whether new variant strains will render approved vaccines less effective.
In summary, I argue that the classic tornado diagram still has its place for displaying optimistic and pessimistic values for those variables and in those situations for which information on probability distributions is lacking. Still, more research needs to be conducted to define the conditions in which it is not appropriate to assume a probability distribution. In this respect, the coronavirus crisis may serve as an opportunity to fine-tune our modelling assumptions and methods.
Change history
24 February 2022
The Funding note is included and deleted from the Declarations section.
References
Vreman RA, Geenen JW, Knies S, Mantel-Teeuwisse AK, Leufkens HGM, Goettsch WG. The application and implications of novel deterministic sensitivity analysis methods. Pharmacoeconomics. 2021;39(1):1–17.
Luce RD, Raiffa H. Games and decisions: introduction and critical survey. New York: John Wiley & Sons; 1957.
Denoeux T. Decision-making with belief functions: a review. Int J Approx Reason. 2019;109:87–110.
Chelst K, Bodily SE. Structured risk management: filling a gap in decision analysis education. J Oper Res Soc. 2000;51(12):1420–32.
Grutters JP, van Asselt MB, Chalkidou K, Joore MA. The Authors’ Reply: Comment on “Healthy Decisions: Towards Uncertainty Tolerance in Healthcare Policy.” Pharmacoeconomics. 2015;33(9):983.
Gandjour A. Comment on: “Healthy Decisions: Towards Uncertainty Tolerance in Healthcare Policy.” Pharmacoeconomics. 2015;33(9):981–2.
Funding
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Availability of data and material
Not applicable.
Conflict of interest
The author declares that he has no competing interests.
Authors' contributions
AG is the sole author responsible for conception, drafting, and approving the submitted version.
Additional information
The original online version of this article was revised : The Funding note was misplaced in Declarations section in the original version.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/.
About this article
Cite this article
Gandjour, A. Deterministic Sensitivity Analysis Under Ignorance. PharmacoEconomics 39, 1197–1198 (2021). https://doi.org/10.1007/s40273-021-01085-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40273-021-01085-8