Abstract
Optimal control theory can be a useful tool to identify the best strategies for the management of infectious diseases. In most of the applications to disease control with ordinary differential equations, the objective functional to be optimized is formulated in monetary terms as the sum of intervention costs and the cost associated with the burden of disease. We present alternate formulations that express epidemiological outcomes via health metrics and reframe the problem to include features such as budget constraints and epidemiological targets. These alternate formulations are illustrated with a compartmental cholera model. The alternate formulations permit us to better explore the sensitivity of the optimal control solutions to changes in available budget or the desired epidemiological target. We also discuss some limitations of comprehensive cost assessment in epidemiology.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Due to systematic pervasive nonlinearities and heterogeneities underlying infectious disease transmission at the population level, epidemiological modeling is a key tool for exploring the potential impact of candidate intervention strategies (Bubar et al. 2021). Questions surrounding the best timing, target populations, and treatment effort to limit the transmission of communicable diseases—whether through drug treatment, vaccines, or non-pharmaceutical interventions—have been central in epidemiological modeling since the publication of the first malaria models by Ross and MacDonald more than a century ago (Smith et al. 2012). Optimal control techniques can be used to address such questions (Sharomi and Malik 2017).
Identification of optimal control strategies for managing diseases has long been pursued informally. In the absence of health interventions or natural immunity, an infectious disease follows its epidemic curve and may cause substantial morbidity and mortality (Bjørnstad et al. 2020), but by investing resources in control and treatment, it is possible to reduce the burden of disease (Keeling and Rohani 2008), even as the decision space in selecting certain strategies is large with uncertain impacts. Larger control and efforts can lower disease burden while increasing costs of control, yet these relationships are often nonlinear, with decreasing returns on investment—i.e., health benefits per dollar invested—at high control effort (Ozawa et al. 2021). Accordingly, extreme control efforts may be infeasible (possibly due to vaccine hesitancy Sallam et al. 2022) or prohibitively expensive (Ozawa et al. 2021). In addition, extreme control effort can be associated with undesirable outcomes, such as intervention fatigue (Rypdal et al. 2020). Intermediate levels of control may help manage costs of disease control when accounting for both the intervention costs (which depend nonlinearly on the control effort) and the cost associated with the burden of disease, but establishing the specific effort levels across strategies remain challenging.
Optimal control (OC) theory has proved to be a particularly powerful mathematical tool to identify the best strategies for the control of infectious diseases. In contrast to some optimization approaches that assume control variables to be constant in time, the admissible control variables in OC (e.g., vaccination rate, drug treatment) can vary in time, allowing dynamic minimization of both the cost of disease burden and the costs of intervention.
In the last two decades, the number of published papers on OC applications to the control of infectious diseases of public health or veterinary importance has increased exponentially. In the majority of these papers, disease burden is typically represented by the integral of I(t), the number of infectious individuals at time t, and the objective functional to be optimized is generally formulated in monetary terms as the sum of intervention costs plus the cost associated with the burden of disease, assuming that both can be estimated monetarily with a reasonable degree of precision (Asano et al. 2008; Neilan et al. 2010; Agusto et al. 2020; Jung et al. 2009; Lee et al. 2020b). This formulation has been very useful to formalize the trade-off between the goal to minimize disease burden and, at the same time, the intervention costs. Yet, as discussed below, accounting for disease burden and loss of human lives in monetary terms is fraught with uncertainties and generally considered not ethically and socially acceptable from a public health perspective (Baltussen 2006; Brock and Wikler 2006; Jiang et al. 2020; Morrow and Byrant 1995; Rutstein et al. 2017).
Minimization of the cost of disease burden represented by the integral of I(t) combined with minimizing the cost of control action has been common, but there are notable exceptions. Examples of alternative formulations of the optimal control problem that do not require monetization of health outcomes are available in the literature. Some solve the OC problem under budget constraints and optimize disease burden. Others change the representation of the goal in terms of other disease outcomes. For instance, Hansen and Day optimized the total number of cases in an \({\textit{SIR}}\) model with control of vaccination, subject to resource constraints (Hansen and Day 2011). Also Bolzoni and collaborators used OC theory to minimize the time to drive the current prevalence level to a desired threshold under resource limitation (Bolzoni et al. 2017, 2019). Using vaccination as a control in a \({\textit{SIRV}}\) model, Laguzet and Turinici minimized infections under the public health budget constraints (Laguzet and Turinici 2015). Campo-Duarte et al. (2018) used optimal control techniques for a trade-off between achieving a fixed desired level of mosquitoes not infected with \({\textit{Wolbachia}}\) in minimum time and minimizing the cost of control interventions. Yet, until recently, these examples have been rare and most of the OC published papers keep formulating OC problems with the traditional approach, i.e., minimization of the combined monetary costs of control measures and disease burden, without discussing the practical challenges of estimating the monetary cost of health outcomes.
Recently COVID-19 modeling research has pursued approaches aligned with those proposed herein. With a goal of minimizing the costs of implementing transmission rate control (social distancing measures), Miclo et al. (2022) included an upper on the number of infecteds in an \({\textit{SI}}\) model; this upper bound was interpreted as intensive care unit constraint (with infinite time horizon). Also using an \({\textit{SI}}\) system, Bliman and Duprez (2021) used social distancing controls on finite time intervals (using one dimensional optimization) to minimize the final epidemic size. Using optimization of finite duration time intervals for reduction of the transmission rates, Morris et al. (2021) minimized epidemic peak size and demonstrated optimal and nearly optimal solutions for an \({\textit{SIR}}\) model. Angulo et al. (2021) studied a control problem to steer an SI model trajectory to a target set with a low I level in minimum time (using transmission reduction control). In summary, some of these papers kept the control level low (to minimize the economic and social costs of limiting economic activity) while achieving the epidemiological constraints (upper bound on number of infecteds) (Bliman and Duprez 2021; Morris et al. 2021). Another included choices of intervals to implement the control at its maximum level while imposing an upper bound on the number of infecteds (Miclo et al. 2022). Matrajt et al. (2020) used discrete optimization to find optimal vaccination levels (taking on discrete values) for different age groups in a deterministic COVID-19 system. These examples show that, when epidemiological modeling is guided by a pressing public health problem and is used to inform the decision-making process, it is crucial to formulate the OC problem to closely reflect the specific goals and constraints.
In this paper, after briefly reviewing the practical and ethical challenges of assessing the monetary value of disease burden and of human life, we propose a simple but general framework to reformulate optimization problems to leverage the strengths of optimal control theory and, at the same time, to explicitly account for real-world features such as budget constraints and epidemiological goals that public health authorities struggle with in their daily work. We demonstrate the new framework and discuss the pros and cons of alternative formulations of OC problems using a simple but general cholera model.
1.1 The Challenges of Estimating the Monetary Value for Health Outcomes
In industrialized countries with developed public health, welfare and demographic reporting systems, both the private and public sectors (i.e., health insurance and health departments/agencies) routinely assess the economic impact associated with diseases, accounting for the cost of over-the-counter and prescribed drugs, medical visits, hospital admissions, special treatments, loss of productivity, etc. This systematic monitoring and reporting effort provides a reasonable foundation to comprehensively estimate the monetary costs of disease outbreak and a range of public health interventions, which can be used to explore alternative control scenarios within an OC theory framework. However, this approach is not exempt from limitations. First, monetization of human health is ethically debatable, especially in the case of fatal diseases and when diseases disproportionately affect a specific segment of the population, such as underrepresented minorities, essential workers, or the homeless population (Scovronick et al. 2020).
Second, monetization of human health and/or loss of productivity becomes particularly problematic in low-income countries, plagued by a wide range of Neglected Tropical Diseases (including several infectious diseases, such as schistosomiasis, Chagas disease, dengue fever, guinea worm disease, echinococcosis, human African sleeping sickness, and leishmaniasis) and other diseases of poverty (such as malaria, tuberculosis, cholera and HIV) affecting altogether more than 2.5 billion people worldwide (Ntuli 2020). For these countries, where a non-marginal fraction of the population is often engaged in a rural, cashless economy, using the same monetary currency to contrast the costs of public health interventions with the burden of disease—which may include also reduced capacity to perform physically engaging activities in subsistence agriculture, or the long-term impact of chronic infections—is highly challenging and debatable. Some studies have explicitly recognized these challenges and analyzed OC problems as a function of loosely defined “relative costs of disease burden” with respect to the cost of intervention (see for instance, Bolzoni et al. 2014). Yet, the problem of integrating economic and public health metrics and indicators measured on vastly different units in a single monetary scale remains unsolved. Cost-effectiveness analysis partially circumvents these ethical shortcoming and practical challenges, as it abstains from measuring the monetary value of health outcomes but does not leverage the power of OC theory to identify time-varying control strategies superior to time constant strategies. While assessing the monetary value of human health is fraught with uncertainties, even the estimation of intervention costs and its translation into model parameters present formidable theoretical and practical challenges that are rarely recognized and accounted for in the published literature on OC applications to infectious disease management.
1.2 Reframing Optimal Control Problems
Public health authorities do not operate in a vacuum, but need to plan health interventions under budget constraints and/or to achieve specific health outcomes, such as to avoid overwhelming the health system by limiting the number of people requiring hospitalization or use of the intensive care units (Miller et al. 2020). Accordingly, OC problems can be reformulated to fully embrace this more practical approach to the control of communicable diseases. Specifically, we argue that OC problems can be restated in one of the two following broad and more meaningful ways, namely:
-
Alternative 1: minimize disease burden (the goal), measured in the most suitable, non-monetary units for the specific disease system under study for a given budget (the constraint) (Murray et al. 2012);
-
Alternative 2: minimize the monetary cost of interventions (the goal) to achieve a specific epidemiological outcome, set as a requirement of the control strategies to be admissible (the constraint).
The first alternative recognizes that budget constraints are a significant factor in choosing control strategies in low-income countries. In the second alternative, the emphasis is on achieving a desired epidemiological target in a cost effective way. In the 2021 United Nations Political Declaration on HIV and AIDS, for example, there is a goal of reducing annual new HIV infections to under 370,000 and annual AIDS-related deaths to under 250,000 by 2025 (Assembly 2021). Optimal control policies that do not exceed regional hospital capacity and the number of available ICU beds at the peak of a COVID-19 outbreak are examples of the second alternative formulation (Miller et al. 2020; Miclo et al. 2022). Other examples include minimization of the time to a desired reduction in disease prevalence or incidence (Angulo et al. 2021; Bolzoni et al. 2017, 2019).
Here below, we present the application of these two alternative frameworks for the formulation of OC problems by using the control of a cholera outbreak as a reference example. Specifically, we use a classic mechanistic, compartmental model for cholera, based on set of ordinary differential equations, that was kept deliberately simple for demonstration purposes to show that it is not only straightforward to reframe optimal control problems in these two alternative ways, but also more informative. In fact, we show that under the new frame (i) analyses can be easily performed to assess how the outcome changes with respect to, say, increasing budget (Alternative 1) or more ambitious epidemiological goals (Alternative 2) and (ii) the results of these analyses can be presented in terms of marginal utilities, for example, additional lives saved per extra dollar invested with respect to baseline.
The rest of the paper is structured as follows. After describing the cholera model with vaccination rate as the control variable, we first define the classic problem as the minimization of the sum of the cumulative cost of disease, estimated in monetary terms, and of the cost of intervention over a given time period, assuming the conversion factor from number of infected cases to dollars is known. This classical approach is referred to in this paper as the “combined” OC problem.
Then, we reformulate the OC problem in the two alternative ways described above. We show that, if the conversion coefficient to assess the monetary value of disease cost can be estimated with reasonable precision, the outcome of the alternative OC formulations can be mapped back to the combined OC problem with no loss of information.
Lastly, we show that the two alternative formulations permit us to better explore the sensitivity of the OC solutions to changes in available budget or the desired epidemiological target. We conclude the paper by discussing our results and presenting some limitations of comprehensive cost assessment in epidemiology.
2 Cholera Model
We illustrate our points using a basic model of cholera over a short time period. Cholera is a waterborne diarrhoeal illness caused by infection of the intestine with the bacterium Vibrio cholerae with both direct and indirect modes of infection and transmission, primarily through contaminated food and water supplies. The disease causes rapid dehydration and electrolyte imbalances. There are an estimated 3–5 million cholera cases and 100,000–120,000 deaths due to cholera every year (Ali et al. 2015; Legros 2018). The recent outbreak in Haiti caused over 10,000 deaths and over 820,000 cases (Zarocostas 2017; Lee et al. 2020a). The disease has a short incubation period, between two hours and five days, affecting all ages. It is known that cholera persists in an aquatic reservoir and can exist in non-culturable, but viable, state for months to over a year (Melbourne 2011). There are two types of safe and effective oral cholera vaccines, Dukoral and Shanchol (Sèvére et al. 2016).
Tien and Earn (2010) extended the classic Susceptible (S), Infected (I) and Recovered (R) SIR model by Codeco (2001) to include a concentration of vibrios (W) in the water compartment. This allows for both direct (fecal-oral) and indirect (from water-borne pathogen) transmissions pathways for cholera at per capita rates \(\beta _I \) and \(\beta _W\) [\(\text {day}^{-1} \text {individuals}^{-1}\)], respectively. For comparisons of several models applied to outbreaks in Haiti and with vaccination scenarios, see the work of Lee et al. (2020a). Our basic \({\textit{SIRW}}\) model with four state variables is described by the following differential equations:
where \(N = S+I+R\) represents the human population, \(\mu \) is the natural mortality and birth rates of humans [\(\text {day}^{-1}\)], \(\gamma \) is the recovery rate from infected to recovered and resistant class [\(\text {day}^{-1}\)], and \(\delta \) is the death rate due to the disease [\(\text {day}^{-1}\)]. Note that the compartment W has been scaled to be in the same units as the I compartment (individuals). Therefore, \(\xi \) is a parameter representing both the per capita shedding rate of viable infectious propagules in the water by an infected individual I and the decay rate of the pathogen in the environment [\(\text {day}^{-1}\)]; see more details in Tien and Earn (2010) and Kelly et al. (2016). The sum of the terms \(\beta _I I\) and \(\beta _W W\) are the force of infection, representing the rate at which susceptible individuals become infected either via contaminated fecal-oral transmission or the ingestion of contaminated water. The parameters and their units are described in Table .
As in Kelly et al. (2016), we assumed that susceptible individuals S are vaccinated and become resistant R at a rate v(t), with v(t) combining efficacy and rate of vaccine distribution [\(\text {day}^{-1}\)]. The time varying rate v(t) of vaccination is then our control variable. The basic reproduction number for this model in the absence of the control v(t) is
where \(S^*\) is the total susceptible population in the absence of disease.
3 Optimal Control Formulations
We begin with a discussion of formulations of objective functionals. The combined objective functional is the sum of the cumulative costs associated with the disease \( J_b(v) \) and the cost of intervention \( J_c(v) \), which are both functions of control effort v(t) , which, in turn, is a function of time, namely:
with disease dynamics described by the above SIRW model. The control effort v(t) is assumed to be continuous over time and is subject to constraints \( 0 \le v(t) \le v_{\max }\), with \(v_{\max }\) being the maximum vaccination rate that can be deployed during the simulation time \( 0 \le t \le T \).
The total cost of intervention is a nonlinear function of v(t),
where B and C are positive constants accounting for the cost of vaccination. We take a simple cost functional to reflect the possibility of nonlinearity in costs; other formats could used depending on the knowledge surrounding costs. Later in our examples, we will illustrate the case where the nonlinearity plays a role and a case where B is small enough that the optimal control is approximately bang-bang, meaning the optimal control only takes values at its lower and upper bounds. We note that if \(B=0\), then this problem is linear in the controls. In some applications, this could result in possible singular controls as part of the solution, which can occur when the derivative of the Hamiltonian with respect to the control vanishes on a non-empty subinterval of time, requiring further analysis. See the work by Ledzewicz and Schaettler for a simple \({\textit{SIR}}\) model exhibiting singular controls, and see their book for some further background on optimal control problems that are linear in the control (Ledzewicz and Schättler 2011; Schättler and Ledzewicz 2012).
For certain diseases, the infected nature of some individuals, who are able to transmit the pathogen, may not be observed. They could be asymptomatic, and it is possible that those persons might be vaccinated. In such cases, the cost of vaccinations in the objective functional would include a term in the integrand, like \(Cv(t)(S(t) +A(t))\), with A for the asymptomatic class.
The cost associated with the disease depends upon the specific disease system under study and the available information. It can be represented as
where \(A_b\) is a positive scaling coefficient accounting for the aggregated mean monetary value \(c_1\) of each day of illness (including drug treatment and loss of productivity) and, in case of fatal disease, also for the value \(c_2\) of human lives lost, namely:
where \(\delta I\) is the rate of individuals dying because of the disease. In many examples with \(A_b =c_1\), the first term of the objective functional integrates the number of current infected individuals, and we illustrate this frequently used ‘burden’ case in the Appendix.
As an alternative method of computation, the monetary cost of disease only, not accounting for deaths, might be also estimated as a flat rate associated to each infected case and, as such, derived from total disease incidence, i.e., the total number of infected cases during the simulation time, namely:
with the \(\beta \) terms giving the rate of new cases.
The precise form of optimal control is affected by the relative values of the coefficients \(A_b, A_n, B, C, c_1\) and \(c_2\). Integrating the sum of the rates of the new infections gives us the total number of new cases. The optimal control can be found using the Pontryagin’s Maximum Principle (Pontryagin et al. 1962) as described later in “Appendix A”. We would also note that one can solve for the value function and the resulting optimal control using Hamilton–Jacobi–Bellman partial differential equation approach (with viscosity solution) as in (Laguzet and Turinici 2015; Bardi and Capuzzo-Dolcetta 1997; Piunovskiy and Clancy 2008).
One could include terms in the objective functional at the final time, like I(T), which would affect the optimal control levels at the final time. There may situations where an infinite time horizon makes sense for particular applications; optimal control problems on infinite horizons are frequently used in economic models (Miclo et al. 2022; Naveed et al. 2019; Kamien and Schwartz 1991).
3.1 Combined Formulations
Combined Objective Functional with Disease Burden and Cost
The objective functional for minimizing both disease burden and cost is defined as
where the control set is
Combined Objective Functional with New Infections and Cost
The objective functional for minimizing both the total number of new infections and intervention cost is defined as
3.2 Alternate Formulation 1: Minimize New Infections under Fixed Budget
Given a fixed budget G for the cost of the control interventions, we define the admissible control set (with constraint)
where S(t) is the susceptible compartment of the population corresponding to the control v(t). The objective functional to minimize the new infections is defined as
subject to the above SIRW model.
To illustrate the process of obtaining the adjoint functions and the optimal control characterization, we use Pontryagin’s Maximum Principle (Pontryagin et al. 1962). To handle the budget constraint, we introduce a fifth state variable, \(x_5\), that satisfies
with the boundary conditions \(x_5(0)=0\) and \(x_5(T)=G\). Then, the Hamiltonian using this principle for our optimization is
with the following adjoint differential equations and boundary conditions
The adjoint system is formed by \( \lambda _i' = - \frac{\partial H}{\partial x_i}\), where \(x_i\) represents the ith state variable. Note \(\lambda _5\) takes no boundary condition since \(x_5\) takes two boundary conditions.
We can then characterize the optimal control on the interior of the control set using the optimality condition
at the corresponding optimal states, adjoints, and control bounds to yield
3.3 Alternate Formulation 2: Minimize Intervention Costs with Fixed New Infections
Assuming the desired outcome is the cost of number of new infection cases being bounded above by P, we define the admissible control set
The objective functional, defined to minimize the costs of implementing the control, is
subject to the above SIRW model. To handle the constraint on the new cases, we introduce a new state variable, \(x_5\) with
similar to the previous formulation.
Note that \(A_n\) is included in (4) and (5) to show the connection between the combined and alternate formulations and allow us to demonstrate how the results of the combined case can be recovered from the alternate cases. However, to achieve the goal of reformulating optimal control problems to avoid the monetization of human health, this constant can be redefined so that the objective functional represents minimizing only the total number of new infections and not the monetary cost associated with these infections. In the numerical illustrations below, we will set \(A_n=1\), since the relative sizes of the coefficients determine the optimal control.
4 Results
Note that in the results below, we choose a variety of parameters to illustrate various points.
4.1 OC Solutions are Superior to the Best Alternative with Constant Control
Modulating control variables in time may provide a more efficient and cost-effective intervention strategy than the best alternative with a constant control. This point is illustrated by using our example cholera model and a combined objective functional (3) to minimize the cost of interventions plus the cost of the cumulative number of new cases using parameters as given in Table . Our numerical results in this paper use the forward-backward sweep algorithm (Lenhart and Workman 2007). Other algorithms and software programs such as GPOPS and PASA have been developed to handle particular types of optimal control problems (Hager and Zhang 2016; Patterson and Rao 2014).
We first illustrate the advantage of time varying control as compared to a constant control. Note that quadratic coefficient B in our objective functional is large enough that our optimal control is not bang-bang; it takes values from its upper bound to lower bounds, connecting in between. The results shown in Fig. and Table 2 show that the combined cost of intervention and new cases, \(J_n(v)+J_c(v)\), is much larger in the case of no control than the cases of optimal or constant control. We choose the constant control to have approximately the same intervention costs as the optimal control. The controlled cases perform equivalently in terms of intervention costs, though with the optimal control case giving better health outcomes. In the optimal control case, the control is time varying, staying at the maximum for approximately 40 days and then decreasing linearly to zero over the next 20 days. A constant vaccination rate \(v \equiv 0.032\) leads to approximately the same intervention cost (black line in the plot) but with a higher number of cases. Decreasing the number of deaths by 10% by using our optimal time-varying control seems valuable. Also note that the maximum number of infected individuals occurs at about 30 days at a level of about 3800, but in the uncontrolled case, the number of infected individuals is about 10,000 near the final time and still rising. There is no constant value of vaccination rate for which the combined costs is lower than in the case of the optimal control solution, since all constant controls are admissible controls but the optimal control is not constant with this choice of parameters.
4.2 Simulations of Alternative Formulations
The optimal control policy is sensitive to how we estimate the monetary cost of the disease, which is notably fraught with uncertainty. For instance, following Castonguay et al. (2020), we can reasonably assume that each sick day entails a productivity loss equal to the daily per-capita Gross Domestic Product (GDP). However, in Africa, the per-capita GDP spans over a 15 fold difference between the richest countries (Gabon, Botsawa, Equatorial Guinea, Mauritius and Seychelles, all above $15,000 per year) and the poorest ones (Burundi, Somalia, Central African Republic, all below $1000/year) (The World Bank 2021). Keeping this in mind, to illustrate the implications of such a wide variation in the per-capita, daily productivity loss, we simulated cholera epidemic dynamics and derived the OC policy with the combined formulation (3) under two alternative assumptions, namely that the unit cost \(A_n\) of each new case is assumed to be equal to either 1 or 10 in suitable monetary units to measure the average productivity loss while sick. For the results shown in Fig. and Table , we use the same \(\beta _I\), \(\beta _W\), \(\mu \), \(\xi \), and \(\delta \) parameters, initial conditions, and time horizon as in Table 2 and set \(\gamma = 0.25\), \(B=100\), \(C=0.1625\), \(v_{\max }=0.03\). Due to the change in \(\gamma \), the \(R_0\) for these simulations is 0.59. Note that due to the parameters chosen, especially a smaller B, the optimal controls in these simulations are approximately bang-bang (the optimal control only takes values at its upper and lower bounds). Also the vaccination campaign starts near the time of the peak of infected population, and thus the optimal controls do not affect that peak but do affect the number of deaths and vaccinations, depending on the size of \(A_n\).
Simulations with our cholera model show that, when the unit cost of disease is undervalued by an order of magnitude (when \(A_n=1\)), the optimal strategy entails 64% less vaccinations, a 71% reduction in the number of days in which vaccines are deployed at the maximum rate, 40% more lives lost, and 51% more cases (Fig. 2/Table 3). Reformulating the OC problem to minimize disease incidence under budget constraints or, in alternative, to minimize intervention costs with a cap on cumulative incidence, allows us to overcome the challenges in estimating the monetary value of the disease, as well as the ethical perils and the uncertainties in the combined formulation.
Do the alternative formulations systematically lead to different control policies than the OC policy derived by solving the combined formulation? Our analysis shows that, in the special (and unrealistic) case of perfect information—i.e., when the monetary cost of the disease can be derived with sufficient precision—budget or epidemiological constraints can be set so that the OC policy resulting from the alternative formulations is the same as the one computed with the combined formulation (3). Specifically, we solved the OC problem for the first alternative formulation (i.e., minimize new infections under a budget constraint (4)) by setting the budget constraint to the intervention costs resulting from solving the combined OC problem (3). As shown in Table and Fig. , the OC policy and health outcomes (such as the maximum number of infected individuals, the number of deaths, and the number of vaccinations) are the same for the combined and the alternative formulations since the intervention cost budget is the same for both. We see the same for the second alternative formulation (i.e., minimize intervention costs to achieve a desired epidemiological outcome (5)) when the epidemiological outcome is constrained to the value derived by solving the combined OC problem (see Table and Fig. ).
The simulations shown in Fig. 3 (corresponding to Tables 4 and ) and Fig. 4 (corresponding to Tables 5 and ) were performed using the same \(\beta _I\), \(\beta _W\), \(\mu \), \(\xi \), and \(\delta \) parameters, initial conditions, and time horizon as in Table 2 and setting \(\gamma = 0.25\), \(A_n=1\), \(B=100\), \(C=0.0813\), \(v_{\max }=0.03\).
Then what advantage do the alternative formulations provide? As commented above, in the majority of cases, the per-capita cost of disease is not known and cannot be estimated with any degree of confidence, especially in the case of fatal or debilitating chronic diseases. Therefore, it is more meaningful and less debatable from the perspective of health equity to frame the optimization problem by explicitly acknowledging the fundamental reality any decision maker has to face. That is, to operate in conditions of limited budget, and/or, to have ethical, political, practical or humanitarian reasons to achieve a desired public health outcome. The alternative formulations allow us to incorporate these considerations much more straightforwardly than the combined formulation does. For instance, in the last two columns of Table 6 we report the breakdown for costs and epidemiological outcomes associated with a 50% budget decrease or a 25% increase, respectively, compared to the intervention cost found in the combined OC solution. The last two columns of Table 7 report the breakdown of the same information when the epidemiological target is 8% lower or 25% higher than the baseline number of new infections (as derived from the combined OC solution).
Note that, in the simulation results shown in Tables 6 and 7, the maximum number of new infections does not change between the different budget and epidemiological constraints. This is because the peak of infections in those simulations occurs early in the simulation and the optimal controls in all of the cases shown are at the maximum control value from the beginning of the simulation until after the peak of infection. Therefore, in these cases, the maximum number of infected is not reduced by increasing budget or setting a more ambitious epidemiological goal. To reduce the maximum number of infected individuals in these simulations, an increase in the maximum vaccination rate is required. An example of a simulation with a different maximum number of infected individuals can be seen in Fig. 1, where a constant control is applied at a lower maximum vaccination rate than the combined optimal control case. Therefore the constant control case displays an increased maximum number of infected individuals.
A sensitivity analysis can be conducted using alternative formulation 1 (minimizing new infections under a budget constraint (4)) by systematically exploring changes in health outcomes of interest over a wide range of budget constraints (see Fig. ). Our analysis shows that the relationship between budget and health outcomes are, for this model, approximately linear until one reaches the budget that uses the maximum vaccination rate over the entire simulation time. The analysis reveals that above this level, what is limiting is not vaccination budget but, rather, vaccination capacity (for instance, the number of vaccination centers). Therefore, above this limit, further health benefits could be achieved by a structural improvement of the vaccination system.
For alternative formulation 2 (minimizing intervention cost to achieve a desired epidemiological outcome (5)), the sensitivity analysis shows that the total intervention cost linearly increases/decreases, respectively, with more ambitious public health targets (percentage reduction in new cases with respect to the no control case—Fig. ). The maximum value of new daily infections drops linearly with more ambitious epidemiological outcome but levels off, in this case, when the required reduction in the cumulative number of cases exceed ca. 30% of that at baseline (no intervention).
Simulations related to the optimal control formulations minimizing disease burden instead of new infections (2) can be found in “Appendix B”.
5 Discussion and Conclusions
Using a cholera model as a prototypical example of environmentally related infectious disease, we have illustrated the benefit of using time-varying controls to achieve the goals and incorporate the constraints of disease management. We have proposed valuable alternatives to the frequently-used objective functional (the minimization of the sum of the total number of new cases (or total disease burden) and the cost of management actions) and showed the advantages of these alternatives in terms of outcomes (in costs, new cases and/or burden levels). Depending on the situation, one should consider minimizing the number of new cases given a fixed intervention budget (4) or minimizing the intervention costs given a fixed epidemiological constraints (such as keeping the total number of new cases under a fixed level (5)). These formulations can be valuable to apply to models of other diseases, such as schistosomiasis (Sokolow et al. 2015), West Nile virus (Abdelrazec et al. 2016), and Zika virus (Valega-Mackenzie and Ríos-Soto 2018; Miyaoka et al. 2019). We also note that other types of budget constraints, like time-varying upper bounds on costs or subsidies, may be interesting to consider; the budget for each year could be different and could have the option of rolling over each year (Drohan et al. 2021; Rowthorn et al. 2009). If the values of specific epidemiological variables such as the fraction of the population with acquired or vaccine-induced immunity near the end of reference time horizon are important, it is possible to reformulate the objective functional to explicitly account for those variables (with terms at the final time in the objective functional).
We analyzed changes in health outcomes of interest over a range of budget constraints. For the specific model and the parameterization in this paper, this relationship between budget and health outcomes was found to be approximately linear until one reaches the budget level that allows the maximum vaccination rate to be used over the entire simulation time. The upper bound on the vaccination rate may be linked to the capacity of the health-care infrastructure (that is, the amount of vaccinations that can be completed per unit time given the current delivery system with a fixed number of workers). Similar results were obtained for changes in intervention costs over a range of fixed health outcomes. Overall, our research outlines the power of reformulating combined optimal control problems and using health outcomes measured in metrics with which public health decision makers are more familiar.
In some of our results, the differences in vaccination effort and epidemiological outcome merely reflect differences in the economic strengths of nations, and have little to do with the much less tangible value of the quality of life lost due to infection. This problem can be partially addressed by conducting a sensitivity analysis of how the OC solution changes as a function of the unit cost of disease, if such costs are known (see one such analysis in Bolzoni et al. 2014).
While the proposed reformulation of the optimization functional addresses some problems of the combined formulation and allows us to avoid the practical and ethical challenges of monetizing human health, it does not solve other problems inherent in the evaluation of health outcome and equity issues. For instance, Weintraub and Cohen (2009) have clearly outlined several of these limitations for the cost-effectiveness analysis that apply also to optimal control problems in general and, specifically, to the newly proposed formulations. Though monetizing the value of human health comes with uncertainties, estimating intervention costs and translating those costs into model parameters also present challenges. In addition, for practical and computational reasons, mechanistic models tend to simplify the complexity of real-world social systems, and optimal control theory cannot overcome the limitations deriving from oversimplified models that lack relevant details (such as spatial structure or social interaction) or are not properly parameterized because of a lack of the necessary data.
Without dismissing the power of OC applications, we are aware that, during an actual epidemic outbreak and in practical terms, simplified protocols with constant intervention rates might work equally well, if not better, than protocols that require continuous adjustment of the intervention effort. This may be especially true when it is difficult to monitor the health state of the population, testing capacity is low, or there are delays in reporting infected cases in remote, rural areas and/or in countries with limited health care and reporting systems. An optimal control can frequently be adjusted (every few months or so) to give a close approximate control that is more feasible to implement (Morris et al. 2021; Demir and Lenhart 2020).
OC applications to the control of communicable diseases often remains so technical that no actual use of them becomes realized as a guideline for action. While some barriers due to the inherent complexity of OC theory cannot be overcome, there is an untapped potential that should be leveraged to inform the decision making process on pressing public health problems. In spite of the limitations listed above, we believe that the scientific community proficient in optimal control theory could easily formulate optimization problems for the control of communicable diseases with our proposed approach. It would reflect more closely the framework in which public health decisions are usually taken in the full spirit of contemporary user-inspired research, as argued in National Research Council (2008) and Stoles (1997).
Data Availability
There are no associated data with this manuscript.
References
Abdelrazec A, Lenhart S, Zhu H (2016) Optimal control of West Nile virus in mosquito, birds and humans with seasonality. Can Appl Math Q 23(3):355–375
Agusto F, Pantha B, Elmojtaba I (2020) Optimal control applied to a visceral leishmaniasis model. Electron J Differ Equ 80:1–24
Ali M, Nelson AR, Lopez AL, Sack DA (2015) Updated global burden of cholera in endemic countries. PLoS Negl Trop Dis 9(6):1–13
Angulo MT, Castaños F, Moreno-Morton R, Velasco-Hernández JX, Moreno JA (2021) A simple criterion to design optimal non-pharmaceutical interventions for mitigating epidemic outbreaks. J R Soc Interface 18(178):20200803
Asano E, Gross LJ, Lenhart S, Real LA (2008) Optimal control of vaccine distribution in a rabies metapopulation model. Math Biosci Eng 5(2):219–238
Assembly UNG (2021) Political declaration on HIV and AIDS: ending inequalities and getting on track to end AIDS by 2030
Baltussen R (2006) Priority setting of public spending in developing countries: do not try to do everything for everybody. Health Policy 78(2–3):149–156
Bardi M, Capuzzo-Dolcetta I (1997) Optimal control and viscosity solutions of Hamilton–Jacobi–Bellman equations. Systems and control: foundations and applications. Birkhäuser Boston Inc
Bjørnstad ON, Shea K, Krzywinski M, Altman N (2020) The SEIRS model for infectious disease dynamics. Nat Methods 17(6):557–558
Bliman PA, Duprez M (2021) How best can finite-time social distancing reduce epidemic final size? J Theor Biol 21(511):110557
Bolzoni L, Tessoni V, Groppi MV, De Leo GA (2014) React or wait: which optimal culling strategy to control infectious diseases in wildlife. J Math Biol 69:1001–1025
Bolzoni L, Bonacini E, Soresina C, Groppi M (2017) Time-optimal control strategies in SIR epidemic models. Math Biosci 292:86–96
Bolzoni L, Bonacini E, Della Marca R, Groppi M (2019) Optimal control of epidemic size and duration with limited resources. Math Biosci 315(108232):1–12
Brock DW, Wikler D (2006) Ethical issues in resource allocation, research, and new product development, Chap 4. In: Disease control priorities in developing countries, 2nd edn, vol 2. Oxford University Press, pp 259–270
Bubar KM, Reinholt K, Kissler SM, Lipsitch M, Cobey S, Grad YH, Larremore DB (2021) Model-informed COVID-19 vaccine prioritization strategies by age and serostatus. Science 371(6532):916–921
Campo-Duarte DE, Vasilieva O, Cardona-Salgado D, Svinin M (2018) Optimal control approach for establishing \(\mathit{wMelPop}\)\(\mathit{Wolbachia}\) infection among wild \(\mathit{Aedes}\)\(\mathit{aegypti}\) populations. J Math Biol 70(7):1907–1950
Castonguay FM, Sokolow SH, De Leo GA, Sanchirico JN (2020) Cost-effectiveness of combining drug and environmental treatments for environmentally transmitted diseases. In: Proceedings of the Royal Society B, vol 287. https://doi.org/10.1098/rspb.2020.0966
Codeco CT (2001) Endemic and epidemic dynamics of cholera: the role of the aquatic reservoir. BMC Infect Dis 1:1–14
Demir M, Lenhart S (2020) Optimal sustainable fishery management of the Black Sea anchovy with food chain modeling framework. Nat Resour Model 33(2):1–29
Drohan SE, Levin SA, Grenfell B, Laxminarayanc R (2021) Incentivizing hospital infection control. PNAS 108(34):14366–14370
Hager WW, Zhang H (2016) An active set algorithm for nonlinear optimization with polyhedral constraints. Sci China Math 59(8):1525–1542
Hansen E, Day T (2011) Optimal control of epidemics with limited resources. J Math Biol 62:423–451
Jiang S, Chen Z, Wu J, Zang X, Jiang Y (2020) Addressing methodological and ethical issues in practicing health economic evaluation in China. J Glob Health 10(2):020322
Jung E, Iwami S, Takeuchi Y, Jo T (2009) Optimal control strategy for prevention of avian influenza pandemic. J Theor Biol 260(2):220–229
Kamien MI, Schwartz NL (1991) Dynamic optimization: the calculus of variations and optimal control in economics and management, 2nd edn. North-Holland, Amsterdam
Keeling MJ, Rohani P (2008) Modeling infectious diseases in humans and animals. Princeton University Press, Princeton
Kelly MR, Tien JH, Eisenberg MC, Lenhart S (2016) The impact of spatial arrangements on epidemic disease dynamics and intervention strategies. J Biol Dyn 10(1):222–249
Laguzet L, Turinici G (2015) Global optimal vaccination in the SIR model: properties of the value function and application to cost-effectiveness analysis. Math Biosci 263:180–197
Ledzewicz U, Schättler H (2011) On optimal singular controls for a general SIR—model with vaccination and treatment. Discret Contin Dyn Syst Ser A 981 –990
Lee E, Chao DL, Lemaitre J, Matrajt L, Pasetto D, Perez-Saez J, Finger F, Rinaldo A, Sugimoto J, Halloran M, Longini I Jr, Ternier R, Vissieres K, Azman AS, Lessler J, Ivers L (2020a) Achieving coordinated national immunity and cholera elimination in Haiti through vaccination: a modelling study. Lancet 8:1–9
Lee S, Baek O, Melara L (2020b) Resource allocation in two-patch epidemic model with state-dependent dispersal behaviors using optimal control. Processes 8(9):1–19
Legros D (2018) Global cholera epidemiology: opportunities to reduce the burden of cholera by 2030. J Infect Dis 218(suppl 3):S137–S140
Lenhart S, Workman J (2007) Optimal control applied to biological models. Chapman and Hall/CRC, New York
Matrajt L, Eaton J, Leung T, Brown ER (2020) Vaccine optimization for COVID-19: Who to vaccinate first? Sci Adv 7(6):1–11
Melbourne EL (2011) Cholera: Symptoms, diagnosis and treatment. Nova Science Publishers
Miclo L, Spiro D, Weibull J (2022) Optimal epidemic suppression under an ICU constraint: an analytical solution. J Math Econ 101:1–17
Miller I, Becker A, Grenfell B, Metcalf C (2020) Disease and healthcare burden of COVID-19 in the United States. Nat Med 26:1212–1217
Miyaoka T, Lenhart S, Meyer JF (2019) Optimal control of vaccination in a vector-borne reaction-diffusion model applied to Zika virus. J Math Biol 79(3):1077–1104
Morris DH, Rossine FW, Plotkin JB, Levin SA (2021) Optimal, near-optimal, and robust epidemic control. Commun Phys 4:1–8
Morrow RH, Byrant JH (1995) Health policy approaches to measuring and valuing human life: conceptual and ethical issues. Am J Public Health 85(10):1356–1360
Murray C, Vos T, Lozano R et al (2012) Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 380:2197–223
National Research Council (2008) Research and networks for decision support in the NOAA sectoral applications research program. The National Academies Press
Naveed C, Cipriano LE, Enns EA (2019) Dynamics of drug resistance: optimal control of an infectious disease. Oper Res 67(3):619–650
Neilan RM, Schaefer E, Gaff H, Fister KR, Lenhart S (2010) Modeling optimal intervention strategies for cholera. Bull Math Biol 1(4):379–393
Ntuli MM (2020) Ending the neglect to attain the sustainable development goals: a road map for neglected tropical diseases 2021–2030 for neglected tropical diseases. World Health Organization
Ozawa S, Yemeke TT, Thompson JM (2021) Systematic review of the incremental costs of interventions that increase immunization coverage vaccine. Vaccine 36(25):3641–3649
Patterson MA, Rao AV (2014) GPOPS-II: a MATLAB software for solving multiple-phase optimal control problems using Hp-adaptive gaussian quadrature collocation methods and sparse nonlinear programming. ACM Trans Math Softw 41(1):1–37
Piunovskiy A, Clancy D (2008) An explicit optimal intervention policy for a deterministic epidemic model. Optim Control Appl Methods 29(6):413–428
Pontryagin LS, Boltyanskii VG, Gamkrelize RV, Mishchenko EF (1962) The mathematical theory of optimal processes. Wiley, London
Rowthorn RA, Laxminarayan R, Gilligan CA (2009) Optimal control of epidemics in metapopulations. J R Soc Interface 6:1135–1144
Rutstein SE, Price JT, Rosenberg NE, Rennie SM, Biddle AK, Miller WC (2017) Hidden Costs: the ethics of cost-effectiveness analyses for health interventions in resource-limited settings. Glob Public Health 12(10):1269–1281
Rypdal K, Bianchi FM, Rypdal M (2020) Intervention fatigue is the primary cause of strong secondary waves in the COVID-19 pandemic. Int J Environ Res Public Health 17:1–17
Sallam M, Al-Sanafi M, Sallam M (2022) A global map of COVID-19 vaccine acceptance rates per country: an updated concise narrative review. J Multidiscip Healthc 15:21–45
Schättler H, Ledzewicz U (2012) Geometric optimal control theory, methods and examples. Springer, Berlin
Scovronick N, Ferranna M, Dennig F, Budolfson M (2020) Valuing health impacts in climate policy: ethical issues and economic challenges. Health Aff 39(12):2105–2112
Sèvére K, Rouzier V, Anglade SB, Bertil C, Joseph P, Deroncelay A, Mabou MM, Wright PF, Guillaume FD, Pape JW (2016) Effectiveness of oral Cholera vaccine in Haiti 37 month follow up. Am Soc Trop Med Hyg 94(5):1136–1142
Sharomi O, Malik T (2017) Optimal control in epidemiology. Ann Oper Res 251(1):55–71
Smith DL, Battle KE, Hay SI, Barker CM, Scott TW, McKenzie FE (2012) Ross, Macdonald, and a theory for the dynamics and control of mosquito-transmitted pathogens. PLoS Pathog 8(4):1–13
Sokolow S, Huttinger E, Jouanard N, Hsieh M, Lafferty K, Kuris A, Riveau G, Senghor S, Thiam C, N’Diaye A, Faye D, De Leo G (2015) Reduced transmission of human schistosomiasis after restoration of a native river prawn that preys on the snail intermediate host. PNAS 112(31):9650–9655
Stoles DE (1997) Pasteur’s quadrant: basic science and technological innovation. Brookings Institution
The World Bank (2021) GFP per capita for different countries 2021. https://data.worldbank.org/indicator/ny.gdp.pcap.cd
Tien JH, Earn DJD (2010) Multiple transmission pathways and disease dynamics in a waterborne pathogen model. Bull Math Biol 72(6):1506–1533
Valega-Mackenzie W, Ríos-Soto K (2018) Can vaccination save a Zika virus epidemic? Bull Math Biol 80(3):598–625
Weintraub WS, Cohen DJ (2009) The limits of cost-effectiveness analysis. Circ Cardiovasc Qual Outcomes 2(1):55–58
Zarocostas J (2017) Cholera outbreak in Haiti-from 2010 to today. Lancet 389(10086):2274–2275
Acknowledgements
This work was started as a part of the working group on “Optimal Control of Neglected Tropical Diseases” at the National Institute for Mathematical and Biological Synthesis, supported by the National Science Foundation through NSF Award DBI-1300426, with additional support from The University of Tennessee, Knoxville. GADL and SHS were partially supported by NSF Grant ICER-2024383 and DEB-2011179 and, along with JNS, also by NSF Grant 1414102.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A: Hamiltonians, Adjoints, and Optimal Control Characterizations
1.1 Appendix A.1: Combined Objective Functional with Disease Burden and Cost
The objective functional combining disease burden and intervention cost is formulated as
where the control set is
The Hamiltonian is then
with the following adjoints and boundary conditions:
We can then characterize the optimal control, using this optimality condition, on the interior of the control set.
Note \( \frac{\partial ^2 H}{\partial v ^2} = 2B>0\), since \(B>0\). Therefore, using the control bounds gives
1.2 Appendix A.2: Combined Objective Functional with New Infections and Cost
The objective functional combining new infections and intervention costs is formulated as
where the control set V is the same as above.
The Hamiltonian is then
with the following adjoints and boundary conditions:
We can then characterize the optimal control, using this optimality condition, on the interior of the control set.
Note \(\frac{\partial ^2 H}{\partial v ^2} = 2B>0\), since \(B>0\). Therefore, we find that
1.3 Appendix A.3: Minimize Disease Burden under Fixed Budget
Given a fixed budget G, the cost of the control interventions, we define the control sets:
and
where S is the susceptible compartment of the population corresponding to the control v. The alternate objective functional to minimize the disease burden is defined as
To show the process of obtaining the adjoint functions and the optimal control characterization, we use Pontryagin’s Maximum Principle. To handle the budget constraint, we introduce a fifth state variable, \(x_5\), that satisfies
with the boundary conditions \(x_5(0)=0\) and \(x_5(T)=G\).
The Hamiltonian is
with the following adjoint differential equations and boundary conditions
Note \(\lambda _5\) takes no boundary condition since \(x_5\) takes two boundary conditions.
We can then characterize the optimal control, using this optimality condition, on the interior of the control set,
and then obtain
1.4 Appendix A.4: Minimize Intervention Costs with Fixed New Infections
Given an upper bound P on the number of new infections, we define the control set \(F_2\) as
where S is the susceptible compartment, I the infected compartment, and W the environmental contamination compartment of the population corresponding to the control v.
The alternative objective functional minimizing cost is defined as
To handle the constraint on the new cases, we introduce a new state variable, \(x_5\) with
The Hamiltonian is then
with the following adjoints and tranversality conditions:
Note that \(\lambda _5\) takes no boundary condition because \(x_5\) takes two boundary conditions, \(x_5(0)=0\) and \(x_5(T)=P\).
We can then characterize the optimal control, using this optimality condition, on the interior of the control set.
Note that \( \frac{\partial ^2 H}{\partial v ^2} = 2B>0\), since \(B>0\). Therefore, we find that
1.5 Appendix A.5: Minimize New Infections under Fixed Budget
Given a budget constraint G, we use the same control set as “Appendix A.3”,
The alternative objective functional minimizing the new infections is then defined as
To handle the budget constraint, we introduce a fifth state variable, \(x_5\), defined by
with the boundary conditions \(x_5(0)=0\) and \(x_5(T)=G\).
The Hamiltonian is then
with the following adjoints and boundary conditions:
Note \(\lambda _5\) takes no boundary condition since \(x_5\) takes two boundary conditions.
We can then characterize the optimal control, using this optimality condition, on the interior of the control set.
Note that \( \frac{\partial ^2 H}{\partial v ^2} = 2B \lambda _5>0\) if \(\lambda _5>0\).
Therefore, we find that
Appendix B: Combined OC: Minimize the Sum of Disease Burden and the Costs of Interventions
Minimizing the cost of interventions and the integral of the infected compartment, scaled appropriately to represent the cost of disease burden, is commonly seen as the goal in management of disease models. The case of a combined objective functional with burden (Eq. 2) is illustrated here in Fig. with model parameters listed in the caption of Table . In that table, one can see the positive effects of implementing the optimal control on the disease are striking. Since the number of susceptible individuals has become low about half way through the time interval, the constant control (set at the maximum control value, 0.04) has similar effects to the optimal control resulting from the combined objective functional (1% reduction in disease burden), but has a slightly higher intervention cost compared to the optimal control (11% increase). These simulations show that considering a time varying control allows for a better result (i.e. a smaller objective functional) than a constant control at the maximum will allow, by 1.4% in this case.
Using the parameters in Table and a combined objective functional with burden (Eq. 2), we compare the optimal control to a constant control with same cost and to the no control case. The plots shown in Fig. do not show much difference in the dynamics of the compartments when comparing the optimal control case and a constant control of 0.13. The two controls result in similar intervention costs but have 10% difference in the overall combined cost.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Igoe, M., Casagrandi, R., Gatto, M. et al. Reframing Optimal Control Problems for Infectious Disease Management in Low-Income Countries. Bull Math Biol 85, 31 (2023). https://doi.org/10.1007/s11538-023-01137-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11538-023-01137-4