AMA Journal of Ethics^®

Abstract

Novel interventions that are effective and safe but costly suggest the importance of questions about value, accessibility, and affordability. Economic evaluation is one useful tool that health care systems draw upon to help make investment decisions and set priorities. But translating results from economic evaluations into actionable insights about a novel intervention’s value remains elusive, given ethical and practical complexities. This article illuminates 5 key factors to consider in interpretating economic evaluations of novel interventions: the health care decision problem, the design and structure of the mathematical model, characterization of uncertainty, awareness of health outcome measure limitations, and the contrast between optimal decision-making conditions and real-world decision-making conditions.

Economic Evaluation in Health Care

In 2019, the US Food and Drug Administration approved the drug tafamidis as a breakthrough therapy for transthyretin amyloid cardiomyopathy (ATTR-CM) based on a pivotal clinical trial that reported a reduction in all-cause mortality and cardiovascular-associated hospitalizations in patients taking the drug once daily compared with patients who received a placebo over a 30-month follow-up period.¹ However, the manufacturer’s wholesale acquisition cost of tafamidis was $225 000 for 1 year of therapy, making it the most expensive cardiovascular disease drug launched in the United States.¹ Considering the expected substantial clinical benefit of tafamidis, a key question is if it’s worth paying this price. Which principles and methods decision makers adhere to in answering this question and in allocating health care resources depends on their preferences about efficiency, equity, universality, and choice.

Economic evaluation is one tool that is widely used to help answer this question.² In essence, economic evaluation aims to characterize the efficiency of health care interventions: How can finite resources be used to maximize the health gains obtained from them?³ It provides a structured approach to measuring and comparing the health outcomes and costs of competing alternative interventions over time and across populations⁴; to accounting for individual, payer, or societal preferences; and to characterizing the inherent uncertainty in model choices and in the precision of model inputs. The results of economic analyses inform decision makers of interventions that could improve the return on resources expended in health care.

An independent economic evaluation estimated that lifetime treatment with tafamidis compared to usual care for ATTR-CM had an incremental cost-effectiveness ratio (ICER) of $880 000 per quality-adjusted life-year (QALY) gained. The price would have to be reduced by 92% to make it cost-effective at the conventional $100 000 per QALY threshold.⁵ How does this conclusion translate into actionable insights on value and priority setting in health care? To understand this process, we will illustrate 5 key factors that should be considered when interpreting economic evaluations to support decision making in health care. First, economic evaluations are based on underlying value judgments about the goal of resource allocation, which determine how such evaluations can be used to address health care decision-making problems. Second, the design and structure of the economic model used will significantly influence the results. Third, the structure of the model and inputs used to populate it create uncertainty that should be characterized. Fourth, generic measures of health, such as QALYs, have limitations that should be understood when used as an outcome measure. Fifth, there are other factors relevant to health care not accounted for in economic evaluations that should be considered by decision makers.

Understanding the Decision Problem

The theoretical foundation of economic evaluation stems from economic welfare theory, which posits that resource allocation decisions should be assessed based on whether net social welfare is increased.⁶ Applied to health care, this measure of efficiency—known as allocative efficiency—assesses whether the mix of resources being allocated to health care vs the rest of the economy maximizes benefit.⁷ Cost-benefit analysis (CBA) can be used to determine whether an intervention provides an overall net welfare gain and to compare this net gain with that from alternative interventions (see Table), including in other sectors of the economy; implementing the intervention with the greatest net gain will increase efficiency.³ However, CBA is not widely used in health care due to conceptual difficulties in placing monetary value on health or life. The methods widely used in economic evaluation in health care are cost-effectiveness analysis (CEA) and cost-utility analysis (CUA) (see Table), which help assess interventions’ productive efficiency in terms of health outcomes gained for a given cost. When interpreting the results from a CEA or CUA, it is important to keep in mind that these analyses cannot indicate whether the amount of health benefits gained from an intervention truly aligns with societal preferences because they cannot compare health care interventions to interventions in other sectors of the economy. In CEA or CUA, a new intervention that is cheaper and more effective than alternatives is deemed more efficient. However, like tafamidis, new interventions that are more effective than alternatives generally cost more, in which case, the decision maker must consider the trade-off between health gained and higher cost compared to alternative interventions. The acceptability of this trade-off, given competing priorities, is a core problem faced by decision makers.

Structuring the Decision

The intuition behind using models to inform economic evaluation is that the consequences of diseases might affect people over several years before manifesting as health outcomes, during which time costs of care accrue. For example, an individual diagnosed with heart failure may experience declining quality of life and hospitalizations and die from cardiovascular or noncardiovascular-related causes. A decision maker would be keen to understand how an intervention, such as tafamidis, would alter these disease events and their cost implications, although some stakeholders, such as insurers, might not be interested in long-term outcomes due to a greater focus on short-term budgetary impact assessment. Modeling enables evidence of the consequences and costs of a disease to be combined and extrapolated over time—often the entire lifetime of a person.⁸

Common instruments used to measure health-related quality of life may not adequately characterize the nuanced and complex aspects of disabled health states. 

Models, however, are limited by modelers’ choices and by data availability. In designing the model, key considerations include the clinical pathways to be included in the model, the availability of data (eg, about the natural history of the disease), and the requisite computational complexity. In addition, the types of health benefits and costs included in the model should be appropriate to the perspective required by the decision maker—eg, the health care system or society. Finally, an important consideration is the representativeness of the clinical data regarding health outcomes. There is evidence to suggest that people of color, women, people with disabilities, and marginalized groups are underrepresented in clinical trials, raising concerns about the applicability of trial findings to these populations.⁹ In addition, caution should be exercised when models are used to extrapolate outcomes beyond the period of a clinical trial. The evidence used should be appropriate and systematically chosen.

Characterizing Uncertainty

Uncertainty in economic evaluation stems from 2 sources: uncertainty relating to decisions about, and assumptions inherent in, the structure of the model and uncertainty concerning the precision of the inputs used in the model, such as costs, outcomes, and probabilities of disease events.¹⁰ Structural uncertainty can be explored by examining how alternative scenarios, such as other plausible disease states, events, and pathways affect the results. Parameter uncertainty could be characterized using one-way sensitivity analysis, which involves varying each parameter estimate independently and individually assessing the impact on model results. Additionally, multiway sensitivity analysis could be performed by simultaneously varying more than one parameter (eg, to evaluate model results in a best- or worst-case scenario). More advanced probabilistic approaches such as probabilistic sensitivity analysis (PSA), in which uncertainty is characterized by randomly sampling model input parameters (eg, costs and treatment effects) using Monte Carlo simulations,¹¹ are now standard practice in health economic evaluation. Characterizing uncertainty in this manner, rather than relying on a single point-estimate of cost-effectiveness, helps make economic evaluation more reliable. However, it is important that the full scope of the PSA be reported, such that all the relevant input parameters are assessed and made accessible to stakeholders.

Basing Decisions on QALYs

CUA, wherein the QALY is used as the health outcome measure, is the most widely used method of health economic evaluation. The QALY combines the quantity of life with health-related quality of life in a single generic composite measure to enable health outcomes to be compared across a range of diseases.¹²

There are ethical problems in using QALYs that require careful consideration. One central criticism in using QALYs to measure health outcomes is that life-years gained by people in full health through a preventive intervention will be considered more valuable than life-years gained by a person who is chronically ill or disabled, with the result that the cost-effectiveness of an intervention may not align with societal preferences.¹³ In addition, common instruments used to measure health-related quality of life, such as the EQ-5D questionnaire, may not adequately characterize the nuanced and complex aspects of disabled health states.¹⁴ Alternative metrics to address some of the limitations of QALYs have been developed, such as health years in total.¹⁵

Perhaps the most controversial aspect of CEA and CUA is the use of the willingness-to-pay (WTP) threshold—a predefined cost per QALY gained above which the intervention would not be considered cost-effective. In practice, it is difficult to accurately identify the outcomes and costs of all interventions in the health care system, so a customary WTP threshold for an additional QALY is assumed as a simplified decision rule. As a result, WTP threshold values have historically lacked an empirical foundation, although implicit thresholds are apparent from the approval decisions of health technology assessment (HTA) bodies.^16,17 For example, the HTA body in England does not officially identify a WTP threshold above which it would not recommend an intervention, but an intervention with an ICER per QALY gained above £30 000 is generally unlikely to be recommended unless there are other compelling reasons to do so.¹⁸ Similarly, the American College of Cardiology considers an intervention to be of high value if the ICER is less than  $50 000 per QALY gained and of low value if the ICER is greater than $150 000 per QALY gained.¹⁹ Therefore, thresholds should be considered as indicators of efficiency rather than prescriptive rules. There is likely to be broad consensus that an intervention that costs less than $50 000 per QALY gained will be reasonably efficient, but one that costs more than $150 000 will be less so, with exceptions depending on the specific circumstances of the intervention and target population.

Context

Economic evaluation should be viewed as one tool in a decision maker’s toolkit to aid pragmatic decision making. Maximizing health gained from available resources should not be the sole objective of a health care system. Other factors that a decision maker must take into account include equity concerns in the allocation of health care resources, need, and societal values.²⁰ For example, although the inverse association between socioeconomic status and health outcomes is well-known,²¹ economic evaluations generally do not provide information about trade-offs between cost-effectiveness and equity. Even within economic evaluation, countries differ in whether they accept QALYs as a measure of heath gain.²² Economic evaluation should be used and interpreted with a clear understanding of how it can help decision making. It inherently involves generalizations and simplifications, but these assumptions should be made transparent and explicit to ensure that the model used aligns with the requirements of the health care system, which ultimately should aid decision making that reflects societal values and shared priorities for health care.