For those of us working in health economics and outcomes research in the United States, I don’t think it is controversial to say that 2025 is shaping up to be a very uncertain year. The US government is making wholesale, unprecedented changes to almost every federal agency that touches research, regulation, payment, and the delivery of healthcare. Some impacts will be visible relatively quickly, while others may take years or even decades to fully manifest themselves. Tariffs that haven’t existed for half a century are going to impact supply chains for pharmaceutical and device manufacturers. Pharmacy Benefits Managers face regulatory scrutiny and bipartisan legislation that could fundamentally alter their business models. For those of us who work in large academic research institutions, let’s just say that things are particularly interesting right now.
How should HEOR groups strategically adapt in the face of these upheavals in our system? How can we even begin to think about them? Let’s start with a famous quote about uncertainty from a US Secretary of Defense that to me feels applicable:
“Reports that say that something hasn’t happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones.”
Donald Rumsfeld
I’ve been thinking about how this insight may be relevant to HEOR. One approach that HEOR groups could take is to say, “My core functions—patterns of care studies, budget impact models, cost-effectiveness analyses, real-world evidence studies—they are not going to change. So, let’s just keep doing what we are doing.” That’s probably true, but I am quite certain that people in the C-suites of the organizations that house HEOR groups (I’m including industry and academia) are thinking broadly in terms of how anticipated and unanticipated shocks to the healthcare ecosystem will shape the future of their organizations.
One of the great strengths of our discipline is our facility with uncertainty. Because it’s in our DNA, it’s worth thinking through how HEOR groups can add value to leaders facing very big questions in a rapidly changing and unpredictable world.
Rumsfeld’s quote has been framed into a conceptual model, referred to as the Awareness – Understanding Matrix1. It is complementary to the familiar SWOT (Strengths Weaknesses, Opportunities, Threats) approach used by leaders thinking though complex decisions2:
Let’s frame the matrix from the perspective of a CEO for major pharmaceutical company[a]:
[a] Equally applicable to CEOs of device companies, health insurers, PBMs, cancer centers (the list goes on).
Known knowns is the space HEOR specialists, particularly decision modelers, work well in. Using information from practice guidelines, clinical trials, observational studies and RWD, we create models that generate results about treatment patterns, budget impact and cost-effectiveness. Within those models we describe uncertainty in terms of what we can identify and characterize, e.g., parameter uncertainty, are made today. While many of us spend 95% of our time thinking about these issues, my guess is that they represent <10% of all uncertainty in the healthcare ecosystem. Let’s take our skills and apply them to the other quadrants.
Known unknowns contain recognized uncertainties where we don’t have adequate information. For example, the FDA is working on guidance for regulating AI-driven healthcare technologies. Other than knowing it is going to happen, we don’t know exactly what the regulations will look like. Other known unknowns are much broader and harder to characterize. The impact of the internet on human health is one of these. Known unknowns may represent 40% of all uncertainty in the ecosystem.
Unknown knowns are issues that we may implicitly know, but don’t consider in decisions. For example, we know that new health technologies may upend treatment paradigms, but we may not know what they are or how they will impact our business model. A good example is GLP-1 drugs. Bariatric surgeons were aware that drugmakers were working on weight loss drugs but maybe expected that they would be incremental compared to prior therapies and not fundamentally impact the populations they care for. The explosion in availability of XXL size clothing in secondhand clothing stores today is evidence of how these implicit assumptions can lead to problems.3 Perhaps this comprises 50% of uncertainty in the healthcare ecosystem.
Unknown unknowns represent “black swan” events that completely upend paradigms but are so rare or unimagined that they don’t enter routine thinking. The COVID 19 pandemic and its devastating impact on general vaccine acceptance may be one of those events. Unknown unknowns are generally considered the most worrisome (they were to Rumsfeld) because we don’t even prepare for them. Perhaps the most interesting square in the matrix, confronting unknown unknowns forces us to imagine the unimaginable—in essence, moving ideas from this box into the known unknown section of the matrix. Unknown unknowns are the realm of crackpots and visionaries. Unfortunately for us, they’re often difficult to distinguish.
How HEOR Can Help Organizations Move from Uncertainty to Resilience
I believe HEOR groups can go well beyond modeling to help their organizations better prepare for threats and opportunities under uncertainty. Keeping in mind that our space focuses on technology assessment, evidence generation, and the financing and delivery of care, here are a few examples of places where HEOR groups typically have a limited role but can add value:
| Areas for HEOR to provide valuable contributions |
| Purchases of external assets |
| Review of early product portfolios |
| Predicting or understanding the impact of systemwide changes in healthcare financing, coverage, or reimbursement |
| Shifts in societal attitudes (e.g., eroding trust in science and evidence) will impact that impact preferences for specific products and services |
| External technologies that could influence product development (e.g., AI) |
| Assessment of changes in pharmaceutical laws or regulations |
Given the limited bandwidths that most executives have, and the need to be clear and concise, a mix of qualitative and quantitative information is likely to be most effective.
Qualitative Approaches
Step 1: Map the knowns
One important thing to keep in mind is that HEOR knowns may be unknowns to others in the organization. Making sure that everyone is on the same page in terms of boxes on the Awareness – Understanding matrix is an important first step. For example, a core function of almost every HEOR group is characterizing patterns of care; i.e., understanding the treatment landscape and how it relates to practice guidelines. I am continually amazed at the limited level of understanding of variability in care for almost any condition in healthcare. Clinical expert key opinion leaders in particular seem to have a blind spot when it comes to imagining how (let alone why) the practicing community differs from them in approaching care. The tunnel vision seems to get transmitted up the leadership chain, often leading to a very skewed assessment of how patients with a particular condition are treated. HEOR groups add real value when they show variability as it is through real world evidence. They add even more when they convince leaders to understand and respect the preferences underlying the patterns of care that we observe.
“HEOR knowns may be unknowns to others in the organization”
As an example, the figure shows variability in treatment for 6,200 women with estrogen receptor–positive, HER2–negative metastatic breast cancer diagnosis.5 Overall, there were over 1,200 unique treatment sequences observed across the cohort. Compared to the NCCN practice guidelines for this condition, this is far more variability that what most experts recommend or even envision.
Step 2. List the unknowns
From a strategy perspective, it is critical to go through an exercise listing potential unknowns in quadrants 2-4 of the Awareness – Understanding matrix. HEOR groups–with their detailed knowledge of care patterns, facility with health technology assessment and analyzing patient and clinician survey data—can provide critical perspectives on pipeline products.
Step 3. Map the unknowns to the situation (e.g., target product profile)
So many conversations in pharma and device companies center around finding and developing products that can address an “unmet need.” This in my mind is a very narrow conception that adds tremendous risk to go/no-go decision making. HEOR can broaden perspectives by framing unmet need in the context of the larger number of fluidly changing factors that influence clinical decision making in the healthcare ecosystem.
For example, I once participated on an advisory board on a next generation anticoagulant purchased by a pharma company that was intended to replace direct oral anticoagulants (DOACs) for many common situations involving thrombosis. While there is room for improving on DOACs, replacing them may not be that easy. Physicians and patients are risk-averse in these situations. It took a long time for DOACs to replace warfarin and heparin, inferior but much less costly products that had the advantages of years of management experience. Coverage limitations by insurers for anticoagulants would likely be a major factor. Putting these issues into context, and seeing rather mediocre results from early-stage studies, I asked how much gain in “unmet need” (i.e., improved safety and efficacy) would offset practice inertia, known DOAC risks versus uncharacterized risks of the new product, and larger changes in healthcare that might influence their uptake? These were challenging questions that ultimately became moot when the product failed in two key pivotal trials (a known unknown!).
Step 4. Quantify judiciously, if at all
Sometimes it can be helpful to quantify and rank uncertainty. In the case of the Awareness – Understanding matrix, however, finding “priors” to gauge uncertainty could be difficult or impossible. Forcing arbitrary estimates or ranges around those estimates can be misleading or give a false sense of precision, with a result of increasing the potential of harm to organizations. A more helpful approach may be to assign “red-yellow-green” weights to attributes in the matrix based on their potential impact to the organization. With reds and maybe a few yellows in hand, analysts can dig into the unknowns, unpacking them into more manageable submatrices that further leaders’ understanding of factors that impact the situation.
On Humility
Yogi Berra famously and hilariously said that “It’s tough to make predictions, especially about the future.” Yes Yogi, it is. Great baseball players still fail to get a hit 2 out of 3 times they step up to bat. The greatest researchers’ batting averages are much worse. Living with risk and uncertainty is the nature of the pharma and device businesses. In a world where success is uncommon and resources are limited, creating Uncertainty-Understanding matrices around big decisions can be invaluable.
I use “the matrix” for framing discussions with clients when it makes sense. It’s worth noting that sometimes the best advice I can provide from this exercise is to tell the team to abandon a product or initiative and move on. I don’t view that as a negative, because it increases the possible number of tries at getting a hit in the future.[b]
[b] Alas, this often runs straight into the sunk cost fallacy, making me the most unpopular person in the room. See Richard Thaler, Toward a positive theory of consumer choice. J Econ Behav Organ. 1980;1:39-60.
HEOR groups can add a lot to product development conversations and even strategic decisions, but we will be one of many at the table. By turning unknowns into knowns and helping leaders form a wholistic view of remaining uncertainty, I believe that HEOR can provide uniquely valuable insights that can help leaders who operate in worlds well beyond our usual orbit.
– Scott Ramsey, MD, PhD
Senior Partner and Chief Medical Officer, Curta
Acknowledgement
David L. Veenstra, PharmD, PhD, provided important contributions to this article. David Campbell PharmD, MS, provided editorial and logistical support for its publication.
CONNECT WITH US
The experts at Curta are ready to discuss your evidence and access needs. For more information, please contact Scott at Scott.Ramsey@Curta.com.
References
- The Rumsfeld Matrix (Chapter 13) – The Climate Demon, Saravanan R., Cambridge University Press (2021) Online ISBN 9781009039604, DOI https://doi.org/10.1017/9781009039604.
- Versett. Strategy: How We Manage Uncertainty. Available at: https://versett.com/writing/how-we-manage-uncertainty.
- Binns J. How Ozempic is reshaping the resale market. Vogue Business. Available at: https://www.voguebusiness.com/-story/consumers/how-ozempic-is-reshaping-the-resale-market.
- Caswell-Jin JL, Callahan A, Purington N, et al. Treatment and Monitoring Variability in US Metastatic Breast Cancer Care. JCO Clin Cancer Inform. 2021;5:600-614. doi: 10.1200/CCI.21.00031.
- Peace, C. The risk matrix: uncertain results? Policy and Practice in Health and Safety. 2017;15(2):131-144. https://doi.org/10.1080/14773996.2017.1348571