Q&A: Mathematical Modeling and Health Policy with C. Jessica Metcalf

Friday, Nov 7, 2014

C. Jessica MetcalfMathematical modeling has tremendous potential for developing and guiding public health initiatives to address some of the world's most pressing public health issues through cost-effective means. Models can have the power to accurately predict the progression of infectious diseases and potential epidemic outcomes.

Such evidence gives policy makers the tools they need to determine the best public health interventions today and in the future. But like any powerful tool, modeling comes with limitations. Addressing these barriers – along with strengthening the connections between analysts and policy makers – is vital for communicating science in clear and meaningful ways.

This is the crux of an argument presented by C. Jessica Metcalf, an assistant professor of ecology and evolutionary biology and public affairs at Princeton University's Woodrow Wilson School of Public and International Affairs, in the latest issue of Epidemics. Together with John Edmunds from the London School of Hygiene & Tropical Medicine and Justin Lessler from Johns Hopkins University, Metcalf addresses the challenges posed by modeling and provides solutions for how analysts and policy makers can better work together to interpret results and communicate uncertainty. Below, Metcalf discusses her analysis by explaining the six modeling challenges outlined in her paper.

Challenge 1: Communicating the limits of modeling

Metcalf: Public health practitioners may want clear, quantitative statements of the future impact of health threats. But unfortunately, models are rarely in the position to provide this. As scientists, modelers are often uncomfortable with producing multiple scenarios, as these are often based on poorly supported assumptions. Instead, we feel comfortable with a single scenario based on the best evidence. At the same time, policy makers often want a clear, single number to use for concrete actions and may face criticism if this number is wrong. So it can be a double-edge sword; both sides have pressure to come up with numbers that can be perceived as predictions. Even if the model performs well in contrasting scenarios, the next challenge is communicating the limits of the model's performance. For example, when populations reach small numbers, deterministic models may fail spectacularly.

Challenge 2: Maintaining the value of models in the face of long-time horizons

Metcalf: Models often make predictions that play out over decades. This is because the natural history and pathogenesis of diseases may take generations for the effects of interventions to be seen. Long timeframes make it really hard to validate predictions, seriously affecting what modelers can and can't say. Modelers must identify approaches to clarify the value of such predictions despite their likely inaccuracies and highlight predictions that must be revisited in the face of situational changes.

Challenge 3: Usefully deploying modeling in the context of 'black swans'

Metcalf: Models can be used to predict low probability, high-impact events, which we call 'black swans.' A classic example of this is a rare, lethal pandemic. Take, for example, the influenza pandemic of the last decade. Most models for pandemic preparedness were centered on a black swan event – the emergence of an extremely virulent pandemic influenza strain. When the awaited pandemic occurred in 2009, it proved to be mild and had already spread widely in Mexico by the time it was detected. As a result, governments and modelers were subject to criticism for unnecessarily stockpiling antivirals and causing panic – even though this might have been the best possible plan for a rare catastrophic event – that has happily not yet occurred. By definition, little is known regarding what a black swan event will be. Therefore, appropriately helping in planning while not fuelling misconceptions of risk is a difficult challenge. Values and perception of values are a crucial consideration in this dialogue. Both modelers and policy makers should work hard to frame rare, probabilistic outcomes in terms that are desirable.

Challenge 4: Integrating modelers and model-building into the policy process

Metcalf: Reducing the divide between policy makers and modelers is essential going forward. An environment should be created in which modelers can successfully communicate how models can and should be used. In turn, policy makers should successfully communicate questions they want answers to. Creating an ongoing interaction between modelers and policy makers will contribute greatly to this goal. Ongoing interactions create the possibility for more effective use of both scenario-based and predictive models.

Challenge 5: Economic analysis and decision support

Metcalf: In public health policy, the real question is simple: "What should I do?" A cost and benefits analysis of possible policies to guide decision-making is required to try and answer this. Weighing costs and benefits is often used in economics, but this dimension is generally lacking in most infectious disease models. This area is ripe for development and will play an essential role in making models more useful going forward. It's very important that policy makers understand that if they want models that give "an answer" to a policy decision, they must identify priorities and identify costs and benefits important to their decision so that modelers and economist can work together.

Challenge 6: Creating a cycle where results inform decisions and vice versa

Metcalf: Each of the challenges we present have both a technical and communication component. Ultimately, communication issues are best resolved by creating long-term relationships that foster understanding and trust. The idealized relationship is not merely a cultural challenge, but also requires technical innovation. There are many benefits to an ongoing cycle of decision and reanalysis in terms of how to interpret results and how those might be relevant to policy decisions.

Full descriptions of these challenges can be found here: http://wws.princeton.edu/news-and-events/news/item/qa-mathematical-modeling-and-health-policy#sthash.zG8h33KE.dpuf