Peter Gluckman: Reflections on the evidentiary-politics interface

Peter Gluckman is ISC President-elect, Chair of INGSA
and Director of Koi Tū: Centre for Informed Futures

While in general science has been accepted as a trusted institution in the context of the pandemic, that acceptance has not been universal: indeed the central tension is now being increasingly framed as a debate between those prioritizing economic re-opening and those prioritizing the continued social distancing called for by the science community.

What is striking is the diversity of decisions that have been made in different jurisdictions from early and total lockdown to the imposition of social restrictions late in the first phase, from intensive and early testing to little focus on contact tracing, from seeking elimination to seeking herd immunity.

While many of the lessons from this pandemic are generalisable across the evidence-to-policy interface, what distinguishes a crisis such as the pandemic is the much more direct connection between the provision of evidence and the political decisions that are made, decisions which can have extraordinary consequences for citizens and the economy.   They should weigh heavily on all that are involved.

The long-term consequences of these different approaches to the provision of science inputs will not be known for some time, perhaps not until vaccines are widely available. But the variability itself, despite well-distributed, if still uncertain information, compels a focus on the interface between evidence and political decisions in a way that is only possible given this unique situation.

INGSA is collating information using its highly collaborative policy making tracker to enable formal research in due course on how this interface has operated. Early data collected from a wide variety of global jurisdiction is already showing that there are at least seven dimensions that merit reflection and analysis. This essay does not attempt to resolve the issues but suggest areas which need to be interrogated.

1. What kind of evidence is presented or proactively sought?

There have been enormous variations in the range of ‘experts’ who sit at the table. In some countries, epidemiologists and virologists have understandably been the key actors, in others it is mathematicians and economists acting as modelers who have had much attention. But the extent to which countries relied on singular analysis has varied. The need for diverse disciplinary inputs is obvious but this has not always played out in practice. Social scientists have been heavily and formally involved in some countries such as Germany, but in others they are largely excluded. Yet much of the debate over various projection models has reflected this variation in the types of input. Indeed, we have seen models based on normative assumptions rather than relevant evidence. Models, figures and graphs are important heuristics. However, when they are presented without their assumptions clear, without sensitivity analysis and without providing any sense of uncertainty or probability, their projections can be called into question. In some cases, there has been a positioning of models as a description of reality, rather than recognising their limitations. This has led to confused public debate and to their misuse in advancing arguments. Arguments about herd immunity have been made in the absence of data as to whether immunity to the virus is long-lasting or not. Policy makers and politicians cannot be expected to be scientific referees. It is therefore important to consider how these different lines of evidence are brought together and integrated to inform decision making.

2. What processes and institutions are used to provide evidence?

Different jurisdictions have very different ways by which evidence is brought to the policy and political tables. The distinction between policy and politics collapses somewhat I emergencies, but given the length of the crisis it is inevitable that that fusion of interests diverges over time. In some countries science advisory ecosystems were well developed and in others they are essentially non-existent. There is little in the pandemic response to suggest that one model is superior to another. The variety of models have developed in different institutional, cultural and historical contexts. Such ecosystems have diverse components ranging from scientists and experts within ministries, especially public health agencies, through to academies, and science advisors.  All have had to bring domain experts to the table, many of whom have not necessarily had deep experience at the evidence -policy interface. A point of tension, discussed, below is the nature of the transmission mechanism from the expert community to the political and policy communities. The very public debate over who attended the United Kingdom’s SAGE meetings is an example of the issue.

3. What are the effective attributes of those individuals within science advisory institutions?

There have in general are two non-exclusive perspectives on science advisory ecosystems. One view is that advisory mechanisms need institutional framing – that is there a need for formal institutions and processes by which the policy community and science community interact and that it is these processes that are key and creates validity (see above). The other is that while the institutional framing needs to be there to validate who has access, but the key to a successful system is the skills of those engaged. The skills of evidence brokerage are particular and not necessarily possessed by every expert. The broker role is to take the expert evidence and transmit it in ways that are understood, are integrated, and that respect, identify and explain uncertainties. Brokers need to be trusted communicators to both the public and decision-makers. They need to avoid being seen as part of the political process. This raises the question – does evidence brokerage need to be a trained skill?

4. Formal and informal advice

Science advice occurs through two major routes. Formal processes of committees, panels, commissions and advisors and informal processes of discussion between key actors. Formal processes tend to be well documented and deliberative. They are ideal for dealing with complex analysis and data interpretation and for integrating knowledge across disciplines. They can be relatively transparent, at least in retrospect. Yet the reality of political decision-making is very much dependent on informal advice. These are the unscripted conversations between senior officials and politicians, of which scientists may be part.. Informal advice by its nature is essential at least when it involves those scientists in roles specifically designed to provide it, such as a senior government scientist or science advisor. Advice from these roles is common and highly influential. By its very nature it is more obscure and relies on the integrity and the skills of the advisor.   The relative role of these forms of advice in decisions made is an area meriting research. But policy makers and politicians may also reach outside to their informal connections for input. In the case of scientific input by such means, there are perhaps special obligations on the scientist, as discussed below

5. Integration of scientific evidence with the normative arguments of politics?

Even at this stage in the pandemic, there remain many scientific unknowns. Science advice mechanisms need not to be afraid of acknowledging these unknowns and uncertainties. Indeed their messaging is more trusted when such uncertainties are openly expressed. Ultimately, however, the decisions governments must make in the pandemic are based on tradeoffs that cannot be reduced to simplest equations. The politician will be adjudicating between health, social impacts and economic impacts (not to mention their interdependencies), expert opinion, public opinion and their own political destinies (and likewise, their interdependencies). No decision is made in the absence of political calculus, and the pandemic is no different. Clearly the evidence base is a key input but not the only input into those decisions. There has been very different calculus between countries that went into early rather than late lockdown and the contexts of decision being made as to when and how to exit social constraint vary accordingly, but also are heavily influenced by broader policy and political considerations. The interface thus between the expert input, the policy input and political decision-making is critical. The nature of that interface depends on the integrity of the science advice, the perceptions held by the political community, and the quality and independence of the policy community. The interface cannot function on the assumption of purely technocratic input but equally it cannot function in the absence of that input. It may well be important that scientific and technical debate is conducted independent of the policy community, but there are also arguments for why the policy community needs to understand the different perspectives and uncertainties.  Technical understandings are important but can get lost at the interface. The key issue in science advice must be to avoid ‘policy-driven evidence’ where the quality of the advice is undermined by a predetermined political lens.  The diplomatic skills of scientists acting at the interface become critical – it is much more complex that just speaking ‘truth to power”.

6. What ethical issues merit reflection?

There is a dearth of guidelines related to the role of science in emergencies and crises. The OECD undertook related work but did not address the specific issues of the conduct of scientists in emergencies.  The Foreign Ministry Science and Technology Advisory Network (FMSTAN) identified this as an area to be addressed and INGSA and ISC have started discussions on working together to consider the issues. Decisions made as to who is at the table, what disciplines are represented, how uncertainty is expressed, how to deal with conflicting views, and how to interface with the policy and political community and with the public all have ethical dimensions. Exploring these might lead to specific guidelines.  Equally there may be a need for guidance to those scientists who are not at the table. A key issue which has merged is that of transparency of advice and more particularly, clarity about whom is giving advice. This issue is key to ensuring the legitimacy and integrity of advice given.  Transparency is essential trust. While transparency can never be absolute or as timely as many would wish for on some issues, there is no justification for not being transparent as to whom is providing advice – yet that vagueness has been apparent in many countries.

7. The conduct of individual scientists?

The comments above highlight the many obligations on experts called upon to assist in an emergency and on the need for the brokerage to be conducted to a very high standard with high integrity. Scientists outside the formal advice system also have critical roles and obligations. Their views in public can be highly influential on both the public and the policy maker. Therefore, they should reflect on their public responsibilities and the professionalism of their conduct. Academic disputes inappropriately carried out in public can reduce trust in the evidence more generally, Yet, where there is a solid academic basis for disagreement the public has a right to know, The question is how and when such dialogue occurs. Further some scientists will want to give their views and feel involved, driven by other considerations (not least, the incentives of the system they trained in or work in). The media fuels the debate by seeking experts preferably with contrary or controversial views. Guidelines for scientific communication in emergencies may be needed.

8. Before and after

In due course, there will be extensive analysis of how different countries handled the crisis. There will likely be many commissions of inquiry and in some countries, this may put scientists and policy makers on the defensive, which could make the robust analysis of the responses more difficult. One major question will be the role of preemptive planning, the use of risk registers, pandemic planning exercises etc. Some of the issues related to these questions are discussed elsewhere. In such reviews the role of scientific (as opposed to strictly policy) input into planning have to be explored. In turn this may lead to more general questions as to the nature of science advisory mechanisms and whether they are fit for dealing with emergencies.

No doubt there are many more questions that merit consideration once the pandemic settles. But the questions above suggest a major agenda for the community of researchers and practitioners interested in the science policy interface. INGSA’s role is to provide the forum where these issues can be explored and discussed. Over the next 12 months, we will be conducting seminars and webinars to both explore these issues and to hear and learn from researchers addressing them.

Visit the INGSA Policy-Making Tracker to compare government responses around the world

To see more debates and discussion on COVID-19, visit the ISC’s Global Science Portal.