EVIDENCE IS IMPORTANT, BUT IT IS NOT ENOUGH TO GUARANTEE BETTER POLICY
A new government often brings a refreshing commitment to the pursuit of evidence-based policy. The subtle implication is that dirty politics, and not noble evidence, motivated the previous holders of office.
Gaining political advantage is an obvious driver of government behaviour. Recent work by Kate Griffiths, Anika Stobart and Danielle Wood from the Grattan Institute found that spending on government advertising spikes massively in the lead up to elections. This builds on previous Grattan work around grant funding that dirty politics explains well, and noble evidence poorly.
That government acts self-interestedly in a robustly-contested democracy is disappointing, but it is not surprising. Power is hard to obtain and difficult to give up. It is why the formal and informal guardrails protecting democracy are so important. Ours, as Grattan and others have ably demonstrated, need strengthening. Using evidence as a basis for policy is one of those informal guardrails.
When in power, government is meant to serve the interests of the people - both individually and as a collective – and not those of the ruling party. This is harder than it sounds. We do not live in Rousseau’s wished for world where the interests of ruler and citizen are “one and the same”.
One big challenge is that the interests of ‘the people’ vary greatly. As an individual, what is in my interest might not be in yours. What is good collectively, might not be good for you or I personally. There is overlap. But there is also tension. This tension fuels the best in our democracy, but it also opens the door for political opportunism.
Serving the interest of the people requires government to fairly resolve what former PM&C Secretary, Mike Keating, politely calls the competing claims of its citizens. Gary Banks sees the task more viscerally. He argues that serving the best interest of the community as a whole occurs in a “maelstrom of political energy, vested interests and lobbying”.
To navigate this maelstrom, the policy wonks argue, government must rely on the evidence.
The case for evidence-based policy was taken up recently in a review of the Australian Public Service. It argued that APS advice “needs to be informed by the best available evidence and expert analysis”. The review made three recommendations designed to reverse a long-term decline in public service evidence-building and using capability. These recommendations will doubtless be pursued diligently by PM&C Secretary, Glyn Davis, one of the report’s authors. Signs from government are positive.
More emphasis on the creation and use of evidence in government is welcome and long overdue. To make the most of the opportunity, evidence needs to become more than what Gerry Stoker and Mark Evans call “window dressing” for pre-determined policies. It must help decide what we should do, and how we should it.
Perspective is also needed. Those who expect evidence to result in a ‘eureka’ like understanding of what policies government should adopt are going to be disappointed. Evidence is important to good policy making. But it is not enough to resolve the competing claims of our citizens fairly and well.
Defining and using evidence
Practical and philosophical limits exist to the use of evidence in policy making. Even defining evidence can be a problem. Historian and philosopher R G Collingwood argues that defining what constitutes evidence is actually “very difficult”. Philosophers are, of course, disposed to finding things complicated. But our conception of evidence matters, and Collingwood’s reminder is worth heeding.
The policy literature typically refers to evidence as combination of accepted fact and objective analysis. Reality is, however, somewhat different. What is commonly presented and used as ‘evidence’ involves an undifferentiated bundle of facts, analysis, theory, values and ideology. Evidence is almost always presented with a world-view built-in for free.
Many think that a broad view of evidence is problematic. Banks, a former Chair of the Productivity Commission, favours a narrow definition. He emphasises objectivity and sees values and ideology as lying outside the policy evidence tent. Banks has a case. But defining evidence too narrowly is also problematic, and separating your world view from how you see the evidence-base is harder than many think.
Dunghutti and Biripi man, Craig Ritchie, sees policy as a profoundly cultural endeavour. He thinks that our current approach to policy knowledge is “universal and imperious”. Ritchie argues that metis (practical knowledge or intelligent ability that is deeply embedded in local practices) is a missing piece of the evidence/policy puzzle. Ritchie too has a case.
Even when we are clear on what evidence is, finding good evidence and using it well can be difficult.
Statistician, Paul Goodwin, outlines a plethora of problems with the way we use the flood of numerical data now available. Goodwin describes the numbers we often rely on for evidence as “at best a simplification … and at worst a gross distortion of” the real world. The recent establishment of an APS data profession to help navigate these issues is a good intiative.
Policy evidence is also regularly mixed, and often hard to interpret. Evaluations of the controversial income management program show the policy harming some people, helping others, and making little difference for the remainder. This same mixed evidence has been used by successive governments to support quite different policy directions.
Availability is a major practical issue. Evidence-based policy requires high-quality evidence to be available when needed. Almost always, it is not. Stoker and Evans acknowledge that “rarely will the conditions emerge for a pure [evidence-based] problem-solving model”. This is likely to remain true, even with a concerted push for more and better evidence from government.
The biggest challenge in using evidence may well be temporal. The nature of policy means that evidence, which is based on an understanding of the past, is used to make decisions about the future. Bridging this temporal divide requires an assumption that past events meaningfully describe future ones.
To meet this challenge, models are often used. Modelling is regularly taken as powerful evidence by government, even when results are driven by the model’s assumptions. Assumption-driven models can be useful, but can also steer us away from reality. Over-reliance on modelling explains why some economists express more confidence predicting the future than they do explaining the past.
Paul Ormerod argues that assuming the future will be like the past is a mistake, and is “why most things fail”. The future, he says, is inherently complex and uncertain, and past results are almost always deeply contextual. Ormerod’s conclusions about the implications for government probably go too far, but his underlying concern is valid.
Bridging the evidence gap
The bottom line for government is that a gap exists between any conception of evidence and policy meaning. More and better evidence helps. But the gap will never be filled completely by even the very best evidence. How this gap is bridged determines how fairly and well government meets the competing claims of its citizens, and the space available for dirty politics.
For Banks, the answer lies partly in who fills the gap. In effect, his call is for the right people – those with independence and objectivity - to be responsible for interpreting the evidence base and presenting what it means to government. Independent and unbiased analysis is obviously desirable. But Banks’ vision may fly too close to creating an anointed class of Confucian ‘superior men’ for complete comfort
At the end of the day, a simple contrast between noble evidence and dirty politics is unfair. Politics certainly has a dirty side, where selfish political advantage trumps public good. But it also has a practical side, where compromises are made to accommodate differing views. And it has a noble side, where (competing) values and principles guide decision making.
Better connecting noble evidence and noble politics is key. The case of income management is instructive, partly because it is common. The same evidence sits behind different approaches. Neither approach can be fairly said to flow from ‘dirty’ politics. Both, instead, reflect alternative values-based propositions about what is best for people directly affected and for society as a whole.
For the public service, this creates a challenge. Developing better and more actively-used evidence is important. But in converting evidence into policy meaning, public servants (and others) need to more transparently identify and articulate the values being used to fill the gap between evidence and policy. Doing so, gives us a better chance of connecting noble evidence and noble politics, and avoiding the worst of dirty politics.