Evidence, expertise, trust: What politics really needs today

Evidence-informed policy is more than a buzzword – it is a political programme that was originally developed in health policy and now has an impact across policy fields. At the same time, it is precisely in Austrian science, technology and innovation policy (STI) that it becomes clear that this ideal cannot simply be transferred “one to one”, but must be adapted to specific contexts.

Evidence is also only one possible way – among several – of preparing scientific knowledge for political action. Another format is expertise. There are, of course, many transitions between the two. We at FORWIT are an expert body. But we also claim to work with evidence. What we see is that policy advice, including scientific advice, often operates in the format of expertise and relies heavily on trust in advisers; ideally, much evidence and little self-interest are involved. The boundary with lobbying has to be judged on a case-by-case basis – examples might include various think tanks with a clear bias. It is in this tension between aspiration and reality that FORWIT positions itself, contributing independently, systematically and on a sound basis to the further development of Austrian STI policy.

It quickly becomes clear that there is no single form of evidence. Evidence is always embedded in a system of interpretation. There is certainly an ideal that evidence should be as hard and as clear as possible – measurable, quantitative, apparently indisputable. But, first, such “hard” evidence is often unavailable – and then there is a risk that attention is paid only where this kind of “hard evidence” exists, while other problems and challenges are ignored. Second, even “hard” evidence is not, on closer inspection, all that hard – the key issue being region-specific definitions or system boundaries. And third, there is also a range of “soft” evidence that is equally important: observations, historical developments, discourses and opinions. Beyond this, every political decision ultimately also has to take other areas of life into account, not just the RTI sphere. A constant example at present is the interlinkage of economic, industrial and RTI policy. Or, indeed, when scientific advice is sought in an acute crisis.

The art of evidence-informed policy lies in combining these different forms of knowledge productively, rather than setting them hierarchically against one another.

How does hard evidence become politically effective?

FORWIT’s STI Monitor is an illustrative example of how this can work – and where the challenges lie. With more than 200 indicators – that is, very “hard” evidence – it maps the performance of the Austrian STI system each year in an international comparison. As a data-based foundation, it creates transparency about strengths and weaknesses, trends over time and the implementation status of STI Strategy 2030. It therefore provides a key basis for strategic decisions and effective STI policy. But these – at first sight – very “hard” forms of evidence only become politically effective when they are interpreted, contextualised and made plausible.

This is exactly where FORWIT’s further work begins: annual reports, analyses and recommendations translate numbers into guidance for decision-makers and show which questions the data actually raise – and which political actions can be derived from them. It is this “soft” work that makes hard evidence relevant in the first place. And even then, one question remains: is it noticed politically at all?

At the same time, practice shows that scientific policy advice does not operate only through publicly visible products such as reports, dashboards or studies, but – as noted above – is also shaped to a significant degree by trust. FORWIT works as a formally established body that advises the federal government on key issues of RTI policy and develops recommendations. Confidential discussions with political decision-makers are not an “informal back channel”, but a necessary space in which expertise can become effective: one speaks openly about observations, doubts and blind spots – and on that basis develops trust in the quality of the advice. It only becomes problematic where informal networks, personal contacts and ad hoc communication replace formal structures.

Trust is a core criterion, but internationally there are different models for how it can be established. One model that has existed since the 1960s is the position of Government Chief Scientific Adviser (GCSA) in the United Kingdom. Such a structure makes the selection of the people who provide advice transparent. At the same time, it explicitly allows trust relationships to be built.

This does not have to be copied one to one; Austria needs its own ecosystem. What I also want to say, however, is this: in Austria we like to lament informality, or what is colloquially called *Verhaberung*, yet we still leave it room because we have not really established official structures through which trusted exchange under controlled conditions can take place, or if we have, then only ad hoc, sporadically and without the opportunity to learn.

Quantitative overload through AI

A future-proof ecosystem of evidence-informed policy would therefore need to combine both: recognition of diverse forms of evidence and a shared understanding of quality and integrity in the way such evidence is collected, processed and used. Evidence ultimately does not tell us which questions should be asked politically – but it does provide good, often instructive answers to the questions we ask of it. In a diverse landscape of actors – politics, administration, science and civil society – there therefore needs to be repeated efforts to reach agreement on which issues should be addressed on the basis of which evidence, and how transparent these processes should be externally. Integrity then arises not only from the “hardness” of the data, but from understandable procedures in which data are interpreted, compared with other forms of knowledge and translated into political programmes. The STI Monitor can serve as a common point of reference here: it makes developments visible, discloses indicators and thus offers a shared data basis that a very wide range of actors can draw on.

With the rapid spread of AI, this whole situation becomes even more complex. On the one hand, because a kind of saturation may set in: through AI we will see more and more “hard” evidence, perhaps to the point where this political programme of evidence-based policy loses plausibility and persuasive power.¹ Why? Because AI-based evidence production is not only quantitatively overwhelming, but also incredibly fast. The validation mechanisms for evidence of this sheer quantity and speed may no longer be able to keep up.

On the other hand, the opposite effect could occur: paralysis through a failure to classify. If hard evidence becomes ever easier to obtain, directly via AI and available to everyone, the skills needed to interpret it may become more important – what is generally grouped under the name of expertise: namely contextual knowledge, plausibility checks and the weighing of different forms of evidence in the light of the political processes within which they operate. I am convinced that the main “playing field” here would perhaps be better described as a “battlefield”.

Leaving behind the concept of hard evidence

In conclusion: policy design that commits itself to evidence-based policymaking should bear in mind that what evidence belongs to matters. Evidence per se does not answer which questions should be asked, nor how evidence should then be translated into political programmes.

In light of the diversity of actors mentioned above, there therefore needs to be repeated agreement on which issues should be addressed on the basis of which evidence. In this way, integrity and quality can be achieved. A vision for a meaningful ecosystem would therefore be one in which those involved are willing to respect different forms of evidence, are open to uncomfortable objections and are prepared to reframe questions or allow different questions to be asked. It is also probably necessary to move away from the idea of evidence as something chiselled in stone; critical scrutiny is always needed, because capacities are limited.

Editorial note: This contribution reflects the personal views of the author, and not necessarily those of FORWIT.

1. Even without AI, there are already a number of institutions in this country that each operate within their own subsystem – and often silo – and generate evidence: in addition to FORWIT, for example, the Fiscal Advisory Council, the Productivity Board, the Bioethics Commission, the Commission on the Long-Term Financing of the Pension System, ASCII, the Creative Industries Council, the Startup Council, the Advisory Board on the Hydrogen Strategy, the platform of the Action Plan for Sustainable Public Procurement, or the Disaster Competence Network Austria.↩