Blog post by Scot Hunter | s.p.hunter@stir.ac.uk | twitter: _hunterscot
What is the point of research if academic work is not being cited in policy? Why are academics told we need to achieve impact when policymakers do not cite it? As social scientists, why are we pitted against each other to gain impact when policymakers aren’t that interested? – all questions I have been wrestling with since finishing an internship with the Scottish Government (SG) and working to develop the New Scots Refugee Integration Strategy.
My background is in migration studies and researching outcomes for marginalised groups. So, I jumped at the chance to work with the SG on asylum seeker and refugee policy and, overall, found it an enriching experience. The internship included analysing some application data, but most importantly, conducting a review of the New Scots policy to see what types of sources were being used to inform its development. Conducting this review challenged a few of my assumptions. For one, I thought the relationship between policy and research was more straightforward than in practice, but I was shocked to find academic work is rarely cited. Second, I realised my understanding of ‘gaining impact’ and influencing policy is individualistic and rooted in personal gain rather than the greater good of the policy area. This blog is a response to these challenges, where I will reflect on my experience of seeing policy in action and attempt to answer the burning questions that emerged.
At first, identifying and reviewing “policy documents” was quite daunting. Yes – the importance of policy was frequently mentioned throughout my studies, and to be honest, I’ve lost count of the number of times I’ve been told that as a researcher, I need to achieve “impact” with my work. Yet, the idea of influencing policy (what that means and how that is achieved) remained abstract, like something I’d figure out as soon as I work through my PhD and start doing more high-quality research as a “proper academic”. The expectation that my research would magically feed into policy as I advance in my career meant that I wasn’t engaging with the policy itself. Instead, I focused on the theoretical and empirical studies in the field, leaving any consideration of how it was to engage with the policy on the backburner.
Despite the initial anxiety, I identified several documents for review and examined what type of sources were being used to inform them. Independent academic journal articles were hardly cited throughout the policy documents – a somewhat alarming revelation considering the deluge of “training” and guidance I had received, preaching the importance of influencing policy with my research. To be fair, some academic work was cited, but only when commissioned by either a government body or NGO. There was also a reliance on wordings from official legislation and sources to provide definitions for terms. Overall, this meant New Scots policy was primarily informed by the same sources as previous documents, without much consideration of the current empirical research in the field.
The observation that academic work was hardly cited in policy challenged everything I thought I knew about gaining “impact” and influencing policy with my work, leaving me confused. Why were academic journals not cited when I’ve been told they’re important? How can policy effectively address the needs of people without including the empirical findings of research that explores their experience? Both issues I thought could only be addressed simply by recommending that policymakers fix this outrage and cite more academic work.
However, this internship has shown me that academic work is only one piece of the policy puzzle. Yes, our academic work is important, and we should strive to publish and gain impact, but I was unaware of so many other facets of policymaking. It also got me thinking about my understanding of how my research should fit into the policy and the broader consequences of how we’re taught about “impact” in academia.
Admittedly, I was a bit naive in thinking the relationship between academia and influencing policy was clean-cut.
In reality, policymakers are not that interested. Academic journals are dense, full of technical jargon and inaccessible – not just in terms of content, but hard to access. Policymakers work to stricter deadlines and have less time to contemplate the nuances of complicated social issues, a common characteristic of academic work. This is especially true for policy domains that deal with complex social relations, such as refugee integration, where simple solutions to problems are not possible. Cairney (2017) suggests policymakers are much more likely to cite academic work underpinned by random control trials and systematic review in domains characterised by self-evident findings such as medicine or the environment. In these domains, the cause of problems is more self-evident, such as the link between smoking and cancer. Citing research that doesn’t involve any ‘singular’ or ‘root’ causes, and involves work on issues that are often polarising, such as asylum, are not as appealing or efficient. But the argument about policymakers’ preference for STEM research is for another day.
It is also important to remember that just because policymakers are not directly citing academic work does not mean they have not been influenced by it somehow. Talks with senior colleagues made me realise that there are other ways of achieving impact – for example, through word of mouth. So some academics influence policymakers even if they are not cited. I think this deserves further investigation, and it would be interesting to interview academics about their views on this [adds to the list of research ideas].
In addition to my naivety, the internship and discussions with colleagues at the Scottish Policy & Research Exchange have made me consider some of the ethical issues tied up with the importance placed on achieving the impact drilled into my peers and me. This consideration also helped me understand why I was so shocked after discovering that hardly any academic work was cited in New Scots. As researchers, we are told to “publish or perish” – our research needs to achieve impact, we need to influence policy and the more publications we have, the better our job prospects will be. However, embedded within all this guidance is a sentiment of competitiveness – I feel in competition with my colleagues, that rather than achieving impact for the greater good of our policy areas. We become fixated on individual gain, having the most impressive list of publications, and achieving more impact than each other.
Although I am still rethinking my conception of impact and how my research should fit into the policy, I realise my experience and training have obscured the bigger picture. For example, although it is important, policymakers are not only interested in academic journals but also look at various other sources. Also, as discussed above, policymakers are not academics, and perhaps we need to do more to try and make our research accessible and compatible with their working patterns.
The more I talked through these findings with colleagues, the less shocking and confusing my “revelation” became. Despite the emphasis on policy, discussions with my colleagues slowly shifted my mindset and revealed that academic work only achieves high levels of impact in exceptional circumstances. It became clear that impact is just a buzzword that was continually thrown about when in reality, we should be focusing on conducting high-quality research instead.
Overall, the internship has been an eye-opening experience for me, expanding my (mis)understanding of the relationship between academia and policy. As social scientists, directly influencing policy with our work seems to be a grey area, something we are told to strive for but something that (from my experience with refugee policy in Scotland) does not seem to happen at all (or in rare circumstances). Is this similar to other policy areas or within different academic fields? Have other academics had a similar experience to me?
These considerations are critical in the context of my work for the British Academy funded project looking at out-migration in Armenia, in which we plan to engage with policymakers and present our work to them. As academics looking to influence policy to help improve the conditions for young people and newcomer groups, what can be done to strengthen the relationship between policy and research? How can our empirical findings make a positive contribution with our work?
In terms of how we are taught about impact, is it potentially time for a change? Do we need to move away from the individualistic model and start thinking of the bigger picture when it comes to influencing policy?
A work in progress…