Today’s blog is by NAEE Fellow Melissa Glackin from King’s College, London.
‘Does the policy-making process engage authentically with practice?’ This is one of the important questions that colleagues at King’s College London are exploring in the ESRC-funded study: Rethinking Impact, Evaluation and Accountability in Youth Work. This week I had the good fortune to attend a seminar where project lead, Tania de St Croix, and co-researcher, Louise Doherty, presented some emerging findings. While Tania was keen to highlight that the research was still in its infancy, it struck me that whilst they are exploring, what they described as, the ‘open access’ youth work sector, their insights resonate with policy formation and practice within the environmental education and outdoor learning sectors. As these three sectors are close cousins, with many organisations and individuals working across the informal context landscape, I thought to share with NAEE members their findings but from my own (environmental education) perspective, alongside weblinks to this project.
The Study & Findings
The study explores the youth ‘impact’ agenda through a policy network ethnography. That is, from what I understood, the agenda being the prevailing policy ideology whereby what can be measured is valued and what is valued is funded. The result is a shift in youth-work from being fluid, flexible and responsive to a more structured practice driven by pre-determined outcomes. This all sounded familiar to me from an environmental education programme perspective, I just hadn’t put words to it before. Their work is novel in that it explores the ‘impact’ agenda’s antecedents and the role multiple organisations and individuals have had in its formation, transformation and regeneration. In what they describe as a ‘discursive ensemble’ they isolate several factors which have fed into the rise of the ‘impact’ agenda. Here are three which I felt resonated with the environmental education sector: practice as unproven, practitioners as antagonistic, and, policy as well-meaning.
- Practice as unproven
In the past, anecdotal accounts and participant discussions were key methods to capture the worthwhileness of youth work programmes; a state similar across the environmental education sector. However, in a climate of measurable outcomes, traditional approaches to evaluation have been demoted making way for the search for the perfect comparative scientific instrument (several environmental education examples spring to mind). The youth work sector, as does the environmental education sector, finds themselves being judged in a climate of ‘random control trials’ and ‘quantitative evaluation tools’ requiring the production of proof that their programme matters.
- Practitioners as antagonistic
Practitioners questioning the legitimacy of the foci of funders or government programmes on measuring ‘impact’ are often labelled as antagonistic and stuck-in-the-past. I recognise this finding. I have been at conferences where during the Q & A sessions practitioners are left feeling like isolated voices, perceived by policy makers to be folk behind the times and preventing change. This perception, of being hostile to change, whilst not true, is also one that the majority of practitioners do not want to be tarred with. So, whilst many practitioners are insulted by the direction, for example, of programme funding calls, they first and foremost want to ensure the necessary resourcing for young people today. Hence, although practitioners find themselves frustrated by the culture of ‘impact’, more pressing is ensuring that they are able to access funding to provide something reassembling a service. Perhaps this resonates for you?
- Policy as well-meaning
Whilst the agenda might be malign, policy and policy-makers are well-meaning. What this finding is highlighting is the potential frustration you might feel when you consider questioning the programme aims or direction. That is, we all want the best for young people – and no one would argue with the necessity for reasonable accountably and even more so when it concerns tax-payer’s monies. However, perhaps what society deems reasonablerequires a fresh debate? As whilst policy might be well-meaning, this study’s findings suggest that the current ‘impact’ ideology is having a substantial influence on the quality of services young people receive.
These are just some insights I captured from the seminar, filtered by my experience of working in environmental education. The authors will have emphasised many things I have missed and will not necessarily agree with my interpretation. However, this study is important, and the wider informal sector might want to take notice, as it shines a light on the mechanisms of policy formation and enactment offering a method to understand how we got to where we are and possibly illuminating pathways for a positive way forward.
Project Website: Rethinking Impact, Evaluation and Accountability in Youth Work
Related article: Youth work, performativity and the new youth impact agenda
Join project mailing: list
Tania de St Croix firstname.lastname@example.org
Melissa Glackin can be contacted at: email@example.com