LEG Seminar Series: Session 1

3rd June 2024

We welcomed Jonathan Schulte and Binda Patel to the first of the Lancaster Evaluation Group’s (LEG) hybrid seminar series, discussing “Evolving Approaches and Expectations of Evaluation in Higher Education.” 

Jonathan Schulte presented his work on “The troubles of ‘good’ evaluation: Identifying practical tensions between participatory approaches, certainty, and usefulness.” In the philosophy of science, a distinction is commonly made between epistemic and non-epistemic aims. Applied to evaluative practice, this paper argues that evaluation has both epistemic and non-epistemic goals: that beyond mere empirical adequacy (epistemic aim) evaluations also aim to produce morally adequate and useful findings (non-epistemic aims). However, in practice, tensions between these aims emerge. Specifically, on the level of method, there are trade-offs between practices considered to further the robustness of findings and those that may seek to reflect on the aims of projects or make findings more actionable. Discussing two examples of this tension, the paper concludes by suggesting that firstly, purely epistemic schemes for the appraisal of evaluative practice are inadequate, and secondly, that reflective tools such as Murray Saunders’ RUFDATA framework are important to surface these tensions and identify approaches adequate to project specific purposes.

Binda Patel presented her work on “An Evolving Approach to Evaluating Educational Programmes.” The Sutton Trust has dedicated 25+ years to delivering high-impact programmes to over 50,000 young people. Our approach to evaluation has had to evolve in this time – from the needs of our young people changing, to increased focus on demonstrating impact, to our curiosity to understand ‘what works’. My presentation will cover how our approach has changed over the years, the tough questions we have had to ask ourselves, and how we test and evaluate innovative projects where the outcomes aren’t always clear from the outset.