This page is dedicated to capturing all the activities the Lancaster Evaluation Group has planned, organised and participated in since its creation.
AUGUST 2024
JULY 2024
LEG Writing Retreats
We created LEG to have the opportunity to foster collaboration between different disciplines, department and partners in and around Lancaster University. As well as our series of fantastic seminars, we also wanted to use this time to focus on the future and decide the next steps for LEG. This July, we have engaged in a series of bid writing workshops with the LEG core team, using this time to decompress after a very busy June and strategize about what comes next.
We're really excited about the future of LEG, so keep watching this space!
JUNE 2024
Lancaster Evaluation Group Event Series
Session 4 - 27th June 2024
We welcomed Barbara Befani and Jessica Ozan to the final of the Lancaster Evaluation Group's (LEG) hybrid seminar series, discussing "Suitability and creativity in evaluation design: prioritising the needs of participants."
Barbara Befani presented her work on "Hybridity and creativity in evaluative practice." Using single methodologies in evaluation is increasingly rare. There is now widespread awareness that designs ought to be bespoke and adapted to an evaluation's specific circumstances. Various terms have been used to describe these designs, but what does creating bespoke or hybrid designs mean and entail? Mixing, combining, adapting, comparing? This seminar attempts to clarify the ambiguity by distinguishing between four actions: comparing or selecting appropriate methods, co-designing evaluations by engaging stakeholders, combining or mixing otherwise self-standing options, and using hybrid (or inherently quali-quanti) methodologies. The discussion will include how to ensure quality under all of these circumstances.
Jessica Ozan presented her work on "Putting Children and Young People at the heart of evaluations." Some of the first evaluations in the United Stated focused on youth and educational policies (e.g., the Cambridge-Somerville Youth Study in the 1930s, the Eight-Year Study 1932-1940). The studies aimed to capture the effectiveness of different juvenile delinquency interventions or curricula and very much focused on administrative data. Practice has drastically changed over the last century. Alongside the “new sociology of childhood” and children’s rights movement, evaluators are now trying to create safe spaces for children and young people to participate meaningfully in studies that concern them. Dr Ozan will explore what this means for evaluation practice today in the UK. What does participation look like? Is this compatible with all methodologies? Who commissions evaluations focusing on children and young people? What are they key challenges and pitfalls? And most importantly, why are we putting children and young people at the heart of our evaluations?
Lancaster Evaluation Group Event Series
Session 3 - 19th June 2024
We welcomed Nathaniel Pickering, Samantha Child, Jonathan Schulte and Emma McDowell to the third of the Lancaster Evaluation Group's (LEG) hybrid seminar series, discussing "Exploring spaces for collective learning in evaluative practice."
Nathaniel Pickering, Samantha Child, and Jonathan Schulte presented their work on "A manifesto for evaluation: An introduction to the Evaluation Collective." During the presentation, we provide an introduction to the Evaluation Collective, we talk through the Evaluation Collective manifesto and then discuss the Wicked Issues project.
Emma McDowell presented her work on "Creating space for collective learning: Centre for Cultural Value." Dr Emma McDowell from the Centre for Cultural Value will discuss the work behind the recently-launched Evaluation Learning Space, which uncovers learning hidden in evaluations from the cultural sector. As a tool to share insights and knowledge to support cultural organisations and practitioners, the resource hub has placed an initial focus on what we can learn from the evaluations of UK-based Cities and Capitals of Culture. Building on the work of the Centre in evaluation practice methodologies and training, these initial resources shed light on who was involved, what methods and frameworks were used and the challenges evaluation teams faced.
Lancaster Evaluation Group Event Series
Session 2 - 10th June 2024
We welcomed Gemma Moore and Rick Davies to the second of the Lancaster Evaluation Group's (LEG) hybrid seminar series, discussing "Participatory and collaborative approaches to evaluation."
Gemma presented her work on "Coproducing evaluations: Evaluation that Engages." Voluntary and community organisations in the UK want, and are under pressure to, improve their capacity to evaluate their work, whilst universities have corresponding research skills and expertise. Within this session, we share a case study of how a university has respond to this need: the Evaluation Exchange. The Evaluation Exchange is a partnership between UCL and a community-infrastructure organisation Compost London . It is a structured 6-month programme that matches interdisciplinary teams of researchers with organisations to improve their capacity to evaluate their work. Rather than seeking to develop or promote one specific evaluation method we bring together different stakeholders with a range of skills, knowledge, and expertise to co-produce a bespoke solution that best fits the circumstances. Within this session we provide an introduction to the principles underlying the Evaluation Exchange, define what we mean by ‘co-productive evaluation’ , and outline some tensions of putting the principles to practice. We share the learning from delivering the programme, reflecting on the building capacity and capabilities in evaluation practice, supporting a shift in traditional evaluation practice.
Rick Davies presented his work on "The Most Significant Change" (MSC) technique: Its origins, how it works, and how it has been used 1994-2024." This presentation will describe the context in which MSC was developed and used, initially in a PhD thesis focused on field work conducted in 1994 Bangladesh. It will then explain the 10 steps in the design and use of a MSC process, as described in detail in the 2005 MSC Guide, describe some of the ranges of use of MSC since then, as accessible in the Zotero Online MSC Bibliography and present Rick's own assessment of the uses of the method, in light of its original design intentions.
Lancaster Evaluation Group Event Series
Session 1 - 3rd June 2024
We welcomed Jonathan Schulte and Binda Patel to the first of the Lancaster Evaluation Group's (LEG) hybrid seminar series, discussing "Evolving Approaches and Expectations of Evaluation in Higher Education."
Jonathan Schulte presented his work on "The troubles of ‘good’ evaluation: Identifying practical tensions between participatory approaches, certainty, and usefulness." In the philosophy of science, a distinction is commonly made between epistemic and non-epistemic aims. Applied to evaluative practice, this paper argues that evaluation has both epistemic and non-epistemic goals: that beyond mere empirical adequacy (epistemic aim) evaluations also aim to produce morally adequate and useful findings (non-epistemic aims). However, in practice, tensions between these aims emerge. Specifically, on the level of method, there are trade-offs between practices considered to further the robustness of findings and those that may seek to reflect on the aims of projects or make findings more actionable. Discussing two examples of this tension, the paper concludes by suggesting that firstly, purely epistemic schemes for the appraisal of evaluative practice are inadequate, and secondly, that reflective tools such as Murray Saunders’ RUFDATA framework are important to surface these tensions and identify approaches adequate to project specific purposes.
Binda Patel presented her work on "An Evolving Approach to Evaluating Educational Programmes." The Sutton Trust has dedicated 25+ years to delivering high-impact programmes to over 50,000 young people. Our approach to evaluation has had to evolve in this time – from the needs of our young people changing, to increased focus on demonstrating impact, to our curiosity to understand ‘what works’. My presentation will cover how our approach has changed over the years, the tough questions we have had to ask ourselves, and how we test and evaluate innovative projects where the outcomes aren’t always clear from the outset.
APRIL 2024
Announcement! LEG is hosting a series of evaluation seminars!
This series of hybrid seminars and networking sessions will explore different topics in evaluative practice and research. The sessions will include presentations from visiting speakers, lunch and a networking session; you can also register to watch the presentations online. To get more details and register on each session please follow the links below. The sessions are open to anyone with an interest in evaluation, they are all free to register and attend.
Session 1: Evolving approaches to and expectations of evaluation in education
Session 2: Participatory and collaborative approaches to evaluation
Session 3: Exploring spaces for collective learning in evaluative practice
Session 4: Suitability and creativity in evaluation design: prioritising the needs of participants
Session 2: Participatory and collaborative approaches to evaluation
Session 3: Exploring spaces for collective learning in evaluative practice
Session 4: Suitability and creativity in evaluation design: prioritising the needs of participants
Book your place now!
MARCH 2024
This month LEG is working hard to plan some exciting events. Keep watching this space for updates!
FEBRUARY 2024
The Lancaster Evaluation Group is born!