On Monday (24th April 2025) we heard a fascinating presentation from Professor Heather Brown of Lancaster University about the challenging process of selecting and implementing two quasi-experimental evaluation designs to address questions relating to public health policy and health inequalities.
“Which quasi-experimental approach should I use? Some examples using Interrupted Time Series and Difference in Difference”
In the first of our new seminar series, Heather took us through the steps she followed to select the right methods in two evaluative research projects she lead.
The first examined the relationship that changes in planning policies linked to fast-food outlets had on local food environments in the North East of England, and the second looked more broadly at the effect that cuts to local government expenditure had on mental health inequalities in young people.
As Heather explained, the methods you choose are ultimately determined by:
- the circumstances in which you are hoping to deliver an evaluation (can recruitment to an intervention be controlled, has the intervention already happened, can conclusions be generalised?)
- the counterfactual questions you are intending to answer (e.g. what would the lives of individuals be like if the policy/intervention had not been in place? etc.)
- access to and availability of longitudinal data to construct appropriate test and control groups
- the purpose of the evaluation, what it is intending to achieve, and its intended audience
Such considerations will determine which path you go down when designing your evaluations. In Heather’s case, her expertise is in the deployment of quantitative methods to answer questions of policy impact and health inequality. In the case of the above mentioned projects, Heather deployed Difference in Difference and Interrupted Time Series respectively.
During the seminar, Heather not only provided a step by step guide on how she navigated this process of method selection and design, but also shared the very rich and valuable insights she was able to draw from the analyses these methods permitted.
The findings have proven invaluable when engaging with policy makers to inform decision making as it enabled her to make highly relevant and credible claims about what the data suggests about the effects of different health policies on key groups.
If you’d like to learn more and to watch a recording of her presentation you can request to join our Microsoft Teams channel where we keep all the recordings and slides from our current and past seminar series: link to Teams
Our next seminar will be on the 23rd April, delivered by Dr Mark MacDonald, of Lancaster University School of Mathematics and Statistics. He will be sharing how he has used student module evaluation to enhance teaching practice and educational support provided by the School.
Paper title: How to upgrade Lancaster University Module Evaluation System (LUMES) questionnaires to maximize their benefit to our teaching.
Lancaster Evaluation Group 
Leave a comment