Marion Macleod, Policy Manager at Children in Scotland, explains why we need to really understand what the evidence is telling us when we make decisions.
A recent briefing on the LGiU Scotland website discussed how an evidence base is needed to support policy decisions. At a time when we are faced with the significant and often adverse consequences of austerity measures, it is of vital importance that we use our increasingly limited resources to best effect. Of course we need to be sure that our decision-making and consequent allocation of public funds are underpinned by evidence. First of all, however, we need to be clear about what constitutes evidence that provides a sufficiently reliable basis for making policy choices and service decisions.
Much emphasis has been placed recently in Scotland on ensuring that there is an evidence base for funding decisions. It is not always clear what we mean by this. Randomised control trials are a pertinent example. They have frequently been used as the justification for procuring some very costly interventions. In simple terms, what RCTs provide is a comparison between a group of subjects who received an intervention and a group who did not. In other words RCTs demonstrate whether the intervention delivered better results than doing nothing at all. They do not offer any guidance as to whether the outcomes were ‘good enough’, offered value for money, or compared positively with other approaches. For example, there is a powerful body of research evidence that concludes that the most positive, sustainable and equal outcomes for children are achieved by strong, high quality universal early childhood education and care services. It would of course be difficult to apply RCT methodology to such services, both because they are more likely to be delivered over the long rather than the short term, and because it would be virtually impossible to identify a control group. Nonetheless, substantial amounts of public money have been invested in patented programmes requiring fidelity in replication on the basis of RCT evidence.
The UNESCO Centre for Child and Family Research at Galway University is this week hosting an international expert symposium on what works in promoting child rights and supporting parents’ effectiveness. Evidence will be one of the key topics under discussion. Professor Pat Dolan, who heads the Centre, has developed a pluralistic approach to evidence and evaluation[1], intended to assist in identifying appropriate evidence gathering and analysis approaches, particularly in respect of services that aim to engage with and change human behaviour and relationships.
So I’d suggest we need to be circumspect in our approach to evidence, and to understand what it is really telling us, if we are to make wise decisions about policy and to use our resources most effectively.
[1] Fives, A., Canavan, J. and Dolan, P. (2014) Evaluation Study Design – A Pluralist Approach to Evidence. A UNESCO Child and Family Research Centre Working Paper. Galway: National University of Ireland, Galway.
