Why do we feel safe taking over-the-counter medicines when we have a headache? One reason is that medicines are thoroughly researched, including using control trials.
A control trial compares the impact of a new or different variants of an intervention by giving it to some people but not others. The side-by-side comparison of interventions simplifies evaluation and gives us a powerful method of understanding whether our policies and programmes are really working.
The Civil Service is already a world leader in using control trials to develop policy and services. The Department for Work and Pensions has for many years used control trials to evaluate the effectiveness of policies to support more people into work. The Department for Business, Innovation & Skills is currently conducting one of the largest-ever policy trials, looking at the introduction of Growth Vouchers for businesses; and HMRC are running trials (or A/B tests) to improve new digital services and communications for millions of taxpayers.
Myth busting
As I mentioned in a recent blogpost, the Behavioural Insights Team (BIT) (BIT) have made a big impact in encouraging fleet-of-foot, empirical approaches to test policy and delivery challenges. As part of this work, the Cabinet Office, BIT, Dr Ben Goldacre and Professor David Torgerson collaborated to write Test, Learn and Adapt; an introduction to the power of trials and how they can be delivered in the Civil Service – busting many of the commonly held myths about trials being too expensive, too complex or innately unethical.
Significant progress has undoubtedly been made, but there are still swathes of government policy and practice where we just do not know whether programmes are delivering their desired outcomes, or whether we could introduce variations within the programme to increase impact. In times of tight finances, we simply can’t afford not to know the answer to these questions, and while experimental approaches are not always going to be appropriate, there are many, many cases where we should be striving to test, learn and adapt more effectively.
Cross-government panel
Colleagues have often responded to this challenge in the past by saying it is too difficult to design and implement complex trials. I am delighted that we now have a brilliant group of experts on board to help address this. A couple of months ago, the Cabinet Office What Works team launched the Cross-Government Trial Advice Panel. This panel brings together trialling experts from across government, along with over 20 top academics (thanks to Economic and Social Research Council support) to provide a free-to-use service to all civil servants to help you design and implement effective trials and test whether your interventions really work. It will build on and support the expertise we already have in departments, while making it easier for everyone to access the technical help we need.
This is a fantastic example of cross-departmental collaboration. Nearly all departments have offered up their top trialling experts to be part of this initiative, and the panel is already busy supporting trials in the Cabinet Office, Home Office, BIS, CLG and more.
Chris Wormald, our Head of Policy Profession spoke passionately at the panel launch event about his vision of future policy-making being built on evidence from trials and robust impact evaluation, and Dave Ramsden, Head of Government Economic Service, called this the next milestone of an incredibly important agenda. I strongly echo their enthusiasm, and very much hope that this great initiative will hasten the pace of change, including a rapid growth in the use of trials by the Civil Service.
"Who you gonna call?"
Please have a think about all the policy and programmes you are involved in and consider whether you are really able to say "this works", and whether a trial might help you to answer this critical question. Dr David Halpern has likened the panel to Ghostbusters – “who you gonna call?” – on hand to help with all your trialling questions and queries (of a non-paranormal nature). Draw on the expertise in your departments, work with analyst colleagues, and use the panel to help you devise trials to understand the impacts for citizens of the fantastic work you do.
To contact the Cross-Government Trial Advice Panel, drop an email to trialadvicepanel@cabinetoffice.gov.uk.