I have had two fascinating insights into data science in the past fortnight. First, when I went to speak to the Operational Research Society's annual conference about Operational Research (OR) and data in government, then at a discussion on data science at the weekly Permanent Secretaries meeting.
The OR Society brings together people who work in analysis and data across a mind-bendingly diverse range of topics. Over four days there had been sessions ranging from analytical in credit card fraud to the role of operational research in dog breeding! I thought that analysts in government covered a pretty wide range of topics, but it was eye opening to see how techniques from private equity or climate change could spark ideas for solutions in transport or justice. I was speaking as a non-expert about how we use analysis and operational research in government and several delegates told me how encouraging it was to hear how this work makes a difference in the real world - in helping to prevent re-offending or in reducing the time it takes for a child to be taken into care. The fact that our work makes a real difference to real people's lives is what attracts many people to the Civil Service.
The discussion about data science in government was a chance to hear about how civil servants have delivered pilot projects using linked data sets and clever visualisation tools to help solve complex problems. Examples included real time information maps of flood risk and of fire and rescue incidents, with users able to drill down through maps and charts to understand the problem in more detail. This is not just about clever mapping - the projects are experimenting with software which analyses the data and highlights anomalies or areas that need investigation. This sort of technology will help civil servants to solve problems more quickly, but it will also help the public to understand more about issues of interest to them in their locality. On the Police website you can already search for information about crime in your area and by linking police and wider criminal justice data you can see whether an offender was caught and what punishment they received. The work of data scientists in the Civil Service means we should be seeing more of this sharing of information on what is happening in local communities.
7 comments
Comment by Richard Taylor posted on
Data Science, big data and Geographical Information Systems – sounds like Shangri-La – where do I sign? As a manager whose team spends a significant amount of time sourcing data, negotiating the use of data, transferring data, cleaning data, chasing late data, reformatting data, allowing for and making adjusts for missing data, finding the computing power to analyse data etc – the journey there looks like being a long one. Sorry if I sound a bit sceptical, but my feelings for data are a bit like the Ancient Mariner’s were for water – I’m surrounded by the stuff - the real problem is turning it into something useful.
In truth, we already have masses of data, the problem is that it is rare to find existing data sources that were specified and/or collected in a way that allows for analysts to undertake their work – unless of course – the specification was originally done by analysts with a specific analytical purpose in mind.
We may have the computing power and software for big data but quantity is not the issue – quality, appropriateness and accessibility are the problems that have always bedevilled analysts. In response, Operational Researchers have always been good at improvising solutions based on relatively small amounts of accurate data. Those skills will be as useful in the future as they are now.
Comment by Alec Waterhouse posted on
Much of what we do relies on having customers who are aware of what is possible. So it is good to see that senior managers in the civil service are becoming increasingly aware of how these techniques can help organisations deal with complex problems successfully.
One important point to note is that operational research can deal with data poor as well as data rich problems. It can help where takeholders have different views of what is required or even where they are struggling to articulate the problem in the first place.
Comment by Stephen Lorrimer posted on
Data science and big data carries huge opportunities, but significant risks too. The recent directors of analysis conference on big data highlighted the importance of doing the basics right and these could get overlooked as big data becomes "the thing" to do.
And we shouldn't overlook some of the wider skills that are still so very important; there will always be the need for the quick, simple analysis "by close of play" that is timely enough top inform critical decisions.
That means we need to continue investing in developing analysts, with broad experience and good judgement, as well as investing in the data sets, IT infra structure and tools needed for analytics applications.
Comment by Mike Smith posted on
I am no operational researcher or statistician, though I do work in performance reporting & analysis, but I endorse all that has been said about the value of data science. Unfortunately I have always found it difficult to persuade senior managers that it is critical that operational IT systems include the capacity and capability to provide quality data and management information. Their eyes glaze over and we end up with nothing, having to manually collect data, pay an IT 'service provider' a fortune every time we need new data or something else hopelessly inadequate. If we don't sort out the front end we cannot make the most of the skils of the professionals we are lucky to have working with us. I'm not holding my breath though.
Comment by Rob Solly posted on
As Tom says, Data Science offers us a really exciting opportunity to improve our understanding of "what works" when developing and delivering policies. I'd also like to draw people's attention to the broader spectrum of operational research techniques.
At the other end of this spectrum from Data Science are problem structuring techniques, designed for complex problems where we have few or no data, but where our information comes instead from people. These techniques can help build an understanding of "why things work" as well as thinking through "what might work" in future. At a recent event at the Royal Society, we heard how problem structuring techniques had been used successfully to resolve conflicts over water resources and fears over the health risks of electricity pylons; to turn around poorly performing children's services; and to help planners prepare more robustly for developing crises overseas.
And in the middle of this spectrum we have simulation and systems thinking techniques that can help predict the performance of systems from something as simple as a clock to something as complex as the climate; and decision analysis techniques that can help draw together information on numerous factors to support a difficult decision,
These techniques are already being used by GORS, Dstl and others but combining them with the momentum and exciting potential of Data Science offers the opportunity to help develop and further improve policy across central and local Government.
Comment by Tom Wilkinson posted on
I wholly share the Permanent Secretary's excitement about Data Science in government, even though it's my day job!
There are, of course, important considerations about how data has been generated, before you can put your faith in analyses coming out of it. But, trustworthy admin data is a window on so many of the services we provide to society. What's more, it is generated as a side-product of administering those services, rather than from an expensive survey on a small sample of people.
For many things, Data Science offers the potential to finally get more robust scientific evidence on policy questions that have been asked for years. But, the Justice system is very very big, and very very complex - like many departments' remits. So, some questions will never have been conceived of, because no one person was ever in a position to understand all the relevant factors in enough detail. This is where Data Science offers something dramatically new: Machine Learning. Machine Learning is where we ask a computer to find relationships in massive amounts of data about diverse objects, the same way a person might spot a relationship between everyday objects. Machine learning is being used increasingly by both tech giants and new startups ( see http://www.forbes.com/sites/85broads/2014/01/06/six-novel-machine-learning-applications/ ), and with our increasingly interconnected data it could guide us to institutions that better serve the public.
Comment by Lisa Ollerhead posted on
Really interesting to see a Permanent Secretary's perspective on this work that has such exciting potential across government - the Data Science programme in Cabinet Office has recently written a post for us on the Open Policy Making blog about their progress.