A report into the usefulness of league tables across the public sector has just been published by the British Academy. In it Harvey Goldstein and Ben Folen argue that tables do not well serve the purposes put forward for such data, which are:
- ‘public accountability’- enabling the funders of institutions to determine whether they are receiving value for money
- “supporting choice” – helping the users of services to chose a school or hospital based on its track record
- “control” – to provide levers for government to influence the activity in institutions
The problem with league tables as they stand is that not only do they create perverse side effects (the prime purpose of a primary school seems now to be to ensure it is well placed in the league tables rather than giving a well rounded curriculum), but the very indicators used to define league tables are limited by what can be measured – and it is very hard to measure qualitative things.
More importantly data in league tables is published as fact, with none of the caveats or reliability information around it supplied. Such data is merely a sample. When schools look at assessment data they are usually given the confidence interval around it – the range of scores that a particular result could be indicative of. Such data is not published around league tables, and due to the small sample of a single assessment the confidence interval (or range of possible results indicated) is large.
The Government is seeking to make league tables in education more useful by publishing more data. However this does not make them more reliable. It also does not address the issue of data literacy in the target market – parents and teachers. How many actually understand what Contextual Value Added (CVA) actually means? And how many just look at the position of a school in the league table and take it as a firm indication of the quality of all aspects of a school?
But this issue of data literacy is evident across the profession. It is often considered that the more data that is supplied to schools the better decisions that a school will make. And the data provided in systems such as RaiseOnline is very detailed. However there are very few teachers who are able to take advantage of this data. Across education there needs to be a focus on making data rich and usable, rather than publishing it because it is there.
Harvey Goldstein, one of the authors of the report, was interviewed on the Today Programme on March 29th 2012 – the segment is at 1h32mins here – catch it while you can!