Can we be trusted with data?

A report into the usefulness of league tables across the public sector has just been published by the British Academy.  In it Harvey Goldstein and Ben Folen argue that tables do not well serve the purposes put forward for such data, which are:

  1. ‘public accountability’- enabling the funders of institutions to determine whether they are receiving value for money
  2. “supporting choice” – helping the users of services to chose a school or hospital based on its track record
  3. “control” – to provide levers for government to influence the activity in institutions
The problem with league tables as they stand is that not only do they create perverse side effects (the prime purpose of a primary school seems  now to be to ensure it is well placed in the league tables rather than giving a well rounded curriculum), but the very indicators used to define league tables are limited by what can be measured – and it is very hard to measure qualitative things.
More importantly data in league tables is published as fact, with none of the caveats or reliability information around it supplied.  Such data is merely a sample.  When schools look at assessment data they are usually given the confidence interval around it – the range of scores that a particular result could be indicative of.  Such data is not published around league tables, and due to the small sample of a single assessment the confidence interval (or range of possible results indicated) is large.
The Government is seeking to make league tables in education more useful by publishing more data.  However this does not make them more reliable.  It also does not address the issue of data literacy in the target market – parents and teachers.  How many actually understand what Contextual Value Added (CVA) actually means?  And how many just look at the position of a school in the league table and take it as a firm indication of the quality of all aspects of a school?
But this issue of data  literacy is evident across the profession.  It is often considered that the more data that is supplied to schools the better decisions that a school will make.  And the data provided in systems such as RaiseOnline is very detailed.  However there are very few teachers who are able to take advantage of this data.  Across education there needs to be a focus on making data rich and usable, rather than publishing it because it is there.
Harvey Goldstein, one of the authors of the report, was interviewed on the Today Programme on March 29th 2012 – the segment is at 1h32mins here – catch it while you can!

The demise of Encyclopaedia Britannica – is this the inevitable end for all print?

It has been widely reported today that Encyclopaedia Britannica is ceasing publication of its print version after 244 years.  In the face of competition from initially Microsoft’s Encarta (remember that?) and now Wikipedia, the investment of thousands of pounds in a static repository of knowledge for individuals and institutions no longer makes sense.  But, in a world of Kindles and Apple’s iBook Textbooks, does this point to the inevitable demise of all things print?

The short answer is, I believe, no.  Firstly there is the historical perspective.  As has been noted many times over the years no one new media “class” has replaced another.  Television did not replace radio.  Digital audio has yet to supplant analogue (in fact vinyl sales have been growing recently).  And the internet has not replaced linear TV.  What seems to happen is that each new media makes the mix of consumption more complex.  (Note that within genre of media, there can be replacement – DVD replaced video, and will eventually be replaced by digital streaming – but video delivery remains – ditto the replacement of illuminated manuscript with print …).

However, within print certain classes of “misfit” publishing will vanish.  Encyclopaedias are one such class.  For what is an encyclopaedia other than a database in a book?  It has been crying out for the invention of digital to add the functionality (e.g. cross referencing) that it has tried to implement in a ham-fisted fashion on paper … this was clear from the early days of electronic legal publishing where even traditional, hide-bound barristers found themselves buying databases such as CELEX and the All England Law Reports.

Where does that leave other publishing formats?  Well, an ebook does not add much functionality to a novel – other than portability at the expense of fragility.  My expectation is that the two will co-exist for generations more.  The case against the traditional textbook is easier to make.  These are repositories of knowledge.  They are multimedia – containing images (and would be enhanced by video), contain exercises and assessments, and so are prime for a gradual decline.  However, until the hardware delivering an electronic textbook becomes cheap, reliable and ubiquitous (and Apples iBook for iPad is none of these yet), this cannot happen.  But the market is aware of the potential disruption, and this is causing a systemic decline in sales.

So is print on its way out – no.  But are there more classes of print publishing that will go?  Without a doubt – and the textbook is likely to be one of them.

Gove seeks to sever the links between exam boards and publishers

Ofqual have launched their consultation into education resource development … this will have an impact on the way that education publishers and exam boards conduct themselves…

Over recent years one of the ways that publishers have gained competitive advantage for their resources has been to gain exam board endorsement.  And this has driven relationships such as Pearson’s with Edexcel.  However, in the light of many representations to Goverment and the recent telegraph expose of exam board practice, Michael Gove is looking to sever this relationship.

This will have an impact both on the way that publishers construct their courses (perhaps making them more board independent) and how they market them.

For details of the consultation, please see the Ofqual site:  Qualification support material and services: call for evidence.