Lies, damned lies and statistics

One of my jobs at school involves looking at academic data.  I think pretty well all schools have a role like that, either as a post in its own right or part of a broader remit – being “data-rich” is essential if you want a chance of a decent inspection report, for a start, and hopefully, all the number-crunching may actually yield something useful.

Most schools do some form of “baseline assessment” on pupils entering the school. There are various providers of these – Fischer Family Trust, CATs, MidYIS …. (and probably a few more I haven’t heard of). Some schools use these as a basis for setting – the alternative, if you have a broad ability range and wish to set, is to teach mixed-ability initially and then set based on internal assessments. Basing it on internal assessments has its advantages, obviously – not least you can see the kid’s paper and judge, if s/he is marginal, which is the better class – but it also entails disruption when classes are shifted around, which isn’t ideal.

The other reason is to demonstrate “value-added”. That means that the baseline assessment is used to generate predictions of how the child will perform at the end of Key Stage 3 and/or at GCSE, to which you compare how they actually do perform. Obviously, if they perform better than predicted, you will argue it is all due to your wonderful teaching. You can certainly be sure if they perform substantially worse, that is likely to be considered down to your failure to drag them bodily into school if a persistent absentee, or to physically stand over them in their home in the evening and ensure they do their homework, or if all else fails to don your school uniform and sit the exam for them….

Do I sound like I’m anti the concept of value-added? I’m not, at all  – it’s absolutely vital if you are to avoid judging schools entirely by actual results – and judging by results means that those schools with a naturally high achieving intake to look good, and those in areas with a high proportion of, for example, kids with English as a second language, or kids in difficult circumstances, invariably look bad.  Knowing that you have moved your pupils on from where they started is really important.

You know there’s  a “but” coming, right? Well, not just one, in fact….

Firstly – you can end up relying on the data too much. I completely accept that there needs to be some degree of looking at the results a teacher’s class achieves – there do exist bad teachers, alas, and they shouldn’t be allowed to spoil people’s chances.  But we neither fully deserve the credit for a top achievement by a pupil with a fantastic work ethic and encouraging parents, nor the censure for a disappointing performance from a disaffected child with unsupportive parents who has put minimal effort in. It’s vital to look at what the numbers tell you in association with knowledge of the individual child. If the child shows fantastic value added in all their other subjects and is two grades down for me, then yes, that is almost certainly me. If they are two grades down for everyone else but just one down for me, then that, I think, is to my credit, rather than the reverse!   ( I should hasten to add that my current school does use data in conjunction with knowing the pupil…. but that is by no means universal, alas).

Secondly, the data simply isn’t that reliable. Children change. I’ve found that at absolute best, statistically speaking 16% of the variation in GCSE scores is explainable by the variation in baseline performance – and it’s usually much lower than that. That doesn’t mean it isn’t useful – it can be very helpful in some circumstances to highlight strengths and weaknesses, under-achieving able pupils, the strength or weakness of a whole cohort…. but it does mean it needs to be taken with a hefty pinch of sodium chloride.

Thirdly, giving teachers information on baseline performance is at risk of causing them to limit their expectations of some pupils. I’ve just been doing some figures on this – we do see a number of kids who’d be predicted to be getting a mix of Bs and Cs at GCSE by the baseline  tests ending up with a mix of As and A*s – in some cases, almost all A*.  If you know that this pupil “shouldn’t” be achieving the top grades, mightn’t that affect the feedback you give? Mightn’t you end up feeling that for them, this piece of work is good enough – when actually they could achieve more highly?  Of course we cannot expect all our pupils to achieve A* – but is there not a case for judging that from their work with us rather than performance on a test on one day a couple of years ago?

The flip side of this, however, is that if you have data telling you that the middle-of-the-road kid who you thought was heading for a B should actually be a really high flier, then that might encourage you to get them to raise their game, which has to be good. The tests can also help spot specific learning difficulties or specific needs for support…. there’s lots of good things you can do, potentially.  We just need to remember that they’re no replacement for considering children as individuals.

This entry was posted in Educational Developments, Opinions. Bookmark the permalink.

2 Responses to Lies, damned lies and statistics

  1. sarah evans says:

    Ofsted is saying that all children should move up 3 levels between key stages. As well as everything you say, this new pressure appears to be completely arbitrary. It is a way of setting a bench mark to ensure lots of good schools suddenly become inadequate. It could be decided it was 4 or 5 levels just to create a tighter hierarchy or meritocracy – to what end?? All this has nothing to do with supporting individual children or understanding any holistic view of the function of schools for society at large.

    • teacherposts says:

      I do agree – it’s hard to see that as anything other than another stick to beat us with. It’s also the height of ridiculousness to expect all children to progress at the same rate – children who learn faster…learn faster!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s