Pages

Friday, 2 January 2015

The Standards Puzzle


"Have standards improved over time?" is one of the most persistent questions in education policy. And understandably so - under the last Government spending on education doubled, so it's reasonable to want to know if that's made any difference to the "output" of the education system.

Unfortunately our main measuring tools in education - national exams - are useless for answering the question. First because they've changed so often and secondly because they are used for school accountability and so schools have got better at teaching to the test (this isn't a criticism - it's an entirely rational response to a clear incentive).

Professor Rob Coe at the University of Durham has made the best attempt at using alternative "low stakes" test data to look at standards over time. In his inaugural lecture he set out his analysis of international tests like PISA as well as Durham's own tests, used by many schools. His conclusion: "The best I think we can say is that overall there probably has not been much change."

In the absence of any better evidence I'd have to agree with Professor Coe that this is what the data shows. And yet it feels counter-intuitive. I've worked in education for ten years and it certainly feels to me that schools have improved over that time. Likewise most of the more experienced teachers and headteachers I've discussed this issue with think things have got significantly better too.

Of course this could simply be cognitive biases at work. We all desperately want things to improve so we convince ourselves they have. But a recently published DfE report suggests another possible explanation.

The snappily titled "Longitudinal study of young people in England: cohort 2" (LSYPE2) will track 13,000 young people from year 9 to the age of 20. The first cohort were tracked from 2004-2010. Comparing the two cohorts will show how things have changed over the past ten years. This report looks at the first year's data from the new cohort and compares it with the first year's data from 2004.

And the trends are very clear. Ironically, given the recent obsession with "character building" amongst policymakers, there have been big improvements in a range of "non-cognitive" measures. Reported bullying has fallen; the percentage who claim to have tried alcohol has plummeted; aspiration has increased - higher percentages say they are likely to go to university; and relationships with parents seem to be a lot stronger too.
























This is entirely consistent with other data showing massive falls in risky behaviours by young people over the last decade - including a huge fall in criminal behaviour. As well as big increases in participation in education post-16 and in higher education.

All of this would suggest I, and others, are not imagining it when we claim that schools are - on average - nicer places to be than ten years ago. And that pupils are making more progress, at least in the sense that more are going on to further and higher education.

But here's the puzzle: given the improvements in behaviour; the reduction in criminality; the falls in truancy; the increase in aspiration; the improvements in home lives - all of which are known to link to academic attainment - why haven't we seen a commensurate, observable, rise in academic standards? Either academic standards have actually improved, but we just don't have the measurements available to identify it properly, or something is happening in schools that's preventing us capitalising on these "non-cognitive" improvements to genuinely improve standards. So what's going on?

All thoughts welcome!


11 comments:

  1. I have pondered this as well. I think there is something about participation and progression in play. The school leaving age only went up to 16 in 1972; and now goes to 18 (in the form of RPA which is a bit different) this year. Up to the introduction of GCSEs in 1986, entering for exams wasn't compulsory. So lots of pupils left school with no qualifications at all (and no aspiration apart from getting into a dwindling number of unskilled jobs), let alone CSEs or O levels. Participation in Level 3 quals has increased in line with more pupils staying in education to 18. A higher proportion get into Uni now than got five GCSEs in 1995 (from memory - I'd need to check to be sure, but not far off).

    So each decade, a higher proportion of of the population has participated in a higher levels of education. So it's not just pupils making more progress, it's more pupils making progress.

    As participation comes from the bottom up, the 'standards' (by which I assume we mean performance standards) are being measured over higher proportions of the population. People of my sort of age chunter about dumbing down not realising that in our day the vast majority of people who took any exams at all were from middle class backgrounds.

    ReplyDelete
  2. This is a really interesting set of data with which to formulate the somewhat divisive and emotive standards question!

    I've checked the previous commenter's stats, and indeed, DfE numbers from the 2012 cohort of KS5, non-selective state school leavers reveal that 52% were studying in higher education institutions in the year after their exams. This compares to the 43.5% of the 1995 GCSE cohort who achieved 5 A-Cs.

    Progress is worthy of appropriate celebration, and, having been born in Hackney in the 1980s, I can speak first hand for the positive impact of the shift upwards that Sam identifies - whatever the variable it is that has actually shifted. However, linking these figures to the debate on educational standards, it's worth noting that only 1% of the 2012 cohort were accepted to Oxbridge, and only a further 8% to a Russell Group university (with RG universities representing something like 20% of total undergraduate population).

    If, as both the previous and current governments hold true, 1) one of the central purposes of raising standards is to provide something of a fair equality of opportunity, and 2) access to the best universities represents the best chances of entering elite professions, then the apparent increase in educational attainment is not significantly filtering through into any form of capital that can be spent in the market competition for either good university places or what would have at one time been thought of as education-appropriate occupations/careers.

    This is sketchy and grabbed at data; nevertheless, until there is a levelling across school type of that different variable (whatever it is) that instead improves young people’s chances in the world outside of the controllable and inflationary education sector, I will not be convinced that, despite the hype, standards of attainment have risen. Instead, we can look to New Labour’s policy of HE expansion and easing of A' Level requirements (with the best of social justice and economic intentions), the increase in accountability and measurability prevalent since 1988, and the reduction of much teaching practice to ever more proficient, yet ever narrower, adherence to the aforesaid accountability measures.

    Like any skilled operator within a market, actors and decision makers within the education industry have become adept at offering the semblance of improvement, return and efficiency in their sector; be they government, the exam boards, or schools themselves. And as everyone is working very hard indeed, and the numbers continue to go up and up, it becomes increasingly difficult to believe anything other than the inflated image. That is until - like many a friend of mine - a student emerges from university with a 2:1 in a traditional subject from a non-RG university and - pre crisis - can’t get a graduate job and is instead, ten years later, a ticket collector on the railways.

    In answer to your puzzle, I’d like to believe that we are on the cusp of some form of capitalisation on the improved environmental factors for pupils, a culture change that hasn’t quite got rolling yet. However, until we burst this inflationary bubble, remove the incentives to expertly adhere to some exam syllabus minimum standard, and transform teacher education so that it includes the psychology, philosophy and sociology of education that will help teachers to develop an autonomy of practice and awareness beyond delivery of content, we can’t even begin the conversation.

    ReplyDelete
    Replies
    1. In that last paragraph, I should have said "that will help A MAJORITY RATHER THAN A MINORITY OF teachers to develop an autonomy of practice and awareness beyond delivery of content, we can’t even begin the conversation". :-)

      Delete
  3. If I can summarise your thoughts here, Sam, you seem to be saying that, subjectively, schools have improved in the past ten years. They seem to be better run, with more motivated children who generally make more positive social decisions. Given all the improvements, ‘why haven't we seen a commensurate, observable, rise in academic standards?’

    Your two possible explanations (standards have improved or something in schools is preventing this) both make the assumption that the input (better schooling) automatically leads to a better output (higher academic standards) – that ‘school effects’ outweigh ‘pupil effects’.

    I would suggest that this assumption – which underpins a great deal of thinking about school systems – is based on little more than wishful thinking, and that there is virtually no evidence that ‘better school input’ automatically equals ‘better pupil output’. Pupils have a great deal of agency and react very differently to the educational ‘inputs’ to which they are subjected.

    My alternative hypothesis to your two possible explanations for the lack of a rise in academic standards is as follows: Pupil’s academic outcomes are driven by pupil effects rather than teacher or school effect ‘inputs’. Whilst ‘Pupil effects’ is a broad term, I use it to mean pupil prior attainment, parental education, cultural and social capital and so on. Additionally, as Steve Higgins and Peter Tymms say in the recent Demos ‘A Tale of Two Classrooms’ report, "We differ at a psychological level in our capacity to learn, and our capability to learn to learn," which significantly underpins observable pupil effects.

    None of this is to suggest that we shouldn’t aim for the highest possible academic standards for all children. There might, however, be alternative explanations as to why, despite the view that schools have improved over that time, and that pupils are choosing to make more positive social decisions, academic standards have not automatically risen as a result.

    Jack Marwood
    http://icingonthecakeblog.weebly.com

    ReplyDelete
    Replies
    1. Thanks for your comment Jack but I think you've got my reasoning the wrong way round.

      I'm arguing that the "pupil effects" have changed - i.e. social capital has improved and that this - all things being equal - should lead to increased attainment. This is regardless of whether school has any effect or not.

      In trying to understand why this expected increase in attainment hasn't happened I suggest that something schools are doing may be getting in the way. But this is just one hypothesis. If it isn't true we still need an explanation for why improved social capital isn't leading to increased attainment.

      Delete
  4. And the trends are very clear. Ironically, given the recent obsession with "character building" amongst policymakers, there have been big improvements in a range of "non-cognitive" measures.

    10th Class Date Sheet 2015 BISE Multan Board

    ReplyDelete
  5. I started teaching in a maintained comprehensive school in 1980, and it seems to me that pupils and teachers get a far better deal now - we know so much more about teaching and learning and how to motivate people; teacher's professional development is taken far more seriously; there is greater emphasis on collaboration and working together within and across schools.

    But I fully accept that this could be cognitive bias at work - as you say, Sam, "We all desperately want things to improve so we convince ourselves they have."

    I need to give more thought to this....

    ReplyDelete
  6. It could be that something has affected all those life choice decisions that has not had the same effect on academic standards - if the social expectations of communities has changed maybe, or the behaviour of role models. As an example, attitudes towards smoking in most areas of society have changed but perhaps attitudes to homework haven't. This isn't necessarily at odds with the increased expectations about higher education - more young people are going to university so the reality has changed here, but that doesn't mean that more young people choose to work harder in order to achieve their, new, higher education goal. The recent research on children from different cultural groups (that shows that the same ethnic / cultural groups excel on PISA whether they are in Shanghai, Sydney or Slough) suggests, as Jack points out, that where kids are coming from may matter a whole lot more than what their schools do.

    As far as the Coe inaugural address is concerned, I think the discussion of the poor understanding of what makes a school effective, or even how to identify such schools (or teachers), is at least as important as the part about the lack of convincing improvement. I too think that schools have improved on balance but I wonder if the things we really notice, like atmosphere, behaviour, bullying (identified above) and teachers' workload have changed but not so much the fundamental attitudes to work and achievement held by pupils.

    I tend to think that policy changes take months, school changes take years, and society the best part of a generation. Maybe it's coming...?

    ReplyDelete
  7. Interesting and thought provoking. The thing is, the only evidence presented by anyone that standards haven't risen (since results in national tests such as GCSE and A level seem to imply that they have) is tests such as the PISA test, with which I have many serious issues: it is simply not, to me, a credible means of anything beyond the ability to do well on the sort of old-fashioned and highly artificial "real world problems" it presents.

    I wrote a blog post on this, and on the broader issue of the current government's apparent lack of interest in an evidence base to support policy decisions. http://theyweewords.blogspot.co.uk/2014/09/the-astonishing-lack-of-evidence-base.html

    ReplyDelete
  8. The global level college students are connected in their
    custom essay writing The post graduate and under graduate student’s writing papers are supplying in global level

    ReplyDelete
  9. Have you considered risk homeostasis- a theory generally considered with respect to traffic accidents. However, the "concept applies to all behaviour domains" (http://riskhomeostasis.org/about-risk-homeostasis). I am suggesting that educational achievement is akin to the Munich taxi drivers: "Some of the cars were outfitted with ABS, other cars were not but in all other respects identical. The drivers anticipated the effect of ABS on the controllability of their vehicles and thus their safety, and adapted their behaviour accordingly" (same website). If students have an educational set point (possibly of socio-economic determination) any positive change will be offset by a negative change elsewhere in the system.

    ReplyDelete