What gets counted, and who gets missed

Written By Amber Windsor

What addiction services taught me about measuring “success”, and what it could mean for children under the SEND reforms

My background is in addictions. Before moving into SEND, I was doing a PhD exploring the lived experiences of people who use crack cocaine and heroin, looking at their lives as a whole rather than in black-and-white terms, including how much of the harm they faced came from punitive responses and the way services treated them, as well as from the drugs themselves, and how to make that kind of research meaningful beyond academia. Alongside that, I worked as a care coordinator and neurodivergence lead in a drug and alcohol service.

That role, together with what was happening for my own child at school at the time, is what led me into SEND. Day after day I was sitting with adults whose needs had been missed or misunderstood since childhood, and I couldn't shake the sense that many of them might never have needed to sit in front of me if those needs had been understood and met when they were at school. That's a big part of why I pay such close attention to how the SEND system decides what counts as success.

What "success" looked like where I used to work

Some of what I'm now seeing in councils' SEND reform plans takes me straight back to that service, so I want to explain how it measured "success", because I think it helps make sense of what could happen to children's support over the next few years.

Like many services funded through public contracts, it had to show the organisations paying for it that it was doing a good job. One of the main ways drug and alcohol services in England are judged is by counting "successful completions", which nationally means people who leave treatment free of the substance they were dependent on and who don't come back to treatment within six months.

Six months without coming back might sound like a fair test, until you look at what it leaves out. It only measures whether someone returns to treatment, so a person who relapsed and never came back, or who ended up in A&E, with a crisis team or in prison instead, could still be counted as a success. Six months is also a very short time in recovery from long-term dependence, and the check on whether someone has returned only happens months after they've left, so the pressure in my team was on getting people to that discharge point as quickly as possible.

The colleagues who moved people through fastest, getting them to the point of reducing or stopping (safely, with medical oversight where alcohol dependence was involved) and then discharging them, were the ones seen as doing the best work and given the most praise. Whether that person might be struggling again a few months later, or back through someone else's door, wasn't part of the picture the figures showed.

I worked differently, as did some colleagues like me. Most of the people I supported were neurodivergent, whether diagnosed, undiagnosed or waiting for assessment, and many had spent years being misunderstood by the services around them. I was focused on helping people reach a point of recovery they could sustain, whatever that looked like for them, so I often took much longer to reach a stage where I felt comfortable discharging someone. My caseload stayed bigger, my completion figures looked lower, and I was regularly pulled up by management about how long people were staying with me.

The research supports that approach. A national five-year study of people in treatment for opioid dependence in England found that those who stayed in treatment longer were more likely to complete it successfully and to keep the benefits afterwards.

I argued repeatedly for the service to recognise how many of the people we worked with were neurodivergent, and for an approach that gave people the time they needed to make changes that would last.

When a number becomes the target

Nobody in that service set out to give people worse support. The difficulty was in what got counted: the organisations funding the service needed a quick way to see whether it was working, and completions were easy to count, whereas whether someone was still doing well a year later was much harder to track, so it mostly wasn't.

There's a well-known name for what happens next. Goodhart's law is usually summed up as the idea that once a measure becomes a target, it stops being a good measure, because when people are judged on a number, the number starts to shape the work, even when everyone involved means well.

Think of a customer helpline judged on how short its calls are. The quickest way to improve that figure is to wrap calls up fast, which can leave people's problems unsolved and have them ringing back the following week, yet on the helpline's own figures it looks like a great improvement. The staff who take the time to sort things out properly end up looking like the slowest on the team, and the cost of the rushed calls lands somewhere else, often back on the customer.

I saw exactly that pattern in drug and alcohol services, and I think it's well worth keeping in mind as councils start reporting on how their SEND reforms are going.

We've already seen this happen in education

Education has a long record of this. When schools are judged heavily on test results and league tables, some of that pressure turns into practices that improve the figures without helping children. The House of Commons Education Committee found the accountability system was a major factor in "off-rolling", where pupils are moved off a school's roll before their exams, often without any formal exclusion, and Ofsted's chief inspector at the time described schools pushing vulnerable pupils out through the back door. The More Than a Score campaign, a coalition of parents, teachers and child development experts, has spent years making the case that high-stakes testing in primary schools narrows what children learn and piles pressure on them, largely to produce figures for judging schools. Many schools and teachers push back against this every day, and the problem sits with how the system judges them.

SEND has its own history too. The Education Committee's 2019 inquiry into SEND found that the 2014 reforms had led, at times, to unlawful practice, including councils using their own criteria for agreeing to assess children that the law doesn't allow.

The closest example is the Safety Valve programme, under which councils with the biggest SEND deficits received extra Government money in return for agreeing to bring their spending under control. The Government consistently said these councils had no targets to cut EHCPs, yet research the DfE itself commissioned found that EHCPs grew noticeably more slowly in Safety Valve areas than elsewhere, some school leaders and parents linked the agreements to tougher decisions on assessments and plans, and council leaders acknowledged the scheme had come to be seen as one that meant cutting costs and reducing support. IPSEA and Contact both concluded that the agreements centred on cost-cutting rather than children's needs and legal rights.

Local SEND Reform Plans, with Government financial help tied to plans being approved and progress reported every three months, look to me very much like the Safety Valve approach being extended to every council in England.

What councils' SEND reform plans are measuring

Children and young people with special educational needs can have an Education, Health and Care Plan (EHCP), a legal document setting out their needs and the support they must receive, which the council then has a legal duty to make sure is in place. Families usually get one by asking the council for an EHC needs assessment.

As part of the Government's SEND reforms, councils in England have been drawing up Local SEND Reform Plans. The Department for Education has confirmed that councils must keep these plans up to date and send it monitoring returns every three months, and that Government help with councils' historic SEND deficits depends on their plans being approved.

Matt Keer at Special Needs Jungle has now looked at 60 of these plans that he was able to get hold of, some of them drafts. Common aims include fewer requests for EHC needs assessments, slower growth in the number of EHCPs, more children placed in mainstream schools and less use of independent specialist schools. He also looked at the data templates the DfE asks councils to fill in, and found they track spending, where children are placed and how much assessment activity is happening, without directly measuring the quality of the education and support children actually receive.

It's worth being clear about what that does and doesn't tell us. This is an independent analysis covering fewer than half of England's councils, some of the documents were drafts, and none of it creates new legal requirements. What it does show is the kind of figures councils expect to be judged on.

My reading is that some of those aims could reflect real progress. If children get the right support in school earlier, fewer families may need to ask for an assessment, and many schools I work with already do a huge amount for children and push hard on their behalf, often getting stuck with the council themselves. Fewer requests could equally mean families are being discouraged, told it isn't worth applying, or simply giving up. On the figures alone, those two very different situations would look exactly the same, which is the same blind spot I saw with "successful completions". Given what happened under Safety Valve, I don't think it's safe to assume the more hopeful reading.

What families are starting to report

Every EHCP has to be reviewed at least once a year at an annual review, where the child's progress and needs are looked at and the plan can be changed if the evidence supports it.

In a recent post, Michael Charles described having to stop a council from amending a child's EHCP ahead of an annual review, and raised concerns about council officers becoming more involved in reviews, possibly with the aim of reducing support or ending plans altogether. Many of the replies came from families and professionals describing similar experiences in their own areas.

These are individual accounts, and on their own they don't show that every council is putting the expected reforms into practice early. They are, however, consistent with what families and advocates have described for years, particularly in Safety Valve areas, and with what I'd expect to see when councils are judged mainly on their numbers. It's well worth keeping a careful record of what happens at your child's reviews, including who attends and what reasons are given for any proposed changes.

Whatever the reason, the law hasn't changed. A council can amend or end an EHCP only where the legal process is followed and the evidence about the child supports it. A council wanting fewer plans, or having targets in its reform plan, doesn't change that. In practice, if a council proposes reducing or removing your child's support, you can ask what evidence about your child has changed, and you can challenge the decision, whether through mediation, an appeal to the SEND Tribunal, a complaint or other routes.

What happens after the plan ends

The part of my old job I think about most is what happened after someone was discharged. Were they still well, still safe, able to get help again if they needed it? The completion figures didn't tell us, because the counting stopped at the point of discharge.

The same question applies to children. If a child's EHCP is ended, or a request for an assessment is turned down, what is in place to find out how they're doing afterwards? Is the child still in school and coping, or have they ended up out of school, in burnout or unable to attend? Is the family now paying privately for therapy the council used to fund, and are they back asking for an assessment a year later? If the monitoring stops when the plan stops, a council could record a drop in EHCPs without ever knowing whether the children behind that figure are getting what they need. I haven't seen anything published about longer-term follow-up of these children, though that is something to ask the DfE about rather than assume.

There's also the question of where the costs go. At the drug and alcohol service, a quick discharge could shrink one caseload while pushing more people towards crisis services or back into treatment later on. With SEND, money saved from one council budget could reappear in health services, social care, schools, the Tribunal or families' own pockets, which could look like a saving on one spreadsheet without anything being cheaper overall.

The adult end of the system is where I first saw what unmet needs from childhood can cost, years after anyone had stopped counting.

Questions worth asking

There's already plenty of evidence about where a system judged on these kinds of figures can lead, so the real question is whether the DfE has learned from Safety Valve, and whether the new system can tell the difference between children's support improving and children's support simply being harder to get. If you're writing to your MP or a local councillor about the reforms, these are questions you could ask:

  • How will the success of the SEND reforms be measured, beyond fewer assessment requests, fewer EHCPs and lower spending?

  • What will stop councils treating a fall in EHCP numbers as proof of better inclusion, without showing that children's needs are being met?

  • Will children whose EHCPs are ended, or whose assessment requests are turned down, be followed up over time to check they're still getting suitable support?

  • How will the DfE spot councils whose finances look better while complaints, exclusions, children out of school or unmet needs rise?

  • Will the DfE publish how it assesses Local SEND Reform Plans, including how much weight it gives to children's experiences compared with financial figures?

At the drug and alcohol service, I was judged on how quickly I could close a case, when what I cared about was whether the person would still be doing well long after they'd left. For children with SEND, I think the measure of the reforms should be whether children's needs are met, and whether they're still being met a year or two on, rather than how many plans a council has managed to close.

Further help

Sources

Amber Windsor
Next

Sitting in a Classroom and Belonging in It: What Inclusion Should Mean Under the SEND Reforms