Confidence Intervals and Raiseonline
Confidence Intervals and Raiseonline

Confidence Intervals and Raiseonline

It has been a long time since I had to actually calculate a confidence interval, my research statistics class at SIUE to be precise. It’s hard to forget such riveting lectures such ‘Surely God loves .06 as much as he loves .05?’ I know, exciting heh. Now it’s time to work with them again in Raiseonline. More importantly I wanted to find a simple way to explain them why they were important to colleagues.

More than half (up to 60%) of the variation in pupils’ KS4 results can be explained in terms of their prior attainment and contextual factors. The remaining 40-50% of variation is due to factors for which we have no data. We might hypothesise that this could be down to attendance, or parental and carer attitudes and support, or variations in the quality of teaching and learning for example. All of which gives rise to a school’s value added score. If we could explain 100% of variation in pupils’ results, all schools in the country would have a value added score of 100.

In other words, the variation in results that cannot be explained by differences between schools in terms of prior attainment and pupil contexts is attributed to the school and presented as a value added score. But if 40-50% of variation cannot be explained, how confident can we be that the figures produced by CVA are reliable?

The statistical solution to this dilemma is to calculate confidence intervals. Confidence intervals allow us to determine a range of values we can be 95% confident the school’s CVA score falls between, given that we cannot account for all the variation in pupils’ KS4 results in the CVA methodology. In RAISEonline, confidence intervals are shown on reports as numbers on reports and as whiskers on charts.

OR

The uncertainty of a contextual value added score as a measure of school effectiveness can be presented as a confidence interval.

For example a school’s CVA score is 100.9 and confidence interval is 0.8. This means that we can be 95% confident that the school’s score lies between (101.1 – 0.8) and (101.1 + 0.8), i.e. between 100.3 and 101.9. In either case, we can be 95% sure that the score is above 100.0, above average, and therefore sig+.

Alternatively, if a school’s CVA score is 100.5 and the confidence interval is 0.6 (100.5 – 0.6) and (100.5 + 0.6), i.e. between 99.9 and 101.1, the lower number is below 100.0, and we cannot be confident that the school’s score is significantly different from average. If both the lower limit and upper limit were below 100.0, below average, and therefore sig-

And as I have learnt, as with most school statistics, smaller schools, small data sets, tend to have larger confidence intervals, since we are estimating achievement on a smaller number of results, so we have less evidence on which to judge a school’s effectiveness. Right, that is about it, I hope.

[qr_code_display]

Leave a Reply