Denver Post writer Eric Gorski wrote up a story on the best performing (alleged) schools according to Colorado School Grades when their latest rankings came out in December.
As I looked over the (supposed) best performing schools, I suspected a systemic bias to be present related to the kinds of students these schools served.
Acknowledging this bias is important, as we have literally decades of evidence telling us that overwhelmingly the largest determinants of the variance in student achievement outcomes (as measured on standardized tests) are related to out-of-school factors.
As a quick analysis, I looked up the percentage of students on Free/Reduced lunch at the top rated schools according to Colorado School Grades and then compared those percentages to the Free/Reduced lunch percentages in the district that school was located within.
Because of the propensity for Free/Reduced lunch reporting to be under-represented at high school, I restricted my analysis to just those (allegedly) top performing elementary and middle schools.
Perhaps not surprisingly, the difference between top schools according to Colorado School Grades and those communities in which they are located is quite large.
The top schools, according to Colorado School Grades, average 13.1% students who are eligible for Free/Reduced Lunch.
By contrast, the average for their communities is 32.6%.
Apparently, one major secret to being a high scoring school (using the Colorado School Grades methodology) is to systemically exclude disadvantaged students from your student population.
*Note* – this is not a personal vendetta against the folks at Colorado School Grades, or the slew of edu-non-profits that support them.
In fact, as Superintendent of Eagle County Schools, I’m particularly proud (although a little confused) of the 2014 “B+” grade given to Minturn Middle School, which has been closed since 2011 (D’oh!).
*Edit* Ben Degrow, with the Independence Institute, noted that MMS hadn’t had a “grade” since 2010. That is accurate – however, they still appear in the search when you look up our district. You can see the results of the search I did here, which clearly shows the now closed Minturn school still showing up, with a B+. Mea culpa – I made the mistake in assuming they received a current “grade” when I searched on this year’s results. However, I would humbly suggest that Colorado School Grades revise their search system to avoid such confusion in the future.
The point is this: we should be asking some critical questions about this data, how it is being used, and what it means.
We might also consider how we can really identify the real pockets of excellence; instead of just identifying pockets of affluence.