The choice of where to send your child to school can feel unbelievably consequential. School is where they will spend the majority of their waking hours, for at least 12 or so years. It’s where they’ll (hopefully) learn to read, to do math, to interact with other people and exist in a constrained social environment. It’s where they will find their friends and, come middle school, their archnemeses. Given all of this, it’s unsurprising that it is a hugely stressful choice for many parents. You do not want to mess up and risk your child’s hating learning or, perhaps worse, finding the wrong archnemesis. These are formative years!
This stress translates into parents looking to find the “best” school for their child. And that goal often leads them to school rankings. These are everywhere — GreatSchools.org, Niche, U.S. News & World Report. Websites like these grade and rank schools. For example, you can go to U.S. News & World Report and see a ranking of public or charter schools in your state. (Here’s the data for Rhode Island, where I live.)
Rankings like these influence the school districts we move to, the schools we choose within those districts, and parental decisions about outside options like private school. Despite their influence, I think the data cautions strongly against overuse of rankings like these. They get more attention than they deserve in parental decision-making.
This is true for two specific reasons, both rooted in economics. One is that rankings do not take into account the variation in needs and desires across families. The second is that the rankings are subject to selection bias. Both of these issues boil down to the need to consider what you are looking to achieve with your school choice.
There is no simple bottom line here, but the bottom line is that school rankings should — at best — be one input to your decision-making. They are absolutely no substitute for thinking this through deliberately in the context of your particular kid.
The limited value of a single ranking
The first issue in using these rankings is that — in this case, as in many others — they collapse a lot of data into a single ranking that may not be relevant for all, or even any, people.
This issue relates to a paper that my husband, Jesse Shapiro, wrote with Isaiah Andrews. Jesse and Isaiah’s paper is called “A Model of Scientific Communication” (which is one of their shorter and more understandable titles). The basic idea is that the most helpful way to communicate data to people depends on how similar their preferences are.
Online school rankings take into account many factors. Nearly all of them use student test scores as a key input, but some of them also take into account student-teacher ratio, ethnic diversity, how test scores vary by demographic group, school funding, and other variables.
If everyone agreed on how much each of these components should be valued in the ranking, then the simplest way to communicate the information to people is with a single ranking (which reflects these universal values). This would be great, because looking at one number is simple, and if we all agree on how to construct that number, we are all set.
However: if people do not agree — if some people put greater value on diversity and others greater value on math test scores, and still others disagree on how to trade off class size and school funding — then this simple “one number” approach to data communication doesn’t work. It would be much better (this is the point of Jesse and Isaiah’s paper) to provide people with the individual components and let them decide for themselves how to combine them.
Whether the single ranking or multiple data points is better depends on how similar people’s preferences are. My view is that in the context of schools, the answer may be “not very similar,” in which case the single ranking is not very useful. Put differently: The rankings in, say, U.S. News & World Report are most valuable to you if your preferences align strongly with what U.S. News thinks is important. If they do not, then these rankings on their own may not really help at all.
What do the rankings really capture?
A second core issue here is the question of what these rankings actually capture. The most important component in nearly all of them is state test scores. In the U.S., every state tests all students in grades 3 through 8 in math and in English language arts every year. Test scores — typically reported in pass rates (defined by the state standard) — are available by school and, usually, demographic group. One school may have, say, 40% of their students proficient in math, while another has 70%. The school rankings will generally be higher for those with the 70% rather than the 40%.
To reiterate some of the points above, it’s not at all clear that these scores are what every parent cares about, certainly not exclusively. But it turns out even if all you care about is your child’s performance on these tests, these rankings may not be sufficient.
As a parent, thinking about my child’s learning and test performance, my goal may be to pick the school that delivers the largest increase in test performance. This concept is sometimes called “value added” — with this test-focused preference set, you’d want to pick the school that has the highest value added in scores. The rankings, though, focus on levels. And it turns out that those numbers are not necessarily the same.
The reason, which is perhaps obvious, is selection into schools. Schools pull from local areas. Areas with higher-income households, where parents have more education themselves and where resources are overall greater, tend to have schools with higher scores. But this would be true even if all schools were identical in their teaching and learning, because we know family background matters for test scores.
A school in which 70% of the children are testing proficient in math in a very poor school district is, on average, going to be delivering greater learning than a school with the same proficiency in a rich district. The rich school district is starting with kids who have more advantages coming in — more tutoring resources, more books at home, and on and on — and their scores are reflecting that even if they do nothing in school.
We know this issue is true in theory, but a new paper shows how it works in practice. The authors (economists, including my wonderful Brown University colleague Peter Hull) show that school rankings are highly correlated with the share of white students in the school (with schools with more white students being ranked better). When they isolate the “value added” of the schools, though, they find no relationship between racial composition and value added. All of the higher ranking of the schools with greater white enrollment was a result of bias.
From a personal standpoint, what does this mean? First: Anytime you see these rankings, remember they reflect, in large part, differences in income and other demographics and their impact on test scores. This can help contextualize.
Second, more practically: all of this means that differences in test scores across schools are almost certain to overstate the impact of the school on your individual child’s test scores. If your child goes to a school with a 10% higher math pass rate on the state exams, you should expect significantly less than a 10% increase in the chance that they pass the exams. The influence of the school on your child is simply less than the average difference.
Is any of the data useful?
The first discussion above suggests that you may want to seek out a range of information on schools to make decisions that align with your preferences. The second point is that test scores themselves may matter less for your individual child. With this in mind: is there anything you should look at?
In the data, there are a couple of school-level inputs, or characteristics, that do seem to be causally linked to student learning. One is student-teacher ratio: fewer students per teacher has been shown to increase learning (it’s true that richer districts have a smaller number of students per teacher, but other analyses that are able to isolate causality also show this). A second input is accountability — schools in which there is more teacher training and follow-up accountability tend to have better outcomes. This evidence is largely based on studies of what works in charter schools.
Beyond these considerations, this is a choice that is going to come down to a lot of family-specific questions. What does your child need? What’s the feel of the school? Is a smaller house in a more expensive school district right for you, or a larger house in a less expensive one? How much do you value your child being able to walk to school versus being driven or taking the bus?
These are all extremely important, and this is an example of a decision I talk about in The Family Firm as being amenable to the “Four Fs” framework for hard choices. School ranking may be one input into this hard decision, but it is only one, and probably not the most important.
Log in