The question of how to make the college admissions process as fair as possible brings intense controversy. The lack of a shared understanding about what it means for something to be “fair” only makes the topic more difficult to navigate. Debates span subjects ranging from the elimination of test scores (see here and here) to the extent to which race should be considered in admissions decisions. Stakeholders include groups and individuals that often have competing interests and divergent priorities — current students, applicants, parents, and the broader university communities. For all these reasons, frank conversations about the future of college admissions require an openness to a wide range of viewpoints and a willingness to see the related challenges from multiple perspectives. And yet, discourse on this topic has largely been highly constrained, which has put up barriers to real engagement.

While much of the focus on the admissions process has been on how various criteria (e.g., test scores, GPA, essays, extracurriculars) are weighted, an equally important component is the pool of students against which each applicant is competing. That pool directly shapes any single applicant’s probability of acceptance. And one of the factors that shapes the applicant pool is college rankings.

Students and their families use college rankings to assemble a list of potential schools to apply to. When applicants combine knowledge of their own credentials with the ranking of (and information about the credentials required by) an institution, they can better select an appropriate balance of “safety” schools, where they have a decent chance of being accepted, and “reach” schools. Rankings convey information about selectivity and prestige. The prestige of Ivy League schools is widely known and recognized. For non-Ivy League schools, rankings can separate institutions hierarchically that may otherwise be difficult to distinguish from one another.

In this context, a recent sociological research paper considered whether rankings themselves have important negative consequences. The paper, titled “Cameras of Merit or Engines of Inequality? College Ranking Systems and the Enrollment of Disadvantaged Students,” was published in the American Journal of Sociology in May 2021.

The paper’s author, James Chu of Columbia University, looked at the expansion of ranking systems with the goal of addressing the concern that these systems “reduce the accessibility of goods and services for individuals of lower socioeconomic status backgrounds.” Using administrative data from U.S. News & World Report (USNWR) from 1997 and 2015, he leveraged a scoring change that occurred at two different time points.

USNWR divides schools into four tiers. In 1997, only tier 1 schools received an institution-level ranking. In 2004, tier 2 schools, which had previously been unranked numerically at the institution level, became ranked, and in 2011, tier 3 schools were ranked. Tier 4 institutions remained unranked throughout this period. Leveraging these changes, Chu compared enrollment patterns within institutions before and after being ranked at the institution (rather than the broader tier) level.

Chu proposed and tested two possible mechanisms that might worsen inequality. Positional disadvantage is based on the idea that when an institution rises in the rankings, more people want to apply and attend that school. Given that most colleges cannot quickly expand capacity for their incoming class, this increased demand results in a more selective admissions process. This more stringent selection process, in turn, makes it harder for students of lower socioeconomic status (SES) to compete for slots. Algorithmic tradeoffs, on the other hand, consist of potential shifts in how organizations think about priorities for admissions. Chu posits that a desire to continue to improve in rankings creates an incentive for schools to prioritize factors, most notably test scores, that contribute significantly to the USNWR rankings.

The results of his analysis found no evidence that positional disadvantage played a role in increasing inequality and some support for a detrimental role of algorithmic tradeoffs. Chu writes that “exposure to rankings led colleges to reduce their share of enrolled Pell grant recipients and first-generation students, and mediation analyses suggest that this was driven by optimization on student test scores.” In other words, rankings ultimately harm some of the most disadvantaged students.

Chu’s study on rankings and inequality is both timely and important. The college admissions process in general, and the use of test scores in particular, was thrown off for the class applying for fall 2021 admission, due to limitations imposed by Covid-19. It remains to be seen what it will look like going forward. This makes having a comprehensive understanding of both access and opportunity all the more important. In other words, though we don’t know what the future looks like for rankings, or perhaps because we don’t know, Chu’s conclusions should be considered.

Important Questions to Consider

At the core of a discussion about rankings and inequality are questions about rankings’ costs and benefits, which are worth considering fully. For instance, do rankings, as suggested in the paper, “neutrally reflect the merit of organizations” or are they a force that “actively reshape[s] and reconfigure[s] society?” This framing, however, is an unnecessary oversimplification. Although Chu touches on some of the complexity, the stakes are high and the topic so fraught that it’s worth exploring more deeply. For instance, in some ways, rankings serve as an equalizer of information such that they enable applicants from all backgrounds to make conscious decisions about which schools they want to target.

We might also zoom out and consider the purpose of rankings more generally. Rankings are often presumed to reflect something about the prestige of the institution. The prestige of the institution is, in turn, seen as tied to the quality of the educational experience and of post-college career opportunities. One can imagine that all three factors — the prestige of the school, the quality of the education, and career opportunities — matter for student outcomes. They matter through the direct transfer of skills, the perception of capability and competence that is conferred to potential employers, and the long-term network benefits of having highly capable, accomplished, and ambitious peers. (It goes without saying that students with those characteristics can be found at all institutions.)

Although certainly other questions remain, I’ll entertain one more:

What happens when an unranked school becomes ranked? Chu focuses on students from disadvantaged backgrounds who are subsequently squeezed out — surely a matter of concern. However, what about the outcomes for the students who do enroll? Are the students who remain, including those from disadvantaged backgrounds, better off after the rankings? Over time, do they end up with a greater skill set or more employment opportunities? Second — and Chu provides some thoughts on this point — what happens to the students who would have previously gotten into a particular school and didn’t because of the shift in focus to test scores? Do they go to school somewhere else? Do they not go to school at all? What are their outcomes? The well-being of all students seems important to consider.

One of the possible solutions suggested by Chu is that the USNWR ranking system could give more explicit attention to socioeconomic diversity, thus creating an incentive for institutions to explicitly factor this into admissions decisions. Weighting socioeconomic diversity in rankings brings us back to one of the questions posed here about the purpose of rankings more generally. What information does socioeconomic diversity convey about the prestige of an institution or the quality of an education?

The issues around the college admissions process in general and the relationship between rankings and inequality specifically are complicated. While Chu’s paper is an important contribution to this dialogue, it certainly isn’t the end of the story. Whatever the path forward is, it should be the result of thoughtful discussions from people with a wide range of ideological and political orientations. We may never agree on what fairness is, but we can start with constructive engagement.