This post was originally published at Psychology Today.

A few months ago, I posted a blog post that raised the question, “What explains racial, gender, and other group-based gaps?” After acknowledging the existence of all sorts of gaps across all sorts of groups, I ended that blog post posing this question:

THE QUESTION

If a university admitted 70% of the men who applied, and only 30% of the women who applied, and the men and women were exactly equally qualified, would that be conclusive evidence that that university was engaging in sex discrimination?

The answer is “no.” Before I explain why, let’s have some fun first by giving a concrete example of why answering this type of question is so important.

AAAS CONFERENCE ENDORSES IMPLICIT BIAS AS EXPLAINING GAPS IN PEER REVIEW

Historically, discrimination has surely played a role in many gaps. And discrimination is not dead, and probably never will be. However, gaps can occur for lots of reasons other than ongoing discrimination at the point at which the gap occurs, such as, aftereffects of historical discrimination, self-selection, prior differences in prior educational attainment (which themselves may but do not necessarily reflect discrimination), inherited wealth (and the benefits obtained therefrom), etc.

Nonetheless, across many fields, there is a frequent knee-jerk assumption – among both laypeople and scientists — that almost any demographic gap, especially if it involves some group my colleagues consider “protected” or whose interests they wish to advance — is discrimination.

If one has any doubt, see this report by the American Association for the Advancement of Science, titled, “Journals and Funders Confront Implicit Bias in Peer Review.” It reports presentation after presentation by famous, influential, and prestigious scientists. Many of these make explicit statements presuming the existence, power, and importance of implicit biases in peer review.

“Implicit bias” generally refers to biases of which people are unaware and is more or less synonymous with unconscious bias, and a close cousin of subtle bias. This “family” of biases is generally distinguished from explicit biases which are of the cruder sort (advocating, e.g., for gender or racial segregation, inequality, etc.).

AAAS CONFERENCE PRESENTS NO EVIDENCE THAT IMPLICIT BIAS EXPLAINS GAPS IN PEER REVIEW

It also did not present a shred of evidence of implicit bias in peer review. Just read it – you can see for yourselves. Or, more exactly, you will NOT be able to see such evidence for yourselves, because no such evidence was presented there to see. There was lots of evidence of demographic differences in academic success rates (publication rates, first authorship rates, funding) – but not a shred on where those gaps come from. Of course, if “everyone knows” gap=discrimination, then evidence is “not necessary.” But how could they know that without evidence? And if they had the evidence, why wouldn’t they present it? And if they did present it, why didn’t the article report it?

Source: Alain Riazuelo, obtained via NASA

This reminds me of what I have been calling the black hole at the bottom of declarations of stereotype inaccuracy. For decades, researchers have been declaring stereotypes inaccurate without providing a shred of evidence that they are actually inaccurate. In the same way, apparently, there was a strong consensus at this conference that implicit bias leads to demographic gaps in scientific publishing and funding, but that consensus seems to have been based on … nothing at all directly linking implicit biases to discrimination in peer review.

Now, to be fair, they did present ample evidence of fairly large racial and gender gaps in some outcomes in some fields. And documenting such gaps is important in itself. But this should be obvious: documenting the existence of some pattern or difference provides no information about what explains that pattern or difference. Most readers of this blog will know that the Amazon gets much more rain than does the Sahara. Does knowing this tell you why? Most readers know that Google is a larger and more profitable company than is Ben and Jerry’s Ice Cream. Does knowing this tell you why? This is really obvious in nearly all contexts other than demographic gaps; knowing some difference exists provides no evidence about the explanation for that difference.

So why is this point – knowing some difference exists provides no explanation for that difference — so not obvious when it comes to understanding demographic group differences? I can think of a semi-innocent explanation and a less innocent explanation. The semi-innocent explanation is that everyone knows that discrimination can cause gaps; the social sciences spend a lot of effort trying to understand discrimination; so discrimination is the most readily available, top of the head “explanation.” Of course, part of social scientists’ job is to try to get past such top of the head explanations, so this explanation is, at best, semi-innocent.

The less innocent explanation is that this is motivated distortion of science. If social scientists have a political agenda of advancing the interests of certain “protected” groups, then casting such results as discrimination can advance that agenda. Indeed, leaping to the “discrimination” conclusion is a great way to signal to the world (or, at least, your overwhelmingly leftist colleagues) that you are a good, decent unbigoted egalitarian, fighting the good fight against racism, sexism, and all the other isms.

Furthermore, kneejerk assumptions of discrimination imply “answers” that require resources. If lower proportions of women or blacks are getting grants or publications, and if this is presumed to reflect bias (implicit/subtle/unconscious or explicit/blatant/conscious), then resources need to be funneled into programs to combat such biases.

WHAT’S SO BAD ABOUT THAT?

“Now,” you might say, “what’s so bad about that? Even if they are wrong, even you, Lee, agree that there are gaps, so what is the problem with funneling money and effort to programs to reduce those gaps?” The most obvious issue is the “curing the wrong disease” problem. If the gaps result primarily from things other than discrimination, targeting discrimination will probably fail to reduce those gaps. If one is serious about reducing some gap, one might also want to get serious about using time, effort, and money wisely.

A secondary issue is the self-selection problem. If gaps result primarily from unfettered** self-selection, they can only be reduced by shunting some people into activities they would rather not engage in.

** By “unfettered,” I mean truly freely chosen. If people self-select out of a field, e.g., because they fear harassment or discrimination, this is not what I mean by “unfettered.”

So, this leads me to two conclusions:

Source: Jean-Pierre Dalbera, via Wikimedia Commons

1. If we want to eliminate a gap, it behooves us to work quite hard to understand its sources, and then to target those sources.

2. If we had a better understanding of gaps, there are some we might not want to eliminate. Many of us would rather not be in the business of pushing people into fields they would rather not enter in the name of some egalitarian ideal.

THE ANSWER TO MY ORIGINAL QUESTION

The answer is “no.” As in “admitting 70% of the men who apply and 30% of the women who apply is not conclusive evidence that a university has engaged in sex discrimination.” It could be discrimination, but it is not conclusively discrimination. First, you would want to be sure to compare equally qualified men and women. If men have stronger qualifications, it could account for all or some of the gap at admissions. However, in the modern era, this is extremely unlikely, because girls do better at every level of schooling than do boys. (Wait! That’s a gap! Why is AAAS not holding conferences about the growing female advantage in school achievement?).

But here is the kicker. Even if women and men were, on average equally qualified, with nearly identical distributions with respect to qualifications, and the university accepted 70% of the men and 30% of the women, it would still not be conclusive evidence of discrimination. (Actually, this is even true if women were more highly qualified, but the math gets more complex, so let’s simplify and assume men and women are exactly equal in qualifications).

This is because of Simpson’s Paradox: A valid statistical conclusion for an entire sample may be invalid for all subsamples. Even though more men are admitted overall, it is possible that every program in the university admits more women. Here is an exaggerated hypothetical example:

Source: Lee Jussim

Even though men are admitted at a 71% rate and women at a 29% rate, women are more likely than men to be admitted both to competitive (22% [200/900] vs. 10% [10/100]) and to easy departments (90% [90/100] vs. 78% [700/900]).

You might need some time to digest this table. Feel free to take your time, this blog is not going anywhere…

Although my example is hypothetical and purposely extreme for clarity, it was inspired by a very real case that occurred at UC Berkeley in the 1970s. Berkeley was sued for sex discrimination because only 35% of women were admitted to its graduate schools, whereas 44% of men were admitted. However, Berkeley ultimately won the suit because of Simpson’s Paradox. Women were as or more likely than men to be admitted to nearly every department.

Berkeley had 85 departments; details regarding the six largest departments are available on Wikipedia under “Simpson’s Paradox”. Interested readers can also consult Bickel et al. (1975) for more details.

Now, nothing here explains why women were more interested in the more difficult and competitive programs. Perhaps discrimination did play a role earlier in the system somehow. Of course, that, too, would require evidence, and we cannot conclude that discrimination did play a role just because it might have.

Next time you see some headline statistic, “Women make 77%** of what men make” or “Only 3% of college students are African American” or some such, remember Simpson’s paradox. The headline number may be true, and it certainly indicates a gap. But whether or not it reflects discrimination at the same level of the headline (wages for women by job in the first example; admissions of African Americans to college in the second) requires evidence that goes well beyond the headline.

** I pulled 77% out of my ear because this number has ranged between 75% and 80% in the U.S. for a long time.

For example, that headline sex difference mostly reflects women working in lower paying jobs than men are working in. Christine Hoff Summers has a great blog post in the Huffington Post on this issue. Now, it is a reasonable question to ask, “Why are women disproportionately in those lower paying jobs?” and, again, the answer could involve discrimination, but, if it does, we need to see the evidence. Also, even after accounting for factors such as status of job and experience, women are still often paid slightly less. The more factors that are ruled out or controlled for, as gaps remain, the more likely discrimination at the level of pay for work done does indeed play some role (albeit typically a much smaller one than the headline number indicates).

Nonetheless, Simpson’s paradox does indeed seem to be playing some role in explaining that headline number. Women may earn 77% of what men make, but there are very few occupations that pay women 77% of what they pay men for the same work.

EPILOGUE

This is a true story. It is an anecdote. I am making ZERO claims that this generalizes to the wider world. Still, some of you may find this interesting.

When I chaired Rutgers Psychology Department (2010-2013), I received a call from some Rutgers administrative office, probably HR, which has an “Office of Employment Equity Investigations,” seeking explanations for Rutgers’ Psychology gender pay gap among its full professors.

I did not even know we had one, so I asked our departmental administrators to track down the salary data of the full professors. They did. As it turned out, the women were making considerably more than the men. In fact, that was what UHR or whoever called were inquiring about — they wanted me to explain why the women were making so much more than the men.

This, however, was not difficult to do. The department keeps each person’s vita on file. In general, our female full professors were more productive (more publications, more grants). Rutgers gives both across-the-board raises (same for all faculty) and merit raises (higher raises for productive faculty), so, in general, more productive faculty make more money. In one case, the person was hired with both a PhD and MD, and the MD gave her a considerably higher starting salary than other assistant professors. Because raises at Rutgers are a percentage of the base salary, a higher starting base salary would produce a higher subsequent salary, even if their productivity was the same.

I did not catch a whiff of discrimination, but, then, maybe I was just a blind fool, and should have raised Hell over the mistreatment of men in my department. After all, “everyone knows” gap=discrimination, right?

REFERENCES

Bickel, P. J., Hammel, E. A., O’Connell, J. W. (1975).Sex bias in graduate admissions: Data from Berkeley. Science, 187, 396-404.

Simpson, E. H. (1951). The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society, Series B, 13, 238-214.