By Sean Stevens (HxA Research Director) and Jonathan Haidt (HxA Director)
This post is an addendum to our post: The Google Memo: What Does the Research Say About Gender Differences. Please read the original post first, before consulting this one.
In this addendum we focus on the Greater Male Variability Hypothesis – the idea that men are more variable than women on a variety of abilities, interests, and personality traits – and the possibility that males are overrepresented in the upper and lower tails of such distributions. This hypothesis was first proposed by Ellis over 100 years ago, in 1894. It is also the hypothesis that Lawrence Summers was referring to in 2005 when, at the National Bureau of Economic Research Conference, he weighed in on the gender gap in STEM professions. Like Damore’s memo, Summers’ comments spurred controversy.
As we noted previously we think the central empirical claim of Damore’s memo was: “When addressing the gap in representation in the population, we need to look at population level differences in distributions.” Damore then attempted to clarify his position with this figure:
Yet, research on the Greater Male Variability Hypothesis suggests that Damore’s figure does not correctly capture the state of affairs for some variable. This because on a variety of abilities, interests, and personality traits, there is no mean difference, but males are more variable than females, and they are more likely to be overrepresented in the tails of a distribution. To visually demonstrate this we present Figure 1 from Hyde and Mertz (2009), which displays theoretical distributions for males and females and takes a closer look at the upper right tail of this theoretical distribution:
In Hyde and Mertz’s Figure 1, green represents the theoretical distribution for females, orange represents the theoretical distribution for males, and brown represents the area of overlap between the 2 theoretical distributions. Hyde and Mertz also present a magnified image of the upper tail (3.8 to 4.2 standard deviations above the mean) where green represents females, red represents males, and brown represents the overlap of the 2 theoretical distributions. The Greater Male Variability Hypothesis would also predict a similar ratio of males to females in the lower tail (3.8 to 4.2 standard deviations below the mean). In other words, according to the Greater Male Variability Hypothesis males are expected to be overrepresented in the upper and lower tails.
Greater male variability, particularly in spatial abilities, mechanical reasoning, and mathematics, is likely to be relevant to the hiring practices of tech companies in general, and Google in particular. Simply put, Google is one of the most successful companies in the world. Because of this, they are likely to be hiring employees from the upper tail of distributions measuring abilities that the company considers relevant and important. Furthermore, because Google is a tech company, it is likely that spatial abilities, mechanical reasoning, and mathematics skills are important for success, and are likely to play some role in the interview and screening process.
Thus, if males are overrepresented in the upper tail of the distributions for spatial abilities, mechanical reasoning, and mathematics, it would be possible for Google to end up hiring more males and, at the same time, not be discriminatory in their hiring practices. This is because the pool of potentially qualified applicants may contain more males.
For this post we set aside a discussion over what factors may produce greater male variability to focus on research that investigates this hypothesis with a variety of measures. We adopt the practice from our initial blog post. Findings that support the Greater Male Variability Hypothesis and emphasize potential biological factors for this variability are presented in green font, while findings that contradict the Greater Male Variability Hypothesis and emphasize cultural factors are presented in red font. (The idea here is that the currently greater male variability may be relevant to current hiring outcomes, but if the variability difference could be reduced to zero in a decade or two, then it counts against those who argue that biology may place a constraint on achieving future gender parity).
We have also emphasized other important areas of the text in bold font. We do this to draw attention to other important findings that may not be directly relevant to the Greater Male Variability Hypothesis, but are relevant to the controversy over the Damore memo. Finally, beneath each article citation we list the measure(s) employed by each study. We do this to demonstrate that a variety of abilities, interests, and personality traits have been subjected to empirical tests of the Greater Male Variability Hypothesis.
|Arden, R. & Plomin, R. (2006). Sex differences in variance of intelligence across childhood. Personality and Individual Differences, 41, 39-48.|
Measure(s): g (extracted from standardized scores on verbal and nonverbal cognitive abilities tests using principal components analysis)
|Why are males over-represented at the upper extremes of intelligence? One possibility for which there is some empirical support is that variance is greater among adult males. There is little published evidence of the development of that variability – is it manifest in early childhood or does it develop later?|
We explored sex differences in phenotypic variance in scores on a general ability factor extracted from several tests of verbal and non-verbal ability at ages 2, 3, 4, 7, 9 and 10 (Ns from > 10,000 to > 2000) in a sample of British children.
We found greater variance, by Levene’s test of homogeneity of variance, among boys at every age except age two despite the girls’ mean advantage from ages two to seven. Girls are significantly over-represented, as measured by chi-square tests, at the high tail and boys at the low tail at ages 2, 3 and 4. By age 10 the boys have a higher mean, greater variance and are over-represented in the high tail. Sex differences in variance emerge early – even before pre-school – suggesting that they are not determined by educational influences.
|Borkenau, P., McCraw, R.R., & Terracciano, A. (2013). Do men vary more than women in personality? A study in 51 cultures. Journal of Research in Personality, 47(2), 135-144.|
Measure(s): 3rd person version of the NEO Personality Inventory
|Do men vary more than women in personality? Evolutionary, genetic, and cultural arguments suggest that hypothesis. In this study we tested it using 12,156 college student raters from 51 cultures who described a person they knew well on the 3rd-person version of the Revised NEO Personality Inventory. In most cultures, male targets varied more than female targets, and ratings by female informants varied more than ratings by male informants, which may explain why higher variances for men are not found in self-reports. Variances were higher in more developed, and effects of target sex were stronger in more individualistic societies. It seems that individualistic cultures enable a less restricted expression of personality, resulting in larger variances and particularly so among men.|
|Ceci, S.J., Williams, W.M., & Barnett, S.M. (2009). Women’s underrepresentation in science: Sociocultural and biological considerations. Psychological Bulletin, 135(2), 218-261.|
Measure(s): None, this is a theoretical paper.
|The underrepresentation of women at the top of math-intensive fields is controversial, with competing claims of biological and sociocultural causation. The authors develop a framework to delineate possible causal pathways and evaluate evidence for each. Biological evidence is contradictory and inconclusive. Although cross-cultural and cross-cohort differences suggest a powerful effect of sociocultural context, evidence for specific factors is inconsistent and contradictory. Factors unique to underrepresentation in math-intensive fields include the following: |
(a) Math-proficient women disproportionately prefer careers in non-math-intensive fields and are more likely to leave math-intensive careers as they advance;
(b) more men than women score in the extreme math-proficient range on gatekeeper tests, such as the SAT Mathematics and the Graduate Record Examinations Quantitative Reasoning sections;
(c) women with high math competence are disproportionately more likely to have high verbal competence, allowing greater choice of professions; and
(d) in some math-intensive fields, women with children are penalized in promotion rates. The evidence indicates that women’s preferences, potentially representing both free and constrained choices, constitute the most powerful explanatory factor, a secondary factor is performance on gatekeeper tests, most likely resulting from sociocultural rather than biological causes.
|Deary, I.J., Thorpe, G., Wilson, V., Starr, J.M., & Whalley, L.J. (2003). Population sex differences in IQ at age 11: The Scottish mental survey 1932. Intelligence, 31(6), 533-542.|
|There is uncertainty whether the sexes differ with respect to their mean levels and variabilities in mental ability test scores. Here we describe the cognitive ability distribution in 80,000+ children—almost everyone born in Scotland in 1921—tested at age 11 in 1932. There were no significant mean differences in cognitive test scores between boys and girls, but there was a highly significant difference in their standard deviations (P<.001). Boys were over-represented at the low and high extremes of cognitive ability. These findings, the first to be presented from a whole population, might in part explain such cognitive outcomes as the slight excess of men achieving first class university degrees, and the excess of males with learning difficulties.|
|Deary, I.J., Irwing, P., Der, G., & Bates, T.C. (2007). Brother-sister differences in the g factor in intelligence: Analysis of full, opposite-sex siblings from the NLSY1979. Intelligence, 35(5), 451-456.|
Measure(s): Armed Services Vocational Aptitude Battery (ASVAB); Armed Forces Qualification Test (AFQT)
|There is scientific and popular dispute about whether there are sex differences in cognitive abilities and whether they are relevant to the proportions of men and women who attain high-level achievements, such as Nobel Prizes. A recent meta-analysis (Lynn, R., and Irwing, P. (2004). Sex differences on the progressive matrices: a meta-analysis. Intelligence, 32, 481–498.), which suggested that males have higher mean scores on the general factor in intelligence (g), proved especially contentious. Here we use a novel design, comparing 1292 pairs of opposite-sex siblings who participated in the US National Longitudinal Survey of Youth 1979 (NLSY1979). The mental test applied was the Armed Services Vocational Aptitude Battery (ASVAB), from which the briefer Armed Forces Qualification Test (AFQT) scores can also be derived. Males have only a marginal advantage in mean levels of g (less than 7% of a standard deviation) from the ASVAB and AFQT, but substantially greater variance. Among the top 2% AFQT scores, there were almost twice as many males as females. These differences could provide a partial basis for sex differences in intellectual eminence.|
|Dykiert, D., Gale, C.R., & Deary, I.J. (2009). Are apparent sex differences in mean IQ scores created in part by sample restriction and increased male variance? Intelligence, 37, 42-47.|
|This study investigated the possibility that apparent sex differences in IQ are at least partly created by the degree of sample restriction from the baseline population. We used a nationally representative sample, the 1970 British Cohort Study. Sample sizes varied from 6518 to 11,389 between data-collection sweeps. Principal components analysis of scores obtained on four cognitive tests administered at age 10 was used to obtain estimates that we name ‘IQ’. These age-10 scores were then used to estimate the sex differences at age 10, and also among participants in the two later waves, at age 26 and 30. At age 10, there was a small but significant advantage for boys (Cohen’s d = 0.081). Boys had greater variability in these IQ scores. We then investigated how this very small male advantage at 10 changed with sample restriction. We used the same IQs obtained at age 10, but considered only those subjects who returned for data-collection sweeps at ages 26 and 30 years. Subjects returning at age 26 and 30 were more likely to be females and to have higher age-10 IQ scores. Attrition at age 30 was 28% and the male advantage in IQ scores increased by 15%. Attrition at age 26 was 43% and the male advantage in IQ scores increased by 48%. The findings underline the importance of monitoring attrition in longitudinal studies, as well as emphasising the need for representative samples in studying sex differences in intelligence. A proportion of the apparent male advantage in general cognitive ability that has been reported by some researchers might be attributable to the combination of greater male variance in general cognitive ability and sample restriction, though this remains to be tested in a sample with an appropriate mental test battery.|
|Feingold, A. (1994). Gender differences in variability in intellectual abilities: A cross-cultural perspective. Sex Roles, 30(1/2), 81-92.|
Measure(s): Calculated variance ratio (VR) for post 1980 studies in which ability tests were administered to examinees from outside the US
|A cross-cultural quantitative review of contemporary findings of gender differences in variability in verbal, mathematical, and spatial abilities was conducted to assess the generalizability of U.S. findings that (a) males are more variable than females in mathematical and spatial abilities, and (b) the sexes are equally variable in verbal ability. No consistent gender differences (variance ratios) were found across countries in any of the three broad ability domains. Instead, males were more variable than females in some nations and females were more variable than males in other nations. Thus, the well-established U.S. findings of consistently greater male variability in mathematical and spatial abilities were not invariant across cultures and nations.|
|Halpern, D.F. (1997). Sex differences in intelligence: Implications for education. American Psychologist, 52(10), 1091-1102.|
Measure(s): None, this is a review paper.
|Sex differences in intelligence is among the most politically volatile topics in contemporary psychology. Although no single finding has unanimous support, conclusions from multiple studies suggest that females, on average, score higher on tasks that require rapid access to and use of phonological and semantic information in long-term memory, production and comprehension of complex prose, fine motor skills, and perceptual speed. Males, on average, score higher on tasks that require transformations in visual-spatial working memory, motor skills involved in aiming, spatiotemporal responding, and fluid reasoning, especially in abstract mathematical and scientific domains. Males, however, are also over-represented in the low-ability end of several distributions, including mental retardation, attention disorders, dyslexia, stuttering, and delayed speech. A psychobiosocial model that is based on the inextricable links between the biological bases of intelligence and environmental events is proposed as an alternative to nature-nurture dichotomies. Societal implications and applications to teaching and learning are suggested.|
|Halpern, D.F., Benbow, C.P., Geary, D.C., Gur, R.C., Hyde, J.S., & Gernsbacher, M.A. (2007). The science of sex differences in science and mathematics. Psychological Science in the Public Interest, 8(1), 1-51.|
Measure(s): None, this is a review paper.
HxA Note: See this post for more detail on the conclusions of this review.
|Amid ongoing public speculation about the reasons for sex differences in careers in science and mathematics, we present a consensus statement that is based on the best available scientific evidence. Sex differences in science and math achievement and ability are smaller for the mid-range of the abilities distribution than they are for those with the highest levels of achievement and ability. Males are more variable on most measures of quantitative and visuospatial ability, which necessarily results in more males at both high- and low-ability extremes; the reasons why males are often more variable remain elusive. Successful careers in math and science require many types of cognitive abilities. Females tend to excel in verbal abilities, with large differences between females and males found when assessments include writing samples. High-level achievement in science and math requires the ability to communicate effectively and comprehend abstract ideas, so the female advantage in writing should be helpful in all academic domains. Males outperform females on most measures of visuospatial abilities, which have been implicated as contributing to sex differences on standardized exams in mathematics and science. An evolutionary account of sex differences in mathematics and science supports the conclusion that, although sex differences in math and science performance have not directly evolved they could be indirectly related to differences in interests and specific brain and cognitive systems. We review the brain basis for sex differences in science and mathematics, describe consistent effects, and identify numerous possible correlates. Experience alters brain structures and functioning, so causal statements about brain differences and success in math and science are circular. A wide range of sociocultural forces contribute to sex differences in mathematics and science achievement and ability—including the effects of family, neighborhood, peer, and school influences; training and experience; and cultural practices. We conclude that early experience, biological factors, educational policy, and cultural context affect the number of women and men who pursue advanced study in science and math and that these effects add and interact in complex ways. There are no single or simple answers to the complex questions about sex differences in science and mathematics.|
|Halpern, D.F. & LaMay, M.L. (2000). The smarter sex: A critical review of sex differences in intelligence. Educational Psychology Review, 12(2), 229-246.|
Measure(s): None, this is a review paper.
|Although there are no sex differences in general intelligence, reliable differences are found on some tests of cognitive abilities. Many of the tasks that assess the ability to manipulate visual images in working memory show an advantage for males, whereas many of the tasks that require retrieval from long-term memory and the acquisition and use of verbal information show a female advantage. Large effects favoring males are also found on advanced tests of mathematical achievement, especially with highly select samples. Males are also overrepresented in some types of mental retardation. Effects sizes are variable and often large. These differences are generally found cross-culturally and across the lifespan. The nature–nurture dichotomy is rejected as an interpretive framework. In light of recent findings that environmental variables alter the biological underpinnings of intelligence and individuals actively participate in creating their environments, we prefer a psychobiosocial model for understanding sex differences in intelligence.|
|He, W. & Wong, W. (2011). Gender differences in creative thinking revisited: Findings from analysis of variability. Personality and Individual Differences, 51(7), 807-811.|
Measure(s): Test for Creative Thinking-Drawing Production (TCT-DP)
|This study investigated gender differences in creativity among 985 schoolchildren (499 boys, 486 girls) by analyzing both means and variability. A relatively new creativity test, the Test for Creative Thinking-Drawing Production (TCT-DP), was employed to gain a more refined understanding of gender differences in creativity using a gestalt approach. Whereas the results of analyses of means generally supported the Gender Similarities Hypothesis, the variability analyses tended to support the Greater Male Variability Hypothesis and the Gender Difference Hypothesis. Analyses of the TCT-DP subscales revealed that both genders have their relative strengths and weaknesses in creative thinking. Whereas girls outperformed boys in thoroughness of thinking, boys outperformed girls in boundary-breaking thinking. Variability analyses further showed that more boys clustered in the two extremes of the composite score. Significantly greater variability was found for males on five criteria of the TCT-DP. The educational implications of such a complex pattern of gender differences are discussed. With a view to searching for an explanation for gender differences, several lines of further research are proposed.|
|Hedges, L.V. & Nowell, A. (1995). Sex differences in mental test scores, variability, and numbers of high-scoring individuals. Science, 269(5220), 41-45.|
Measure(s): Project Talent; National Longitudinal Study of the High School Class of 1972 (NLS-72 data set); National Longitudinal Study of Youth (NLSY data set); High School and Beyond (HS&B data set); National Educational Longitudinal Study of the Eighth Grade Class of 1988 (NELS:88 data set); National Assessment of Educational Progress (NAEP data set)
|Sex differences in central tendency, variability, and numbers of high scores on mental tests have been extensively studied. Research has not always seemed to yield consistent results, partly because most studies have not used representative samples of national populations. An analysis of mental test scores from six studies that used national probability samples provided evidence that although average sex differences have been generally small and stable over time, the test scores of males consistently have larger variance. Except in tests of reading comprehension, perceptual speed, and associative memory, males typically outnumber females substantially among high-scoring individuals.|
|Hyde, J.S., Lindberg, S.M., Linn, M.C., Ellis, A.B., & Williams, C.C. (2008). Gender similarities characterize math performance. Science, 321(5888), 494-495.|
Measure(s): Minnesota state assessments
|[HxA Note: There is no abstract for this paper, we, therefore, present the conclusion] |
Our analysis shows that, for grades 2 to 11, the general population no longer shows a gender difference in math skills, consistent with the gender similarities hypothesis. There is evidence of slightly greater male variability in scores, although the causes remain unexplained.
Gender differences in math performance, even among high scorers, are insufficient to explain lopsided gender patterns in participation in some STEM fields. An unexpected finding was that state assessments designed to meet NCLB requirements fail to test complex problem-solving of the kind needed for success in STEM careers, a lacuna that should be fixed.
|Hyde, J.S. & Mertz, J.E. (2009). Gender, culture, and mathematics performance. Proceedings of the National Academy of Sciences, 106(22), 8801-8807.|
Measure(s): Minnesota state assessments (see Hyde et al., 2008); Program for International Student Assessment (PISA; see Guiso et al., 2008); Trends in International Mathematics and Science Study (TIMSS; see Machin & Pekkarinen, 2008)
|Using contemporary data from the U.S. and other nations, we address 3 questions: Do gender differences in mathematics performance exist in the general population? Do gender differences exist among the mathematically talented? Do females exist who possess profound mathematical talent? In regard to the first question, contemporary data indicate that girls in the U.S. have reached parity with boys in mathematics performance, a pattern that is found in some other nations as well. Focusing on the second question, studies find more males than females scoring above the 95th or 99th percentile, but this gender gap has significantly narrowed over time in the U.S. and is not found among some ethnic groups and in some nations. Furthermore, data from several studies indicate that greater male variability with respect to mathematics is not ubiquitous. Rather, its presence correlates with several measures of gender inequality. Thus, it is largely an artifact of changeable sociocultural factors, not immutable, innate biological differences between the sexes. Responding to the third question, we document the existence of females who possess profound mathematical talent. Finally, we review mounting evidence that both the magnitude of mean math gender differences and the frequency of identification of gifted and profoundly gifted females significantly correlate with sociocultural factors, including measures of gender equality across nations.|
|Johnson, W., Carothers, A., & Deary, I.J. (2008). Sex differences in variability in general intelligence: A new look at the old question. Perspectives on Psychological Science, 3(6), 518-531.|
|The idea that general intelligence may be more variable in males than in females has a long history. In recent years it has been presented as a reason that there is little, if any, mean sex difference in general intelligence, yet males tend to be overrepresented at both the top and bottom ends of its overall, presumably normal, distribution. Clear analysis of the actual distribution of general intelligence based on large and appropriately population-representative samples is rare, however. Using two population-wide surveys of general intelligence in 11-year-olds in Scotland, we showed that there were substantial departures from normality in the distribution, with less variability in the higher range than in the lower. Despite mean IQ-scale scores of 100, modal scores were about 105. Even above modal level, males showed more variability than females. This is consistent with a model of the population distribution of general intelligence as a mixture of two essentially normal distributions, one reflecting normal variation in general intelligence and one reflecting normal variation in effects of genetic and environmental conditions involving mental retardation. Though present at the high end of the distribution, sex differences in variability did not appear to account for sex differences in high-level achievement.|
|Lakin, J.M. (2013). Sex differences in reasoning abilities: Surprising evidence that male-female ratios in the tails of the quantitative reasoning distribution have increased. Intelligence, 41, 263-274.|
Measure(s): Cognitive Abilities Test
|Sex differences in cognitive abilities, particularly at the extremes of ability distributions, have important implications for the participation of men and women in highly valued and technical career fields. Although negligible mean differences have been found in many domains, differences in variability and high ratios of males to females in the tails of the ability distribution have been found in a number of studies and across domains.|
A few studies have also observed trends over time, with some noting the decreasing ratios of boys to girls in the highest levels of mathematics test performance. In this study, sex differences in means, variances, and ratios were evaluated in four cohorts (1984, 1992, 2000, and 2011) in verbal, quantitative, and nonverbal/figural reasoning domains as measured by the Cognitive Abilities Test. Samples included US students in grades 3–11.
Overall, the results were consistent with previous research, showing small mean differences in the three domains, but considerably greater variability for males. The most surprising finding was that, contrary to related research, the ratio of males to females in the upper tail of the quantitative reasoning distribution seemed to increase over time. Explanations for this finding are explored.
|Lohman, D.F. & Lakin, J.M. (2009). Consistencies in sex differences on the Cognitive Abilities Test across countries, grades, test forms, and cohorts. British Journal of Educational Psychology, 79, 389-407.|
Measure(s): Cognitive Abilities Test
|BACKGROUND: Strand, Deary, and Smith (2006) [HxA Note: see below for the abstract and link to Strnad et al. (2006)] reported an analysis of sex differences on the Cognitive Abilities Test (CAT) for over 320,000 UK students 11-12 years old. Although mean differences were small, males were overrepresented at the upper and lower extremes of the score distributions on the quantitative and non-verbal batteries and at the lower extreme of the verbal battery.|
AIMS: We investigate whether these results were unique to the UK or whether they would be seen in other countries, at other grades, cohorts, or forms of the test.
SAMPLE: The sample consisted of three nationally representative cohorts of US students in grades 3 through 11 (total N=318,599) for the 1984, 1992, and 2000 standardizations of the US version of the CAT.
METHODS: We replicated and extended the Strand et al. (2006) results by comparing the proportions of males and females at each score level across countries (UK vs. US), grades (3-11), and cohorts/test forms (Forms 4, 5, and 6 standardized in 1984, 1992, and 2000, respectively).
RESULTS: The results showed an astonishing consistency in sex differences across countries, grades, cohorts, and test forms.
CONCLUSIONS: Implications for the current debate about sex differences in quantitative reasoning abilities are discussed.
|Machin, S. & Pekkarinen, T. (2008). Global sex differences in test score variability. Science, 322(5906), 1331-1332. |
Measure(s): Trends in International Mathematics and Science Study (TIMSS)
|[HxA Note: There is no abstract for this paper, we therefore present the conclusion] Our analysis of international test score data shows a higher variance in boys’ than girls’ results on mathematics and reading tests in most OECD countries. How this translates into educational achievement is a matter open for discussion. Higher variability among boys is a salient feature of reading and mathematics test performance across the world. In almost all comparisons, the age 15 boy-girl variance difference in test scores is present. This difference in variance is higher in countries that have higher levels of test score performance.|
Sex differences in means are easier to characterize: It is evident from the PISA data that boys do better in mathematics, and girls do better in reading. This has a compositional effect on the variance differences as well. The higher boy-girl variance ratio in mathematics comes about because of an increased prevalence of boys in the upper part of the distribution, but the higher variance in reading is due to a greater preponderance of boys in the bottom part of the test score distribution. Because literacy and numeracy skills have been shown to be important determinants of later success in life (for instance, in terms of earning higher wages or getting better jobs), these differing variances have important economic and social implications.
We therefore confirm that 15-year-old boys do show more variability than girls in educational performance, with specifics that differ according to whether mathematics or reading are being studied and tested. These results imply that gender differences in the variance of test scores are an international phenomenon and that they emerge in different institutional settings.
Please see the supplementary materials here: http://science.sciencemag.org/content/suppl/2008/11/24/322.5906.1331.DC1
|Makel, M.C., Wai, J., Peairs, K., & Putallaz, M. (2016). Sex differences in the right tail of cognitive abilities: An update and cross cultural extension. Intelligence, 59, 8-15.|
Measure(s): Duke University Talent Identification Program (Duke TIP); SAT; ACT; EXPLORE; ASSET
|Male–female ability differences in the right tail (at or above the 95th percentile) have been widely discussed for their potential role in achievement and occupational differences in adults. The present study provides updated male–female ability ratios from 320,000 7th grade students in the United States in the right tail (top 5%) through the extreme right tail (top 0.01%) from 2011 to 2015 using measures of math, verbal, and science reasoning. Additionally, the present study establishes male-female ability ratios in a sample of over 7000 7th grade students in the right tail from 2011 to 2015 in India. Results indicate that ratios in the extreme right tail of math ability in the U.S. have shrunk in the last 20 years (still favoring males) and remained relatively stable in the verbal domain (still favoring females). Similar patterns of male-female ratios in the extreme right tail were found in the Indian sample.|
|Nowell, A. & Hedges, L.V. (1998). Trends in gender differences in academic achievement from 1960 to 1994: An analysis of differences in mean, variance, and extreme scores. Sex Roles, 39(1/2), 21-43.|
Measure(s): Project Talent; Equality of Educational Opportunity; National Longitudinal Study of the High School Class of 1972 (NLS-72 data set); National Longitudinal Study of Youth (NLSY data set); High School and Beyond (HS&B data set); 1992 follow-up to the National Educational Longitudinal Study of the Eighth Grade Class of 1988 (NELS:92 data set); National Assessment of Educational Progress (NAEP data set)
|Gender differences in academic achievement have been studied extensively. While it is generally agreed that females have a slight advantage on average in verbal abilities and males have a slight advantage on average in mathematics, it is unclear whether these differences have changed over time. In this paper evidence from seven surveys representative of the United States twelfth grade student population and the National Assessment of Educational Progress (NAEP) long term trend data is brought to bear on the magnitude of gender differences in achievement, the level of agreement among different indices of difference, and the stability of these differences over time. These data provide the unique opportunity to not only empirically estimate mean differences, differences in variance, and differences in extreme scores, but also to estimate change over time in all three indices using both the same and different tests over time. Results show that gender differences in mean and variance are small, while differences in extreme scores are often substantial. None of these differences have changed significantly since 1960 with the possible exception of mean differences in mathematics and science. Each of the datasets reflects the racial composition of the national population when properly weighted (i.e. White = 70%, Black = 15%,Hispanic = 10%, Other = 5%).|
|Paessler, K. (2015). Sex differences in variability in vocational interests. European Journal of Personality, 29(5), 568-578.|
Measure(s): Explorix (online-based interest inventory); was-studiere-ich (online-based interest inventory)
[HxA Note: both inventories contain Holland’s (1997) RIASEC scales]
|Greater male variability has been established in cognitive abilities and physical attributes. This study investigated sex differences in variability in vocational interests with two large samples (N > 40 000 and N > 70 000). The results show that although men varied more in Realistic and Enterprising interests, women varied more in Artistic and Conventional interests. These differences in variability had considerable influence on the female–male tail ratios in vocational interests that have been found to contribute to reported gender disparities in certain fields of work and academic disciplines. Moreover, differences in means and variability interacted non-linearly in shaping tail-ratio imbalances. An age-specific analysis additionally revealed that differences in variability diminished with age: Older samples showed smaller differences in variance in Realistic, Artistic, and Social interests than younger samples. Thus, I found no evidence that greater male variability applies for vocational interests in general.|
|Reilly, D., Neumann, D.L., & Andrews, G. (2015). Sex differences in mathematics and science achievement: A meta-analysis of National assessment of Educational Progress assessments. Journal of Educational Psychology, 107(3), 645-662.|
Measure(s): National Assessment of Educational Progress
|Gender gaps in the development of mathematical and scientific literacy have important implications for the general public’s understanding of scientific issues and for the underrepresentation of women in science, technology, engineering, and math. We subjected data from the National Assessment of Educational Progress to a meta-analysis to examine whether there were sex differences in mathematics and science achievement for students in the United States across the period 1990-2011. Results show that there were small but stable mean sex differences favoring males in mathematics and science across the past 2 decades, with an effect size of d = 0.10 and 0.13, respectively, for students in 12th grade. Furthermore, there were large sex differences in high achievers, with males being overrepresented by a factor of over 2:1 at the upper right of the ability distribution for both mathematics and science. Further efforts are called for to reach equity in mathematics and science educational outcomes for all students.|
|Strand, S., Deary, I.J., & Smith, P. (2006). Sex differences in Cognitive Abilities Test scores: A UK national picture. British Journal of Educational Psychology, 76, 463-480.|
Measure(s): Cognitive Abilities Test
|Background and aims. There is uncertainty about the extent or even existence of sex differences in the mean and variability of reasoning test scores (Jensen, 1998; Lynn, 1994, 1998; Mackintosh, 1996). This paper analyses the Cognitive Abilities Test (CAT) scores of a large and representative sample of UK pupils to determine the extent of any sex differences.|
Sample. A nationally representative UK sample of over 320,000 school pupils aged 11–12 years was assessed on the CAT (third edition) between September 2001 and August 2003. The CAT includes separate nationally standardized tests for verbal, quantitative, and non-verbal reasoning. The size and recency of the sample is unprecedented in research on this issue.
Methods. The sheer size of the sample ensures that any sex difference will achieve statistical significance. Therefore, effect sizes (d) and variance ratios (VR) are employed to evaluate the magnitude of sex differences in mean scores and in score variability, respectively.
Results. The mean verbal reasoning score for girls was 2.2 standard score points higher than the mean for boys, but only 0.3 standard points in favour of girls for non-verbal reasoning (NVR), and 0.7 points in favour of boys for quantitative reasoning (QR). However, for all three tests there were substantial sex differences in the standard deviation of scores, with greater variance among boys. Boys were over represented relative to girls at both the top and the bottom extremes for all tests, with the exception of the top 10% in verbal reasoning.
Conclusions. Given the small differences in means, explanations for sex differences in wider domains such examination attainment at age 16 need to look beyond conceptions of ‘ability’. Boys tend to be both the lowest and the highest performers in terms of their reasoning abilities, which warns against the danger of stereotyping boys as low achievers.
|Wai, J., Cacchio, M., Putallaz, M. & Makel, M.C. (2010). Sex differences in the right tail of cognitive abilities: A 30 year examination. Intelligence, 38(4), 412-423.|
Measure(s): SAT; ACT
|One factor in the debate surrounding the underrepresentation of women in science technology, engineering and mathematics (STEM) involves male–female mathematical ability differences in the extreme right tail (top 1% in ability). The present study provides male–female ability ratios from over 1.6 million 7th grade students in the right tail (top 5% in ability) across 30 years (1981–2010) using multiple measures of math, verbal, and writing ability and science reasoning from the SAT and ACT. Male–female ratios in mathematical reasoning are substantially lower than 30 years ago, but have been stable over the last 20 years and still favor males. Over the last two decades males showed a stable or slightly increasing advantage in science reasoning. However, more females scored in the extreme right tail of verbal reasoning and writing ability tests. The potential role of sociocultural factors on changes in the male–female ability ratios is discussed and the introduction of science reasoning as a potential new factor in the debate is proposed. The implications of continued sex differences in math and science reasoning is discussed within the context of the many important interlocking factors surrounding the debate on the underrepresentation of women in STEM.|
|Wai, J., Putallaz, M., & Makel, M.C. (2012). Studying intellectual outliers: Are there sex differences, and are the smart getting smarter? Current Directions in Psychological Science, 21(6), 382-390.|
Measure(s): Duke University Talent Identification Program (Duke TIP); SAT; ACT; EXPLORE; ASSET
|By studying samples of intellectual outliers across 30 years, researchers can leverage right-tail data (i.e., samples at or above the 95th percentile on tests of ability) to uncover missing pieces to two psychological puzzles: whether there are sex differences in cognitive abilities among smart people, and whether test scores are rising (a phenomenon known as the Flynn effect) among smart people. For the first puzzle, data indicate that the high male-to-female ratio among extremely high scorers on measures of math ability has decreased dramatically, but is still likely one factor among many explaining female underrepresentation in some professions. For the second puzzle, data indicate that the right tail has risen at a similar rate as the general (or middle portion of the) distribution; it is thus likely that the entire curve is rising at a relatively constant rate, consistent with the Flynn effect, which may explain why a greater number of gifted students have been identified in recent years. However, the causes for these gains and whether they reflect real gains in intelligence continue to remain a mystery. We show how these two puzzles are linked and stress the importance of paying attention to the entire distribution when attempting to address some scientific questions.|
Our Conclusions about the Greater Male Variability Hypothesis:
- On average, male variability is greater than female variability on a variety of measures of cognitive ability, personality traits, and interests. This means men are more likely to be found at both the low and high end of these distributions (see Halpern et al., 2007; Machin & Pekkarinen, 2008 and, especially, the supplementary materials; for an ungated summary click here). This finding is consistent across decades.
- The gender difference in variability has reduced substantially over time within the United States and is variable across cultures. It is clearly responsive to social and cultural factors (see Hyde & Mertz, 2009; Wai et al., 2010); Educational programs can be effective. It is also clear that there are cultural/societal influences, as the male:female variability ratios can vary considerably across cultures (e.g., Machin & Pekkarinen, 2008).
- While the gender difference in the male:female ratio for the upper tail of the distribution of math test scores (SAT, ACT) narrowed considerably in the United States in the 1980s, it appears to have remained steady since the early 1990s. This can be seen visually in Figure 1 from Wai et al. (2010):
- Therefore at the top end of any distribution of test scores where men have higher variability, we’d expect men to make up more than 50% of the upper end of the tail. Thus, any company drawing from the top 5% is likely to find a pool that contains more males. As one goes further out into the tail (i.e. becomes even more selective) the gender tilt becomes larger.
- Further compounding the gender tilt: the women in this elite group generally have much better verbal skills than the men in that elite group (see Reilly, 2012). This means that these women may be better employees than men who match them on quantitative skills, but because they have such superior verbal skills they have more choices available to them when selecting a profession.
Our Revised Conclusions About the Damore Memo
We maintain that the research findings are complicated. This is evident in both this post and our original one. There are many abstracts containing both red and green text, and some of the top researchers in psychology are represented on both sides of the debate. Furthermore, many of the experts have concluded that:
… early experience, biological factors, educational policy, and cultural context affect the number of women and men who pursue advanced study in science and math and that these effects add and interact in complex ways. There are no single or simple answers to the complex questions about sex differences in science and mathematics (Halpern et al., 2009).
In light of of the research on the Greater Male Variability Hypothesis however, we have revised our original conclusions:
- Gender differences in math/science ability, achievement, and performance are small or nil. (See especially the studies by Hyde; see also this review paper by Spelke, 2005). There are two exceptions to this statement:
- Men (on average) score higher than women on most tests of spatial abilities, but the size of this advantage depends on the task and varies from small to large (e.g., Lindberg et al., 2010). There is at least one spatial task that favors females (spatial location memory; see e.g., Galea & Kimura, 1993; Kimura, 1996; Vandenberg & Kuse, 1978). Men also (on average) score higher on mechanical reasoning and tests of mathematical ability, although this latter advantage is small. Women get better grades at all levels of schooling and score higher on a few abilities that are relevant to success in any job (e.g., reading comprehension, writing, social skills). Thus, we assume that this one area of male superiority is not likely to outweigh areas of male inferiority to become a major source of differential outcomes.
- There is good evidence that men are more variable on a variety of traits, meaning that they are over-represented at both tails of the distribution (i.e., more men at the very bottom, and at the very top), even though there is no gender difference on average. Thus, the pool of potentially qualified applicants for a company like Google is likely to contain more males than females. To be clear, this does not mean that males are more “suited” for STEM jobs. Anyone located in the upper tail of the distributions valued in the hiring process possesses the requisite skills. Although there may be fewer women in that upper tail, the ones who are found there are likely to have several advantages over the men, particularly because they likely have better verbal skills.
- Gender differences in interest and enjoyment of math, coding, and highly “systemizing” activities are large. The difference on traits related to preferences for “people vs. things” is found consistently and is very large, with some effect sizes exceeding 1.0. (See especially the meta-analyses by Su and her colleagues, and also see this review paper by Ceci & Williams, 2015).
- Culture and context matter, in complicated ways. Some gender differences have decreased over time as women have achieved greater equality, showing that these differences are responsive to changes in culture and environment. But the cross-national findings sometimes show “paradoxical” effects: progress toward gender equality in rights and opportunities sometimes leads to larger gender differences in some traits and career choices. Nonetheless, it seems that actions taken today by parents, teachers, politicians, and designers of tech products may increase the likelihood that girls will grow up to pursue careers in tech, and this is true whether or not biology plays a role in producing any particular population difference. (See this review paper by Eagly and Wood, 2013).
In conclusion, based on the meta-analyses we reviewed and the research on the Greater Male Variability Hypothesis, Damore is correct that there are “population level differences in distributions” of traits that are likely to be relevant for understanding gender gaps at Google and other tech firms. The differences are much larger and more consistent for traits related to interest and enjoyment, rather than ability. This distinction between interest and ability is important because it may address one of the main fears raised by Damore’s critics: that the memo itself will cause Google employees to assume that women are less qualified, or less “suited” for tech jobs, and will therefore lead to more bias against women in tech jobs. But the empirical evidence we have reviewed should have the opposite effect. Population differences in interest and population differences in variability of abilities may help explain why there are fewer women in the applicant pool, but the women who choose to enter the pool are just as capable as the larger number of men in the pool. This conclusion does not deny that various forms of bias, harassment, and discouragement exist and may contribute to outcome disparities, nor does it imply that the differences in interest are biologically fixed and cannot be changed in future generations.
If our conclusions are correct then Damore was drawing attention to empirical findings that seem to have been previously unknown or ignored at Google, and which might be helpful to the company as it tries to improve its diversity policies and outcomes.
We again thank Alice Eagly for her helpful input.