Over the past year, Helen Pluckrose, James Lindsay & Peter Boghossian published a series of hoax papers in humanities journals oriented towards the “critical study” of gender and sexuality. Their plot was discovered midway through, forcing them to go public prematurely.

In a journal-length essay explaining their methods, findings, and intentions, “Academic Grievance Studies and the Corruption of Scholarship,” the authors hold up their publications as proof that research on race, gender, sexuality – along with the discipline of sociology – are defined by groupthink, confirmation bias and motivated reasoning, which allow bogus work to regularly be published in these (sub)fields.

Dubbed “Sokal Squared,” or the “Grievance Studies hoax,” the incident has become something of a Rorschach Test within academic and media spaces: Those already disposed towards skepticism of “critical” scholarship on race, gender or sexuality view the incident as damning proof of a deep rot within these subfields, and perhaps with social research or academia more broadly. Those more sympathetic to “critical studies” instead see a deeply flawed and limited experiment — and accuse the authors of overstating their findings, speaking beyond their data and, intentionally or not, feeding into the agenda of right-wing reactionaries. There have been relatively few unexpected validators or critics.

The purpose of this essay is not to adjudicate between these rival perspectives, but to provide some critical context on the background of these types of experiments and this particular study, explore the relevant ethical considerations, and provide a concise description and analysis of what the trio found (click to expand).

Background

NYU physicist Alan Sokal was disturbed and annoyed by postmodern social critiques and, especially, the (mis)appropriation of concepts from the “hard sciences” into social research. And so, in 1996, he crafted a paper which he viewed as not just weak, but essentially nonsensical: “Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity,” and managed to get it published Social Text – a journal which, while not peer-reviewed, regularly published essays by social criticism luminaries of the time. Shortly after it was published, Sokal penned an essay for the American literary magazine Lingua Franca exposing his original paper as a hoax; he then co-authored a book with fellow physicist Jean Bricmont, Impostures Intellectulles, expanding his critique of postmodernism, the misuse of scientific and mathematical concepts in social research, and the lack of intellectual rigor in “critical studies.”

This was a significant departure from the standard operating procedure:

Within the social sciences, there was already a well-established tradition of sending problematic papers up for peer review, with false attribution of authorship, in order to better understand how editors exercise discretion in selecting and evaluating work. This line of research dates back (at least) to Abramowitz, Gomes & Abramowitz’s 1975 paper in the Journal of Applied Social Psychology, “Publish or Politic: Referee Bias in Manuscript Review.”

Most of these have included a control group, tested a fairly large number of papers (typically submitted to journals in the authors’ own fields), and – because they were merely testing whether editors would accept or reject the works – researchers did not allow accepted fake papers to proceed to publication, so as to avoid polluting the knowledge environment. Scholars have also tended to be fairly circumspect in detailing the findings and implications of their studies (e.g. here, here, here, here, here, here).

In contrast, Sokal wrote a single paper, submitted to a single journal. He was an outsider to the field he was testing. The paper was allowed to proceed all the way to publication, forcing the journal to retract it when the hoax was exposed. And rather than drawing narrow, careful conclusions from the experiment, Sokal held it out as an indictment of the entire “critical studies” genre.

As a result of this novelty, and the significance of the journal involved, the incident became a major story in mainstream media outlets in France, the U.S. and beyond. This, in turn, prompted robust criticism of the hoax from Jacques Derrida, Bruno Latour, Gabriel Stolzenberg and others – who argued that the conclusions Sokal was drawing were far too wide given he only published one single (non-peer reviewed) essay; critics also argued that Sokal seemed to be misunderstanding, misrepresenting, and/or uncharitably reading most of the works he (and Bricmont) singled out.

In 2008, reflecting on these criticisms, and with the perspective distance provides, Sokal wrote a book, Beyond the Hoax: Science, Philosophy and Culture, wherein he conceded that his initial criticisms may have been overly dismissive, and perhaps even insufficiently informed. He had come to believe that much of the work he and Bricmont were criticizing at the time was not outright wrong or nonsensical – instead, the dispute turned on deep (even reasonable) disagreements about the nature of scientific knowledge.

However, in that same book, Sokal highlighted at length the ways in which bad actors — from industry executives to mass media outlets to cynical politicians — have re-appropriated tenets of postmodernism in order to render the population more vulnerable to propaganda and advertisements, and less likely to believe expert criticism of the state or corporations. As a result, in many respects, critical theory may be accomplishing the opposite of its stated purpose (a point previously emphasized by Sokal’s critic, Bruno Latour, in his 2004 Critical Inquiry essay “Why Has Critique Run Out of Steam? From Matters of Fact to Matters of Concern”).

Notwithstanding Sokal’s own evolution on these issues, his 1996 hoax reenergized criticism, skepticism and resistance towards critical theory – particularly within the field of philosophy. In 1997, the journal Philosophy and Literature initiated a “Bad Writing Contest,” highlighting the most obtuse or meaningless selections of the year – generally targeting critical theorists. The contest achieved peak notoriety in 1999 when Judith Butler took the top prize (she was less than amused).

In his 1998 book, Achieving Our Country: Leftist Thought in Twentieth-Century America, pragmatist Richard Rorty urged social scientists and activists to kick their philosophy habit (hat tip to Conor Friedersdorf):

“Recent attempts to subvert social institutions by problematizing concepts have produced a few very good books. They have also produced many thousands of books which represent scholastic philosophizing at its worst. The authors of these purportedly ‘subversive’ books honestly believe that they are serving human liberty. But it is almost impossible to clamber back down from their books to a level of abstraction on which one might discuss the merits of a law, a treaty, a candidate, or a political strategy. Even though what these authors ‘theorize’ is often something very concrete and near at hand—a current TV show, a media celebrity, a recent scandal—they offer the most abstract and barren explanations imaginable. These futile attempts to philosophize one’s way into political relevance are a symptom of what happens when a Left retreats from activism and adopts a spectatorial approach to the problems of its country” (pp. 93-4).

Incidentally, in that same work, twenty years before Plukrose, Lindsay & Boghossian (or John McWhorter before them), Rorty also noted the apparent religiosity of critical theorists:

“The Ubiquity of Foucaldian power is reminiscent of the ubiquity of Satan, and thus of the ubiquity of original sin—that diabolical stain on every human soul… in committing itself to what it calls ‘theory,’ this Left has gotten something which is entirely too much like religion. For the cultural Left has come to believe that we must place our country within a theoretical frame of reference, situate it within a vast quasi-cosmological perspective. What stories about blue-eyed devils are to Black Muslims, stories about hegemony and power are to many cultural Leftists… To step into the intellectual world which some of these Leftists inhabit is to move out of a world in which the citizens of a democracy can join forces to resist sadism and selfishness into a Gothic world in which democratic politics has become a farce… in which all the daylight cheerfulness of Whitmanesque hyper-secularism has been lost, and ‘liberalism’ and ‘humanism’ are synonyms for naiveté—for an inability to grasp the full horror of our situation” (pp. 95-6).

There were many more attempts at Sokal-style hoaxes attempting to delegitimize certain lines of scholarship within the discipline. Perhaps the most prominent of these occurred right around the 20-year anniversary of the Sokal Affair: philosophers Phillipe Huneman & Anouk Barberousse pseudonymously published what they perceived to be a nonsensical article in the Journal of Badiou Studies: “Ontology, Neutrality and the Strive for (non-) Being-Queer.” The authors claimed they perpetrated the hoax in order to help dispel the cult around prominent philosopher Alain Badiou (for whom the journal was named). In their view, many credit him a genius because they like his politics, but understand or engage little with his metaphysics – the primary subject of most of his work – which the hoaxers viewed as subpar. Philosopher Justin E.H. Smith (University of Paris 7 – Denis Diderot) wrote in response:

“But what sort of victory is it? Here my laughter on first reading Huneman and Barberousse’s text quickly gave way to two concerns. One is that the joke is not so much on the abstruse theory-heads, as had surely been the case in the Sokal incident. The joke isn’t on anyone who is committed to any particular ideology or style of thinking at all. The joke is, rather, on the folks running these pop-up online journals with their ludicrously low editorial standards. Remarkably, the editors of the Journal of Badiou Studies even admitted as much when they complained of Huneman and Barberousse’s ‘dated’ method of attack ‘in an age when the pressures on independent Open Access publishing include underfunding and time-pressured staff.’ In other words, the editors effectively confess that they do not have the resources to produce a decent journal on their own, and so must rely on the good will of the contributors to not send them crap. But many people who submit to journals are not in a position to know of their own work whether it is crap or not, and for this reason alone a journal that does not have the resources to weed out crap would be doing scholarship a far greater service by simply not existing. The problem is compounded in the context of continental European publishing in English, where often, at every stage of production, from writing to proofreading to publishing, all of the people involved speak English, at best, as a second language. What slipped through at the Journal of Badiou Studies does not in fact look so different from what slips through on a regular basis at Springer or Brill.”

Notwithstanding these concerns, the following year, Portland State philosopher Peter Boghossian and mathematician James Lindsay pseudonymously published “The Conceptual Penis as a Social Construct” in the journal Cogent Social Sciences; they revealed their hoax in Skeptic magazine alongside a lengthy critique of gender studies.

While many who were already critical of gender studies embraced the hoax as further evidence of what they already believed, the duo were also met with widespread criticism (including from Sokal himself!) – most trenchantly for their decision to target an obscure pay-to-play journal that lacked specialists from the field they were targeting. In other words, they could not confidently assert that their work was actually representative of the literature they were trying to critique – and as a result, the strong claims they made about gender & sexuality studies on the basis of the hoax seemed to be ill-substantiated.

Humanities scholar Helen Pluckrose rose to the defense of Boghossian & Lindsay in the online magazine she oversees (Areo). Even prior to the “Conceptual Penis” controversy, James Lindsay was a semi-regular contributor to this publication — and ultimately served as the bridge between Pluckrose and his Cogent Studies co-conspirator, Boghossian. After some discussion, the three of them decided to work together to replicate the hoax in a way that would render many of the criticisms of the “Conceptual Penis” paper moot:

They chose to target well-regarded peer-review journals specializing in the critical study of gender, sexuality… and race. Of course, a single published paper — even if placed in a high-quality specialized journal – could still be written off as non-representative: an artifact of a single set of bad reviewers. So the trio attempted to compose and place dozens of papers at well-respected journals that focused on gender, sexuality and/or race.

Yet, ironically, their hoax was prematurely exposed by others who were also highly critical of the “critical studies” approach to race, gender and sexuality. It began when Robby Soave published an article in Reason arguing that their essay on canine rape culture (“Human reactions to rape culture and queer performativity at urban dog parks in Portland, Oregon”) was so absurd he hoped it was a Sokal-style hoax – although it didn’t seem to be one.

This drew the attention of Toni Ariaksinen, who tried to identify and contact others at the Portland Ungendering Research Institute, which the author claimed to be affiliated with, as background for a Campus Reform story on the paper. She could not find any other scholars affiliated with the “institute,” and the website seemed to be thrown up just prior to the paper’s publication. She reached out to the handful of programs offering a PhD in feminist studies (which the author claimed to hold) — and none could confirm “Helen Wilson” had ever been a student there. And so, in July 2018 Ariaksinen published her article — arguing that the “dog rape culture” essay was either a hoax, or its author had lied about her credentials and affiliations. The essay was subsequently flagged by the journal it was published in (Gender, Place and Culture), pending an investigation into its authorship.

This piqued the interest of Wall Street Journal reporter Jillian Kay Melchior, who contacted “Helen Wilson” and conveyed that she planned to write a major story about the incident, and the possibility the paper was a hoax. The trio realized the jig was up and “came clean to the Wall Street Journal at the beginning of August and began preparing a summary as quickly as possible” despite the fact they “still had several papers progressing encouragingly through the review process.” In turn, WSJ apparently agreed to hold off on publishing the story, in order to allow the trio time to get their summary together.

Ethics

In principle, the “right” way to demonstrate flaws within a given literature would be to single out several prominent examples (works that were published in major journals and/or are highly-cited, especially from prominent scholars within that subfield) and show that they are deeply flawed. Ideally, the errors would seem to point in the same general direction, suggesting a systemic issue with published research in the subfield. These criticisms would, themselves, be subjected to the peer-review process and published in a scholarly journal – ideally, a journal within the subfield under criticism (rendering it more likely that the critique will reach the people who need to hear it most, and helping to ensure that the criticisms of the subfield are well-informed and fair-minded).

There seemed to be several compelling reasons why the authors did not go about criticizing “grievance studies” in the “right way.” Among them:

• The authors seemed to believe that many of the claims, and much of the evidence, presented in “critical studies” literature is so vague, esoteric or abstract that it cannot even be said to be “wrong.” Consequently, it becomes difficult to point out errors – not because the arguments are correct, but because there is so little “there” to refute.
• The authors seemed frustrated by the fact that, often, when outsiders do try to refute “critical studies” scholarship, these critiques are largely waved away on the grounds that their interlocutors just don’t “get it.” This kind of move becomes much more difficult when the critics have published multiple studies in respected peer-reviewed journals in the subfield, been asked to serve as peer-reviewers for other journals in the subfield on the basis of their contributions / demonstrated expertise, and even won awards for their research in the subfield (the “dog rape culture” essay won a prize from the journal it was published in). Indeed, it only makes the problem worse if the authors truly don’t “get it” but still managed to achieve these feats. Hence the utility behind allowing the fake papers to proceed all the way through publication.
• The process of identifying the ideal journal, submitting a paper, then having it reviewed, edited and published, typically takes months (sometimes even years). Given that their plot was discovered midway through the experiment, and they were about to be “scooped” on their own story (and lose control over the narrative) – they could not wait to publish their report in a peer-reviewed journal. As an aside, given that most published academic research are never cited and barely read, it is likely that their work would have reached a far smaller audience/ made much less impact had it been published in a peer-reviewed academic journal rather than Areo.
• Plurkrose studies literature and philosophy (epistemology); Boghossian is a philosopher; Lindsay is a mathematician trained in physics (similar to Sokal). Hence, one reason they seemed to lack the background or inclination to follow the same protocols as social scientists may be that… they are not, in fact, social scientists.

However, even granting these caveats, a number of other significant ethical issues remain.

For instance, the authors misrepresented their identities, credentials and affiliations, put forward bad-faith arguments and (perhaps more troubling) phony data – and allowed these to actually be published, and to stand for some time. If their plot had not been exposed prematurely, many other researchers could have been led (even further) astray through other works that cited the trio’s research, believing it to be reliable (in part because it had been published in these respected journals – and therefore, presumably vetted by authorities in the subfield). This is very different, for instance, from when Science magazine carried out a similar hoax to test the rigor of open-access journals, sending fatally-flawed papers to hundreds of journals to see how many would accept (most did): Science did not allow these flawed papers to be published. Plukrose et al. did.

One could add, it would be very bad if this kind of hoaxing became a widespread practice (for instance, if the often well-credentialed contributors to Areo, Quillette and other sympathetic outlets decided to also flood social research journals with fake studies, allow accepted papers to proceed all the way through publication, and expose them as fakes, making sweeping claims against the relevant fields in the process) – this would greatly undermine the fidelity of peer review or citation, and public trust in social research. In Kantian terms, the trio’s actions could not be successfully scaled into a categorical imperative.

There are also financial considerations. Coming up with, researching for, writing, editing and placing such a large number of academic articles in such a short time became something like a full-time job over the course of the experiment. The trio received significant financial support from an unknown benefactor, allowing them to waive many of their other commitments and dedicate themselves fully to this project. Typically, research funding is disclosed — both at the time of article submission and (especially) at publication. This is important because, even outside of direct pressure or oversight from donors, a desire to “stay on their good side” or “demonstrate value” can influence how data are collected, interpreted and presented in a host of ways, both subtle and obvious (intentionally or not). Disclosure helps editors and readers better understand and account for these possible sources of bias — and therefore, more effectively evaluate research.

Pluckrose, Lindsay & Boghossian have asserted that their donors wished to remain anonymous out of fear they would be smeared as racist, sexist, etc. for being associated with the project — and face social and financial sanctions as a result. This seems like a legitimate concern given how the authors themselves have been depicted in many corners of academia and the media for carrying out the hoax. As the Wall Street Journal reports:

Mr. Boghossian doesn’t have tenure and expects the university will fire or otherwise punish him. Ms. Pluckrose predicts she’ll have a hard time getting accepted to a doctoral program. Mr. Lindsay said he expects to become ‘an academic pariah,’ barred from professorships or publications.”

And to be clear: the “dark money” associated with the project provides no reason to write it off out of hand; after all, funding the authors received could not plausibly have had any significant or direct influence on what they were most keen to measure: reviewers’ evaluation of their presented arguments and evidence (although, depending on the donor, the journal editor might have been more inclined to ultimately reject certain essays due to apparent conflicts-of-interest were the funding disclosed). Yet, significant anonymous funding does open the door to legitimate scrutiny about the motivations of the donors and authors, and how those might have informed the targets they chose, and the ways they opted to interpret and present their results (again, it does not in any way establish that there actually were ulterior motives or biases on the part of the authors, it merely elevates that possibility). And to her credit, Pluckrose recognizes this reality:

Update: Dr. Boghossian is currently under investigation by his university’s institutional review board for having carried out experiments on humans (journal editors) without IRB approval, and for falsifying data. For the record, here is my own view on the matter, as conveyed to Reason:

“Peter Boghossian is a philosopher, not a social scientist. There is a long tradition within the field of philosophy in carrying out hoaxes like these. They virtually never involve IRB approval. None of Boghossian’s collaborators were social scientists either. Ms. Pluckrose is a humanities scholar gearing up for grad school, and Mr. Lindsay is a mathematician and physicist by training. It is likely that none of them are well-versed in IRB requirements and protocols. In short, to the extent that normal social science protocols were not met, it was likely an honest mistake rather than a willful violation. Nor is likely that there was serious harm inflicted upon any of the ‘human subjects’ involved. Hence, I can see no reason why Dr. Boghossian’s career should be menaced for taking part in the hoax. And I’ll add in closing, given the bias in various parts of the standard research process (which is what their experiment was about!) it seems highly plausible that *had* they followed standard protocol, the IRB board would have rejected their proposal for political/ ideological reasons.”

For what it’s worth, Alan Sokal (who, again, was critical of the “Conceptual Penis” attempt) has also come to Boghossian’s defense. He claims the hoax and resultant discussions ultimately served the public interest, and that Portland State University would become a “laughingstock” in academia and to the broader public if it punished Boghossian on the grounds of carrying out an unsanctioned experiment on human subjects — and disagrees with classifying journal editors as human subjects here.

Jesse Singal offers a compelling counter-perspective: We need to separate the question of whether some on the left are responding to the hoax in a bad way from the question as to whether the IRB complaints are sound. As it relates to this latter question: Boghossian clearly seemed to have violated rules and norms. As such, sanctions of some kind are probably appropriate — but should not be severe.

Another concern is that the “Sokal Squared” hoax, much like its “Conceptual Penis” predecessor, provides plenty of “red meat” for reactionaries who want to discredit the left — or work on gender, race or sexuality – or even social research more broadly. This was a consequence the authors anticipated in advance (as their essay and interviews allude to); yet some have argued they did not do enough to hedge against this possibility (in their view the trio framed their study and described its findings in an irresponsible manner), or to push back against misuses. As a result, good research on gender, sexuality, race or inequality could find itself discredited or dismissed alongside the bad (to which the authors would likely reply, “this is already happening irrespective of the hoax – all the more reason for social research fields to better identify and purge bad scholarship”).

Some have alleged that the trio seem to be “punching down” by targeting lines of scholarship that are already marginal within social research fields – and which also happen to be oriented towards empowering historically disadvantaged and marginalized groups (in principle).

The authors have addressed this charge at length, arguing:

1. “Critical studies” of race, gender or sexuality have had a huge impact – both within the academy and in the broader culture – particularly within elite academic and media circles, and among left-leaning activists.
2. The activist-orientation of “critical studies” may provide a principled reason to be suspicious of their intellectual rigor. Attempting to promote a particular outcome, narrative or ideology seems, prima facie, incompatible with following the truth wherever it leads. Hence, there is good reason to suspect that these subfields may be more prone to errors than others, which warrants their being singled-out.
3. The authors claim they are not out to discredit research on gender, sexuality, or race. On the contrary: precisely because these issues are so contentious and important, it is essential for research on these topics to be as objective, rigorous and precise as possible. Put another way: the more important these lines of research are held to be, the more necessary it becomes for someone to push back on bad scholarship within them.

One could add that it is not unheard of for “critical studies” scholars to pull this kind of hoax on one-another. For instance (as reported by The Guardian), pseudonymous authors published a “study” in the journal Totalitarianism and Democracy “proving” that the dogs East German border guards had used to patrol the Berlin Wall were direct descendants of those Nazis relied on in two of their concentration camps – thereby maintaining a “tradition of violence” among these animals and their descendants. The authors claimed to have perpetrated the hoax in order to discredit the “animal turn” in critical scholarship.

Hence, Pluckrose, Lindsay & Boghossian seem to be on solid ground for many of the moral objections people have raised with regards to their study – for others, apparently less so. Yet, as Bret Weinstein argued, whatever ethical concerns people may have over how the experiment was carried out and presented – the moral “rightness” of the authors is a separate question from “what does the experiment reveal?”

It is to this question we now turn: