Trigger Warning: Empirical Evidence Ahead

Every year, thousands of individuals receive trigger warnings on college campuses, on social media, and even in casual conversation. The vast majority of these individuals do not have any form of PTSD. What is the effect of trigger warnings on such individuals? Do trigger warnings, in the words of Haidt & Lukianoff, “coddle the American mind?” Or do they prepare individuals to brace themselves for distressing events, allowing them to face difficulties head-on? Along with my colleagues at Harvard Psychology Benjamin Bellet and Richard McNally, I conducted an experiment to get a first look. Our paper, “Trigger Warning: Empirical Evidence Ahead,” was recently published in the Journal of Behavior Therapy and Experimental Psychiatry. While this is only a preliminary look at trigger warnings in a non-traumatized population, it provides some insight into how trigger warnings might affect the mind. We are currently conducting replication studies in college students and in previously traumatized individuals to see if the results will replicate and generalize.

Methods

The study consisted of a very simple randomized experiment. A group of 270 individuals recruited from Amazon Mechanical Turk read passages from literature. A power analysis determined that this sample was large enough to detect a small effect size in our analyses (1 – β error probability = 0.96). Half of the sample was given trigger warnings before some of the passages, and the other half received no trigger warnings at all. Trigger warnings were only given for passages that were rated as “distressing” in a pilot study. After each passage, individuals rated their own emotions. Although they rated a variety of emotions (so as not to clue them in to the true purpose of the study), we were interested in their rating of “anxiety”. Individuals rated their emotions on three passages before and after the section including trigger warnings, to establish a baseline anxiety rating and a follow-up anxiety rating. After reading passages, the participants completed a few follow-up measures. The questions of the study were extremely simple, and the methods and statistical models reflect this experimental simplicity.

Results

We had five questions. In each case, the independent variable was the binary condition of trigger warnings. The results were mixed, with trigger warnings affecting about half of the outcome measures.

Did receiving trigger warnings alter participants’ perceptions of their own posttraumatic vulnerability? (e.g., “[If I were to suffer a trauma], I would not be able to work a job, or take care of myself”) Yes. Individuals who received trigger warnings rated themselves as more vulnerable (p < 0.05).
Did receiving trigger warnings alter participants’ implicit perceptions of their own vulnerability? (tested by an implicit association test) No. There was no significant difference between the groups.
Did receiving trigger warnings alter participants’ perceptions of others’ posttraumatic vulnerability? (e.g., “[If an average person were to suffer a trauma, that person would] not be able to work a job or take care of himself/herself”) Yes. Individuals who received trigger warnings rated an average person as more vulnerable (p < 0.05). This outcome was highly correlated with the outcome from the first question (r = 0.74).
Did trigger warnings affect immediate anxiety response? Was this moderated by belief that “words can harm”? No and yes. Trigger warnings did not significantly affect immediate anxiety response. However, this effect was significantly moderated by the belief that words can harm (p < 0.01), such that individuals with a high belief that words can harm showed increases in anxiety if they received trigger warnings (p < 0.05).
Did trigger warnings affect anxiety at follow-up? Was this moderated by belief that the world is predictable and controllable? No and no. Trigger warnings did not significantly affect anxiety at follow-up, and there was no evidence for moderation.

Discussion

This is the first scientific look at trigger warnings. It answers some questions, does not answer others, and raises additional ones. We will outline some of the major questions here: What is the overall message? Do trigger warnings help or harm?

In this study, we found that trigger warnings occasionally harm, and the effects of this harm were small. However, it is important to remember that there is a third option: help, harm, or do nothing. For several of our outcomes, we found that trigger warnings do nothing (i.e., nothing statistically significant). In no cases did we find that trigger warnings help, but we also did not measure the sample that trigger warnings are intended to help (trauma survivors).

I’m a teacher. Should I stop using trigger warnings?

The evidence in this study is not strong or generalizable enough to make a conclusion about whether or not to use trigger warnings in the classroom. But it is also important to ask yourself the following question: if there is no evidence regarding whether or not a practice works, should I be using it? This is a personal question that you will have to answer for yourself (the microaggressions literature faces a similar challenge).

I have seen researchers claiming that these results are “p-hacked”, not generalizable, or too small to care about. Is this true?

The analyses were incredibly simple, leaving relatively little room for “p-hacking” (performing many analyses until statistically significant results are found). The study reports controlled and uncontrolled analyses, both of which gave the exact same pattern of results. The pattern of p-values is also unsurprising given that the two “marginal” p-values (0.01 < p < 0.05) in question came from highly correlated dependent variables (r = 0.74). The results are not immediately generalizable to individuals who have suffered from trauma. But it is important to remember that we have noevidence on these individuals – so although we shouldn’t jump to the conclusion that trigger warnings harm these individuals, we also shouldn’t jump to the conclusion that trigger warnings help these individuals.

Are the effects “too small to care about”?

It is hard to say. We gave only a handful of trigger warnings over a short period of time. At this point, it is impossible to know whether these effects accumulate over time. However, it is important to remember that even if trigger warnings did absolutely nothing, this is still something important enough to care about.

Are trigger warnings a violation of free speech?

This is a question that cannot be answered by scientific inquiry, as it depends on moral values rather than statistical outcomes.

I want to learn more about the science behind trigger warnings. Is there anything else I can read?

To the best of our knowledge, this was the first published empirical study on trigger warnings. However, there is a yet unpublished study which is available as a poster. The third author of the current study, an expert in PTSD research, has also written an op-ed in the New York Times on the topic.

Citation: Benjamin W. Bellet, Payton J. Jones & Richard J. McNally. “Trigger Warning: Empirical Evidence Ahead.” Journal of Behavior Therapy and Experimental Psychiatry 61 (2018): 134-41. Editor’s note: This post was submitted, edited, and slated for publication weeks ago. However, a second peer-reviewed study on trigger warnings and their effects has just been published. Citation:

Izzy Gainsburg & Allison Earl. “Trigger Warnings as an Interpersonal Emotion-Regulation Tool: Avoidance, Attention and Affect Depend on Beliefs.” Journal of Experimental Social Psychology 70 (2018): 252-63.

The findings are broadly consistent with Bellet, Jones & McNally’s, as Jones describes here. One interesting wrinkle is as follows: for those who were exposed to trigger warnings and believed that trigger warnings actually protect people, these respondents experienced increased anticipated negative affect (i.e. more anxiety about engaging the material) — and trigger warnings did a worse job for them at reducing experienced negative affect post-reading. This is broadly consistent with Bellet, Jones & McNally’s findings that those exposed to trigger warnings reported increased perceptions of vulnerability (of self and others), and that those who strongly believe “words can harm” experienced increased anxiety when exposed to trigger warnings — while there was no significant effect on anxiety among those who did not hold this belief. However, somewhat ironically perhaps, the Gainsburg & Earl study shows that those who dismissed trigger warnings as “coddling” had smaller increases in anticipated negative affect after being given a warning. And for them, being provided a trigger warning actually did correlate with decreased negative affect, and to an extent significantly stronger than in those who actually believed trigger warnings protect (again, Bellet, Jones & McNally found no effect for trigger warnings on reducing negative affect post-exposure to the material). Of course, the caveats mentioned above still hold fully: this is very preliminary research. Indeed, these mark the first empirical studies on the effects of trigger warnings. Much more work needs to be done in order to be confident in the results of these papers and any apparent implications. However, tentative conclusions from the two studies discussed here may include: to the extent that trigger warnings are actually effective at their intended purpose (reducing negative affect), it seems as though the effect is small, and may be most pronounced among those who believe in them the least. For those already predisposed to believe in trigger warnings and the logic behind them (they “protect” because “words can harm”), theyseem to experience significantly higher anxiety and perceptions of vulnerability after being given trigger warnings — and for them, warnings have a significantly smaller impact on actually reducing negative affect post-exposure to controversial material.