Hofmann, V., et al. (2024).
Nature, 633(8028), 147–154.
Abstract
Hundreds of millions of people now interact with language models, with uses ranging from help with writing to informing hiring decisions. However, these language models are known to perpetuate systematic racial prejudices, making their judgements biased in problematic ways about groups such as African Americans. Although previous research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time, particularly in the United States after the civil rights movement. It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice, exhibiting raciolinguistic stereotypes about speakers of African American English (AAE) that are more negative than any human stereotypes about African Americans ever experimentally recorded. By contrast, the language models’ overt stereotypes about African Americans are more positive. Dialect prejudice has the potential for harmful consequences: language models are more likely to suggest that speakers of AAE be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death. Finally, we show that current practices of alleviating racial bias in language models, such as human preference alignment, exacerbate the discrepancy between covert and overt stereotypes, by superficially obscuring the racism that language models maintain on a deeper level. Our findings have far-reaching implications for the fair and safe use of language technology.
Here are some thoughts:
This research article demonstrates that artificial intelligence language models exhibit covert racism through deep-seated dialect prejudice against speakers of African American English. By evaluating language variations, the authors found that these models attach negative stereotypes to African American English that are more severe than any human stereotypes ever recorded experimentally, even while their overt statements about Black individuals appear positive. For psychologists, this study is highly important because it reveals how systemic racism and implicit bias can be stealthily automated and amplified within technology. It underscores that human preference training merely masks superficial bias while leaving harmful, underlying prejudices fully intact. Insightfully, the findings warn that relying on artificial intelligence for clinical diagnostics, forensic evaluations, or employment screening can lead to discriminatory outcomes, such as harsher legal judgments or lower prestige job recommendations. Psychologists must therefore spearhead critical evaluations of these tools to ensure digital assessments do not reinforce historical inequities.
