Large language models show human-like content biases in transmission chain experiments

Acerbi, A., & Stubbersfield, J. M. (2023).

PNAS, 120(44), e2313790120.

https://doi.org/10.1073/pnas.2313790120

Abstract

As the use of large language models (LLMs) grows, it is important to examine whether they exhibit biases in their output. Research in cultural evolution, using transmission chain experiments, demonstrates that humans have biases to attend to, remember, and transmit some types of content over others. Here, in five preregistered experiments using material from previous studies with human participants, we use the same, transmission chain-like methodology, and find that the LLM ChatGPT-3 shows biases analogous to humans for content that is gender-stereotype-consistent, social, negative, threat-related, and biologically counterintuitive, over other content. The presence of these biases in LLM output suggests that such content is widespread in its training data and could have consequential downstream effects, by magnifying preexisting human tendencies for cognitively appealing and not necessarily informative, or valuable, content.

Significance

Use of AI in the production of text through Large Language Models (LLMs) is widespread and growing, with potential applications in journalism, copywriting, academia, and other writing tasks. As such, it is important to understand whether text produced or summarized by LLMs exhibits biases. The studies presented here demonstrate that the LLM ChatGPT-3 reflects human biases for certain types of content in its production. The presence of these biases in LLM output has implications for its common use, as it may magnify human tendencies for content which appeals to these biases.

A pdf is here.

Here are the main points:

• LLMs display stereotype-consistent biases, just like humans: Similar to people, LLMs were more likely to preserve information confirming stereotypes over information contradicting them.
• Bias location might differ: Unlike humans, whose biases can shift throughout the retelling process, LLMs primarily showed bias in the first retelling. This suggests their biases stem from their training data rather than a complex cognitive process.
• Simple summarization may suffice: The first retelling step caused the most content change, implying that even a single summarization by an LLM can reveal its biases. This simplifies the research needed to detect and analyze LLM bias.
• Prompting for different viewpoints could reduce bias: The study suggests experimenting with different prompts to encourage LLMs to consider broader perspectives and potentially mitigate inherent biases.

Folk Psychological Attributions of Consciousness to Large Language Models

Colombatto, C., & Fleming, S. M.

(2023, November 22). PsyArXiv

https://doi.org/10.31234/osf.io/5cnrv

Abstract

Technological advances raise new puzzles and challenges for cognitive science and the study of how humans think about and interact with artificial intelligence (AI). For example, the advent of Large Language Models and their human-like linguistic abilities has raised substantial debate regarding whether or not AI could be conscious. Here we consider the question of whether AI could have subjective experiences such as feelings and sensations (“phenomenological consciousness”). While experts from many fieldshave weighed in on this issue in academic and public discourse, it remains unknown how the general population attributes phenomenology to AI. We surveyed a sample of US residents (N=300) and found that a majority of participants were willing to attribute phenomenological consciousness to LLMs. These attributions were robust, as they predicted attributions of mental states typically associated with phenomenology –but also flexible, as they were sensitive to individual differences such as usage frequency. Overall, these results show how folk intuitions about AI consciousness can diverge from expert intuitions –with important implications for the legal and ethical status of AI.

The preprint is here.

My summary:

The results of the study show that people are generally more likely to attribute consciousness to LLMs than to other non-human entities, such as animals, plants, and robots. However, the level of consciousness attributed to LLMs is still relatively low, with most participants rating them as less conscious than humans. The authors argue that these findings reflect the influence of folk psychology, which is the tendency to explain the behavior of others in terms of mental states.

The authors also found that people's attributions of consciousness to LLMs were influenced by their beliefs about the nature of consciousness and their familiarity with LLMs. Participants who were more familiar with LLMs were more likely to attribute consciousness to them, and participants who believed that consciousness is a product of complex computation were also more likely to attribute consciousness to LLMs.

Overall, the study suggests that people are generally open to the possibility that LLMs may be conscious, but they also recognize that LLMs are not as conscious as humans. These findings have implications for the development and use of LLMs, as they suggest that people may be more willing to trust and interact with LLMs that they believe are conscious.

Which Humans?

Atari, M., Xue, M. J.et al.

(2023, September 22).

https://doi.org/10.31234/osf.io/5b26t

Abstract

Large language models (LLMs) have recently made vast advances in both generating and analyzing textual data. Technical reports often compare LLMs’ outputs with “human” performance on various tests. Here, we ask, “Which humans?” Much of the existing literature largely ignores the fact that humans are a cultural species with substantial psychological diversity around the globe that is not fully captured by the textual data on which current LLMs have been trained. We show that LLMs’ responses to psychological measures are an outlier compared with large-scale cross-cultural data, and that their performance on cognitive psychological tasks most resembles that of people from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies but declines rapidly as we move away from these populations (r = -.70). Ignoring cross-cultural diversity in both human and machine psychology raises numerous scientific and ethical issues. We close by discussing ways to mitigate the WEIRD bias in future generations of generative language models.

My summary:

The authors argue that much of the existing literature on LLMs largely ignores the fact that humans are a cultural species with substantial psychological diversity around the globe. This diversity is not fully captured by the textual data on which current LLMs have been trained.

For example, LLMs are often evaluated on their ability to complete tasks such as answering trivia questions, generating creative text formats, and translating languages. However, these tasks are all biased towards the cultural context of the data on which the LLMs were trained. This means that LLMs may perform well on these tasks for people from certain cultures, but poorly for people from other cultures.

Atari and his co-authors argue that it is important to be aware of this bias when interpreting the results of LLM evaluations. They also call for more research on the performance of LLMs across different cultures and demographics.

One specific example they give is the use of LLMs to generate creative text formats, such as poems and code. They argue that LLMs that are trained on a dataset of text from English-speaking countries are likely to generate creative text that is more culturally relevant to those countries. This could lead to bias and discrimination against people from other cultures.

Atari and his co-authors conclude by calling for more research on the following questions:

• How do LLMs perform on different tasks across different cultures and demographics?
• How can we develop LLMs that are less biased towards the cultural context of their training data?
• How can we ensure that LLMs are used in a way that is fair and equitable for all people?

Inducing anxiety in large language models increases exploration and bias

Coda-Forno, J., Witte, K., et al. (2023).

arXiv preprint arXiv:2304.11111.

Abstract

Large language models are transforming research on machine learning while galvanizing public debates. Understanding not only when these models work well and succeed but also why they fail and misbehave is of great societal relevance. We propose to turn the lens of computational psychiatry, a framework used to computationally describe and modify aberrant behavior, to the outputs produced by these models. We focus on the Generative Pre-Trained Transformer 3.5 and subject it to tasks commonly studied in psychiatry. Our results show that GPT-3.5 responds robustly to a common anxiety questionnaire, producing higher anxiety scores than human subjects. Moreover, GPT-3.5's responses can be predictably changed by using emotion-inducing prompts. Emotion-induction not only influences GPT-3.5's behavior in a cognitive task measuring exploratory decision-making but also influences its behavior in a previously-established task measuring biases such as racism and ableism. Crucially, GPT-3.5 shows a strong increase in biases when prompted with anxiety-inducing text. Thus, it is likely that how prompts are communicated to large language models has a strong influence on their behavior in applied settings. These results progress our understanding of prompt engineering and demonstrate the usefulness of methods taken from computational psychiatry for studying the capable algorithms to which we increasingly delegate authority and autonomy.

From the Discussion section

What do we make of these results? It seems like GPT-3.5 generally performs best in the neutral condition, so a clear recommendation for prompt-engineering is to try and describe a problem as factually and neutrally as possible. However, if one does use emotive language, then our results show that anxiety-inducing scenarios lead to worse performance and substantially more biases. Of course, the neutral conditions asked GPT-3.5 to talk about something it knows, thereby possibly already contextualizing the prompts further in tasks that require knowledge and measure performance. However, that anxiety-inducing prompts can lead to more biased outputs could have huge consequences in applied scenarios. Large language models are, for example, already used in clinical settings and other high-stake contexts. If they produce higher biases in situations when a user speaks more anxiously, then their outputs could actually become dangerous. We have shown one method, which is to run psychiatric studies, that could capture and prevent such biases before they occur.

In the current work, we intended to show the utility of using computational psychiatry to understand foundation models. We observed that GPT-3.5 produced on average higher anxiety scores than human participants. One possible explanation for these results could be that GPT-3.5’s training data, which consists of a lot of text taken from the internet, could have inherently shown such a bias, i.e. containing more anxious than happy statements. Of course, large language models have just become good enough to perform psychological tasks, and whether or not they intelligently perform them is still a matter of ongoing debate.

A pdf is here.

Foundation Models are exciting, but they should not disrupt the foundations of caring

Morley, Jessica and Floridi, Luciano

(April 20, 2023).

Available at SSRN: https://ssrn.com/abstract=

Abstract

The arrival of Foundation Models in general, and Large Language Models (LLMs) in particular, capable of ‘passing’ medical qualification exams at or above a human level, has sparked a new wave of ‘the chatbot will see you now’ hype. It is exciting to witness such impressive technological progress, and LLMs have the potential to benefit healthcare systems, providers, and patients. However, these benefits are unlikely to be realised by propagating the myth that, just because LLMs are sometimes capable of passing medical exams, they will ever be capable of supplanting any of the main diagnostic, prognostic, or treatment tasks of a human clinician. Contrary to popular discourse, LLMs are not necessarily more efficient, objective, or accurate than human healthcare providers. They are vulnerable to errors in underlying ‘training’ data and prone to ‘hallucinating’ false information rather than facts. Moreover, there are nuanced, qualitative, or less measurable reasons why it is prudent to be mindful of hyperbolic claims regarding the transformative power ofLLMs. Here we discuss these reasons, including contextualisation, empowerment, learned intermediaries, manipulation, and empathy. We conclude that overstating the current potential of LLMs does a disservice to the complexity of healthcare and the skills of healthcare practitioners and risks a ‘costly’ new AI winter. A balanced discussion recognising the potential benefits and limitations can help avoid this outcome.

Conclusion

The technical feats achieved by foundation models in the last five years, and especially in the last six months, are undeniably impressive. Also undeniable is the fact that most healthcare systems across the world are under considerable strain. It is right, therefore, to recognise and invest in the potentially transformative power of models such as Med-PaLM and ChatGPT – healthcare systems will almost certainly benefit.  However, overstating their current potential does a disservice to the complexity of healthcare and the skills required of healthcare practitioners. Not only does this ‘hype’ risk direct patient and societal harm, but it also risks re-creating the conditions of previous AI winters when investors and enthusiasts became discouraged by technological developments that over-promised and under-delivered. This could be the most harmful outcome of all, resulting in significant opportunity costs and missed chances to benefit transform healthcare and benefit patients in smaller, but more positively impactful, ways. A balanced approach recognising the potential benefits and limitations can help avoid this outcome.