Unmasking Cultural Bias in Language Models: A Dive into Mythological Knowledge Retrieval

A groundbreaking study conducted by researchers Iaroslav Chelombitko, Ekaterina Chelombitko, and Mika Hämäläinen reveals a profound cultural bias within open-source large language models (LLMs). The research demonstrates that while these models efficiently recall mythological figures like Zeus and Thor from prominent traditions, they often struggle to identify lesser-known deities from cultures such as Finnish, Slavic, or Chinese mythology. This disparity not only highlights existing biases in AI but also raises important questions about the implications of these biases in real-world applications.

Understanding the Mechanism Behind Cultural Bias

To investigate this phenomenon, the researchers employed advanced mechanistic interpretability techniques. They utilized a comprehensive analysis across 18 different LLMs, demonstrating how these models recognize cultural identities but fail to produce culturally relevant output consistently. Specifically, although the models can identify mythological entities through linear probing — a method that tracks how information flows within the network — the decoding process collapses culturally specific tokens into a dominant tradition. This indicates that the bias originates at the decoders rather than the encoders, meaning that while the models internally recognize these identities, they struggle to express them accurately.

The Significance of Decoding Suppression

The study identifies a key finding termed "Decoding Suppressed," which illustrates that across all examined models, the readout system fails to produce the correct cultural names despite correctly identifying them internally. For instance, when asked about gods from less represented cultures, the models frequently output names associated with well-known traditions, thereby sidelining other mythologies. The researchers found that this suppression is especially pronounced when the models are prompted in English compared to the native languages of those mythologies. This linguistic bias demonstrates that the models' outputs are significantly gated by the language in which they are queried.

Addressing the Cultural Knowledge Gap

The implications of this research stretch beyond academics, posing critical questions for industries reliant on AI technologies, such as education, media, and international relations. The findings suggest that to foster equitable AI, developers must implement targeted interventions that mitigate the decoding bias by enhancing the cultural knowledge that is represented and expressed across all models.

Releasing Tools for Future Research

In response to their findings, the authors introduced a suite of tools aimed at improving the evaluation and understanding of cultural knowledge within LLMs. Among these are a citation-anchored cross-cultural ground truth and a framework for evaluating prompt sensitivity across languages. These resources can assist future researchers in diagnosing and addressing cultural biases in AI models, striving for a more inclusive development process.

As we tread further into an AI-driven future, it is crucial to ensure that our technological creations don’t perpetuate existing cultural disparities. The work of Chelombitko and his colleagues marks a vital step towards reckoning with and correcting inherent biases in AI systems, paving the way for more culturally aware and sensitive applications.

Authors: Iaroslav Chelombitko, Ekaterina Chelombitko, and Mika Hämäläinen