Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias

Findings of the Association for Computational Linguistics: NAACL, pp. 4984–5004

Abstract

Generated texts from large language models (LLMs) have been shown to exhibit a variety of harmful, human-like biases against various demographics.These findings motivate research efforts aiming to understand and measure such effects.This paper introduces a causal formulation for bias measurement in generative language models.Based on this theoretical foundation, we outline a list of desiderata for designing robust bias benchmarks.We then propose a benchmark called OCCUGENDER, with a bias-measuring procedure to investigate occupational gender bias.We test several state-ofthe-art open-source LLMs on OCCUGENDER, including Llama, Mistral, and their instructiontuned versions.The results show that these models exhibit substantial occupational gender bias.Lastly, we discuss prompting strategies for bias mitigation and an extension of our causal formulation to illustrate the generalizability of our framework.1 * Equal contribution. 1 The code and the OCCUGENDER benchmark are available at https://github.com/chenyuen0103/ gender-bias.

Authors 5

1. Yuen Chen

   University of Illinois Urbana-Champaign

   Affiliation as printed

   University of Illinois at Urbana-Champaign ,

2. Vethavikashini Chithrra Raghuram
3. Justus Mattern Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen ,

4. Rada F. Mihalcea

   University of Michigan

   Affiliation as printed

   University of Michigan ,

5. Zhijing Jin

   ETH Zurich · Max Planck Society

   Affiliation as printed

   ETH Zürich ,

   Max Planck Institute