Large Language Models are biased to overestimate profoundness

Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 9653–9661

Abstract

Recent advancements in natural language processing by large language models (LLMs), such as GPT-4, have been suggested to approach Artificial General Intelligence.And yet, it is still under dispute whether LLMs possess similar reasoning abilities to humans.This study evaluates GPT-4 and various other LLMs in judging the profoundness of mundane, motivational, and pseudo-profound statements.We found a significant statement-to-statement correlation between the LLMs and humans, irrespective of the type of statements and the prompting technique used.However, LLMs systematically overestimate the profoundness of nonsensical statements, with the exception of Tk-instruct, which uniquely underestimates the profoundness of statements.Only fewshot learning prompts, as opposed to chain-ofthought prompting, draw LLMs ratings closer to humans.Furthermore, this work provides insights into the potential biases induced by Reinforcement Learning from Human Feedback (RLHF), inducing an increase in the bias to overestimate the profoundness of statements.

Authors 5

1. Eugenio Herrera-Berg
   Affiliation as printed

   Centro Nacional de Inteligencia Artificial (CENIA) , Chile

   Centro Nacional de Inteligencia Artificial (CENIA), Chile

2. Tomás Vergara Browne

   Pontificia Universidad Católica de Chile

   Affiliation as printed

   Centro Nacional de Inteligencia Artificial (CENIA) , Chile

   Pontificia Universidad Católica de Chile , Chile

   Centro Nacional de Inteligencia Artificial (CENIA), Chile

   Pontificia Universidad Católica de Chile, Chile

3. Pablo León-Villagrá

   Brown University

   Affiliation as printed

   Brown University , United States

   Brown University, United States

4. Marc–Lluís Vives Aachen

   Leiden University

   Affiliation as printed

   Leiden University , the Netherlands

   Leiden University, the Netherlands

5. Cristian Buc Calderon
   Affiliation as printed

   Centro Nacional de Inteligencia Artificial (CENIA) , Chile

   Centro Nacional de Inteligencia Artificial (CENIA), Chile