Post-training makes large language models less human-like

arXiv (Cornell University)

Abstract

Large language models (LLMs) are increasingly used as surrogates for human participants, but it remains unclear which models best capture human behavior and why. To address this, we introduce Psych-201, a novel dataset that enables us to measure behavioral alignment at scale. We find that post-training -- the stage that turns base models into useful assistants -- consistently reduces alignment with human behavior across model families, sizes, and objectives. Moreover, this misalignment widens in newer model generations even as base models continue to improve. Finally, we find that persona-induction -- a popular technique for eliciting human-like behavior by conditioning models on participant-specific information -- does not improve predictions at the level of individuals. Taken together, our results suggest that the very processes that are currently employed to turn LLMs into useful assistants also make them less accurate models of human behavior.

Authors 78

1. Marcel Binz

   Helmholtz Association of German Research Centres · Helmholtz Munich

   Affiliation as printed

   Helmholtz Munich ,

2. Elif Akata

   Helmholtz Association of German Research Centres · Helmholtz Munich

   Affiliation as printed

   Helmholtz Munich ,

3. Abdullah Almaatouq

   Massachusetts Institute of Technology

   Affiliation as printed

   Massachusetts Institute of Technology ,

4. Mohammed Alsobay

   Massachusetts Institute of Technology

   Affiliation as printed

   Massachusetts Institute of Technology ,

5. Oleksii Ariasov

   University of Tübingen

   Affiliation as printed

   University of Tübingen ,

6. Franziska Brändle

   University of Oxford

   Affiliation as printed

   University of Oxford ,

7. David Broska

   Stanford University

   Affiliation as printed

   Stanford University ,

8. Jason W. Burton

   University of Copenhagen · Max Planck Institute for Human Development

   Affiliation as printed

   Max Planck Institute for Human Development ,

   University of Copenhagen ,

9. Nuno Busch

   Technical University of Munich

   Affiliation as printed

   Technical University of Munich ,

10. Frederick Callaway

    New York University

    Affiliation as printed

    New York University , 10

11. Vanessa Cheung Aachen

    Leiden University · University of Applied Sciences Leiden

    Affiliation as printed

    Leiden University , 19

12. Brian Christian

    University of Oxford

    Affiliation as printed

    University of Oxford ,

13. Julian Coda-Forno

    Helmholtz Association of German Research Centres · Helmholtz Munich · Technical University of Munich

    Affiliation as printed

    Helmholtz Munich ,

    Technical University of Munich ,

14. Can Demircan

    Helmholtz Association of German Research Centres · Helmholtz Munich

    Affiliation as printed

    Helmholtz Munich ,

15. Vittoria Dentella

    Google (United States)

    Affiliation as printed

    University of Pavia , 12 Google DeepMind ,

16. Maria Katharina Eckstein

    University of Basel

    Affiliation as printed

    University of Basel ,

17. Noémi Éltető

    Max Planck Society · Max Planck Institute for Biological Cybernetics

    Affiliation as printed

    Max Planck Institute for Biological Cybernetics ,

18. Michael Franke

    University of Tübingen

    Affiliation as printed

    University of Tübingen ,

19. Thomas L. Griffiths

    Princeton University

    Affiliation as printed

    Princeton University ,

20. Fritz Günther

    Humboldt-Universität zu Berlin · Ludwig-Maximilians-Universität München

    Affiliation as printed

    Humboldt-Universität zu Berlin , 16 LMU Munich

21. Susanne Haridi

    University of Warwick

    Affiliation as printed

    University of Warwick , 26

22. Sebastian Hellmann

    Technical University of Munich

    Affiliation as printed

    Technical University of Munich ,

23. Stefan Herytash Aachen

    Leiden University · University of Applied Sciences Leiden

    Affiliation as printed

    Leiden University , 19

24. Linus Hof

    Technical University of Munich

    Affiliation as printed

    Technical University of Munich ,

25. Eleanor Holton

    University of Oxford

    Affiliation as printed

    University of Oxford ,

26. Isabelle Hoxha

    École Normale Supérieure

    Affiliation as printed

    École normale supérieure , 18

27. Zak Hussain

    Technische Universität Darmstadt

    Affiliation as printed

    TU Darmstadt , 33 VinUniversity ,

28. Akshay Jagadish

    Helmholtz Association of German Research Centres · Helmholtz Munich

    Affiliation as printed

    Helmholtz Munich ,

29. Elif Kara

    University of Warwick

    Affiliation as printed

    University of Warwick , 26

30. Valentin Kriegmair

    Max Planck Institute for Human Development

    Affiliation as printed

    Max Planck Institute for Human Development ,

31. Evelina Leivada

    Institució Catalana de Recerca i Estudis Avançats · Universitat Autònoma de Barcelona

    Affiliation as printed

    Autonomous University of Barcelona ,

    Institució Catalana de Recerca i Estudis Avanc ¸ats (ICREA) , 22

32. Li Ji-An
33. Tobias Ludwig

    University of Tübingen

    Affiliation as printed

    University of Tübingen ,

34. Maximilian Maier

    University of Warwick

    Affiliation as printed

    University of Warwick , 26

35. Marcelo Gomes Mattar

    New York University

    Affiliation as printed

    New York University , 10

36. Marvin Mathony

    Helmholtz Association of German Research Centres · Helmholtz Munich

    Affiliation as printed

    Helmholtz Munich ,

37. Alireza Modirshanechi

    Helmholtz Association of German Research Centres · Max Planck Society · Helmholtz Munich · Max Planck Institute for Biological Cybernetics

    Affiliation as printed

    Helmholtz Munich ,

    Max Planck Institute for Biological Cybernetics ,

38. Robin Na

    Massachusetts Institute of Technology

    Affiliation as printed

    Massachusetts Institute of Technology ,

39. Mariia Nadverniuk

    University of Tübingen

    Affiliation as printed

    University of Tübingen ,

40. Antonios Nasioulas

    École Normale Supérieure

    Affiliation as printed

    École normale supérieure , 18

41. Surabhi S Nath

    Max Planck Society · Max Planck Institute for Biological Cybernetics

    Affiliation as printed

    Max Planck Institute for Biological Cybernetics ,

42. Helen Niemeyer

    Freie Universität Berlin

    Affiliation as printed

    Freie Universität Berlin , 25

43. Kate Nussenbaum

    University of Warwick

    Affiliation as printed

    University of Warwick , 26

44. Sebastian Olschewski
    Affiliation as printed

    Institute , 37

45. Thorsten Pachur

    Technical University of Munich

    Affiliation as printed

    Technical University of Munich ,

46. Stefano Palminteri

    École Normale Supérieure

    Affiliation as printed

    École normale supérieure , 18

47. Aliona Petrenco

    Humboldt-Universität zu Berlin · Ludwig-Maximilians-Universität München

    Affiliation as printed

    Humboldt-Universität zu Berlin , 16 LMU Munich

48. Camille V. Phaneuf

    City University of New York

    Affiliation as printed

    City University of New York

49. Angelo Pirrone

    University of Warwick

    Affiliation as printed

    University of Warwick , 26

50. Manuel Rausch

    Catholic University of Eichstätt-Ingolstadt · University of Klagenfurt

    Affiliation as printed

    Alpe-Adria-Universität Klagenfurt ,

    Katholische Universität Eichstätt-Ingolstadt ,

51. Laura Raveling

    Humboldt-Universität zu Berlin · Ludwig-Maximilians-Universität München

    Affiliation as printed

    Humboldt-Universität zu Berlin , 16 LMU Munich

52. Shashank Reddy

    University of Oxford · Singapore Management University

    Affiliation as printed

    Singapore Management University , 32

    University of Oxford ,

53. Milena Rmus

    Helmholtz Association of German Research Centres · Helmholtz Munich

    Affiliation as printed

    Helmholtz Munich ,

54. Evan M. Russek

    Princeton University

    Affiliation as printed

    Princeton University ,

55. Tankred Saanum

    Helmholtz Association of German Research Centres · Max Planck Society · Helmholtz Munich · Max Planck Institute for Biological Cybernetics

    Affiliation as printed

    Helmholtz Munich ,

    Max Planck Institute for Biological Cybernetics ,

56. Kai Sandbrink

    University of Oxford

    Affiliation as printed

    University of Oxford ,

57. Louis Schiekiera

    Humboldt-Universität zu Berlin · Freie Universität Berlin · Ludwig-Maximilians-Universität München

    Affiliation as printed

    Freie Universität Berlin , 25

    Humboldt-Universität zu Berlin , 16 LMU Munich

58. Johannes A. Schubert

    Helmholtz Association of German Research Centres · Helmholtz Munich · University of Oxford

    Affiliation as printed

    Helmholtz Munich ,

    University of Oxford ,

59. Luca M. Schulze Buschoff

    Helmholtz Association of German Research Centres · Helmholtz Munich

    Affiliation as printed

    Helmholtz Munich ,

60. Nishad Singhi
    Affiliation as printed

    Institute , 37

61. Leah H. Somerville
    Affiliation as printed

    Institute , 37

62. Mikhail S. Spektor

    City University of New York

    Affiliation as printed

    City University of New York

63. Xin Sui

    Max Planck Society · Max Planck Institute for Biological Cybernetics · University of Tübingen

    Affiliation as printed

    Max Planck Institute for Biological Cybernetics ,

    University of Tübingen ,

64. Christopher Summerfield

    University of Oxford

    Affiliation as printed

    University of Oxford ,

65. Mirko Thalmann

    Helmholtz Association of German Research Centres · Helmholtz Munich

    Affiliation as printed

    Helmholtz Munich ,

66. Anna Isabel Thoma

    Max Planck Institute for Human Development

    Affiliation as printed

    Max Planck Institute for Human Development ,

67. Taisiia Tikhomirova

    Max Planck Institute for Human Development

    Affiliation as printed

    Max Planck Institute for Human Development ,

68. Vuong Truong

    Taipei Medical University

    Affiliation as printed

    Taipei Medical University ,

69. Polina Tsvilodub

    Helmholtz Association of German Research Centres · Helmholtz Munich · University of Tübingen

    Affiliation as printed

    Helmholtz Munich ,

    University of Tübingen ,

70. Konstantinos Voudouris

    Helmholtz Association of German Research Centres · Helmholtz Munich

    Affiliation as printed

    Helmholtz Munich ,

71. Kristin Witte

    Helmholtz Association of German Research Centres · Helmholtz Munich

    Affiliation as printed

    Helmholtz Munich ,

72. Shuchen Wu

    University of Basel

    Affiliation as printed

    University of Basel ,

73. Hua-Dong Xiong

    Georgia Institute of Technology

    Affiliation as printed

    Georgia Institute of Technology , 36 Allen

74. Songlin Xu

    University College London

    Affiliation as printed

    University College London ,

75. Lance Ying

    Massachusetts Institute of Technology

    Affiliation as printed

    Massachusetts Institute of Technology ,

76. Xinyu Zhang

    University of Basel

    Affiliation as printed

    University of Basel ,

77. Jian-Qiao Zhu
78. Eric Schulz

    Boston University · Helmholtz Association of German Research Centres · Helmholtz Munich

    Affiliation as printed

    Boston University ,

    Helmholtz Munich ,