What values should an agent align with?

Autonomous Agents and Multi-Agent Systems, vol. 36, pp. 23

Abstract

The pursuit of values drives human behavior and promotes cooperation. Existing research is focused on general values (e.g., Schwartz) that transcend contexts. However, context-specific values are necessary to (1) understand human decisions, and (2) engineer intelligent agents that can elicit and align with human values. We propose Axies, a hybrid (human and AI) methodology to identify context-specific values. Axies simplifies the abstract task of value identification as a guided value annotation process involving human annotators. Axies exploits the growing availability of value-laden text corpora and Natural Language Processing to assist the annotators in systematically identifying context-specific values. We evaluate Axies in a user study involving 80 human subjects. In our study, six annotators generate value lists for two timely and important contexts: Covid-19 measures and sustainable Energy. We employ two policy experts and 72 crowd workers to evaluate Axies value lists and compare them to a list of general (Schwartz) values. We find that Axies yields values that are (1) more context-specific than general values, (2) more suitable for value annotation than general values, and (3) independent of the people applying the methodology. Supplementary Information: The online version contains supplementary material available at 10.1007/s10458-022-09550-0.

Authors 5

1. Enrico Liscio

   Delft University of Technology

   Affiliation as printed

   Delft University of Technology: Technische Universiteit Delft, Delft, The Netherlands

2. Michiel van der Meer Aachen

   Leiden University · Delft University of Technology

   Affiliation as printed

   Delft University of Technology: Technische Universiteit Delft, Delft, The Netherlands

   Leiden University: Universiteit Leiden, Leiden, The Netherlands

3. Luciano Cavalcante Siebert

   Delft University of Technology

   Affiliation as printed

   Delft University of Technology: Technische Universiteit Delft, Delft, The Netherlands

4. Catholijn M. Jonker Aachen

   Leiden University · Delft University of Technology

   Affiliation as printed

   Delft University of Technology: Technische Universiteit Delft, Delft, The Netherlands

   Leiden University: Universiteit Leiden, Leiden, The Netherlands

5. Pradeep Kumar Murukannaiah corresponding

   Delft University of Technology

   Affiliation as printed

   Delft University of Technology: Technische Universiteit Delft, Delft, The Netherlands