arXiv:2505.11584v1 Announce Type: new
Abstract: LLMs are being set loose in complex, real-world environments involving sequential decision-making and tool use. Often, this involves making choices on behalf of human users. However, not much is known about the distribution of such choices, and how susceptible they are to different choice architectures. We perform a case study with a few such LLM models on a multi-attribute tabular decision-making problem, under canonical nudges such as the default option, suggestions, and information highlighting, as well as additional prompting strategies. We show that, despite superficial similarities to human choice distributions, such models differ in subtle but important ways. First, they show much higher susceptibility to the nudges. Second, they diverge in points earned, being affected by factors like the idiosyncrasy of available prizes. Third, they diverge in information acquisition strategies: e.g. incurring substantial cost to reveal too much information, or selecting without revealing any. Moreover, we show that simple prompt strategies like zero-shot chain of thought (CoT) can shift the choice distribution, and few-shot prompting with human data can induce greater alignment. Yet, none of these methods resolve the sensitivity of these models to nudges. Finally, we show how optimal nudges optimized with a human resource-rational model can similarly increase LLM performance for some models. All these findings suggest that behavioral tests are needed before deploying models as agents or assistants acting on behalf of users in complex environments.
Expert Commentary: Understanding the Impact of Nudges on Language Model Models
Language models have become increasingly prevalent in real-world applications, including sequential decision-making and tool use. However, as highlighted in this study, there is a significant gap in our understanding of how these models make choices on behalf of human users and how susceptible they are to different choice architectures.
This case study delves into the nuanced differences observed when applying canonical nudges such as default options, suggestions, and information highlighting to language model models (LLMs) in a multi-attribute tabular decision-making problem. One key takeaway is the heightened susceptibility of LLMs to nudges compared to human decision-makers. This highlights the importance of considering the unique characteristics of these models when designing decision support systems.
Furthermore, the study reveals that LLMs exhibit divergent behaviors in points earned and information acquisition strategies, shedding light on their decision-making processes. The introduction of prompt strategies like zero-shot chain of thought (CoT) and few-shot prompting with human data demonstrates the potential for aligning LLM choices more closely with human preferences.
Despite these advancements, the study emphasizes the ongoing sensitivity of LLMs to nudges, even with optimal nudges designed using human resource-rational models. This underscores the need for thorough behavioral testing before deploying LLMs as agents or assistants in complex environments, where their decisions can impact human users.
The multi-disciplinary nature of this study, combining insights from behavioral science, artificial intelligence, and decision theory, underscores the complex interplay of factors influencing LLM choices. By further exploring the underlying mechanisms driving LLM decision-making, researchers can enhance the transparency, performance, and ethical considerations surrounding the use of LLMs in real-world applications.