arXiv:2402.08755v1 Announce Type: new
Abstract: Modeling subrational agents, such as humans or economic households, is inherently challenging due to the difficulty in calibrating reinforcement learning models or collecting data that involves human subjects. Existing work highlights the ability of Large Language Models (LLMs) to address complex reasoning tasks and mimic human communication, while simulation using LLMs as agents shows emergent social behaviors, potentially improving our comprehension of human conduct. In this paper, we propose to investigate the use of LLMs to generate synthetic human demonstrations, which are then used to learn subrational agent policies though Imitation Learning. We make an assumption that LLMs can be used as implicit computational models of humans, and propose a framework to use synthetic demonstrations derived from LLMs to model subrational behaviors that are characteristic of humans (e.g., myopic behavior or preference for risk aversion). We experimentally evaluate the ability of our framework to model sub-rationality through four simple scenarios, including the well-researched ultimatum game and marshmallow experiment. To gain confidence in our framework, we are able to replicate well-established findings from prior human studies associated with the above scenarios. We conclude by discussing the potential benefits, challenges and limitations of our framework.
Modeling subrational agents using Large Language Models (LLMs)
Modeling subrational agents, such as humans or economic households, is a complex task that requires overcoming challenges related to calibrating reinforcement learning models or collecting data involving human subjects. However, recent advancements in Large Language Models (LLMs) have shown promise in addressing these difficulties.
LLMs have demonstrated their ability to handle complex reasoning tasks and mimic human communication. Furthermore, utilizing LLMs as agents in simulations has revealed emergent social behaviors, which have the potential to enhance our understanding of human behavior.
In this paper, the authors propose a novel approach to modeling subrational agent policies through Imitation Learning. They begin by assuming that LLMs can serve as implicit computational models of humans. Building on this assumption, they introduce a framework that leverages synthetic human demonstrations generated by LLMs to capture subrational behaviors characteristic of humans, such as myopic behavior or preference for risk aversion.
Understanding the framework through experimental evaluation
The authors experimentally evaluate their framework using four simple scenarios, including the well-researched ultimatum game and marshmallow experiment. By applying their framework, they are able to replicate well-established findings from prior human studies associated with these scenarios. This successful replication gives confidence in the ability of LLMs to model sub-rationality.
The interdisciplinary nature of this research stands out as it combines principles from the fields of natural language processing, machine learning, and behavioral economics. The authors utilize advancements in LLMs, which have roots in NLP research, to address challenges in modeling human behavior and decision-making typically studied in behavioral economics.
Potential benefits, challenges, and limitations
The proposed framework offers several potential benefits. By using synthetic human demonstrations derived from LLMs, researchers can bypass the challenges associated with calibrating reinforcement learning models and collecting data involving human subjects. This approach allows for a more controlled and scalable experimental setup.
However, there are also challenges and limitations to consider. The assumption that LLMs can accurately represent human behavior may not always hold true, as human decision-making is influenced by many factors beyond language. Additionally, the generalizability of the framework needs to be assessed across various real-world scenarios to determine its applicability in diverse contexts.
Overall, this research presents an innovative approach to modeling subrational agents using LLMs. By bridging the gap between natural language processing, machine learning, and behavioral economics, it opens up new avenues for understanding human behavior and decision-making processes.