article · draft
Generative Agent Simulations of 1,000 People
Stanford-led study, 2024, revised 2026: agents built from two-hour interviews with 1,052 Americans predicted their survey answers at 83% of the people's own two-week retest consistency.
What is reusable: open sourcerestricted dataAPI unknownunknown participation MIT (the code repository)Interview transcripts are held in a restricted-access "agent bank"; aggregated survey responses are open. The GitHub repository was last pushed in November 2024.
Most simulated respondents are built from a few demographic facts: age, sex, party, state. This study, led by Joon Sung Park and Michael S. Bernstein at Stanford University with colleagues at Northwestern, the University of Washington, Google DeepMind and Sciences Po, built each agent from much more: a two-hour interview with the real person it stands for.1 A survey firm recruited 1,052 Americans; an AI interviewer, speaking aloud, took each through the American Voices Project interview schedule, and each also answered surveys and took part in experiments.1 GPT-4o then answered new questions in the voice of each person, given their transcript.1 The paper was posted in November 2024 as "Generative Agent Simulations of 1,000 People" and revised in June 2026 under a new title, "LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals".1,2
The yardstick is the person. Each participant retook the same surveys two weeks later and matched their own earlier answers 79.5% of the time on the General Social Survey, a long-running US survey of attitudes.1 Interview agents matched 65.7%, which is 83% of what the person managed. Agents built from survey answers reached 82%, agents with both 86%, and agents given demographics alone 74%.1 (The 2024 version reported 85% for interview agents.)2 For the Big Five personality test the interview agents reached 80% of the people's own consistency.1 In five classic social-science experiments, the agents' effects pointed the same way as the participants' and were highly correlated with them (r from 0.91 to 0.99), but larger.1 In economic games played for real money, no kind of agent did significantly better than the demographics-only agents.1
Agents built from interviews also narrowed, without closing, the accuracy gaps between racial and ideological groups seen with demographic agents.1,2 The study does not show whether a whole set of such agents reproduces how attitudes go together across a population, something earlier work found language models overstate.1 It tested only OpenAI models, not open ones such as Llama or Qwen, and only English speakers. Five experiments are too few to judge predicted effect sizes.1
The point of the paper is that depth of information about a real person matters more than clever persona prompts.1 The cost is that every person simulated must first be interviewed or surveyed, so it does not create people who were never asked. The interview data are hard to anonymise and sit in a restricted "agent bank" for researchers;1,3 the code for building agents and the AI interviewer is on GitHub under an MIT licence.1,3
- url
- https://arxiv.org/abs/2411.10109
- published
- 2024-11-15
- publisher
- arXiv (preprint; revised 2026-06-28)
- authors
- Joon Sung Park, Carolyn Q. Zou, Jonne Kamphorst, Niles Egan, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Percy Liang, Robb Willer, Michael S. Bernstein
refers to this entry as a source
-
Political Analysis paper, 2023, from Brigham Young University: GPT-3, given the backgrounds of real US election survey respondents, reproduced the overall pattern of their vote choice and attitudes.
open data
-
ICML 2023 paper that compared nine language models with 1,498 Pew survey questions answered by 60 US demographic groups and found the models' opinions far from the public's.
open data
-
Political Analysis paper, 2024: ChatGPT personas matched US survey respondents' average feelings toward 11 groups but gave too little spread, wrong regression results and shifting answers.
open data
cited by
-
Models of a population at the level of its people: residents built from the census, run through their days, and given a voice by language models, with what the evidence says about each layer.
Tags: generative agents, interviews, synthetic respondents, General Social Survey, simulation of individuals, language models