Social Simulacra: populated prototypes for social computing systems

Many of the problems of an online community, such as trolling or a slide off topic, only show up once it has a lot of members, and a designer testing a new one can usually recruit only a handful of people.¹ This paper asks whether the members can be simulated instead. A designer describes the community's goal, its rules and a few example members, and the system, called a social simulacrum, writes thousands of distinct members and the posts and replies they would exchange, including the anti-social ones.¹

The authors built it as SimReddit, a tool for designing a new subreddit, on GPT-3.¹ Their premise is that a large language model's training data already holds a wide range of good and bad behaviour from social media, so it can be prompted to produce both.¹ The designer can then ask "what if?": change a rule, have a moderator or a member step in, and watch how the generated conversation shifts.¹

They tested it two ways. They took 50 subreddits created after GPT-3's release, regenerated each from its description alone, and showed people a real and a generated conversation from each. People picked the wrong one 41 percent of the time on average, close to chance.¹ In a second study, 16 designers of social computing systems used SimReddit to design a subreddit they wanted to create, and refined their designs after seeing what the simulated members did.¹

The paper was presented at UIST in October 2022.² It came a few months before the same group's generative agents paper, which moved the simulated people from a message board into a small town with a daily life, and later work by Joon Sung Park, Percy Liang and Michael S. Bernstein built agents of 1,000 real people from interviews.¹ Park and Bernstein went on to found Simile, which sells simulations of how people would respond to a change.