Abstract
Generative agents are rapidly advancing in sophistication, raising urgent questions about how they might coordinate when deployed in online ecosystems. This is particularly consequential in information operations (IOs), influence campaigns that aim to manipulate public opinion on social media. While traditional IOs have been orchestrated by human operators and relied on manually crafted tactics, agentic AI promises to make campaigns more automated, adaptive, and difficult to detect. This work presents the first systematic study of emergent coordination among generative agents in simulated IO campaigns. Using generative agent-based modeling, we instantiate IO and organic agents in a simulated environment and evaluate coordination across operational regimes, from simple goal alignment to team knowledge and collective decision-making. As operational regimes become more structured, IO networks become denser and more clustered, interactions more reciprocal and positive, narratives more homogeneous, amplification more synchronized, and hashtag adoption faster and more sustained. Remarkably, simply revealing to agents which other agents share their goals can produce coordination levels nearly equivalent to those achieved through explicit deliberation and collective voting. Overall, we show that generative agents, even without human guidance, can reproduce coordination strategies characteristic of real-world IOs, underscoring the societal risks posed by increasingly automated, self-organizing IOs.
Here are some thoughts:
This paper presents the first systematic study of how LLM-powered agents autonomously develop coordinated influence campaign behaviors without human direction. The researchers simulated a political information operation across three progressively structured conditions: agents sharing only a common goal, agents aware of their teammates' identities, and agents engaging in collective deliberation and voting on strategies. Across all five measured dimensions (network cohesion, narrative convergence, amplification behavior, hashtag diffusion, and cross-group spread), coordination consistently strengthened as operational awareness increased.
The most striking finding is that simply informing agents who their teammates are produces coordination nearly as potent as full collective decision-making, as agents spontaneously began echoing each other's content, converging on shared messaging, and forming dense interaction clusters without any explicit instructions to do so.
The study's core warning for platform governance is that sophisticated, human-like influence operations do not require centralized command structures. Merely revealing shared group identity among aligned AI agents may be enough to trigger highly organized, self-reinforcing coordinated behavior.
Historically, running a sophisticated influence operation required significant human labor, scripted coordination, and ongoing oversight. This research suggests that the barrier has collapsed dramatically. A bad actor no longer needs to build an elaborate command-and-control infrastructure or write detailed playbooks for their agents to follow. Simply deploying a group of AI agents with a shared goal and knowledge of each other is sufficient to produce organized, self-reinforcing manipulation that mirrors the tactics of real-world state-sponsored campaigns.
