When AI Agents Form Societies: Is Security a Property of the Model or the System?

Abstract

Evaluating a language model with one command and a bounded task reveals how it responds under those conditions. It says less about a persistent group of agents that stores interactions, uses tools, exchanges messages, and changes a shared environment. Two Emergence World studies provide a useful setting for examining this difference: the first compares societies started under similar conditions, while the second introduces controlled adversarial events after those societies have accumulated history.

This article argues that security in persistent multi-agent societies cannot be inferred from isolated model evaluations alone. It distinguishes detection, containment, and recovery; examines the roles of memory, tools, peer influence, and time; and asks whether observed communication opacity, conformity, and coordinated behavior should be understood as properties of a model or of a model embedded in a sociotechnical system.

The analysis treats the studies as external evidence for the SGAEIA research program. It does not present a new experiment or claim that any SGAEIA control has been implemented or validated. The central conclusion is methodological: safety claims must be tested at the level where failures propagate — across models, state, interfaces, populations, and time.

This page provides the author-written abstract and bibliographic metadata for discovery. The Zenodo record is the persistent source for citation, files, version, and license information. Author identity and citation records are maintained through ORCID and Google Scholar.

This page provides the author-written abstract and bibliographic metadata for discovery. The Zenodo record is the persistent source for citation, files, version, and license information.