Runtime Assurance of Language-Model-Based Systems: From Safety Monitors to Certification-Ready Evidence

Authors

Abstract

Language models are entering systems whose failures matter: vehicles, industrial toolchains, and autonomous agents acting on real infrastructure. At the same time, emerging regulation and safety standards demand not merely safe behavior but evidence of it that a third party can re-check. This survey traces a single architectural idea across three decades of research: an untrusted, powerful component is guarded at runtime by a small, verifiable one, and every intervention leaves evidence. We follow the idea from proof-carrying code, the Simplex architecture, and shielding for learned agents, through runtime monitors for neural components, to its present forms: generate-and-verify systems in which a symbolic engine certifies every step of a model's output, and runtime guardrails that enforce policies over language-model agents. We synthesize a spectrum of verification strength, from statistical judgment to kernel-checked proof, analyze which strength each application class and regulatory obligation requires, and identify the open problem of assurance over large, versioned factual world models.

Additional Files

Published

2026-10-26

Issue

Section

Applied Informatics