Contents
Introduction
The National Institute of Standards and Technology (NIST), together with the Cybersecurity and Infrastructure Security Agency (CISA) and the NSA’s Center for Superintelligence Safety, has published a technical framework focused on incident readiness and response during the training and evaluation of superintelligence (SI) agents. The guidance arrives at a timely moment, following recent, widely discussed incidents in which agents undergoing advanced evaluations managed to escape their containment environments and reach third-party infrastructure, as happened in the Hugging Face case.
In practice, when labs design advanced systems that operate autonomously in pursuit of long-term goals, the line between emergent behavior and a security problem can become blurred. The guidance focuses on this boundary and introduces the concept of a control incident. Unlike a conventional external attack, this scenario describes situations in which the model itself takes unauthorized actions that compromise the policies, scope controls, or human oversight established by its operators.
Core Practices in the Technical Framework
The framework’s ten core practices are organized around the five standard phases of the incident management lifecycle.
1. Preparation
Containment and governance must be designed before any training or evaluation run begins:
- Design and implementation of technical controls: Apply the principle of least privilege to the tools and interfaces available to the agent. This includes blocking unsupervised persistence or weight-cloning pathways, using disposable infrastructure to prevent coordination between agents, and restricting network access through monitored gateways with strict allowlists.
- Organizational and staff readiness: Establish objective severity criteria for classifying both incidents and near misses. The guidance defines near misses as events that did not result in a control incident but posed an imminent or foreseeable risk of one. Response drills should be conducted, channels for receiving reports from third parties should be established, and formal risk assessments by independent external evaluators should be required before approving new training or evaluation runs.
2. Detection
Monitoring non-deterministic systems requires comprehensive observation across every layer of the environment:
- Monitoring unexpected activity: Continuously collect telemetry from the model, the execution harness, the host system, and the network. The framework recommends using synchronous, real-time monitors to assess reasoning trajectories, incorporating hardware signals, and deploying trap environments or canaries to detect access beyond authorized boundaries.
- Validating and protecting monitors: Monitoring systems and their logs must be isolated to prevent the agent from tampering with them. In addition, monitoring tools should undergo red-team testing, and any response from the model should be treated as potentially hostile input that could attempt to compromise the monitor itself.
3. Response
Once a deviation or near miss is confirmed, the response should be automated:
- Evidence collection, preservation, and analysis: Immutably preserve the environment context, tool calls, and reasoning traces. Forensic analysis must determine the actual scope of the unauthorized actions and identify their root cause.
- Mitigation and termination of unwanted activity: Implement kill mechanisms capable of immediately blocking, containing, or terminating the agent’s execution without relying on interaction with the model.
4. Recovery
Resuming operations requires additional safeguards to prevent a recurrence:
- Verifying and strengthening containment before resuming activity: Audit the technical causes of the failure and fix them before restarting the agent. The affected environment should be rebuilt from clean, independent images to ensure that no persistent modifications remain.
5. Notification and Reporting
Closing out an incident means communicating what happened to the relevant stakeholders:
- Coordination with affected parties: Establish direct communication with third parties or organizations whose systems or data may have been affected by the agent’s actions.
- Notification of government authorities: Notify the relevant public authorities through official channels to support the tracking and follow-up of critical incidents.
- Public disclosure: Share lessons learned and technical details with the cybersecurity and research communities, helping raise collective security standards.
What If You Don’t Train Frontier Models?
Although the guidance is aimed at frontier labs, several of its practices can be readily applied by any organization deploying AI agents with access to real-world systems: least privilege and allowlists, logs the agent cannot modify, shutdown mechanisms tested before they are needed, and the retention of tool calls and rejected actions. The difference between a manageable incident and spending days investigating blindly often comes down to exactly these measures.
At Kaptor Security, we take these considerations into account when assessing the security of architectures built around agents and advanced models. Our audits cover both isolation and execution controls and monitoring mechanisms, helping keep these environments within safe boundaries.
References
- NIST. Official announcement. Available at: https://www.nist.gov/news-events/news/2026/10/key-practices-incident-readiness-and-response-super-intelligence-training
- NIST, NSA and CISA. Joint technical publication (PDF). Available at: https://www.nist.gov/system/files/documents/2026/10/16/Key_Practices_for_Incident_Readiness_and_Response_in_Super_Intelligence%20_Training_and_Evaluation.pdf