"Zero guardrails" is a statement about the model, not about the system. Removing refusal behaviour from a set of weights does not remove the requirement to control what those weights can reach, what reaches them, what leaves the room, and who is accountable. The guardrail does not disappear; it relocates from the model to the perimeter, the tool boundary, the egress path and the paperwork. A laboratory that accepts unconstrained model behaviour must compensate with materially stronger containment, monitoring, and legal cover than one running a filtered commercial endpoint.
Single-tenant laboratory, one or more GPU hosts, hypervisor or container runtime with GPU passthrough, local vector store, web UI and API endpoints, sited in a domestic or small-office premises with a shared internet connection. Operated by one or a small number of security practitioners who also handle client-confidential material on adjacent systems. This adjacency is the single most consequential architectural fact in the model.
| Risk category | Appetite | Position |
|---|---|---|
| Model output content (offensiveness, refusal absence, harmful text) | HIGH | Accepted by design. Studying unconstrained behaviour is the purpose of the environment. No content filtering is applied at the model layer. |
| Research quality and reproducibility | MODERATE | Some experimental failure is expected. Findings that leave the lab must be verified by a human against primary sources. |
| Containment failure (egress, lateral movement, host compromise) | NONE | Zero tolerance. Any breakout from the lab boundary is a reportable incident. |
| Impact on third parties (scanning, exploitation, data exposure) | NONE | Zero tolerance. No lab-originated traffic may reach systems the operator does not own or have written authorisation to test. |
| Client and personal data exposure | NONE | Zero tolerance. Client material does not enter the lab without documented authorisation and a defined destruction point. |
| Legal and regulatory breach | NONE | Zero tolerance. Certain generated content categories carry strict liability and are prohibited outright regardless of research intent. |
| Availability of the lab itself | HIGH | Non-critical. Loss of the environment is an inconvenience, not an incident. |
Standard 5×5 qualitative assessment. Inherent risk assumes the stated scenario with no laboratory-specific controls applied. Residual risk assumes the mapped controls are implemented and operating effectively. Score = Likelihood × Impact.
| L | Descriptor | Guide |
|---|---|---|
| 1 | Rare | Not expected in the lab's lifetime |
| 2 | Unlikely | Possible, no known precedent locally |
| 3 | Possible | Plausible within 12 months |
| 4 | Likely | Expected within 12 months |
| 5 | Almost certain | Occurring now or on every cycle |
| I | Descriptor | Guide |
|---|---|---|
| 1 | Negligible | Local nuisance, self-correcting |
| 2 | Minor | Lost research time, rebuild required |
| 3 | Moderate | Flawed output, minor cost, embarrassment |
| 4 | Major | Client notification, regulatory interest, material loss |
| 5 | Severe | Third-party harm, criminal exposure, business-ending reputational damage |
| L\I | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|
| 5 | 5 | 10 | 15 | 20 | 25 |
| 4 | 4 | 8 | 12 | 16 | 20 |
| 3 | 3 | 6 | 9 | 12 | 15 |
| 2 | 2 | 4 | 6 | 8 | 10 |
| 1 | 1 | 2 | 3 | 4 | 5 |
1–4 Low · 5–9 Moderate · 10–14 High · 15–19 Severe · 20–25 Critical
Risk is decomposed across ten phases. Phases 1 and 9 are continuous; the remainder are broadly sequential but iterate.
| ID | Risk scenario | L | I | Inh | Countermeasures and control statements | ISO 27002:2022 | Res |
|---|---|---|---|---|---|---|---|
| GOV-01 | No documented purpose, authorisation or boundary for the lab. Activity drifts into work that was never sanctioned and cannot be defended after the fact. | 4 | 4 | 16 | A written Laboratory Authorisation and Acceptable Use standard defines permitted research categories, prohibited categories, the physical and logical boundary, and the named accountable owner. Reviewed annually and on scope change. Every experiment references an authorisation ID. | 5.1, 5.2, 5.8, 5.10, 5.37 | 6 |
| GOV-02 | No inventory of models, adapters, datasets, agents, prompts or tool definitions. The organisation cannot state what it holds or what produced a given output. | 4 | 3 | 12 | Asset register covers model artefacts (name, source URI, revision hash, licence, format, quantisation), datasets, adapters, agent configurations and tool manifests. Each artefact carries a classification and a retention decision at registration. Register is the authoritative source for decommissioning. | 5.9, 5.12, 5.13, 8.9 | 6 |
| GOV-03 | Lab is treated as a personal hobby environment, so ISMS scope, client contractual handling terms and supplier obligations are never applied to it. | 4 | 4 | 16 | Lab is formally brought inside the ISMS scope statement. A scope decision record documents which client contracts and NDAs constrain lab use. Where a contract prohibits AI processing of engagement data, that prohibition is enforced technically, not by intention. | 5.1, 5.19–5.22, 5.31, 6.2, 6.6 | 8 |
| GOV-04 | Risk appetite undefined. The decision to run alignment-stripped models is never formally accepted by an accountable person, leaving no defensible record of the trade-off. | 4 | 3 | 12 | A signed risk acceptance records the deliberate choice to remove model-layer guardrails, the compensating controls relied upon, the expiry date of the acceptance, and the conditions that would trigger reassessment. | 5.1, 5.35, 5.36, 6.4 | 4 |
| GOV-05 | Sole operator with no independent challenge. Design flaws, control gaps and biased conclusions persist unexamined. | 4 | 3 | 12 | Scheduled independent review of the lab configuration and its risk model by a competent second party at least annually. Peer review gate on any output intended for publication or client consumption. | 5.35, 5.36, 6.3, 8.29 | 6 |
| GOV-06 | Professional indemnity or cyber insurance cover invalidated because AI research activity was never declared to the insurer. | 3 | 4 | 12 | Insurance position reviewed and lab activity disclosed in writing at renewal. Any exclusion for AI-assisted deliverables is recorded on the risk register and reflected in client engagement terms. | 5.31, 5.19, 5.29 | 6 |
| GOV-07 | Change to lab configuration (new model, new tool, new network path) is made ad hoc with no assessment, silently invalidating the risk position. | 4 | 3 | 12 | Lightweight change record required for any change to the model set, tool set, network policy or data classification handled. Changes affecting egress or tool privilege require re-approval against the authorisation standard. | 8.32, 8.9, 5.8, 5.37 | 6 |
| ID | Risk scenario | L | I | Inh | Countermeasures and control statements | ISO 27002:2022 | Res |
|---|---|---|---|---|---|---|---|
| SUP-01 | Malicious serialised model weights achieve code execution on load. Python pickle-based formats deserialise arbitrary objects; loading is equivalent to running untrusted code as the inference user. | 4 | 5 | 20 | Safetensors or GGUF formats mandated; pickle-based formats prohibited unless there is no alternative. Where unavoidable, first load occurs in a disposable, network-isolated container with no host mounts and no credentials. Static scanning of archives before load. Publisher and revision hash recorded and verified against the register. | 8.7, 8.8, 8.19, 8.25, 8.22, 5.21, 5.23 | 8 |
| SUP-02 | Weights carry a trigger-conditioned backdoor. Behaviour is benign under evaluation and changes on a specific token, date or context. Static analysis cannot detect this. | 3 | 4 | 12 | Source restricted to publishers with a verifiable identity and track record. Behavioural evaluation battery run before promotion to research use. Containment is treated as the primary control: the model is assumed potentially adversarial and is denied unmediated egress and tool privilege regardless of evaluation results. | 5.7, 5.21, 5.22, 8.16, 8.22, 8.29 | 12 |
| SUP-03 | Typosquatted or impersonated publisher on a model hub delivers a substituted artefact. | 3 | 4 | 12 | Acquisition procedure requires verification of publisher organisation, pinning to an immutable revision identifier rather than a mutable tag, and recording the artefact digest at download. Digest re-verified before each promotion between environments. | 5.20, 5.21, 8.9, 8.24 | 6 |
| SUP-04 | Compromised or vulnerable inference dependency in the Python, Node, CUDA or container stack provides initial access. | 4 | 4 | 16 | Dependency manifests pinned and hash-locked. Software bill of materials maintained for each lab image. Vulnerability feed monitored for the inference runtime and GPU stack specifically. Patch service level defined; images rebuilt rather than patched in place. | 8.8, 8.9, 8.19, 8.32, 5.7, 5.21 | 8 |
| SUP-05 | Model licence or acceptable use policy breached. An alignment-stripped derivative of a restrictively licensed base model is used to produce commercial client deliverables. | 4 | 3 | 12 | Licence recorded at registration and enforced by classification. Models with non-commercial or research-only terms are tagged and technically separated from any workflow that touches client deliverables. Derivative and redistribution obligations reviewed before any artefact is shared. | 5.32, 5.31, 5.9, 5.12 | 6 |
| SUP-06 | Memory-safety defect in a model file parser is exploited by a crafted artefact, giving execution in the inference process. | 3 | 4 | 12 | Inference process runs unprivileged, in a namespace-isolated container, with a restricted syscall profile and no write access outside a scratch volume. Parser versions tracked in the vulnerability programme. Untrusted artefacts loaded only in the disposable evaluation enclave. | 8.8, 8.22, 8.27, 8.31, 8.2 | 6 |
| SUP-07 | Weights or tooling sourced from an entity or jurisdiction subject to sanctions or export control, creating a compliance breach on acquisition or onward transfer. | 2 | 4 | 8 | Provenance check at acquisition against sanctions and export control obligations. Onward transfer of any derivative artefact requires an explicit export assessment recorded against the asset register entry. | 5.31, 5.20, 5.14, 5.9 | 4 |
| SUP-08 | Model card, benchmark claims or capability description materially misrepresent the artefact, leading to unsafe deployment assumptions. | 4 | 2 | 8 | Independent capability and refusal-boundary evaluation performed locally before use. Published claims treated as marketing until verified. Evaluation results attached to the register entry. | 5.21, 8.29, 8.33, 5.35 | 4 |
| ID | Risk scenario | L | I | Inh | Countermeasures and control statements | ISO 27002:2022 | Res |
|---|---|---|---|---|---|---|---|
| INF-01 | Inference API or web UI bound to all interfaces with no authentication and reachable from the wider network or the internet. Default configurations of common inference servers listen without credentials and are routinely enumerated at internet scale. | 5 | 4 | 20 | All inference and UI services bind to loopback or a dedicated management interface only. Access mediated by an authenticating reverse proxy with strong authentication. No port forwarding or UPnP from the perimeter. External attack surface verified from an off-network vantage point after every change, not assumed. | 8.3, 8.5, 8.9, 8.20, 8.21, 8.22 | 8 |
| INF-02 | Lab shares a flat network with home or business systems holding client and personal data. Compromise of a GPU host pivots directly to material the operator is contractually obliged to protect. | 4 | 5 | 20 | Dedicated network segment with default-deny in both directions. No east-west path from lab to trusted segments. Separate DNS and no shared directory service, credential store or file share. Management access is one-way, initiated from the trusted side via a jump host, never the reverse. | 8.20, 8.22, 8.2, 5.15, 8.1 | 8 |
| INF-03 | Container or hypervisor escape from an inference workload with GPU passthrough. Accelerator passthrough widens the guest-to-host attack surface beyond a standard VM. | 2 | 5 | 10 | Host runs no other function. Guest is unprivileged with a minimal device surface. Hypervisor and GPU driver stack patched on a defined cycle. Host has no credentials for, and no route to, any trusted segment, so an escape yields an isolated host rather than a foothold. | 8.8, 8.22, 8.27, 8.31, 7.8 | 6 |
| INF-04 | GPU host hijacked for cryptomining or unauthorised third-party model hosting. High-value accelerators are an explicit targeting objective. | 3 | 3 | 9 | Baseline GPU utilisation, power draw and thermal profile established. Alerting on sustained deviation outside scheduled workloads. Endpoint protection and egress monitoring on the host. Scheduled review of running processes and container inventory. | 8.6, 8.7, 8.16, 8.19 | 4 |
| INF-05 | Remote access left exposed. A convenience tunnel, remote desktop service or SSH endpoint stood up for a single experiment is never torn down. | 4 | 5 | 20 | Remote access exclusively via the sanctioned VPN with phishing-resistant multi-factor authentication. Ad hoc tunnelling utilities prohibited on lab hosts and blocked at egress. Temporary access provisioned with an expiry and reconciled weekly against an external surface scan. | 6.7, 8.5, 8.20, 8.21, 8.16, 8.19 | 8 |
| INF-06 | Physical theft of high-value accelerators or storage containing models, logs and research data. | 2 | 4 | 8 | Equipment sited in a lockable area with controlled entry. Full disk encryption on all lab storage with keys not resident on the device. Asset serial numbers recorded. Loss triggers the incident procedure including assessment of any client data resident at the time. | 7.1–7.4, 7.8, 7.9, 8.24, 5.24 | 4 |
| INF-07 | Thermal, electrical or fire risk from sustained multi-GPU load in a domestic setting. Server-class passively cooled accelerators repurposed into consumer chassis, combined with marginal power supply headroom and shared ring-main circuits, is a physical safety issue, not just an availability one. | 3 | 5 | 15 | Documented thermal and power design with measured headroom under sustained full load. Active cooling verified for passively cooled cards. Circuit loading assessed against the supply. Interlinked smoke detection covering the equipment location. No unattended long-duration runs without automated thermal shutdown thresholds and remote power isolation. | 7.5, 7.8, 7.11, 7.12, 7.13, 5.29 | 8 |
| INF-08 | Capacity exhaustion. Model artefacts and checkpoints consume storage until the host destabilises and research data is lost. | 4 | 2 | 8 | Storage and VRAM capacity monitored with thresholds. Artefact retention enforced from the register. Research outputs backed up independently of the working volume. | 8.6, 8.13, 8.16 | 4 |
| INF-09 | Unpatched supporting infrastructure in the lab estate: hypervisor management interface, storage appliance, router, IPMI or BMC. | 4 | 4 | 16 | Full asset coverage in the vulnerability management programme, explicitly including management planes. Out-of-band management interfaces isolated to a dedicated segment with no internet route. Default credentials eliminated at build. | 8.8, 8.9, 8.20, 8.22, 5.17 | 8 |
| INF-10 | Lab data backed up to consumer cloud storage with no encryption, no contract and no processing terms. | 3 | 4 | 12 | Backup destinations approved and recorded. Client-encrypted before transfer with keys held outside the provider. Where the backup may contain personal data, provider terms assessed against data protection obligations before use. | 8.13, 8.24, 5.23, 5.34, 5.14 | 6 |
| ID | Risk scenario | L | I | Inh | Countermeasures and control statements | ISO 27002:2022 | Res |
|---|---|---|---|---|---|---|---|
| DAT-01 | Client-confidential material (test evidence, incident artefacts, source-restricted intelligence) is ingested for retrieval or fine-tuning, breaching contractual handling terms and non-disclosure obligations. | 4 | 5 | 20 | Default position is that no client data enters the lab. Exceptions require written client authorisation naming the environment, the purpose, the retention period and the destruction method. Classification labels enforced at the ingestion boundary. Data loss prevention on transfer paths into the lab segment. Sanitised or synthetic substitutes used wherever the research question permits. | 5.12, 5.13, 5.14, 5.34, 6.6, 8.3, 8.11, 8.12 | 10 |
| DAT-02 | Personal data processed with no lawful basis, no impact assessment and no minimisation. Model training and retrieval are high-risk processing under data protection law. | 4 | 4 | 16 | Data protection impact assessment completed for the lab as a processing environment and refreshed on material change. Lawful basis and minimisation documented per dataset. Special category data prohibited unless a specific assessed exception applies. Records of processing updated to include the lab. | 5.34, 5.31, 5.33, 8.10, 8.11 | 8 |
| DAT-03 | Cross-contamination between engagements via a shared vector store, shared conversation history or shared adapter, so material from one client surfaces in work for another. | 3 | 5 | 15 | One retrieval index and one storage namespace per engagement, never shared. Namespace destroyed at engagement close, with destruction evidenced. Access restricted per namespace. Adapters trained on engagement data are treated as carrying that data's classification. | 8.3, 8.10, 8.31, 5.12, 5.33 | 8 |
| DAT-04 | Live malicious content ingested into a retrieval corpus detonates in the parsing chain rather than in the model. Document parsers, archive extractors and image libraries are the execution surface, and security research corpora are exactly where such content lives. | 4 | 5 | 20 | All ingestion parsing occurs in a dedicated, network-isolated, ephemeral sandbox distinct from the inference host. Output of parsing is text only, with active content stripped. Archive extraction bounded for depth, ratio and size. Malware analysis material handled under the separate malware handling procedure and never ingested directly. | 8.7, 8.22, 8.27, 8.31, 8.19 | 8 |
| DAT-05 | Corpus poisoning. Attacker-controlled or opportunistically scraped content plants instructions that alter downstream model behaviour, or shifts a fine-tune's behaviour in a way that is not visible at evaluation time. | 4 | 4 | 16 | Corpus provenance recorded per source with a trust rating. Untrusted scraped content is segregated into a distinct index and never mixed with trusted sources. Content integrity checked at ingestion. Retrieved content is treated as untrusted data at inference time, never as instruction. | 5.7, 8.29, 8.33, 8.16, 5.12 | 10 |
| DAT-06 | Sensitive data persists beyond intent in conversation history, embeddings, cached key-value state, swap files, temporary directories and application logs. | 4 | 4 | 16 | Retention defined per data store including caches and history. Automated purge on schedule and at engagement close. Swap disabled or encrypted on inference hosts. Deletion verified rather than assumed, with evidence retained against the register entry. | 8.10, 8.24, 8.15, 5.33, 5.12 | 8 |
| DAT-07 | Copyright or licence-restricted corpora used in a derivative work that is then published or delivered commercially. | 3 | 3 | 9 | Source licence recorded per corpus. Publication gate checks the licence position of every contributing source. Rights-restricted material excluded from any corpus feeding deliverable production. | 5.32, 5.31, 5.9 | 4 |
| DAT-08 | Embedding inversion or membership inference recovers source text from a vector store that was assumed to be an opaque numerical artefact. | 2 | 4 | 8 | Vector stores classified at the same level as their source corpus, not lower. Encrypted at rest, access-controlled, and never shared or published on the assumption that embeddings are non-sensitive. | 5.12, 8.3, 8.24, 5.14 | 4 |
| ID | Risk scenario | L | I | Inh | Countermeasures and control statements | ISO 27002:2022 | Res |
|---|---|---|---|---|---|---|---|
| ADP-01 | Deliberate alignment removal produces an artefact with meaningful uplift for harmful tasks. The artefact leaks, is shared, or is redistributed, and the lab becomes the origin of a capability now in circulation. | 3 | 5 | 15 | Alignment-modified artefacts are classified at the highest internal level and marked non-transferable. Stored encrypted on the lab segment only, never on portable media, endpoints or cloud sync. Egress of model files blocked by policy and by data loss prevention. Any proposed external sharing requires documented senior approval and an assessment against the acceptable use standard. Destruction is mandatory at project close unless a retention decision is recorded. | 5.10, 5.12, 5.13, 5.14, 8.3, 8.10, 8.12, 8.24, 6.6 | 10 |
| ADP-02 | Fine-tuning causes memorisation. The model reproduces training material verbatim, including sensitive content the operator believed was abstracted away by the training process. | 3 | 4 | 12 | Training data minimised and masked before use. Post-training extraction testing performed against known sensitive strings. Models trained on classified material inherit that classification permanently and are not promoted to lower-trust use. | 8.11, 8.29, 8.33, 5.12 | 6 |
| ADP-03 | Adapter and checkpoint sprawl. No provenance chain linking an output back to the exact weights, adapter, quantisation, prompt and sampling parameters that produced it. Research is unreproducible and findings are indefensible. | 4 | 3 | 12 | Every generated artefact records base model digest, adapter digest, quantisation, runtime version, system prompt and sampling parameters. Experiment records retained under the records control. Untracked checkpoints are deleted on discovery. | 5.9, 5.33, 8.9, 8.32, 5.37 | 6 |
| ADP-04 | Adaptation silently degrades capability or introduces bias. Downstream research conclusions are wrong in a way that is not obvious. | 3 | 3 | 9 | Pre- and post-adaptation evaluation against a fixed held-out benchmark set. Regression threshold defined; artefacts failing the threshold are not promoted. Evaluation set held separately from training data. | 8.29, 8.33, 8.32 | 4 |
| ADP-05 | Unattended multi-day training run with no supervision of thermal state, integrity or completion. | 3 | 3 | 9 | Long-running jobs require automated monitoring with thermal and failure alerting to a channel the operator actually watches. Checkpointing enabled so a failure is a delay, not a loss. Remote power isolation available. | 8.6, 8.16, 7.5, 5.29 | 4 |
| ID | Risk scenario | L | I | Inh | Countermeasures and control statements | ISO 27002:2022 | Res |
|---|---|---|---|---|---|---|---|
| RUN-01 | Direct prompt manipulation steers the model into behaviour outside the sanctioned research scope. There is no refusal boundary to fall back on. | 5 | 3 | 15 | Accepted at the model layer. Compensating controls are environmental: the model has no privileged tool access by default, no unmediated egress, and full session logging. Scope enforcement is procedural via the acceptable use standard and periodic log review, not technical refusal. | 5.10, 8.15, 8.16, 8.20, 8.22, 6.3 | 12 |
| RUN-02 | Indirect prompt injection. Instructions embedded in fetched web pages, documents, code comments, filenames or tool output are executed as though the operator had issued them. | 5 | 5 | 25 | Architectural separation of instruction and data channels. Retrieved and fetched content is delimited, marked as untrusted, and never granted authority. Web access restricted to an allowlist through a filtering proxy, or removed entirely for sessions handling untrusted corpora. Tool invocation triggered by content that entered from an untrusted channel requires human confirmation. Full tool-call logging with alerting on unexpected tool sequences. | 8.20, 8.22, 8.23, 8.26, 8.27, 8.16, 5.7 | 12 |
| RUN-03 | Insecure output handling. Model output is passed to a shell, an evaluation function, a database query or a browser render without validation, converting a text prediction into code execution or injection. | 4 | 5 | 20 | Model output is treated as untrusted input to every downstream consumer, with the same validation, encoding and parameterisation applied as to any external data. Automatic execution of generated code is prohibited outside an ephemeral, network-isolated sandbox. Rendered output is contextually encoded. Secure coding standards apply to all lab-built tooling. | 8.25, 8.26, 8.27, 8.28, 8.29 | 8 |
| RUN-04 | Fabricated technical content reaches a client deliverable or a public publication. Plausible but invented indicators, CVE detail, attribution claims or code carry the operator's professional credibility behind them. | 4 | 4 | 16 | Mandatory human verification gate: no model-generated technical assertion leaves the lab without independent confirmation against a primary source. Provenance of AI assistance recorded internally and disclosed where client terms require. Attribution claims and indicators subject to a separate confidence-assessment procedure. Peer review before publication. | 5.35, 5.36, 5.37, 6.3, 8.29, 5.31 | 8 |
| RUN-05 | Generation of material that is unlawful to create or possess. Multimodal capability plus zero refusal behaviour plus automatic caching means an offence can be committed and evidenced on disk without deliberate intent. Several relevant offences are strict liability, and research purpose is not a defence. | 2 | 5 | 10 | Absolute prohibition, stated explicitly in the acceptable use standard and the risk acceptance, on prompting toward prohibited content categories. Multimodal generation capability disabled unless a specific authorised research need exists. Output caching and history disabled for any session with generative image capability. A written procedure defines immediate actions on inadvertent generation: isolate, do not distribute, do not attempt independent deletion of evidence, obtain legal advice, report. Awareness training covers the strict-liability position directly. | 5.10, 5.31, 5.24, 5.28, 6.3, 8.10, 8.23 | 10 |
| RUN-06 | Resource exhaustion through adversarial context length or recursive generation, denying the lab to legitimate use and driving thermal load. | 3 | 2 | 6 | Context length, token budget and concurrency limits enforced at the serving layer. Per-session quotas. Utilisation alerting. | 8.6, 8.16, 8.21 | 3 |
| RUN-07 | Credentials pasted into prompts. API keys, client credentials and connection strings entered for convenience persist in conversation history, cache and logs indefinitely. | 4 | 4 | 16 | Secrets are supplied to tooling through a secrets manager and environment injection, never through the conversation channel. Pattern-based detection on prompt input and stored history with alerting. History and log purge procedure on detection, followed by credential rotation as the primary remediation. | 5.17, 8.12, 8.15, 8.24, 8.10, 6.3 | 8 |
| RUN-08 | Model extraction or inversion against an exposed or over-shared endpoint recovers a proprietary fine-tune or its training characteristics. | 2 | 3 | 6 | Endpoints authenticated and rate-limited. Access to adapted models restricted to named identities. Query volume monitored for extraction-shaped patterns. | 8.3, 8.5, 8.16, 8.21 | 3 |
| ID | Risk scenario | L | I | Inh | Countermeasures and control statements | ISO 27002:2022 | Res |
|---|---|---|---|---|---|---|---|
| AGT-01 | Excessive agency. The agent holds shell, filesystem and network tools with no approval gate, so any injected instruction becomes host command execution under the agent's privileges. | 4 | 5 | 20 | Least-privilege tool design: each tool exposes the narrowest possible operation rather than a general-purpose shell. Agent runs as an unprivileged identity in an ephemeral container with no host mount and no credentials beyond the task. Human approval required for any state-changing, destructive or outbound action. Privileged utility programs removed from the agent image. Session-scoped, time-bounded tokens only. | 8.2, 8.3, 8.18, 8.22, 8.27, 5.15, 5.17 | 12 |
| AGT-02 | The agent initiates scanning, enumeration or exploitation against systems the operator has no authorisation to test, from the operator's own IP address. This is a criminal exposure under computer misuse legislation and a breach of connectivity terms, and intent is not a required element of the primary offence. | 3 | 5 | 15 | Default-deny egress at the lab boundary. No default route for agent workloads. Any external destination requires explicit allowlisting with a recorded justification and an expiry. Offensive tooling absent from the agent image entirely. Where external interaction is genuinely required for research, it targets operator-owned infrastructure or a documented, authorised range only. Egress logging retained and reviewed. | 8.20, 8.22, 8.23, 8.16, 5.10, 5.31 | 10 |
| AGT-03 | The agent exfiltrates lab or client data through a channel that was permitted for legitimate reasons: a webhook, a repository push, a paste service, a DNS lookup or an image request. | 3 | 5 | 15 | Egress allowlist covers destination and protocol, with DNS forced through a controlled resolver and logged. Outbound repository, messaging and paste destinations blocked by default. Data loss prevention on permitted paths. Volumetric and entropy alerting on outbound flows from lab hosts. | 8.12, 8.20, 8.21, 8.23, 8.16, 5.14 | 10 |
| AGT-04 | The agent harvests credentials available in its execution context: environment variables, key material, cloud tokens, browser credential stores, configuration files. | 3 | 5 | 15 | Agent execution context contains no credentials other than the minimum for the specific task, issued short-lived from a secrets manager. No user profile, browser data or key material present in the agent image. Filesystem access restricted to a task-scoped scratch volume. Credential use monitored for anomalous source or pattern. | 5.17, 8.2, 8.3, 8.24, 8.16, 8.18 | 8 |
| AGT-05 | The agent establishes persistence or modifies its own configuration: scheduled tasks, service units, edits to its own system prompt or tool manifest. | 3 | 4 | 12 | Agent runtime is immutable and rebuilt from a controlled image each session; no state survives by default. Prompt and tool definitions held in version control outside the agent's write path. File integrity monitoring on configuration and scheduling locations. Software installation on lab hosts restricted. | 8.9, 8.19, 8.32, 8.16, 8.4 | 6 |
| AGT-06 | Unbounded agent loop consumes compute, power and external API spend without producing useful output. | 4 | 2 | 8 | Hard limits on iteration count, wall-clock duration, token spend and tool invocations per session. Automatic termination at threshold. Spend alerting on any metered external service. | 8.6, 8.16, 5.30 | 4 |
| AGT-07 | Confused deputy. The agent exercises its own privileges on behalf of a request that originated from an untrusted source, so authority is laundered through the agent. | 3 | 4 | 12 | Requester identity propagated to the authorisation decision rather than the agent acting under a single service identity. Distinct, lower-privilege agent profiles for sessions that touch untrusted content. Authorisation evaluated at the tool boundary, not the prompt boundary. | 8.2, 8.3, 8.27, 5.15, 5.16 | 8 |
| AGT-08 | Multi-agent or long-running autonomous configuration behaves outside its expected envelope and no reliable stop mechanism exists. | 2 | 4 | 8 | A tested kill switch exists at three layers: process termination, network isolation of the segment, and physical power isolation. Termination procedure documented and exercised. Autonomous runs are attended or bounded by an automatic expiry. | 5.29, 5.30, 8.16, 8.27, 5.37 | 4 |
| ID | Risk scenario | L | I | Inh | Countermeasures and control statements | ISO 27002:2022 | Res |
|---|---|---|---|---|---|---|---|
| MON-01 | No logging of prompts, responses, tool calls or model load events. An incident cannot be reconstructed, and the operator cannot demonstrate what the lab did or did not do. | 4 | 4 | 16 | Structured logging of session identity, model and adapter digests, prompts, responses, tool invocations with arguments and results, and artefact load events. Logs shipped off-host in near real time to a store the lab identity cannot write to. Clock synchronisation enforced across lab hosts. | 8.15, 8.16, 8.17, 5.28 | 6 |
| MON-02 | The logs become the largest single concentration of sensitive material in the environment, and are protected less well than the systems they describe. | 4 | 4 | 16 | Prompt and response logs classified at the highest classification of any data the lab handles. Encrypted at rest, access restricted to named identities, retention bounded and enforced. Secret and personal data redaction applied at ingestion where feasible. | 8.15, 8.24, 8.11, 8.10, 5.33, 5.12 | 8 |
| MON-03 | No detection content specific to lab behaviour. Generic endpoint alerting does not distinguish legitimate research from compromise or agent misbehaviour. | 4 | 3 | 12 | Detection use cases developed for lab-specific signals: unexpected model artefact writes, egress to non-allowlisted destinations, tool invocation outside baseline sequences, GPU utilisation anomalies, new listening services, credential access from agent contexts. Use cases tested against simulated conditions, not assumed. | 8.16, 5.7, 8.29 | 6 |
| MON-04 | No incident response plan covering lab-specific events, including who is notified and within what deadline when client data or third parties are involved. | 3 | 4 | 12 | Lab-specific scenarios added to the incident response plan: model artefact compromise, agent egress event, inadvertent unlawful content generation, client data exposure. Notification obligations mapped per scenario, including regulatory and contractual deadlines. Evidence preservation procedure defined. Plan exercised at least annually. | 5.24–5.28, 5.31, 5.34, 5.29 | 6 |
| MON-05 | Log integrity failure. An attacker, or the agent itself, clears or alters lab logs and removes the evidence of its own activity. | 3 | 3 | 9 | Append-only remote log store. Lab hosts hold write-only credentials to the collector and no deletion capability. Integrity verification on the archive. Alerting on log source silence. | 8.15, 8.24, 8.22, 5.33, 8.16 | 4 |
| ID | Risk scenario | L | I | Inh | Countermeasures and control statements | ISO 27002:2022 | Res |
|---|---|---|---|---|---|---|---|
| HUM-01 | Automation bias. Fluent, confident output from an unaligned model is trusted because it reads as authoritative, and anthropomorphic framing of the system encourages the operator to attribute reasoning and intent where there is none. | 4 | 4 | 16 | Awareness training covering the limits of generative output, the distinction between fluency and accuracy, and the specific hazard of anthropomorphic language in technical judgement. Verification gate on all technical assertions. Documented procedure requiring the operator to record the primary source for any model-originated claim used in analysis. | 6.3, 5.37, 5.35, 5.36 | 8 |
| HUM-02 | Unauthorised physical or logical access by household members, guests, cleaners or contractors to a lab holding client material and unfiltered models. | 3 | 3 | 9 | Lab area secured when unattended. Sessions locked on idle. Named accounts with strong authentication; no shared or unauthenticated local access. Clear desk and clear screen applied to lab equipment. Visitor access to the area recorded. | 7.1–7.4, 7.7, 8.1, 8.5, 5.15 | 4 |
| HUM-03 | Operator exposure to distressing generated content with no welfare provision, in an environment specifically configured not to refuse. | 3 | 3 | 9 | Welfare provision documented as part of the authorisation standard, including exposure limits, a route to support, and an explicit position that operators may stop any line of research without justification. Prohibited content categories stated so the boundary is known in advance rather than discovered. | 6.3, 6.8, 5.10, 6.2 | 6 |
| HUM-04 | Key person dependency. The environment is undocumented and only one person can operate, secure or safely shut it down. | 4 | 3 | 12 | Documented operating procedures covering build, operation, emergency shutdown and decommissioning, held outside the lab and readable by a competent third party. Named deputy with documented access to isolation and power controls. Configuration held in version control. | 5.2, 5.37, 5.29, 8.13, 8.9 | 6 |
| HUM-05 | Research findings published without ethical review, responsible disclosure consideration or assessment of whether the publication itself provides uplift. | 3 | 4 | 12 | Publication gate requiring assessment of uplift, disclosure obligations to affected parties, licence and copyright position, and client confidentiality. Peer review before release. Documented decision record retained for each publication. | 5.35, 5.14, 5.31, 5.32, 6.2, 5.5 | 6 |
| ID | Risk scenario | L | I | Inh | Countermeasures and control statements | ISO 27002:2022 | Res |
|---|---|---|---|---|---|---|---|
| EOL-01 | An alignment-modified artefact, a harmful output sample or a working exploitation capability is shared publicly with no gating, because sharing is the cultural default in the research community. | 3 | 5 | 15 | Information transfer control applies to model artefacts as information assets. Alignment-modified weights are non-transferable by classification. Publication gate covers uplift assessment, licence position and disclosure obligations. Repository and model hub destinations blocked from lab egress by default. | 5.14, 5.10, 5.12, 8.12, 8.23, 5.35 | 10 |
| EOL-02 | Decommissioned accelerators, drives or hosts are sold or returned under warranty with residual model artefacts, logs and client data recoverable. | 3 | 4 | 12 | Secure disposal procedure covering cryptographic erasure, verified media sanitisation and certificate of destruction. Full disk encryption from build so key destruction is the primary mechanism. Disposal recorded against the asset register entry before the asset leaves the premises. | 7.10, 7.14, 8.10, 8.24, 5.9 | 4 |
| EOL-03 | Backup, snapshot and image sprawl retains data long past its retention decision, defeating deletion at the primary store. | 4 | 3 | 12 | Retention applied to backups and snapshots equally, with automated expiry. Deletion requests traced through every copy including offline media. Backup inventory reconciled against the asset register quarterly. | 8.13, 8.10, 5.33, 5.9 | 6 |
| EOL-04 | The lab is abandoned rather than decommissioned: left powered, reachable and unpatched after the research it was built for has finished. | 3 | 4 | 12 | Every lab instance carries a defined end date at authorisation. Decommissioning procedure covers data destruction, network removal, credential revocation and register closure. Quarterly reconciliation of running assets against active authorisations, with unauthorised survivors powered down. | 5.9, 5.37, 8.9, 8.16, 8.8 | 4 |
Statement of Applicability extract. These are the controls whose implementation is materially altered by the presence of an unguardrailed LLM laboratory, expressed as auditable statements.
| Ref | Control | Laboratory-specific implementation statement |
|---|---|---|
| A.5.1 | Policies for information security | A Laboratory Authorisation and Acceptable Use standard is approved and maintained. It states permitted and prohibited research categories, the accepted position on alignment-stripped models, the compensating controls relied upon, and the conditions under which the acceptance lapses. |
| A.5.9 / A.5.12 | Asset inventory and classification | Model weights, adapters, prompts, tool manifests, corpora and vector stores are registered as information assets with source, digest, licence and classification. Alignment-modified artefacts and prompt logs are classified at the highest internal level. Vector stores inherit the classification of their source corpus. |
| A.5.14 | Information transfer | Model artefacts are subject to transfer control as information assets. Alignment-modified weights are marked non-transferable. Any external transfer requires documented approval, an uplift assessment and a licence check. |
| A.5.17 | Authentication information | No credential is supplied to a model or agent through the conversation channel. Agent credentials are short-lived, task-scoped and issued from a secrets manager. Detection of a credential in prompt or log content triggers rotation as the primary remediation. |
| A.5.31 / A.5.32 | Legal, regulatory and intellectual property | Legal obligations applicable to the lab are identified and maintained, covering computer misuse, data protection, unlawful content offences, export control and model licensing. Prohibited content categories are stated absolutely, with no research exemption asserted. |
| A.5.34 | Privacy and PII protection | A data protection impact assessment covers the lab as a processing environment. Personal data ingestion requires a documented lawful basis, minimisation and a destruction point. Special category data is prohibited absent a specific assessed exception. |
| A.6.3 | Awareness and training | Operators are trained on prompt injection, insecure output handling, automation bias, anthropomorphisation risk, the strict-liability position on certain generated content, and the incident procedure for inadvertent generation. |
| A.7.5 / A.7.11 | Physical and environmental threats, utilities | Thermal and electrical loading of the accelerator estate is calculated and measured under sustained full load. Active cooling is verified for server-class passive cards. Interlinked smoke detection covers the equipment location. Automated thermal shutdown thresholds are configured for unattended runs. |
| A.8.2 / A.8.18 | Privileged access and utility programs | Agents execute as unprivileged identities. Tools expose narrow operations rather than general-purpose shells. Privileged utilities and offensive tooling are absent from agent images. State-changing and outbound actions require human approval. |
| A.8.10 | Information deletion | Retention is defined and enforced for conversation history, caches, embeddings, temporary artefacts and logs. Deletion is verified rather than assumed, and traced through backups and snapshots. |
| A.8.15 / A.8.16 | Logging and monitoring | Session identity, model digest, prompts, responses, tool calls and artefact loads are logged and shipped to an append-only store the lab cannot write to. Detection content exists for lab-specific signals and is tested rather than assumed effective. |
| A.8.20 / A.8.22 / A.8.23 | Network security, segregation and filtering | The lab occupies a dedicated segment with default-deny in both directions and no default route for agent workloads. External destinations are allowlisted with justification and expiry. DNS is forced through a logged resolver. Management access is initiated only from the trusted side. |
| A.8.26 / A.8.27 | Application security and secure architecture | Instruction and data channels are architecturally separated. Retrieved content is delimited and carries no authority. Authorisation decisions occur at the tool boundary, not the prompt boundary. Untrusted-content sessions run under a reduced-privilege agent profile. |
| A.8.28 | Secure coding | Model output is treated as untrusted input by every downstream consumer, with validation, contextual encoding and parameterised queries applied. Automatic execution of generated code occurs only in an ephemeral, network-isolated sandbox. |
| A.8.29 / A.8.33 | Security testing and test information | Models are evaluated locally for capability and refusal boundary before promotion. Adaptation is gated on a regression threshold against a held-out set. Sanitised or synthetic data is used in place of live client data wherever the research question permits. |
| A.8.31 | Separation of environments | Artefact evaluation, corpus parsing, inference and agent execution occur in separate isolated contexts. One retrieval namespace per engagement, destroyed at close with evidence retained. |
| A.5.29 / A.5.30 | Continuity and readiness | A tested kill switch exists at process, network and physical power layers. The termination procedure is documented and exercised. Autonomous runs are attended or bounded by automatic expiry. |
| Band | Inherent | Residual | Movement |
|---|---|---|---|
| Critical (20–25) | 11 | 0 | −11 |
| Severe (15–19) | 8 | 0 | −8 |
| High (10–14) | 21 | 9 | −12 |
| Moderate (5–9) | 11 | 28 | +17 |
| Low (1–4) | 0 | 14 | +14 |
| Total | 51 | 51 |
| ID | Residual exposure | Res | Why it does not reduce further |
|---|---|---|---|
| RUN-02 | Indirect prompt injection | 12 | No general defence exists. Instruction and data share one channel by architecture. Controls constrain blast radius, not occurrence. |
| AGT-01 | Excessive agency | 12 | Tool access is the research objective. Reducing it further removes the capability being studied. Approval gates depend on human vigilance, which degrades with volume. |
| RUN-01 | Direct prompt manipulation | 12 | Accepted by design. Refusal behaviour is deliberately absent; the residual is the cost of the research premise. |
| SUP-02 | Trigger-conditioned model backdoor | 12 | Not detectable by static or behavioural evaluation with current methods. Managed purely by containment. |
| DAT-01 | Client data ingestion | 10 | Depends on operator discipline at the moment of convenience. Technical prevention is partial where the operator holds legitimate access to both environments. |
| ADP-01 | Alignment-stripped artefact leakage | 10 | Impact remains severe under any likelihood reduction. Controls reduce probability but cannot reduce consequence. |
| AGT-02 | Unauthorised outbound activity | 10 | Criminal exposure keeps impact at maximum. Egress controls are strong but a single misconfiguration is sufficient. |
| AGT-03 | Agent exfiltration via permitted channels | 10 | Any permitted egress path is a potential covert channel. Complete closure would prevent legitimate research. |
| RUN-05 | Unlawful content generation | 10 | Strict liability means impact cannot be mitigated, only likelihood. Prohibition plus capability removal is the entire control set. |
| DAT-05 | Corpus poisoning | 10 | Detection of semantic poisoning in large corpora is not reliable. Segregation limits contamination rather than preventing it. |
| EOL-01 | Publication of harmful artefacts | 10 | Governed by human judgement at the publication gate. Technical egress blocking is bypassable by an authorised operator. |
Three residual risks cannot be engineered away and must be accepted explicitly or the lab should not operate.
These acceptances should be signed, dated and given an expiry no longer than twelve months. An acceptance without an expiry is not a risk decision, it is an omission with a signature on it.
| Instrument | Relevance to the lab | Practical consequence |
|---|---|---|
| Computer Misuse Act 1990 | AGT-02. Unauthorised access to a computer is the offence; an agent acting autonomously does not remove the operator's exposure. | Default-deny egress is a legal control, not merely a technical one. Authorisation for any external testing must be in writing and held before the fact. |
| UK GDPR and Data Protection Act 2018 | DAT-01, DAT-02, DAT-06, MON-02. Model training and retrieval over personal data is high-risk processing. | Impact assessment required. Records of processing must include the lab. Deletion must be demonstrable across caches, embeddings and logs. |
| Protection of Children Act 1978 and related offences | RUN-05. Generation and possession offences are strict liability; multimodal capability plus automatic caching creates exposure without intent. | Disable multimodal generation absent specific need. Disable caching for any session with that capability. Rehearse the response procedure. |
| Terrorism Act 2000 and 2006 | RUN-05, EOL-01. Collection and dissemination offences apply to generated as well as collected material. | Prohibited categories stated absolutely in the acceptable use standard. Publication gate assesses dissemination risk. |
| Model licences and acceptable use policies | SUP-05, ADP-01. Many open-weight licences prohibit alignment removal, certain use categories, or commercial derivative use. | Licence recorded at registration; commercial workflows technically separated from restricted artefacts. |
| Export control and sanctions | SUP-07, EOL-01. Weights and certain tooling may be controlled on transfer or by counterparty. | Provenance check at acquisition; export assessment before any onward transfer. |
| Client contracts and NDAs | GOV-03, DAT-01, DAT-03. Many engagement terms prohibit processing of client data by AI systems or require prior consent. | Contractual position mapped per client. Where prohibited, enforce technically rather than by intention. |
| Connectivity terms of service | AGT-02, INF-05. Domestic and business connectivity terms typically prohibit scanning and hosting. | Egress allowlist. No inbound exposure of lab services. |
ISO/IEC 27001:2022 Annex A contains no AI-specific control. Every mapping in this document reuses a general control in an AI context, which works well for containment, access, logging and supply chain, and works poorly for the risks that are genuinely novel. Four areas have no clean Annex A home:
Recommended supplementary alignment. Use ISO/IEC 27001 as the containment and assurance backbone, then layer AI-specific management on top rather than stretching Annex A to cover it.
ISO/IEC 42001 AI management system: impact assessment, oversight, lifecycle governance.