When Can an Action Shield Support a Safety Claim?

A shield that rejects unsafe actions every run can still support no safety claim at all.

The log looks clean. The shield blocked the bad action again, the episode ended without a violation, and the team writes "safety verified" in the summary. I have watched this exact reasoning survive a design review, and it is wrong in a way that is hard to see from inside the runtime output. A shield is a filter. A filter is only as honest as the model it consults and the specification it was handed. Blocking a known-unsafe action is a local, per-step decision. A safety claim is a statement about every step the shield approves — including the steps where it never had to block anything.

This article audits the three assumptions that convert "the shield blocked a bad action" into "each approved step stays safe." We will state the claim precisely, verify it against a small transition table, then break it twice: once by omitting an unsafe state, once by introducing a single optimistic transition in the shield's model. The failures have different signatures, and knowing which one you are looking at changes who needs to fix it.

What a Shield Actually Claims

A shield is a function from state to an allowed action set. It constrains the support of the policy — the set of actions the policy can actually select — rather than rewriting the environment or the reward. If you already know masking and rejection-based filtering from invalid-action handling, that is the same runtime shape: the policy proposes, the shield disposes.

What differs is the claim people attach to it. Two shield families are worth naming as boundary markers:

• Absolute shields treat safety as qualitative. The environment is modeled adversarially, no risk is tolerated, and an action is safe only if the agent can continue to stay in safe states regardless of what the environment does next. The guarantee is "never."
• Probabilistic shields assign a risk budget. An action is rejected when its probability of violating the specification exceeds a threshold. The guarantee becomes "with probability at least," which is a different claim with different assumptions about how the model was estimated.

This article stays on the absolute, finite, deterministic case. That is the cleanest setting in which to see exactly which assumptions carry the guarantee, and the probabilistic case changes the wording rather than the structure.

The distinction that matters: a shield can be perfectly effective at rejection and still support no claim at all. It may simply never have been tested against the states where its model is wrong.

Notation for a Finite Safety Problem

Fix the objects before stating anything. Let the environment be an MDP $(S, A, P, s_0)$ with finite state set $S$, finite action set $A$, and transition kernel $P(s' \mid s, a)$ — the distribution the environment actually uses.

Let $U \subseteq S$ be the unsafe set. Safety is a property of states, not actions. An action is unsafe only because of where it can lead; the state is the thing that violates the specification.

The shield consults its own transition model, written $\hat{P}(s' \mid s, a)$. It is not required to equal $P$. That gap is the whole story.

The shield permits:

$\text{Allowed}(s) = \{\, a \in A : \text{every } s' \text{ with } \hat{P}(s' \mid s, a) > 0 \text{ satisfies } s' \notin U \,\}$

Read it aloud once: an action is allowed only when the shield's model says every possible successor lies outside the unsafe set. Not the likely successor. Every successor with nonzero probability under $\hat{P}$.

Two assumptions are baked into that definition and deserve names.

Feasibility. $\text{Allowed}(s)$ is non-empty for every reachable safe state. If it is empty, no shield exists at that state and the claim is vacuous. This is an assumption, not a footnote — a shield that blocks everything keeps the agent safe and useless.

Soundness of the model. $\hat{P}$ never assigns zero probability to a transition that $P$ permits. Soundness is one-directional and easy to get backwards: the shield's successor set must be a superset of the true successor set. The shield must be pessimistic, never optimistic.

The Bounded Claim and Its Three Assumptions

Here is the claim, in one sentence:

If (1) $s_0 \notin U$, (2) $\hat{P}$ is sound with respect to $P$, and (3) $U$ contains every state that violates the safety specification, then every state reached by a shield-approved action is outside $U$.

Each assumption is load-bearing.

Assumption 1 is about the starting point. A shield cannot un-violate a state it was handed. If the agent begins inside $U$, the claim is false before the first decision.

Assumption 2 is about the model. Soundness means the shield reasons over a superset of the true successors. If the shield's model is optimistic — if it misses a transition the environment permits — the shield can approve an action that lands in $U$.

Assumption 3 is about coverage. This is the assumption teams forget, because it lives in the specification, not in the code. The set $U$ must contain every state the safety requirement actually cares about.

The derivation is three lines. Base case: $s_0 \notin U$ by assumption 1. Inductive step: suppose $s \notin U$ and $a \in \text{Allowed}(s)$. By the shield rule, every $s'$ with $\hat{P}(s' \mid s, a) > 0$ satisfies $s' \notin U$. By soundness, every $s'$ with $P(s' \mid s, a) > 0$ also has $\hat{P}(s' \mid s, a) > 0$, so every true successor is in that same set. Therefore every true successor lies outside $U$. Induction closes.

Keep it short, because the value is in seeing which assumption each line consumes. Line one spends assumption 1. Line two spends the shield rule and assumption 2. The conclusion is only as strong as assumption 3, which never appears in the derivation at all — it appears in the interpretation of $U$ as "the unsafe states."

State plainly what the claim is not: it says nothing about reward, optimality, task completion, or whether the policy learns anything useful. It is a statement about reachability of $U$.

Worked Example: A Four-State Transition Table

Take a corridor with states $s_0, s_1, s_2, s_3$ and $U = \{s_3\}$. Two actions, $a_1$ and $a_2$. The environment's true transitions:

At $s_1$, action $a_2$ has a true successor in $U$, so the shield rejects it. Action $a_1$ keeps every successor in $\{s_0, s_1, s_2\}$, so it is approved. At $s_2$, both actions stay outside $U$ and both are approved. At $s_0$, $a_1$ leads to $s_1$, which is safe, so it is approved.

Now the shield's own table $\hat{P}$. In this example it agrees with $P$ exactly — this is the case where the guarantee holds. Trace a short approved trajectory: $s_0 \xrightarrow{a_1} s_1 \xrightarrow{a_1} s_2 \xrightarrow{a_2} s_2 \to \cdots$. Every state is outside $U$. You can reproduce this on paper in under two minutes, and that is the point: the claim is checkable by hand in a finite setting.

Note the conservatism cost. The shield reasons over all successors rather than the likely one, so it may reject actions that are actually fine. Absolute guarantees buy safety with permissiveness. That trade is the price of the "never," and it is worth naming before someone tries to tune it away.

Breaking It: One Omitted State

Change one thing. The specification's $U$ omits a state the real safety requirement cares about. Call it $s_4$, reachable from $s_2$ by an approved action. Add it to the true dynamics: at $s_2$, action $a_2$ now leads to $s_4$ with probability 1.0 instead of looping back to $s_2$.

The shield still behaves correctly with respect to its own $U$. Every approved step avoids the states it knows about. The log stays clean. The guarantee fails anyway, because assumption 3 was false. The shield was never wrong; the specification was incomplete.

This is the failure mode that hides best, because nothing in the runtime output looks broken. Coverage gaps are invisible from inside the shield. The shield cannot report a state it was never told to avoid.

The practical consequence is uncomfortable: coverage is a specification-review problem, not a code-review problem. It is found by asking what states the safety requirement implies but the set does not list. That question is answered by reading the requirement, not the implementation.

Breaking It: One Model-Error Transition

Change a different thing. Let $\hat{P}$ assign zero probability to a transition that $P$ actually permits. Say at $s_1$, the shield's model claims action $a_2$ leads only to $s_1$. The true kernel gives $s_3$ probability 0.7.

Walk the step. The shield's successor set is $\{s_1\}$. The true successor set is $\{s_1, s_3\}$, and $s_3 \in U$. The shield approves $a_2$, and the approved action lands in $U$. The guarantee fails at the first step where the model is optimistic. One optimistic transition is enough; soundness is not a statistical property you can average over.

The two failures have different signatures, and the difference determines who fixes them:

Model error is detectable by comparing $\hat{P}$ against observed transitions — but only in one direction. An observed disagreement refutes soundness and points to a specific transition. Passing sampled comparisons does not prove soundness, because the transitions you never observed are exactly the ones that can hide an optimistic gap. Treat the comparison as a counterexample search, not as evidence that the assumption holds.

Coverage gaps require reasoning about the requirement itself. Different tools, different people, different review.

What This Claim Does Not Cover

The claim is conditional and local. It holds relative to a model, a state set, and a starting point. Remove any one and it evaporates.

It says nothing about the policy's behavior outside the shield, about sensor error, about partial observability, or about whether the shield's inputs are trustworthy. It says nothing about liveness — feasibility is an assumption, not a guarantee. Probabilistic shields trade the absolute "never" for a bounded violation probability, which is a different claim with different assumptions about how the model was estimated.

When the claim is the right tool: you have a finite state abstraction, a specification you can enumerate, and a model you can argue is pessimistic. When it is overkill or misleading: continuous or partially observed state, learned models with no soundness argument, or any setting where "safe" is a business judgment rather than a reachability property. In those settings the shield may still be useful, but the claim you can support is weaker, and you should say so out loud.

Auditing a Shield Before You Trust It

Turn the three assumptions into a checklist you can run against your own setup.

• Assumption 1: Is the initial state provably outside $U$, and is that checked at runtime rather than assumed?
• Assumption 2: What evidence supports soundness of $\hat{P}$? Enumerate the transitions where $\hat{P}$ and observed behavior disagree, and treat each disagreement as a potential guarantee break. Remember that agreement on the transitions you happened to observe is not soundness.
• Assumption 3: Who owns the mapping from the written safety requirement to the set $U$, and when was it last reviewed against the requirement rather than against the code?

Write the claim down in one sentence with its assumptions attached, and put it next to the shield. A guarantee nobody can restate is a guarantee nobody can falsify.

The smallest useful drill: take your own transition table, delete one successor from $\hat{P}$, and check whether your test suite notices. If it does not, the suite is testing the shield's behavior, not its assumptions.

A clean rejection log proves the shield is doing its job. It does not prove the safety claim holds. The claim is conditional, and the conditions are the part you have to earn. Write it down, then deliberately break one assumption in a transition table and watch the guarantee disappear. That exercise teaches more about action shielding safety assumptions than any amount of passing tests — because a filter's guarantee is only as strong as the model and the specification it was handed.