Formalize Information Constraints in Centralized Training and Decentralized Execution

Your team scores well in the training harness. You strip the centralized critic, deploy the actors, and the score collapses — even though the environment never changed. The plumbing changed. Somewhere in the actor's input vector, a variable that only existed during training was quietly doing the work.

That failure is not a tuning problem. It is a specification problem. "Centralized training, decentralized execution" gets treated as a schedule — a phase where you use global information, followed by a phase where you don't. The schedule is the weak model. The stronger model is two information sets, and the deployment contract is written entirely in the second one.

Two Information Sets, Not Two Phases

The common phrasing — train with global info, execute with local info — hides the actual constraint. It sounds temporal, so it invites a temporal fix: train, then remove the extra inputs. But the centralized critic is discarded the moment training ends. What survives is the actor's learned dependence on whatever it was fed. If the actor's input signature included the global state, removing the critic does not remove that dependence. You shipped a function that expects an argument the deployment environment cannot supply.

So separate the variables, not the phases:

• I_train — everything the learning machinery may read. Global state, joint actions, joint observations, per-agent rewards, constraint evaluations, dual variables, simulator-side quantities. This set is large by design; that is the point of centralized training.
• I_i — everything agent i may read at execution. This is a design decision, not an emergent property. It might contain the local observation $o_i$, the action-observation history $h_i$, and any messages actually delivered under your stated communication model. Nothing else.

The boundary between them is a property of the deployed artifact, not of the training run. Draw it as a hard vertical wall: on the left, everything training may touch; on the right, one column per agent listing exactly what its actor reads at decision time.

This is the piece that a workflow-level tour of CTDE leaves open. Tracing information flows tells you where data moves. Formalizing the constraint tells you which flows are admissible once the harness is gone.

Notation: I_train, I_i, and the Decentralized Rule

Set the vocabulary before deriving anything. A cooperative CTDE system lives on a Dec-POMDP: agents $A = \{1,\dots,N\}$, joint state $s$, joint action $a = (a_1,\dots,a_N)$, local observation $o_i \sim O(s,i)$, shared reward $r(s,a)$, discount $\gamma$. You have seen this skeleton; we only need it as scaffolding.

Now the two sets, stated as objects rather than vibes:

$I_{\text{train}} = \{\text{global state } s,\ \text{joint actions } a,\ \text{joint observations } \{o_j\},\ \text{rewards},\ \text{constraint signals},\ \dots\}$

$I_i = \{o_i,\ h_i,\ \text{messages received under the stated channel}\}$

The decentralized rule is a conditional policy whose argument is $I_i$:

$\pi_i(a_i \mid I_i)$

Read the subscript on the conditioning bar carefully. $I_i$ is not a suggestion about what the actor should prefer. It is the function's input signature. Anything outside $I_i$ is not merely discouraged — it is undefined at execution. A function with a fixed arity cannot be called with a missing argument.

The critic, by contrast, is a separate function with a strictly larger signature:

$V(s, a) \quad \text{or} \quad Q(s, a) \quad \text{— training only}$

Component	Input signature	Survives deployment?
Actor $\pi_i$	$I_i$ only	Yes
Critic $V$ / $Q$	$I_{\text{train}}$ (superset)	No
Mixing network	joint utilities	No

The admissibility test falls straight out of the table: a variable $v$ may appear in the deployed actor's input only if $v \in I_i$. Everything else belongs to $I_{\text{train}}$ and must be consumed by training-only components.

Why a Centralized Variable Cannot Be an Execution Input

Do not take the exclusion on faith. Derive it.

Example 1 — global state. Suppose you train an actor as $\pi_i(a_i \mid s, o_i)$, with $s$ handed over by the simulator. At execution, $s$ does not exist. The call $\pi_i(a_i \mid o_i)$ is not a degraded version of the trained function — it is a different function, evaluated on an input the network never saw alone. The failure is an undefined call, not a performance dip.

Example 2 — teammate action. Suppose the actor consumes $a_j$ at decision time. Under simultaneous action selection, $a_j$ is not produced yet when $a_i$ is chosen. You are reading a variable from the future. If $a_j$ is genuinely needed, it must arrive as a message, which means it belongs in $I_i$ explicitly — with the latency and drop model that channel implies.

The subtle case. A variable can be technically reachable and still outside $I_i$. Privacy requirements, bandwidth limits, or missing infrastructure can exclude a variable by design. The constraint is normative, not physical. A shared memory bus in simulation does not make the global state admissible.

Common mistake: "It worked in training" is not evidence. The training harness supplies $I_{\text{train}}$, so any dependence on $I_{\text{train}}$ is invisible until the harness is removed. Training returns measure the function you built, not the function you can deploy.

Translate this back to system behavior: the deployed policy is a function with a fixed arity. Adding a training-time input changes the arity. The deployment environment cannot satisfy the new signature, and no amount of retraining fixes a contract that was never written down.

Assumptions You Must State Before Claiming Decentralization

The meaning of $I_i$ changes completely depending on which assumptions hold. State them before you claim anything.

Observability. Is $o_i$ a sufficient statistic for the decision, or must $I_i$ include history $h_i$? Partial observability is the default in MARL. Say whether the policy is memoryless or recurrent. A memoryless policy trained on a Markov observation fails the moment the observation stops being sufficient.

Communication. Is there a channel at execution? If yes, $I_i$ includes received messages, and message content, latency, and drop behavior become part of the information set. If no, $I_i$ is strictly local. A policy trained under an assumed channel degrades to an undefined input when the channel is absent.

Timing. Simultaneous or sequential action selection? This determines whether a teammate's current action can ever be in $I_i$. Under simultaneous selection, it cannot.

Homogeneity and permutation. If agents are interchangeable, the ordering of variables inside $I_i$ is a modeling choice that affects generalization. Permuting the order can increase robustness; fixing it can overfit to one arrangement.

The decision boundary: a richer $I_i$ buys coordination at deployment cost. When the cost exceeds the benefit — or when the centralized information is genuinely present at execution — the honest answer is that the task is not decentralized, and you should stop calling it that.

Diagnosing a Leak: A Worked Case

Here is a leak I have watched people ship. The actor's input vector is built by concatenating the agent's own observation with a "context" vector. In the training code, that context vector happens to include the global state.

Symptom. Strong training-harness returns. Sharp drop at deployment. The tell: the drop appears even when the environment is unchanged, because only the input plumbing changed.

Root cause. The actor's input signature includes a variable from $I_{\text{train}}$. The critic was correctly training-only. The actor was not.

Diagnostic procedure.

1. List every tensor the deployed actor reads.
2. List $I_i$.
3. Diff the two lists.
4. For each extra variable, trace its runtime data dependency: does the deployed computation require a value that is unavailable at execution? A fixed learned parameter — a weight, a bias, a normalization constant baked in at export — is not a leak. A live input that carries unavailable runtime information is.

Common disguises. A "team context" embedding that is really the global state in disguise. A teammate's action passed through a channel that exists only in the simulator. A normalization layer that recomputes statistics from the current global batch at inference time, rather than using frozen training statistics. A shared encoder that reads joint observations at runtime, not just during training.

Note the distinction in that last group. A shared encoder trained on joint observations is fine if, at deployment, it runs only on admissible local inputs. It becomes a leak when the deployed forward pass still requires joint observations to produce the actor's input. The question is never "where did this parameter come from?" It is "what does the deployed computation read right now?"

Fix patterns. Route the privileged variable to the critic only. Replace the actor's dependence with a learned estimate from $I_i$. Freeze any preprocessing statistics at export time. Or, if the variable is genuinely needed, widen $I_i$ and accept the deployment cost explicitly.

# The leak, in miniature.
actor_input = torch.cat([obs_i, context], dim=-1)   # context includes global state

# The fix: split the signatures.
critic_input = torch.cat([obs_i, context], dim=-1)  # training only
actor_input  = obs_i                                 # matches I_i

The diff is the whole procedure. Write $I_i$ down. List the tensors. Trace each one to its runtime source. Any variable that requires unavailable execution-time information is a leak, regardless of how well training went.

What This Constraint Costs You

The benefit is concrete: a policy whose input signature matches $I_i$ is deployable by construction, and its evaluation numbers mean something outside the training harness.

The cost is equally concrete. Restricting $I_i$ removes information the actor could have used, which can slow learning or cap achievable coordination. That is the price of decentralization, not a bug.

When the constraint is overkill. Single-machine simulation, research settings where "deployment" is the same process, or any case where the centralized information is genuinely present at execution. In that last case, you are not doing decentralized execution — say so.

When it is non-negotiable. Physical robots, privacy-sensitive multi-party systems, bandwidth-limited or intermittently connected deployments, and any setting where the training simulator is not the deployment environment.

Keep this separate from a neighboring constraint. Value decomposition and monotonic mixing constrain the argmax structure of action selection — they ensure decentralized greedy actions are consistent with joint maximization. This article constrains inputs. Different object, different failure mode. Do not conflate them.

Before you ship any decentralized policy, write down $I_i$, list the tensors the deployed actor actually reads, and diff the two. The next durable question is not which information the actor may read, but how the joint value function must be structured so that decentralized action selection stays consistent — a constraint on the argmax, not on the input.