Skip to content

use-case

Multi-Agent Learning

Learning and decision-making involving multiple interacting agents, including coordination, competition, non-stationarity, communication, and agent-level credit.

Tagged articles

Multi-Agent Learning articles

26 articles in this tag.

A child focuses on a chess game with blurry background outdoors.

advanced

12 min read

Update a Bayesian Model of an Opponent’s Actions

Frequency counting answers with a single number and no error bar. It cannot tell you how much evidence you actually have, and it has no place to put what…

Close-up of a hand moving a chess piece during an intense game.

advanced

13 min read

Test Prior Sensitivity in a Bayesian Opponent Model

Same opponent. Same six observed actions. Same code. Two different decisions — because the prior changed.

Telecommunication antennas on a rooftop in Istanbul, showcasing modern urban infrastructure.

advanced

11 min read

Centralized Training and Decentralized Execution in Multi-Agent RL

The agent that acts at deployment is not the agent that learns. Training sees a richer world than execution ever will.

Electric blue wires connected to network adapter plugged in socket on shabby brown wall of building on street with shadow

advanced

8 min read

Communication and Coordination in Multi-Agent RL

More communication is not automatically better coordination. In cooperative multi-agent reinforcement learning, a naive message-passing channel can inject…

Elegant chess pieces on a board, featuring two knights and a king, symbolizing strategy and planning.

advanced

11 min read

Cooperative vs Competitive Multi-Agent RL: How Incentives Change Learning

"Cooperative" and "competitive" describe the reward wiring, not the environment—and the same physical scene can host either regime depending on how you…

Man diving into the ocean from a ledge with others swimming, showcasing thrilling summer fun.

advanced

11 min read

Counterfactual Credit Assignment in Multi-Agent RL: A Worked Derivation

A team wins. The shared reward is +1. Every agent's gradient gets the same number, so the agent who stood still learns exactly as much as the agent who…

A modern server room featuring network equipment with blue illumination. Ideal for technology themes.

advanced

11 min read

Credit Assignment in Multi-Agent RL: Which Agent Caused the Outcome?

Your team just won. The shared reward arrives, positive and unambiguous. Every agent updates its policy as if the victory were its own doing—and that is…

Close-up of a humanoid robot in motion, showcasing modern robotics innovation.

advanced

9 min read

Formalize Information Constraints in Centralized Training and Decentralized Execution

Your team scores well in the training harness. You strip the centralized critic, deploy the actors, and the score collapses — even though the environment…

Detailed image of a server rack with glowing lights in a modern data center.

advanced

10 min read

Audit Centralized Training and Decentralized Execution in a Toy Task

A CTDE run can post a beautiful training curve while the deployed policy quietly cheats. The critic saw the joint state. So did the actor. Nobody noticed,…

Boy in a blue shirt intently playing chess at a wooden table in a bright room.

advanced

11 min read

Evaluating Multi-Agent Policies: Partners, Opponents, and Exploitability

A policy that crushes its self-play twin can still collapse against a slightly different opponent. That single blind spot is why multi-agent reinforcement…

From below of long thin identical blue cables connected to small round electrical connectors

advanced

13 min read

Markov Games and Moving Learning Targets in Multi-Agent RL

The game never changes. Your experience of it changes every time another agent updates.

Close-up image of ethernet cables plugged into a network switch, showcasing IT infrastructure.

advanced

10 min read

Why Monotonic Value Factorization Supports Decentralized Greedy Actions

A team where every agent picks its own best action can still lose. Monotonic value factorization is the constraint that rules out one specific failure —…

A young couple deeply focused on a chess game in a cozy indoor setting.

advanced

9 min read

Analyze Cooperation, Competition, and Mixed Motives With Payoff Matrices

Two agents converge. Both policies are stable. Both agents are, by their own local test, optimal. The team still loses.

A focused software engineer working on a laptop in a server room, reflecting dedication in tech.

advanced

9 min read

Multi-Agent Reinforcement Learning Explained: What Changes When Other Agents Learn Too?

Single-agent RL treats the environment as a fixed target. Add a second learning agent, and the target starts moving.

Drone shot of a blue tractor on geometric salt flats, showcasing farming in a unique landscape.

advanced

9 min read

Test How Message Bandwidth Changes Multi-Agent Coordination

A team that coordinates perfectly with unlimited chatter can fall apart the moment you cap messages per step or flip a few bits. That collapse is the…

Close-up of an elegant greenfinch bird in mid-flight with wings spread.

advanced

9 min read

Compare Shared and Counterfactual Credit Signals in a Toy Team Task

Two agents, one shared reward, and a number that refuses to say who earned it. The fastest way to understand that refusal is to force it into arithmetic:…

A close-up of a vintage blue toy car on a wet surface with a blurred green background.

advanced

10 min read

Exploitability in Multi-Agent Reinforcement Learning: Best Responses and a Worked Game

A policy that wins 90% of its self-play games can still lose to an opponent that never appeared in training.

Detailed image of illuminated server racks showcasing modern technology infrastructure.

advanced

9 min read

Measure Stale Experience When an Opponent Policy Changes

A return drop after an opponent switches is not evidence of a moving target. It is evidence that you have not yet measured the noise floor.

Operator in a modern control room managing technological systems in El Agustino, Lima.

advanced

11 min read

Non-Stationarity in Multi-Agent Reinforcement Learning: Why Yesterday’s Target Moves

Your training curves look beautiful. Reward is climbing, losses are shrinking, and the agent seems to have mastered its task. Then you deploy it against a…

A vibrant display of fresh fish in a blue barrel at a market in Dhaka, Bangladesh.

advanced

8 min read

What One Bit of Communication Enables in a Two-Agent Coordination Game

Two agents want the same thing. Both know the rules. Both are fully cooperative. And they still fail half the time — until someone is allowed to say a…

Group of adults playing craps around a casino table, engaging in betting and socializing.

advanced

13 min read

Opponent Modeling in Multi-Agent RL: Predicting the Other Learner

The moment you treat another learning agent as part of the environment, you are betting on a target that moves. Opponent modeling is the attempt to stop…

An intense poker game featuring players and poker chips on the table in an indoor setting.

advanced

11 min read

Test a Simple Opponent Model in a Repeated Game

An adaptive agent that beats a fixed opponent in one run can lose to the same opponent after a single behavior shift. The frequency estimate it trusts is a…

Rear view of a man driving a car with focus on interior details and steering.

advanced

9 min read

Test Self-Play for Cycling and Exploitability in a Small Game

A self-play win rate can climb for a thousand iterations and still be lying to you. Here is the smallest instrument that catches it.

Silhouetted antennas on building roof against an orange sunset sky creating a striking scene.

advanced

11 min read

Self-Play in Reinforcement Learning: How Agents Learn From Evolving Opponents

The opponent is part of the environment, and the opponent keeps learning. That single fact turns self-play into a moving-target problem where your own…

Young man in white playing chess indoors, demonstrating concentration and strategy.

advanced

11 min read

Value Decomposition in Multi-Agent RL: Learning Team Actions From Local Views

A team shares one reward, but at execution time each agent must pick its own action from only its own observation. Value decomposition answers that…

Detailed shot of Ethernet cables connected to server ports highlighting technology infrastructure.

advanced

11 min read

Test Value Factorization in a Small Multi-Agent Coordination Game

Two agents. Three actions each. Nine joint outcomes. If you cannot find the coordination failure in a table that small, you will not find it in a training…