RecursiveMAS: Multi-Agent Coordination Through Latent States

A multi-agent system usually makes neural networks translate their internal states into prose for other neural networks. RecursiveMAS tests a direct latent-space connection instead.

A planner writes a proposal, a critic responds, and a solver reads both messages. This familiar multi-agent design repeatedly converts dense hidden states into tokens and then encodes those tokens again. The transcript is useful to people, but the conversion adds computation and latency.

Recursive Multi-Agent Systems, posted to arXiv in April 2026 by researchers from UIUC, Stanford, NVIDIA and MIT, replaces those intermediate messages with adapted latent states. Across nine benchmarks, the authors report 8.3% higher average accuracy, 1.2× to 2.4× faster inference, and 34.6% to 75.6% fewer tokens than an otherwise equivalent text-communicating system.

Replacing the text channel

A sending model normally projects a hidden state of dimension d_h onto a vocabulary of size |V|, one token at a time. The receiver then tokenizes and encodes the resulting sequence. In the paper's complexity model, decoding m reasoning tokens includes a vocabulary-projection cost of m·|V|·d_h; latent transfer costs m·d_h². Because the hidden dimension is much smaller than the vocabulary, the latent route avoids much of that projection work.

The calculation is only one part of the cost. A continuous representation must also be serialized through a discrete, autoregressive channel. Text can carry detailed reasoning, but the sender has to make that reasoning explicit as a sequence and the receiver has to reconstruct a useful representation. Those operations recur with every message and every collaboration round.

The connection is a small adapter called RecursiveLink: two linear layers, a GELU activation and a residual path.

An inner link maps a model's last hidden state back into its input embedding space: R(h) = h + W₂ σ(W₁h). The next forward pass consumes this vector rather than a generated token. The paper calls each step a latent thought. An outer link transfers a state between different models, adding a projection for mismatched dimensions: R(h) = W₃h + W₂ σ(W₁h).

The residual path lets the trainable branch concentrate on the shift between embedding spaces. In the reported ablation, this design scored 88.0 on MATH500, compared with 85.6 for the same two-layer adapter without the residual connection.

One recursive computation

The first agent reads the question and produces latent thoughts. An outer link converts them into the next agent's embedding space, where they accompany that agent's instructions. The final agent's latent output returns to the first agent for another round. Each model can then revise its contribution using the state produced by the whole chain in the previous pass. Text is decoded only from the last agent at the end.

This makes the agent chain resemble a recursive model whose layers happen to be separate LLMs. It extends work on looped and recursive language models, including LoopLM, which reuses model layers to add computation without adding parameters. The authors describe RecursiveMAS as the first application of that scaling approach at system level.

Training only the links

The LLM weights remain frozen. About 13 million RecursiveLink parameters are trained in two stages.

First, each agent's inner link learns to align its latent thoughts with the embedding of a ground-truth answer, using cosine regression. The complete chain is then unrolled for several rounds; cross-entropy on the final text answer propagates through all the links. Under the assumptions of the paper's Theorem 4.1, this differentiable latent path also preserves learning signals better across rounds than supervised training through increasingly confident token predictions.

The reported training cost was $4.27, updating 0.31% of the system's parameters. The paper compares this with $6.64 for LoRA and $9.67 for full supervised fine-tuning; average downstream accuracy was 74.9 for RecursiveMAS, 66.9 for LoRA and 68.6 for full fine-tuning. These figures describe the authors' experimental setup rather than a general price for adapting an arbitrary agent system, but they show how little of the model stack needed to change in this evaluation.

Results across nine benchmarks

The evaluation covers mathematics (MATH500, AIME2025 and AIME2026), science and medicine (GPQA-Diamond and MedQA), code (LiveCodeBench-v6 and MBPP+), and search-based QA (HotpotQA and Bamboogle). The key control is Recursive-TextMAS, which keeps the same agents, structure and recursion budget but communicates in text. That comparison is more informative than a leaderboard result alone because it isolates the communication channel within the authors' implementation. It still remains an experimental comparison: the result depends on the selected models, training data, task mix and recursion budgets.

Latent communication improved average accuracy by 3.4 points with one round and 7.2 points with three. On several benchmarks, text recursion declined after the second round while the latent system continued to improve. At three rounds, RecursiveMAS used 75.6% fewer tokens and was 2.4× faster because it did not decode and re-encode an intermediate transcript at every step.

The broader comparison includes fine-tuned single agents, Mixture-of-Agents, TextGrad and LoopLM. RecursiveMAS's largest margins over the strongest baseline were 13.4 points on AIME2025 and 10.0 on AIME2026.

Training and inference recursion also produced separate gains, with the strongest result when both used the largest tested number of rounds. More inference rounds helped even when the training depth was fixed, while deeper training improved results at a fixed inference depth. The grid therefore supports recursion as a test-time and training-time variable within the measured range; it does not show that performance will continue rising beyond it.

Different collaboration patterns

The authors tested four configurations rather than one fixed chain:

Gains appeared in all four. In the mixture configuration, the recursive system exceeded the strongest individual specialist by 6.2% on average. A small model in the distillation configuration retained a 1.5× speed advantage while recovering much of the larger expert's performance. Latent deliberation improved a tool-calling agent by 4.8%.

These experiments also connect model families with different embedding dimensions, including BioMistral and Qwen. Each pair still needs a trained outer link. The authors released code under the MIT license, checkpoints and training data, with links on the project page.

What the results do not establish

Latent communication removes the readable intermediate transcript. In a text-based system, an operator can inspect what a planner proposed or what a critic rejected, even if that transcript is not a perfect account of the models' internal reasoning. Here the native exchange is a vector. A hidden state can be decoded for inspection, but that adds cost and produces an approximation rather than a complete account of the internal representation. Selective decoding may help debugging, but the paper does not establish a mature interpretability method for these channels. This limits auditability, especially in regulated or high-consequence uses.

The adapters are also domain- and pair-specific. Every connection has to reconcile the embedding spaces of its sender and receiver, so replacing an agent requires training a new outer link. A text protocol can connect a new model through prompts as long as it understands the shared language. The reported low training cost does not make the latent system plug-and-play, and the experiments do not measure the operational cost of maintaining many links as models or domains change.

Most benchmarks have verifiable answers, a setting well suited to the final cross-entropy objective. The paper does not establish equivalent gains for open-ended writing, design work or long-running agents that use tools and change external systems. Its tool-calling experiment is narrower. Performance also flattened at roughly 80 latent steps per turn in the budget ablation. Latent transfer is therefore more efficient in this setup, but it does not provide unlimited useful collaboration depth. The saturation point may also vary outside the tasks and models included in the study.

RecursiveMAS therefore provides strong evidence for a particular claim: natural language is not required as the internal protocol of a multi-agent system. In the tested settings, latent transfer made recursive collaboration more accurate as well as cheaper. Whether the approach holds for production workloads depends on broader tasks, adaptable links and practical methods for inspecting what agents exchange.

Sources: Recursive Multi-Agent Systems (arXiv 2604.25917), project page, official GitHub repository, and the linked Hugging Face checkpoints and datasets.