Sakana AI's MASS: A 27B Open Model Improves Its Own AI Team — and the Blind-Spot Risk

Posted on 12th Oct 2026 12:05:44 in Artificial Intelligence, Machine Learning

Tagged as: Sakana AI, MASS, Self-Improving AI, AI Agents, AI Research

Researchers at Japan's Sakana AI, working with the University of California, Berkeley, have demonstrated a deceptively simple idea: an AI model can improve itself by running a team of AI workers, studying what its own team did, and training on the work it judged to be best. The method is called Multi-Agent Self-Supervision, or MASS, and the paper landed on arXiv on October 8, 2026. In two rounds of this loop, a single open-weight, 27-billion-parameter model — Qwen3.6-27B — improved its performance per token by up to 1.6 times on research benchmarks. The researchers are equally clear about what did not work: on two coding benchmarks, the model slipped slightly below where it started.

The result arrives in the middle of a loud industry debate about whether today's AI systems can meaningfully improve themselves, and what happens if they can. MASS does not settle that debate — it is a small-scale, carefully measured lab demonstration — but it is one of the clearest practical recipes so far for self-improvement on tasks where no automatic "right answer" check exists.

How MASS Works: One Model, Two Loops, Three Roles

Most self-improvement experiments need a strong supervisor: humans, a verification tool, or a larger model to check the work. MASS removes that crutch. In what the authors call a homogeneous loop, the same model plays all three roles — the executor (a team of agents solving the task), the workflow optimizer (which redesigns how the team works), and the evaluator (which judges the results).

The process runs in two nested loops. In the inner loop, the model evolves a written workflow — a plan describing each subagent's role, instructions, the contract of what it must return, and the order in which agents are called. A finance example from the paper shows the flavor: the data specialist's contract requires explicit checks that a backtest does not let historical decisions peek at future information, a classic way a result can look convincing while being wrong. The team executes the workflow, the model compares the new workspace against the best one so far, and then proposes the next version. The winner is kept as the champion.

In the outer loop, the model turns the team's work into lessons for itself. It runs the champion workflow 18 times per training task, ranks the workspaces produced, and keeps up to 16 execution records — 15 for training, one for validation. Then it strips the workflow text out of the training prompt — forcing the model to learn organization from the task itself — and fine-tunes the shared model with LoRA on both the orchestrator's and the subagents' conversations. The updated model then returns to all three roles, and the next round begins.

The experimental setup: a 12-task synthetic research suite spanning finance, robotics, and pharmacy, each task requiring code, quantitative results, and a report. Eight tasks supplied training data and three were held out. The full paper runs 39 pages, the base model was Qwen3.6-27B-FP8 in the qwen-code agent environment, and the code is open on GitHub.

What Two Rounds of Self-Training Achieved

The numbers — judged by outside referees, GPT-5.5 and Claude Opus 4.8, whose verdicts were kept out of the training process — show real movement. On the three held-out tasks, the model's win rate against its own starting point rose from 53.9 percent after one round to 69.9 percent after two, and the second-round model beat the first-round model in 60.8 percent of comparisons. On the training tasks, win rates climbed from 65.6 percent to 82.5 percent.

On public benchmarks, the standout metric is efficiency. Score per output token reached 1.60 times the base model's level on MLR-Bench, 1.35 times on DSBench, 1.28 times on AstaBench, and 1.20 times on ScienceAgentBench. On MLR-Bench, the absolute task score climbed from 1.58 to 2.40. In other words, the model got more done with less generated text — a meaningful cost signal in an industry where tokens equal money.

The loop also improved roles it was never directly trained for. MASS fine-tunes only on task-solving conversations, yet the workflow optimizer and the evaluator both got better on their own: the model's agreement with the external judges climbed from 73 percent to 93 percent across generations. The authors call this role generalization, and it is what makes the recursion self-sustaining rather than a one-off trick.

Then there is the honest part. On Terminal-Bench 2.0 and SWE-bench Verified — two hard software-engineering tests — score per output token landed at 0.96 and 0.94 times the baseline, meaning the model became slightly less efficient at coding after training. The researchers believe adding software-engineering tasks to the training mix would address this, but that is a hypothesis, not a demonstrated result. As Superpower Daily put it in its write-up, the findings do not yet show that research-task gains extend to software engineering.

One more finding deserves a highlight for anyone building agent systems. When the researchers swapped in GPT-5.5 to assist, upgrading the evaluator alone helped a little, but upgrading the optimizer as well found winning workflows for all 12 tasks within five search iterations. Their interpretation — that the bottleneck in self-improvement can be the optimizer, not the evaluator — is a practical lesson for teams designing multi-agent pipelines today.

Why This Matters: The Race to Build AI That Improves AI

Recursive self-improvement — a system that bootstraps its own capabilities — has moved from thought experiment to research agenda. Recent analyses, including a 2026 paper on whether automating AI research could trigger an intelligence explosion (cited by the MASS authors), have put the question in front of labs and regulators alike. The MASS authors frame their work around what they call AI's Odyssean moment: the stage at which continued learning must proceed without a stronger supervisor in the room, because the model's outputs already exceed what human experts can reliably assess.

What makes this week's result notable is the scale and the openness. The improvement loop ran on a 27B open-weight model — the class of model a well-funded startup or university lab can actually run — and the code was released. If self-supervision loops work at this size, they become an option not just for frontier labs but for anyone with a modest cluster and a solid orchestration stack.

There is an efficiency angle too. The study found that multi-agent traces are a denser training signal than single-agent ones: a student trained on multi-agent trajectories beat a single-agent student that consumed about 1.4 times more training tokens, reaching a 68.3 percent win rate at 28.8 million supervised tokens versus 64.0 percent at 42 million. Learning how to organize work — not just how to do it — appears to be the more valuable lesson.

The Catch: Self-Judging Loops and Shared Blind Spots

The paper does not hide the problem at the center of its own method. When the model's judgments both guide training and go largely unchecked, the correctness of the training signal is unknown. A flawed result could be produced, accepted as an improvement, and folded into the next generation — and if the weakness that produced it is shared across the loop, each iteration can reinforce it.

The authors are direct about the stakes, writing that shared blind spots may be an Achilles' heel of recursive self-improvement, and that the same weakness can compromise both the system's behavior and the mechanism meant to correct it. Their proposed guardrails are the ones enterprises should note: independent checks outside the loop, constraints on what autonomous systems may do, and clear conditions for human intervention. They also connect the work to OpenAI's research on whether models' chain-of-thought reasoning can be monitored for misbehavior — useful, but, as that research itself notes, incomplete evidence.

None of this amounts to a runaway machine. Two rounds, a synthetic suite of a dozen tasks, and a mid-size model is a careful lab result, not an intelligence explosion. But it is the mechanism class that safety researchers have been warning about, demonstrated at a scale many organizations can now replicate.

What It Means for Developers and Small Teams

For developers and smaller companies — including the many AI-curious firms in India building on open models — the practical takeaway is that orchestration itself is becoming a learned skill. Open weights, released code, and LoRA-based fine-tuning put experiments like this within reach of modest budgets. The cost story matters more than the headline: when a model achieves the same result per token, generation after generation, the price of every agent workflow built on top of it trends down.

The caveat travels with the opportunity. Any team building self-improving or self-evaluating pipelines should keep an evaluation layer the system cannot edit, log what changes between rounds, and treat benchmark numbers — especially score-per-token — as directional signals rather than proof. The researchers' own honesty about coding regressions is the template: measure, publish, and fix by adding data, not by trusting the loop blindly.

Sources

• Sakana AI — MASS: Recursive Self-Improvement through Multi-Agent Self-Supervision
• arXiv — Recursive Self-Improvement through Multi-Agent Self-Supervision (arXiv:2610.12176)
• Superpower Daily — Sakana AI reports research gains from self-trained AI teams, but coding efficiency slips
• DAIR.AI Academy — Recursive Self-Improvement through Multi-Agent Self-Supervision
• GitHub — SakanaAI/mass: MASS code and models