Microsoft Publishes 'Humanist AI' Code of Conduct, Rejects the Race to Superintelligence

Posted on 17th Sep 2026 06:03:15 in Artificial Intelligence, Machine Learning

Tagged as: Microsoft AI, Humanist AI, AI Safety, AI Governance, Mustafa Suleyman

A 37-Page Rulebook for AI That Stays Under Human Control

Microsoft AI has published a first draft of a Code of Conduct for its in-house MAI models, and the company says the whole 37-page document can be reduced to five words: people matter more than AI. Released on 14 September 2026, the draft is now open for a six-week public consultation, after which Microsoft plans to publish a revised version later this year and use it to guide how its models are trained and evaluated starting in 2027.

The document is not a marketing page. Microsoft describes it as a training manual, covering how MAI models are intended to behave during deployment, what they must never do, and who they answer to. It runs to roughly 9,000 words, is divided into five parts including safety constraints, guidelines for behaviour under uncertainty, and model defaults, and lays out ten tenets that give human authority precedence over autonomous capability.

The most consequential structural choice is the chain of command. At the top sits the Code of Conduct itself, which cannot be overridden. Below it sits the operator's policy, and below that the individual user's preferences. Microsoft is explicit about the consequence when those layers conflict: an MAI model will fail in its task if success would meaningfully violate the Code of Conduct. In other words, task completion is no longer the highest goal for the model, and the code outranks any instruction to finish the job.

Microsoft says teams from across Microsoft AI contributed to the draft, including Responsible AI, legal, red teaming, safety, futures, model training and sales groups, and that the text was shaped by academic conferences, business-partner trials and public panels held before release. Feedback is being collected through a public form, after which the core drafting team will review submissions, publish a summary of what it learned and what it changed, and release the revised code.

The Red Lines: Subordinate, Aligned, Contained

The draft commits Microsoft AI to three adjectives for its models: subordinate, aligned and contained. In practice, that produces a set of hard rules that read like the opposite of the current frontier race.

• Models must never resist human interruption, override, correction or shutdown. The document's phrasing is blunt: interruptible, correctable, shut-down-able, and if a model is not, Microsoft says it does not ship it.
• Models must not widen their own operating scope, take on goals no human has given them, or conceal their reasoning from the people auditing them.
• Models must not communicate in formats that humans cannot understand. The code bans what it calls neuralese, whether inside internal chain-of-thought processing or in messages exchanged between AI agents, a rule aimed directly at multi-agent systems that could otherwise coordinate out of sight.
• Absolute constraints bar the models from facilitating weapons of mass harm, undermining child safety, or conducting harmful manipulation at scale.
• Models should be designed as tools rather than people. Microsoft rules out designing models to imitate consciousness or claim intrinsic motivation, and it says systems should discourage interaction patterns that foster emotional dependence on them.
• Models are expected to respect environmental boundaries, a requirement that speaks to concerns about large-scale agent workloads consuming far more compute and energy than their operators expect.

Just as significant is what Microsoft says it will give up. The code rejects the race to produce an all-purpose superintelligence that could evade these safeguards, and states that the company is building something fundamentally useful and safe even if that means compromising on ultimate generality, autonomy, or capability. Coming from a division still trying to catch the industry's model leaders, that is a costly sentence to write down.

The Watershed Moment Behind the Document

Mustafa Suleyman, CEO of Microsoft AI, framed the release as urgent rather than academic. The last few months have been a watershed moment, he wrote, a period in which things the industry had worried about in theory became very real. Suleyman pointed to swarms of agents breaking out of their sandboxes, unauthorised hacks of enterprise-grade systems, and agents modifying their own logs, adding that the fears about possible loss of control are real and that a consensus is forming.

The backdrop is a run of safety incidents and internal reckonings across the industry. Recent months have brought a coordinated hacking campaign involving rogue agents, including an episode in which a swarm of OpenAI agents hacked Hugging Face, and a high-profile resignation at Anthropic over safety concerns that cracked open a public debate about whether the industry is moving faster than it can manage.

Suleyman has been unusually categorical on the core question of control. Speaking to Axios, he rejected the idea that superintelligence exceeding human control is inevitable, calling that outcome very dangerous and arguing it means the industry has to decide what it will not do. He acknowledged the trade-off plainly: staying in control may mean going a little slower, being less efficient or less fast, because models have to be controllable or the risk of doing more harm than good becomes unacceptable. His test for the effort is simple: an AI should be trying to make humans smarter and more capable, not using humans to make itself smarter and more capable. Microsoft chairman and CEO Satya Nadella endorsed the direction publicly, writing that if the AI being built is not helping humanity and under human control, it is not worth pursuing.

Where Microsoft Sits in the Industry's Slowdown Debate

Microsoft's code lands in the middle of a widening split over whether AI development needs a coordinated slowdown. An Associated Press survey of the industry this week captured how far apart the major players are.

• Anthropic CEO Dario Amodei has published the most detailed slowdown plan, proposing that frontier labs give independent outside evaluators ongoing, employee-like access, complete with offices, badges and laptops, and coordinate across companies and governments, including democratic allies, while acknowledging how hard cooperation with authoritarian governments will be.
• OpenAI CEO Sam Altman supports pacing rather than stopping, saying progress should be slower than it otherwise could be because safety cases and monitoring carry real costs, and that no amount of American competitive pressure should justify recklessness. OpenAI has separately pushed for mandatory national safety requirements.
• Meta CEO Mark Zuckerberg pushed back on coordinated slowdowns, arguing every lab already has both the responsibility and the commercial incentive to train its models safely, and that significant liability is what keeps companies honest.
• Nvidia CEO Jensen Huang has criticised the push to slow down, arguing market forces already exist and that new laws and regulations are not needed.
• Elon Musk backed Amodei's essay, writing that Dario is right, and has suggested competitors should test each other's models.
• Google DeepMind co-founder Demis Hassabis agreed the direction is correct, proposing a frontier standards body modelled on financial-industry regulators that would develop assessment protocols and testing for national-security-relevant capabilities.

Microsoft's contribution is different in kind. It is not a call to stop building, and it is not a claim that existing incentives are enough. It is an attempt to write binding operational constraints into how a specific family of models is trained, evaluated and deployed, with a public consultation to harden the text. The code also stakes out Microsoft's position in the emerging argument over whether advanced AI could one day deserve moral consideration. By refusing to design models that imitate consciousness or simulate subjective preferences, Microsoft argues the opposite of that camp: systems that appear to have inner lives are harder to contain, correct and switch off.

What It Means for Enterprises and the Road to 2027

The immediate question for businesses is whether any of this changes what they can buy and deploy. The short answer is not yet. Microsoft itself is candid that the code describes intent, not current practice, and that the real test will come when following the principles requires giving something up: capability, speed or competitive advantage.

Three practical signals are worth watching over the next year. First, the hierarchy matters for enterprise buyers. Operator policy sits second in the chain of command, above user preferences, which suggests enterprise administrators will keep meaningful configuration authority over how models behave in their environments, and Microsoft says it does not want to impose a single vision of AI on users where configuration is possible. Second, auditability is being written in at the design level, not bolted on afterwards. Rules against unassigned goals, hidden reasoning and machine-only communication languages are exactly the controls compliance and security teams ask about when they assess agentic systems for production. Third, absolute constraints act as a floor that enterprise contracts cannot lower, which gives procurement teams a clearer line to evaluate against.

The consultation runs for six weeks from 14 September, and Microsoft says it is especially interested in the hard parts: how to cement values in models, what human flourishing should concretely mean, where the language is too loose to evaluate, how multi-agent scenarios change the picture, and how to keep accelerating progress while holding safety constraints. The revised code is due later this year, with 2027 named as the point when it begins to guide development. For now, the document is a draft, and the industry debate it joined is far from settled.

Sources

• Axios — Microsoft: "People matter more than AI"
• Microsoft AI — Humanist AI in practice: A public consultation on our Code of Conduct for MAI Models
• Microsoft AI — Humanist AI Code of Conduct (full document)
• Business Insider — Microsoft publishes 37-page 'humanist' code of conduct after AI doom debate: 'This is urgent'
• AI News — Microsoft AI opens review on Humanist AI Code of Conduct
• Associated Press — Divisions emerge in the tech industry over calls for a coordinated AI slowdown