Fireworks AI Breach Exposes Notion AI Chats: The Third-Party AI Risk Every Business Must Audit

Posted on 10th Nov 2026 11:04:49 in Artificial Intelligence, Machine Learning

Tagged as: Fireworks AI, Notion, AI Security, LLMJacking, Data Breach, AI Supply Chain

Fireworks AI, the Nvidia- and AMD-backed inference platform that powers AI features inside thousands of products, has disclosed a second security breach in under three weeks — and this one reached customer data at Notion, Cursor and other companies that rely on it. The incident is a case study in a risk most businesses have not yet priced in: the AI vendor sitting quietly in your stack is now part of your attack surface.

Fireworks notified affected customers on Monday, November 9, 2026, according to Bloomberg, which first reported the new breach. The company confirmed it is "actively working with our customers to manage the incident, including Notion, Cursor and others." Fireworks lists more than 10,000 customers, among them the legal AI firm Harvey, productivity platform Notion and coding company Cursor.

What makes this incident worth studying is not the size of the leak. It is the shape of it. Two separate intrusion paths, a criminal group monetising stolen model access, and a downstream customer — Notion — that had to make its own judgement call about how much to trust a vendor's assurance.

What Happened: Two Breaches, Three Weeks Apart

The first incident began on October 6, 2026, when Fireworks identified unauthorised use of internal Fireworks credentials. According to the company's own bulletin, an unauthorised third party accessed environment variables used in evaluation jobs inside a limited number of customer accounts. Fireworks began containment the same day, started notifying affected customers on October 7, rotated employee API keys on October 9, and published its initial bulletin the same evening. On October 10 it revoked additional user API keys and rotated service-account tokens identified during a credential review.

At that point the company said it had found no evidence of unauthorised access to customer models, datasets, training data, personal information or inference traffic. That assessment did not hold.

In a subsequent update, Fireworks disclosed that its investigation — conducted with Palo Alto Networks Unit 42 after notifying law enforcement — had uncovered impacts to account and billing information. For a limited set of accounts it also found impact to secrets, datasets and incomplete models. The affected internal application exposed resource metadata such as Fireworks resource identifiers, display names and descriptions; billing information including billing and shipping addresses, invoice and payment history, account balance and credit usage; and, for certain customers, a subset of incomplete model weights and datasets. Payment card numbers are not stored by Fireworks and were not affected.

Then came the second path. A third-party cloud provider that hosted Fireworks research and development machines suffered its own security incident on October 23. On October 24, an unauthorised party used that foothold to reach Fireworks development machines and ran an unauthorised workload that led to further customer data impact. Fireworks says it detected the activity the same day and closed known critical vulnerabilities by October 25.

Bloomberg's reporting adds the scale the bulletins do not: attackers gained access to tens of thousands of files, some containing customer data. The Information characterised the new hack as a threat to a business reportedly valued at $17.5 billion in July, now said to be raising at a valuation of $30 billion to $40 billion.

What Notion Actually Told Its Customers

Notion's disclosure is unusually specific, and it is the most useful document in this story for anyone who runs a SaaS product.

Notion said its security team determined that fewer than 1% of Notion workspaces may have been affected. Some "unattributable" chat sessions with Notion AI may have been exposed. Unattributable, in Notion's definition, means the chats were not labelled with who wrote them or which workspace they came from, and the transcripts did not contain tool call results such as Notion page content or external data pulled in by Notion AI. Beyond the words in the transcript itself, Notion says there is no information tying the chat back to a workspace, user or company, and no other data inside the workspace is affected.

The mechanism matters. Notion used Fireworks for two things: classification jobs behind an Insights feature that helps customers understand their own AI usage, and as a general inference provider serving open-weight models inside the product. When Fireworks warned of potential incidents earlier in the month, Notion shut down all connections to Fireworks systems. By November 3 it had disconnected its systems, deleted all data held in the Fireworks system, stopped sending traffic and halted all training runs. On November 6, Fireworks told Notion that malicious actors may have accessed potential customer data. On November 8, Fireworks confirmed the new dataset to Notion.

Notion's response was to disable the Insights feature and stop serving open-weight models through Fireworks entirely. It also gave affected workspace owners a way to retrieve and review their exposed data inside the product, with CSV exports for Enterprise plans and an annotated CSV adding metadata that was never part of the exposed set. And it told customers the thing most vendors avoid saying: treat this data as if it were exposed, and rotate any passwords, API keys or credentials that anyone pasted into Notion AI during the window.

That last instruction is the crux. "Unattributable" protects Notion from liability arithmetic. It does not protect a user who pasted a production API key into a chat box to debug a function.

The Threat Behind It: LLMJacking

Fireworks attributed the evidence to a financially motivated cybercriminal group aimed at selling tokens from unauthorised access to large language models through illicit routing gateways — a practice the industry calls LLMJacking. That attribution is not speculative. On November 2, 2026, Google Threat Intelligence Group and Mandiant published research documenting the tactic across cloud and LLM providers, with victims in financial services, retail, technology and transportation across North America, South America and Asia Pacific.

The economics explain the aggression. Unlike cryptojacking, where resource consumption creeps upward, LLMJacking can produce enormous fraudulent cloud charges within hours. Mandiant observed single-day fraudulent charges exceeding $100,000 in individual incidents, with cumulative charges at some organisations reaching $1 million before access was revoked.

Mandiant also described the division of labour that has emerged: one cluster establishes initial access, another operates "LLM Mixers" — illicit routing gateways that proxy, aggregate and resell unauthorised LLM access to underground subscribers. The dominant initial access vector across their investigations was not exotic. It was exposed or stolen cloud identity tokens: IAM credentials, service account keys, GitHub personal access tokens and active session tokens. In one mid-2026 case, an actor compromised a cloud instance through a misconfigured service account key. In others, stolen tokens were combined with web application vulnerabilities or used to programme fresh access keys that sidestepped MFA.

Read that alongside Fireworks' findings — environment variables including API keys and tokens stored as secrets, captured in evaluation jobs — and the pattern is the same one that has always worked: credentials sitting where they should not be, reachable by anyone who gets one hop inside.

What Fireworks Has Changed

Working with Unit 42, Fireworks says it has revoked and replaced credentials and service accounts, removed unnecessary elevated privileges, and tightened controls intended to prevent new unauthorised credentials from being created. It restricted unauthorised network pathways and isolated affected systems, reviewed access to data storage and analytics systems, and secured logging and monitoring.

It has also disabled self-serve dataset uploads and downloads used by Batch Inference and dataset-based training while the investigation continues. Inference, the Training SDK and Nexus remain available and are zero data retention by default — meaning prompts and outputs are not stored, which is why Fireworks says it found no impact to inference traffic. That architectural choice is the single biggest reason this breach is not far worse than it is.

What This Means for Indian SMEs and In-House Teams

Most Indian businesses will never buy from Fireworks directly. That is precisely the point. The lesson transfers through every "AI-powered" feature you add to your website, CRM, support desk, HR tool or accounting stack.

Notion is a sophisticated security organisation, and it still only learned the scope of the exposure four days after Fireworks first flagged possible data access. If your business treats an AI vendor's SOC 2 certificate as a substitute for its own controls, you are carrying risk you cannot see.

Five things are worth doing this month:

• Inventory every AI vendor, including the ones you did not buy. Your CRM's AI summary feature, your support desk's suggested replies, your billing tool's invoice extraction — each is a third-party AI dependency. Write them down with the data categories each one touches.
• Never let customer data leave your boundary as raw prompt text. Notion's exposure was chat transcripts. If your team pastes customer records, contracts or database rows into a chat window to get work done, those transcripts exist somewhere else. Redact or use a vendor whose contract guarantees zero retention.
• Assume credentials pasted into AI tools are compromised. Rotate anything that has been typed into a chat interface. Removing a stored secret does not invalidate the credential — you must act with the issuing provider. This is the step most teams skip.
• Ask for the ZDR guarantee in writing. Zero data retention on inference is not a marketing line; it is the difference between an incident that exposes metadata and one that exposes your prompts. Fireworks' own bulletins show the gap.
• Have an exit switch, and test it. Notion could disconnect from Fireworks, delete its data and halt training runs within days because it knew exactly which integrations existed. If you cannot answer "what breaks if we disable vendor X right now," you cannot respond to a breach.

The Bigger Question Nobody Is Asking

Fireworks is investigating, cooperating with law enforcement, and hardening its platform. Its customers are rotating credentials. The immediate incident will resolve. But the structural question remains: the AI infrastructure layer consolidating into a handful of inference providers is becoming a single point of failure for thousands of products, and the security maturity of that layer is being stress-tested in public, twice in three weeks.

For a small business in Mumbai or Pune building AI features into a client product, the practical takeaway is not to avoid third-party inference. It is to stop treating the AI vendor as infrastructure and start treating it as a supplier with a security obligation you have documented, reviewed and rehearsed. Ask what happens on a Tuesday evening when your inference provider emails to say customer data may have been accessed. If the answer is a scramble, the breach already happened — you just have not been told yet.

Sources

• Fireworks AI — October 2026 Security Incident Bulletin
• Fireworks AI — Update on October 2026 Security Incident
• Notion Help Center — Notice about the Fireworks AI incident
• Google Threat Intelligence Group and Mandiant — The Rise of LLMJacking
• UNN, citing Bloomberg — Fireworks AI Hacked for the Second Time in Several Weeks
• Bloomberg — Nvidia-Backed Fireworks AI Notifies Customers of Its Second Hack in Weeks