NVIDIA Cosmos 3 Edge Brings Frontier World Models to Edge Devices for Physical AI
Posted on 6th Aug 2026 06:03:07 in Artificial Intelligence, Machine Learning
Tagged as: NVIDIA, world models, robotics, edge AI, physical AI, SIGGRAPH
At this year's SIGGRAPH conference in Los Angeles, NVIDIA released Cosmos 3 Edge, a 4-billion-parameter world model purpose-built to run in real time on edge devices. The model, unveiled during the NVIDIA keynote on July 20, extends the open Cosmos 3 family launched in May, and gives robots, autonomous vehicles and smart-infrastructure systems the ability to understand a scene, reason about what happens next and generate actions locally — without relying on a cloud connection.
World models are becoming the backbone of physical AI: systems that perceive, predict and act in the real world. Until now, most of these models were too large and too slow to run on the constrained hardware found inside robots and cameras. Cosmos 3 Edge is designed to close that gap, delivering data-center-grade reasoning on memory-constrained edge systems, and it is openly available on Hugging Face alongside the rest of the Cosmos platform.
What Is Cosmos 3 Edge?
Cosmos 3 Edge is a compact open world model that understands and generates text, images, video, ambient sound and action — an "omnimodal" capability that lets a single backbone handle perception, prediction and control. According to NVIDIA, the model ranks first among similar-size 4-billion-parameter models on VANTAGE-Bench for vision analytics and sets a new state of the art for robot policy learning.
Unlike the Cosmos 3 Super and Cosmos 3 Nano models introduced in May, Cosmos 3 Edge was trained from scratch rather than initialized from pretrained vision-language weights. The result is a model that can be post-trained on proprietary robot and sensor data using an NVIDIA DGX Station deskside supercomputer, then deployed on Jetson Thor or the newly introduced Jetson T2000 and T3000 edge modules for real-time control.
NVIDIA says developers can adapt the base model to specific robots, vehicles, sensors and operating environments in about a day using the open Cosmos framework, which includes reference post-trained checkpoints, training recipes and datasets published on Hugging Face and GitHub.
How the Model Works
Cosmos 3 combines two transformer towers that share a common representation. An autoregressive tower processes vision and text tokens for understanding and reasoning, while a diffusion tower processes vision, audio and action tokens for prediction, generation and neural simulation. The two towers maintain separate normalization layers and multilayer perceptrons but share multimodal attention layers that align information across language, video, audio and action.
This architecture lets the model reason about a scene before generating an output. Depending on the task, Cosmos 3 can produce reasoning tokens from the autoregressive tower or denoised video and action tokens from the diffusion tower. Language tokens use causal attention, while diffusion tokens attend more broadly to the available context, supporting coherent prediction and generation.
Physical systems describe actions differently: a vehicle moves through ego pose, a camera moves through camera motion, and a robot arm moves through the pose of its end effector, while a gripper must also track grasp state. Cosmos 3 maps these different embodiments into a common action representation — compact geometric vectors that capture translation, rotation and manipulation state. As NVIDIA's Ming-Yu Liu put it during the keynote: "Every embodiment speaks a different language. Our solution is to build a common vocabulary."
In policy mode, the model predicts an action together with its expected visual consequence, connecting world modeling directly to robot policy training and evaluation. Action can flow through the model in both directions, allowing Cosmos 3 Edge to predict the effects of an action or infer an action from its effects.
Real-Time Robotics at the Edge
As a post-trained world action model, Cosmos 3 Edge operates at robot-control resolution of 640 by 360 observations, generating 32 actions per inference on Jetson Thor while achieving real-time control at 15 hertz. That makes it suitable for manipulation and locomotion policies that must react to a changing environment without round-trip delays to a server.
NVIDIA is also releasing Cosmos 3 Edge Policy (DROID), a robot manipulation policy post-trained on the DROID dataset for pick-and-place tasks, with accompanying post-training scripts. Developers can use a small cluster of H100 GPUs or a DGX Station to fine-tune Cosmos 3 Edge for their target workloads before deploying it to edge and accelerated computing platforms.
A growing set of robotics companies is evaluating the model for production workflows, including Agile Robots, Doosan Robotics, Siemens and Skild AI. For smart infrastructure, the model enables real-time inference on Jetson Thor for vision agents that reason across live video streams for traffic monitoring, public safety, logistics and industrial inspection.
Autonomous Vehicles and Beyond
For autonomous vehicles, Cosmos 3 Edge supports road-scene understanding, traffic reasoning, object-intent prediction and policy-model distillation on resource-constrained hardware. The model can serve as a student backbone for automotive policy distillation, including with NVIDIA Alpamayo vision-language-action models, helping carmakers compress large driving policies into models small enough to run in production vehicles.
The release also came with Cosmos-Dreams, a collection of closed-loop simulators. A demonstration showed a simulator built for autonomous vehicles that generates an entire world from a single frame, running on a single RTX PRO 6000 GPU. Developers can use Cosmos-Dreams to verify model accuracy before deployment in a real fleet, or to train models on AI-generated scenarios that are difficult and expensive to reproduce in the physical world.
An Open Platform and a Growing Ecosystem
Cosmos 3 Edge is part of the broader NVIDIA Cosmos platform for developing physical AI world models, which is now openly available. Alongside the edge release, NVIDIA announced that a group of Japanese manufacturing and technology companies — including Fanuc, Yaskawa Electric, Fujitsu, Hitachi, Kawasaki Heavy Industries, Kubota, NEC, SoftBank, Sony and AIRoA — intends to join the NVIDIA Cosmos Coalition to help build open physical AI models.
Fujitsu is exploring a collaborative control platform for physical AI, while Fanuc, Yaskawa Electric and Kawasaki Heavy Industries are integrating NVIDIA technologies into their robotics systems. Another group, including Honda R&D, Omron, Mitsui, Telexistence and Enactic, is building applications on the Cosmos platform for retail automation, elder-care robotics and industrial inspection. Jensen Huang framed the initiative in the context of Japan's manufacturing heritage: "Japan invented modern manufacturing. Now, it has the opportunity to reinvent it for the age of intelligent industries."
The shift to on-device world models matters for three reasons. First, latency: robot control loops cannot wait for a cloud round trip. Second, privacy and security: sensitive video and sensor data stays on the device. Third, cost: edge inference avoids the constant bandwidth and compute bills of streaming everything to a data center. As world models shrink and edge hardware improves, physical AI systems — from factory robots to traffic cameras to autonomous cars — are moving from cloud-dependent experiments to self-contained, real-time deployments.