NVIDIA explains two Cosmos paths for driving simulation

NVIDIA’s November 2 guide explains how released Cosmos models can generate synchronized camera views and LiDAR observations for driving simulation. It distinguishes preparing synthetic scenes in advance from evaluating a driving policy, the system that chooses driving actions, in a loop. In that loop, updated scenario controls guide generation, and the resulting observations return to the policy. The examples start from released model checkpoints.

The Transfer-Auto description documents the offline option. Text specifies weather, lighting and time of day, while World Scenario Map inputs describe road structure through controls such as lane markings and object shapes for each camera view. This lets developers vary appearance within the same driving scenario when preparing training or evaluation data. Generating LiDAR additionally requires aligned map controls; initial camera images are optional. Adapting the model to a different sensor rig requires further training. The card dates its release to October 20.

The Sim-Auto description covers the interactive option. A renderer converts inputs from an automated driving policy or a human steering wheel into scene controls. The model generates small chunks using stored history and incoming controls, so subsequent observations can reflect changed driving inputs. Its generation speed depends on hardware and configuration; video playback rate is a separate quantity. For a custom sensor rig, the documented order is to train Transfer-Auto first, then derive the incrementally generating Sim-Auto variant from it.