Nvidia releases Flora models for labeling robotics video

Nvidia’s Flora project addresses several jobs in robotics data preparation. Descriptions tied to video times help filter and balance pretraining data. Labels assessing intermediate actions support reinforcement learning, while outcome labels describe whether an episode succeeded. Nvidia presents the release as a way for researchers to scale their own robotics datasets.

The captioning model receives a target interval chosen by the caller. Neighboring events before and after it provide context for interpreting the local operation. The model returns one English action instruction; the surrounding pipeline attaches that text to the supplied time interval. It does not choose or revise the interval’s boundaries. This distinction matters when preparing inputs: temporal context helps describe the selected action, rather than adding actions outside it.

The reward model takes video frames and a prompt specifying the annotation task. It can score an episode from complete failure to complete success, or label short steps as failed, suboptimal or normal, with explanations for non-normal steps. Its documentation limits intended use to research and development: the outputs are annotations, not direct robot control or a sole safety monitor. Commercial and production use are not supported.

The Decoder reported the release and linked Nvidia’s documentation.