Article

RoboJev-27B: An Open-Source Vision-Language Model for Judging Robot Task Success

RoboJev-27B is an open-source vision-language model that takes a task instruction and 10 frames of a manipulation attempt as input and predicts whether the robot completed the task. It achieves 95.2% balanced accuracy and 0.993 AUROC on 268 held-out episodes of the same robot model (SO-101), and an AUROC of 0.890 on 459 episodes from unseen labs or robots. Median inference time is 0.26 seconds per episode on a single H100, and the model, inference code, and evaluation results for 20,628 predictions are released together.

Part of the event: RoboJev-27B Open-Weight Vision-Language Judgment Model Released

This page provides a summary and source link only.