RoboJev-27B: An Open-Source Vision-Language Model for Robot Manipulation Success Detection

RoboJev-27B is an open-weight vision-language model that takes a task instruction and 10 frames of a manipulation attempt to judge whether the robot completed the task. It achieves 95.2% balanced accuracy and 0.993 AUROC on 268 held-out episodes of the same robot model (SO-101), and an AUROC of 0.890 on 459 episodes from unseen labs and robots. Median inference time is 0.26 seconds per episode on a single H100, and the model, inference code, and evaluation results for 29 model/protocol entries are released together.