# RoboJev-27B: An Open-Source Vision-Language Model for Judging Robot Task Success

- Source: arXiv cs.RO
- Published: 2026-10-27T04:00:00.000Z
- RoboHot: https://robohot.news/en/items/nf62rm4hq29rfr0tq84xcfrl1
- Original: https://arxiv.org/abs/2610.33718

## Summary

RoboJev-27B is an open-source vision-language model that takes a task instruction and 10 frames of a manipulation attempt as input and predicts whether the robot completed the task. It achieves 95.2% balanced accuracy and 0.993 AUROC on 268 held-out episodes of the same robot model (SO-101), and an AUROC of 0.890 on 459 episodes from unseen labs or robots. Median inference time is 0.26 seconds per episode on a single H100, and the model, inference code, and evaluation results for 20,628 predictions are released together.