Event

NimCritic Introduces Action-Aware Quantification and Long-Horizon Progress Benchmark

1 reports · 1 sources

Tianchi Xu and other authors propose an action-aware quantification method and build the NimCritic progress model based on it. The method uses a vision-language model to decompose long-horizon manipulation trajectories into object-level subtasks and assign weighted action credit, so as to characterize fine-grained execution dynamics. The authors state that, using VLM automatic annotation plus manual calibration, they built a training video dataset of more than 800 hours, as well as LOOM, a long-horizon progress benchmark containing 19,618 manually calibrated evaluation frames. The authors report that NimCritic matches or outperforms existing process reward models on VPB and LOOM, and is about 65% higher than GVL (GPT-6-Sol) on terminal success accuracy on VPB; on the RoboArena policy evaluation benchmark, its overall recall for retrieving human-selected successful trajectories is about 97%. The above performance figures are all results self-reported by the authors.