Event

AnyRank Reliably Ranks Robot Policies Using Sequential Testing

1 reports · 1 sources

Apurva Badithela et al. propose AnyRank, a sequential method for ranking multiple policies: it adaptively allocates trials to policies that are hard to distinguish, produces a (partial) ranking at any stopping time, and simultaneously bounds the probability of incorrect ranking. The method can optionally leverage prior information (such as expert intuition, small held-out hardware evaluations, simulation, or policy rollouts from an action-conditioned video world model) to improve hardware evaluation efficiency; the proposers state that inaccurate priors do not break its statistical guarantees.