Statistical leverage score experience replay method proposed

Shaifalee Saxena and Alexander Scheinker propose a diversity-aware experience replay method: each transition in the replay buffer is scored using a statistical leverage score, which measures how unusual it is relative to the buffer's own data. The score is computed solely from stored values, without using labels, rewards, or a value network; mini-batches are then drawn by mixing this score with uniform sampling, to alleviate the under-sampling of rare events in robot offline experience. When actions are stored as one-hot encodings, the method instead uses the feature column with maximum kurtosis (heaviest tail); and it abandons the importance-weight correction of prioritized replay, directly optimizing the rebalanced objective. For rare situations related to the target, the same tail rule is applied to short trajectory windows. The authors report that on a three-joint robotic arm task requiring pushing an object around an obstacle, the method raises the success rate on rare tasks from 0.49 to 0.69, while the labeled oracle upper bound is 0.74.

AI 根据报道生成 · 6 days ago更新