Event
OpTTA: Incorporating Planning Objectives into World Model Adaptation
1 reports · 1 sources
Akshay Mete, Dileep Kalathil, and P. R. Kumar propose OpTTA, which adds the planner's goal-achievement cost term to the test-time adaptation objective of world models, making the model optimistic about reaching the goal rather than optimizing only prediction accuracy. Under visual and environmental shifts in manipulation, reaching, and navigation tasks, when adapting only the predictor, OpTTA improves the final planning success rate by an average of 13.5 percentage points over the frozen model, compared with 6.9 percentage points for AdaJEPA, which uses only the prediction objective; when adapting the encoder as well, the two achieve 13.5 and 7.5 percentage points, respectively. The experiments cover three JEPA world models with different encoders and training objectives, and the temporal straightening part uses both CEM and gradient-based planners.
Report timeline
- OpTTA: Test-time adaptation of world models for planning, not just prediction
The paper proposes OpTTA, which adds the planner's goal-reaching cost term to the test-time adaptation objective for world models, keeping the model optimistic about reaching the goal. Under visual and environmental shifts in manipulation, reaching, and navigation tasks, when only the predictor is adapted, OpTTA raises the final planning success rate by an average of 13.5 percentage points, compared with 6.9 percentage points for AdaJEPA, which uses only the prediction objective; when the encoder is also adapted, the two achieve 13.5 and 7.5 percentage points, respectively.
arXiv cs.RO ·