Event
FEC: Conditioning Robot Policies on Latent Representations of Future Video
1 reports · 1 sources
Mohammad Khoshnazar and other authors propose the Future Experience Conditioning (FEC) method: using the latent representation of a short future video clip as a condition for a closed-loop policy, for exploration and policy adaptation in multi-step robotic manipulation. The authors state that, in simulation settings, the future clip is generated in three steps: an LLM reasoner initializes task ontology reasoning based on the current scene state, a robot-free digital twin simulates the target object's motion, and a mask-free video diffusion model synthesizes a future clip consistent with the robot; no segmentation is needed at inference time. On RoboCasa and CALVIN, the authors compare FEC with the future-conditioned Streaming Flow Policy (SFP) baseline, under test conditions including NoFuture, GTFuture, GenFuture, and WrongFuture. These results come from simulation experiments; effects on real robots are not yet stated.
Report timeline
- LLM-Guided Future Hypotheses for Horizon-Aware Exploration in Multi-Step Robot Manipulation
The paper proposes Future-Experience Conditioning (FEC), which uses a latent representation of a short future video as a condition for closed-loop policies, for exploration and policy adaptation in multi-step robot manipulation. Future clips are generated in three stages: an LLM reasoner performs task-ontology reasoning initialized from the current scene state, a robot-free digital twin rolls out the intended object motion, and a mask-free video diffusion model synthesizes a robot-consistent future clip.
arXiv cs.RO ·