FocusGUI: A Focused Perception Adapter for Omni-Modal GUI Agents

FocusGUI proposes a lightweight adapter that achieves focused perception for omni-modal GUI agents by dynamically weighting multimodal channels at each step, compressing key evidence into a compact latent, and injecting it into the frozen decoder's embeddings. Across four omni-modal backbone models, it raises Type Match by an average of 33.2 and Exact Match to 3.0x relative to the frozen backbone, and improves Type Match by 30.1 over the strongest existing enhancement method. Training relies only on action supervision, requiring neither modality-relevance labels nor backbone fine-tuning.