Article

AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-Time Adaptation in Evolving Environments

AdaExploreBench introduces a benchmark that instantiates environmental dynamics as hidden rules, requiring LLM agents to actively acquire evidence and infer the rules, and, under implicit rule drift, to detect that prior knowledge has become stale and re-explore. Experiments on eight leading LLMs show that GPT-6 Sol achieves only 61.9% calibrated accuracy; providing sufficient evidence or removing rule drift improves average performance by close to 10%. The results indicate that agents can reason from valid evidence but struggle to actively acquire evidence and to recognize when evidence becomes invalid.

Part of the event: AdaExploreBench Released to Evaluate Agents' Active Exploration Capabilities

This page provides a summary and source link only.