Event

Scale AI and Elorian Release HSS Benchmark, Models' Visual Intuition Far Below Humans

1 reports · 1 sources

Scale AI, in collaboration with Elorian, released a new benchmark, Humanity's Sixth Sense (HSS). In evaluations testing multimodal models' intuitive visual reasoning, 20 human participants scored 93.1%, while the top model, GPT-6-astra, scored only 53.6% even at maximum reasoning effort; the median across 25 models was 30.9%. HSS contains 522 open-ended tasks, corresponding to 288 images and 234 videos; models answer freely with no options to eliminate, and Claude-Opus-5 scores each response item by item against the scoring criteria, with a response counted correct only if all criteria are met. The research team tested 25 multimodal models from 8 vendors. On a 388-question subset, Claude-Opus-5 rose from 30.8% to 51.3%, and Claude-Fable-5.1 rose from 41.2% to 57.7%; GPT-6-astra achieved 59.3% in Codex under the Agent setting, the highest score in that setting.