Article

Giving Models a Finger: Developmentally Aligned Evaluation of Counting in Vision-Language Models

A study of 9 vision-language models, 682 images, and 7 evaluation conditions found that models' exact-counting ranges differ by nearly two orders of magnitude (R80 from 3 to 150 dots). Using a training-free agent loop that lets models mark images to aid counting, the study found this method mostly failed as an intervention but produced auditable serial reasoning traces that help reveal how models arrive at answers. Experiments also showed that models often change answers when traversal order is reversed, and that a child-like advantage from regular arrangements appeared only under specific conditions.

Part of the event: Research on Developmentally Aligned Evaluation of VLM Counting Ability

This page provides a summary and source link only.