Event
Seven VLMs Found to Exhibit Failure Modes in Thermal Human Activity Understanding
1 reports · 1 sources
A study evaluated seven locally deployable vision-language models (VLMs) on two human-centric thermal imaging datasets and found failure modes in fine-grained action recognition, activity timeline reconstruction, and refusal capability. The study used paired RGB recordings as a performance reference and tested whether these errors could be mitigated by varying frame sampling, thermal input presentation, and inference methods. The researchers report that simple thermal input preprocessing and additional inference yielded only limited, model-dependent improvements.
Report timeline
- Failure Mode Analysis of Vision-Language Models in Thermal Human-Robot Interaction
A study evaluated the human activity understanding capabilities of seven locally deployable vision-language models (VLMs) on two person-centric thermal imaging datasets, with diagnostic tasks covering fine-grained action recognition, activity timeline reconstruction, and the ability to withhold answers.
arXiv cs.RO ·