Event

Method Proposed for Evaluating Transfer of Uncertainty Thresholds in LiDAR Detection

1 reports · 1 sources

Milin Patel and Rolf Jung propose a method for evaluating uncertainty in LiDAR object detection for ISO 21448 SOTIF analysis: an uncertainty threshold selected under one condition is transferred to a given triggering condition, and testing is then used to determine whether the performance target holds under that condition at a given error probability—applicable, not applicable, or undeterminable—and a lower bound is given on the number of independent units required to reach an "applicable" conclusion. The authors tested an ensemble of PointPillars models on KITTI and nuScenes. When counting by recorded independent sequences, neither dataset had enough independent units for any target limit, and no transfer yielded a "not applicable" conclusion; when counting by frame, the same data yielded "not applicable" conclusions for the pre-set limits after transfer to long range, rain, and between two separately trained detectors. The authors also report that thresholds satisfying the target limit after transfer would reject most correct detections, while the area under the ROC curve did not decrease in the rain scenario. On this basis, the authors conclude that a performance target met under one condition can serve as evidence only after being retested under that triggering condition, and that the proportion of accepted correct detections and the number of independent recorded sequences must be reported together.