Jointly with the “Vision for All Seasons” workshop, we organize the “UIoU Dark Zurich” challenge on uncertainty-aware semantic nighttime image segmentation. The challenge uses the Dark Zurich dataset presented in the ICCV 2019 paper “ Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation” and containing a total of 8779 images captured at nighttime, twilight, and daytime, along with the respective GPS coordinates of the camera for each image. Evaluation of semantic segmentation models on the labeled nighttime part of Dark Zurich is based on a novel, uncertainty-aware framework in which corresponding daytime images are leveraged at annotation to assign reliable semantic labels to originally indiscernible image regions beyond human recognition capability and to indeed include such invalid regions in the evaluation jointly with valid regions.
This evaluation framework is highlighted by UIoU (or uncertainty-aware IoU), a new performance metric that generalizes standard IoU and allows the selective invalidation of predictions, which is crucial for safety-oriented systems handling inputs with potentially ambiguous content, as in the adverse conditions scenario. UIoU rewards models which place higher confidence on valid regions than on invalid ones, i.e. exhibit consistent behavior with human annotators.
Please find the details and participate in the challenge here.
The challenge winners will be invited to present their work at the workshop.