The paper “Haystack: A Panoptic Scene Graph Dataset to Evaluate Rare Predicate Classes” by Julian Lorenz, Florian Barthel, Daniel Kienzle, and Rainer Lienhart is accepted at the First ICCV Workshop on Scene Graphs and Graph Representation Learning (SG2RL). The authors present Haystack, a new dataset for scene graph generation that tackles current shortcomings when evaluating with current scene graph datasets. Most notably, Haystack contains rare predicate classes and explicit negative annotations. Only through these properties can rare relationships be reliably evaluated. Based on the design of Haystack, the authors introduce three new scene graph metrics that can be used to gain more detailed insights about the prediction of rare predicate classes.

More information can be found here: https://lorjul.github.io/haystack/