Adjudication of coreference annotations via answer set optimisation


Schueller P.

JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE, vol.30, no.4, pp.525-546, 2018 (Journal Indexed in SCI) identifier

  • Publication Type: Article / Article
  • Volume: 30 Issue: 4
  • Publication Date: 2018
  • Doi Number: 10.1080/0952813x.2018.1456793
  • Title of Journal : JOURNAL OF EXPERIMENTAL & THEORETICAL ARTIFICIAL INTELLIGENCE
  • Page Numbers: pp.525-546

Abstract

We describe the first automatic approach for merging coreference annotations obtained from multiple annotators into a single gold standard. This merging is subject to certain linguistic hard constraints and optimisation criteria that prefer solutions with minimal divergence from annotators. The representation involves an equivalence relation over a large number of elements. We use Answer Set Programming to describe two representations of the problem and four objective functions suitable for different data-sets. We provide two structurally different real-world benchmark data-sets based on the METU-Sabanci Turkish Treebank and we report our experiences in using the Gringo, Clasp and Wasp tools for computing optimal adjudication results on these data-sets.