Metric-Dependent Annotation Saturation for Learning from Label Distributions

«When annotators disagree on a label, the disagreement itself carries signal—and the number of annotators needed to capture it depends on the evaluation metric. We fine-tune NLI models on label distributions subsampled fr...»