Abstract

In a crowd forecasting system, aggregation is an algorithm that returns aggregated probabilities for each question based on the probabilities provided per question by each individual in the crowd. Various aggregation methods have been proposed, but simple strategies like linear averaging or selecting the best-performing individual remain competitive. With the recent advance in neural networks, we model forecasts aggregation as a machine translation task, that translates from a sequence of individual forecasts into aggregated forecasts, based on proposed Anchor Attention between questions and forecasters. We evaluate our approach using data collected on our forecasting platform and publicly available Good Judgement Project dataset, and show that our method outperforms current state-of-the-art aggregation approaches by learning a good representation of forecaster and question.

Date

March 3, 2020

Authors

Yuzhong Huang, Andres Abeliuk, Fred Morstatter, Pavel Atanasov, Aram Galstyan

Journal

arXiv preprint arXiv:2003.12447