Publications

Replacing mechanical turkers? challenges in the evaluation of models with semantic properties

Abstract

Some machine-learning algorithms offer more than just predictive power. For example, algorithms provide additional insight into the underlying data. Examples of these algorithms are topic modeling algorithms such as Latent Dirichlet Allocation (LDA)[Blei et al. 2003], whose topics are often inspected as part of the analysis that many researchers perform on their data. Recently, deep learning algorithms such as word embedding algorithms like Word2Vec [Mikolov et al. 2013] have produced models with semantic properties. These algorithms are immensely useful; they tell us something about the environment from which they generate their predictions. One pressing challenge is how to evaluate the quality of the semantic information produced by these algorithms. When we employ algorithms for their semantic properties, it is important that these properties can be understood by a human. Currently, there are no …

Date
2016
Authors
Fred Morstatter, Huan Liu
Journal
Journal of Data and Information Quality (JDIQ)
Volume
7
Issue
4
Pages
1-4
Publisher
ACM