Max-Planck-Institut für Informatik
max planck institut
mpii logo Minerva of the Max Planck Society

MPI-INF or MPI-SWS or Local Campus Event Calendar

<< Previous Entry Next Entry >> New Event Entry Edit this Entry Login to DB (to update, delete)
What and Who
Title:Survey Equivalence: An Information-theoretic Measure of Classifier Accuracy When the Ground Truth is Subjective
Speaker:Paul Resnick
coming from:University of Michigan, School of Information
Speakers Bio:Paul Resnick is the Michael D. Cohen Collegiate Professor of Information and Associate Dean for Research at the University of Michigan School of Information. He was a pioneer in the fields of recommender systems and reputation systems. He recently started the Center for Social Media Responsibility, which encourages and helps social media platforms to meet their public responsibilities.
Event Type:SWS Distinguished Lecture Series
Visibility:SWS, RG1, MMCI
We use this to send out email in the morning.
Level:Public Audience
Date, Time and Location
Date:Tuesday, 27 November 2018
Duration:90 Minutes
Building:E1 5
Many classification tasks have no objective ground truth. Examples include: which content or explanation is "better" according to some community? is this comment toxic? what is the political leaning of this news article? The traditional modeling approach assumes each item has an objective true state that is perceived by humans with some random error. It fails to account for the fact that people have greater agreement on some items than others. I will describe an alternative model where the true state is a distribution over labels that raters from a specified population would assign to an item. This leads to information gain (mutual information) as a theoretically justified and computationally tractable measure of a classifier's quality, and an intuitive interpretation of information gain in terms of the sample size for a survey that would yield the same expected error rate.
Name(s):Claudia Richter
Phone:9303 9100
EMail:--email address not disclosed on the web
Video Broadcast
Video Broadcast:YesTo Location:Kaiserslautern
To Building:G26To Room:111
Meeting ID:
Tags, Category, Keywords and additional notes
Attachments, File(s):
Claudia Richter/MPI-SWS, 09/11/2018 09:07 AM
Last modified:
Uwe Brahm/MPII/DE, 11/27/2018 07:01 AM
  • Uwe Brahm, 09/12/2018 05:55 PM
  • Claudia Richter, 09/11/2018 09:13 AM -- Created document.