Max-Planck-Institut für Informatik
max planck institut
mpii logo Minerva of the Max Planck Society

MPI-INF or MPI-SWS or Local Campus Event Calendar

<< Previous Entry Next Entry >> New Event Entry Edit this Entry Login to DB (to update, delete)
What and Who
Title:Automatic Methods for Low-Cost Evaluation and Position-Aware Models for Neural Information Retrieval
Speaker:Kai Hui
coming from:Max-Planck-Institut für Informatik - D5
Speakers Bio:
Event Type:Promotionskolloquium
Visibility:D1, D2, D3, D4, D5, SWS, RG1, MMCI
We use this to send out email in the morning.
Level:Public Audience
Date, Time and Location
Date:Monday, 4 December 2017
Duration:60 Minutes
Building:E1 5
An information retrieval (IR) system assists people in consuming a huge amount of data, where the

evaluation and the construction of such systems are important. However, there exist two difficulties:
the overwhelmingly large number of query-document pairs to judge, making IR evaluation a manually laborious
task; and the complicated patterns to model due to the non-symmetric, heterogeneous relationships between a
query-document pair, where different interaction patterns such as term dependency and proximity have been
demonstrated to be useful, yet are non-trivial for a single IR model to encode. In this thesis, we attempt to address
both difficulties from the perspectives of IR evaluation and of the retrieval model respectively, by reducing the
manual cost with automatic methods, by investigating the use of crowdsourcing in collecting preference judgments,
and by proposing novel neural retrieval models.
In particular, to address the large number of query-document pairs in IR evaluation, a low-cost selective labelling
method is proposed to pick out a small subset of representative documents for manual judgments in favour of the
follow-up prediction for the remaining query-document pairs; furthermore, a language-model based cascade measure
framework is developed to evaluate the novelty and diversity, utilizing the content of the labeled documents to
mitigate incomplete labels. In addition, we also attempt to make the preference judgments practically usable by
empirically investigating different properties of the judgments when collected via crowdsourcing; and by proposing
a novel judgment mechanism, making a compromise between the judgment quality and the number of judgments.
Finally, to model different complicated patterns in a single retrieval model, inspired by the recent advances in deep
learning, we develop novel neural IR models to incorporate different patterns of term dependency, query proximity,
the density of relevance, and query coverage in a single model. We demonstrate their superior performances through
evaluations on different datasets.

Name(s):Petra Schaaf
EMail:--email address not disclosed on the web
Video Broadcast
Video Broadcast:NoTo Location:
Tags, Category, Keywords and additional notes
Attachments, File(s):

Created by:Petra Schaaf/AG5/MPII/DE, 11/23/2017 10:35 AMLast modified by:Uwe Brahm/MPII/DE, 12/04/2017 07:01 AM
  • Petra Schaaf, 11/23/2017 10:38 AM -- Created document.