Improving Decision Making with Machine Learning, Provably

However, these systems also need to make human experts understand when and how to use these predictions to update their own 
predictions. Unfortunately, this has been proven challenging. In this talk, I will introduce an alternative type of decision support systems 
that circumvent this challenge by design. Rather than providing a single label prediction, these systems provide a set of label prediction 
values, namely a prediction set, and forcefully ask experts to predict a label value from the prediction set. Moreover, I will discuss how to
use conformal prediction, online learning and counterfactual inference to efficiently construct prediction sets that optimize experts’ performance, 
provably. Further, I will present the results of a large-scale human subject study, which show that, for decision support systems based on 
prediction sets, limiting experts’ level of agency leads to greater performance than allowing experts to always exercise their own agency.