New Abstractions for High-Performance Datacenter Applications

Malte Schwarzkopf is a postdoc at MIT CSAIL, where he is a member of the Parallel and Distributed Operating Systems (PDOS) group. In his research, Malte designs and builds systems that aim to be both efficient and easy to use, and some of these systems have already impacted industry practice. Malte received both his B.A. and Ph.D. from the University of Cambridge, and his research has won an NSDI Best Paper Award and a EuroSys Best Student Paper Award.

I illustrate the impact of such redesigns with Noria, which recasts web application backends-i.e., databases and caches-as a streaming dataflow computation based on a new abstraction of partial state. Noria's partially-stateful dataflow brings classic databases' familiar query flexibility to scalable dataflow systems, simplifying applications and improving the backend's efficiency. For example, Noria increases the request load handled by a single server by 5-70x compared to state-of-the-art backends. Additional new abstractions from my research increase the efficiency of other datacenter systems (e.g., cluster schedulers), or enable new kinds of systems that, for example, may help protect user data against exposure through application bugs.