Beyond Multi-Touch: User Interfaces for Flexible Displays and Surfaces

This paradigm has the potential to revolutionize Human-Computer Interaction. As one example, I envision that future interactive devices will have the form factor of paper-thin stickers. These stickers are deformable, of various shapes and sizes, provide high resolution display output, and support touch and deformation sensing for input. They enable to easily make any surface interactive, including human skin, clothing or physical objects. By radically redefining the form factor of computing devices, flexibility opens up previously impossible scenarios of use for more mobile, more direct and more versatile interaction with computers.

This poses unique scientific challenges regarding interaction technologies and interaction techniques. My research addresses these challenges from a user-centered perspective, driven by visions of future computing.

1) Interaction technologies: Conventional sensors and displays are based on rigid electronics. Substantially different technical solutions are required to meet the demands of flexibility, ultra-thin form factors and arbitrary shapes. Moreover, flexibility yields novel input modalities that need to be captured, e.g. deformation, pressure, or cutting. Informed by user needs that we derive from empirical studies, we develop novel multi-modal sensors and displays to technically enable flexible interfaces.

2) Interaction techniques: Novel modalities of interaction and novel contexts of use open a design space of interaction techniques, which is largely unstudied. We systematically model and study mappings between input and output, to define a vocabulary of interactions and inform applications that are useful and usable, meaningful, and make the most of the differentiating properties of this new paradigm.

In my talk, I will present this research agenda and first results achieved. I will present PrintSense, a paper-thin, fully flexible, multi-modal sensor surface. In addition to multi-touch input, this sensor captures proximity, pressure input, and deformation. Moreover, it can be cut to various shapes while remaining functional. The sensor can be easily produced and deployed in a variety of scenarios, including deformable smart phones and tablets, interactive paper, smart products, and on-body interfaces.