Natural User Interface is recently becoming important according to various uses of computers as shown in the success of Apple's iPhone and Nintendo's Wii. Although various VR systems have been developed for intuitive operations, the cost and setting space issues prevent wide uses. On the other hand, Mixed Reality, which makes reality by projecting CG to the real scene, requires only a web camera at least.
Therefore, it is expected to be widely used. The goal of our research is to develop an inexpensive system that users can naturally interact with computer generated objects using MR technologies. In this talk, we present two novel applications; photo realistic water surface interaction and direct 3D volume segmentation using Mixed Reality. In a water tank application, users can touch a virtual water tank with a real pen and make various wave patterns. For a volume segmentation application, which is an important process to analyze scanned medical volume data, users can control the region to be extracted using a Wii remote controller attached motion tracking markers. This talk also includes a brief introduction of my research activities at RIKEN.