In a keyboard-based task, we experience the inefficiency when moving a hand to mouse and back occasionally. This causes time consumption and mental distraction. On the other hand, we use keyboard shortcuts to replace and accelerate mouse function. However, the shortcuts have a problem that hard to remember, and different over applications. In this paper, we introduce gesture recognition interface with keyboard embedded IR modules. The system replaces functions of mouse, hence reduces the inefficiency of moving hand position back and forth between keyboard and mouse. In addition, because of the characteristics of gesture, the system gives less cognitive load to user than shortcuts. In this paper, we demonstrate our approach to design the gesture recognizing keyboard’s sensor system, and the gestures. We conducted user study to verify intuitiveness of the proposed gestures and usability of the system. We also discuss other applications utilizing the proposed gesture recognizing system.