Open Access Open Access  Restricted Access Subscription Access

Gesture-Controlled Interface for Contactless Control of Various Computer Programs with a Hooking-Based Keyboard and Mouse-Mapping Technique in the Operating Room


Affiliations
1 Medical Information Development Team, Asan Medical Center, Seoul 05505, Korea, Republic of
2 Department of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul 05505, Korea, Republic of
3 Department of Plastic and Reconstructive Surgery, University of Ulsan College of Medicine, Asan Medical Center, Seoul 05505, Korea, Republic of
 

We developed a contactless interface that exploits hand gestures to effectively control medical images in the operating room. We developed an in-house program called GestureHook that exploits message hooking techniques to convert gestures into specific functions. For quantitative evaluation of this program, we used gestures to control images of a dynamic biliary CT study and compared the results with those of a mouse (8.54 ± 1.77 s to 5.29 ± 1.00 s; p < 0.001) and measured the recognition rates of specific gestures and the success rates of tasks based on clinical scenarios. For clinical applications, this program was set up in the operating room to browse images for plastic surgery. A surgeon browsed images from three different programs: CT images from a PACS program, volume-rendered images from a 3D PACS program, and surgical planning photographs from a basic image viewing program. All programs could be seamlessly controlled by gestures and motions. This approach can control all operating room programs without source code modification and provide surgeons with a new way to safely browse through images and easily switch applications during surgical procedures.
User
Notifications
Font Size

Abstract Views: 111

PDF Views: 1




  • Gesture-Controlled Interface for Contactless Control of Various Computer Programs with a Hooking-Based Keyboard and Mouse-Mapping Technique in the Operating Room

Abstract Views: 111  |  PDF Views: 1

Authors

Ben Joonyeon Park
Medical Information Development Team, Asan Medical Center, Seoul 05505, Korea, Republic of
Taekjin Jang
Department of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul 05505, Korea, Republic of
Jong Woo Choi
Department of Plastic and Reconstructive Surgery, University of Ulsan College of Medicine, Asan Medical Center, Seoul 05505, Korea, Republic of
Namkug Kim
Department of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul 05505, Korea, Republic of

Abstract


We developed a contactless interface that exploits hand gestures to effectively control medical images in the operating room. We developed an in-house program called GestureHook that exploits message hooking techniques to convert gestures into specific functions. For quantitative evaluation of this program, we used gestures to control images of a dynamic biliary CT study and compared the results with those of a mouse (8.54 ± 1.77 s to 5.29 ± 1.00 s; p < 0.001) and measured the recognition rates of specific gestures and the success rates of tasks based on clinical scenarios. For clinical applications, this program was set up in the operating room to browse images for plastic surgery. A surgeon browsed images from three different programs: CT images from a PACS program, volume-rendered images from a 3D PACS program, and surgical planning photographs from a basic image viewing program. All programs could be seamlessly controlled by gestures and motions. This approach can control all operating room programs without source code modification and provide surgeons with a new way to safely browse through images and easily switch applications during surgical procedures.