International Journal of Advanced Networking and Applications

Open Access Open Access  Restricted Access Subscription Access

Operating Computer Cursor using Eye and Face Movements


Affiliations
1 Department of Information Technology, Mahatma Gandhi Institute of Technology, Hyderabad-75, India
 

The advent of modern human computer interfaces has seen a considerable progress in Hands-free Human Computer Interaction (HCI) solutions. This project focuses on developing a methodology to facilitate computer cursor control for people with physical disabilities such as Quadriplegics and amputees. The proposed methodology takes real-time video input from the user using OpenCV and performs face recognition. The 68- point landmark algorithm is used to locate the various facial features which can be used for cursor control. Opening/closing the mouth based on Mouth Aspect Ratio (MAR) indicates activation/deactivation of the cursor control. The nose tip is used for controlling and moving the cursor in all 4 directions by moving the head left, right, up and down. Eye Aspect Ratio (EAR) is used to detect eyes and eye flickering. Left and right eye blinks indicate left and right clicks respectively. Squinted eyes indicate scrolling of pages, which is beneficial while working with PDFs and other such documents. The proposed system requires very basic requirements like webcam and a few Python libraries such as OpenCV, Numpy, imutils, dlib and PyAutoGUI. Thus it would help the physically disabled users to efficiently use the computer, thus eliminating the need of a physical mouse interaction.

Keywords

Human Computer Interaction, Face Recognition, 68-point Landmark Algorithm, MAR, EAR.
User
Notifications
Font Size

  • Yuvan.M, Varun Ramesh Kumar, et.al. Real Time Driver Drowsiness Detection Using Open CV, International Journal Of Advanced Networking & Applications (IJANA), 2019, special issue - 2019 : 96-99.

  • V. Khare, S. G. Krishna and S. K. Sanisetty, "Cursor Control Using Eye Ball Movement," 2019 Fifth International Conference on Science Technology Engineering and Mathematics (ICONSTEM), 2019, pp. 232-235, doi: 10.1109/ICONSTEM.2019.8918780.

  • Maruthapillai, Vasanthan & M, Murugappan & Nagarajan, R. & Ilias, Bukhari & Letchumikanth, J.. (2012). Facial expression based computer cursor control system for assisting physically disabled person. Proceeding - COMNETSAT 2012: 2012 IEEE International Conference on Communication, Networks and Satellite. 172-176. 10.1109/ComNetSat.2012.6380800.

  • Mehta, Sukrit and Dadhich, Sharad and Gumber, Sahil and Jadhav Bhatt, Arpita, Real-Time Driver Drowsiness Detection System Using Eye Aspect Ratio and Eye Closure Ratio (March 20, 2019). Proceedings of International Conference on Sustainable Computing in Science, Technology and Management (SUSCOM), Amity University Rajasthan, Jaipur - India, February 26-28, 2019.

  • S. Mohanty, S. V. Hegde, S. Prasad and J. Manikandan, "Design of Real-time Drowsiness Detection System using Dlib," 2019 IEEE International WIE Conference on Electrical and Computer Engineering (WIECON-ECE), 2019, pp. 1-4, doi: 10.1109/WIECONECE48653.2019.9019910.

  • A. ROSEBROCK, FACIAL LANDMARKS WITH DLIB, OPENCV, AND PYTHON, APRIL 2017, AVAILABLE: HTTPS://WWW.PYIMAGESEARCH.COM/2017/04/03/FACIALLANDMARKSDLIB-OPENCV-PYTHON/

  • How to detect eye blinking in videos using dlib and OpenCV in Python, June 2020, Available: http://datahacker.rs/011-how-to-detect-eye-blinking-invideosusing-dlib-and-opencv-in-python/

  • Relangi S.P.K., Nilesh M., Kumar K.P., Naveen A. (2020) Full Length Driver Drowsiness Detection Model—Utilising Driver Specific Judging Parameters. In: Reddy A., Marla D., Simic M., Favorskaya M., Satapathy S. (eds) Intelligent Manufacturing and Energy Sustainability. Smart Innovation, Systems and Technologies, vol 169. Springer, Singapore. https://doi.org/10.1007/978-981-15-1616-0_77


Abstract Views: 231

PDF Views: 2




  • Operating Computer Cursor using Eye and Face Movements

Abstract Views: 231  |  PDF Views: 2

Authors

U. Chaitanya
Department of Information Technology, Mahatma Gandhi Institute of Technology, Hyderabad-75, India
Hansika Garapati
Department of Information Technology, Mahatma Gandhi Institute of Technology, Hyderabad-75, India
S. Priyanka Raj
Department of Information Technology, Mahatma Gandhi Institute of Technology, Hyderabad-75, India

Abstract


The advent of modern human computer interfaces has seen a considerable progress in Hands-free Human Computer Interaction (HCI) solutions. This project focuses on developing a methodology to facilitate computer cursor control for people with physical disabilities such as Quadriplegics and amputees. The proposed methodology takes real-time video input from the user using OpenCV and performs face recognition. The 68- point landmark algorithm is used to locate the various facial features which can be used for cursor control. Opening/closing the mouth based on Mouth Aspect Ratio (MAR) indicates activation/deactivation of the cursor control. The nose tip is used for controlling and moving the cursor in all 4 directions by moving the head left, right, up and down. Eye Aspect Ratio (EAR) is used to detect eyes and eye flickering. Left and right eye blinks indicate left and right clicks respectively. Squinted eyes indicate scrolling of pages, which is beneficial while working with PDFs and other such documents. The proposed system requires very basic requirements like webcam and a few Python libraries such as OpenCV, Numpy, imutils, dlib and PyAutoGUI. Thus it would help the physically disabled users to efficiently use the computer, thus eliminating the need of a physical mouse interaction.

Keywords


Human Computer Interaction, Face Recognition, 68-point Landmark Algorithm, MAR, EAR.

References