Indian Journal of Science and Technology

Open Access Open Access  Restricted Access Subscription Access

Investigating Hand Gestures for Interactivity in Legacy Notice Board System


Affiliations
1 Department of Software Engineering, Mehran University of Engineering and Technology, Jamshoro, Pakistan
 

Objectives: This research has been carried out to add interactivity and information transfer in digital format by adding technology in legacy paper based notice board system. Methods/Statistical Analysis: To achieve the above-mentioned objective, we have used Hand gesture recognition Technology. A Microsoft Kinect sensor is placed in front of the notice board to detect hand gestures which serves as the medium of interactivity. Through specified American Sign Language Number gestures user can make selections and interact with the system. For the detection of gestures Visual gesture Builder has been used which implements AdaBoost Trigger Algorithm. This framework uses Data Driven Machine Learning Algorithm to detect gestures. For the analysis, training and testing of the framework we have collected Gestures data for each predefined American Sign Language Number gestures from 0–9, from both Left and right hands, from 49 people. The machine learning algorithm was trained by 80% of the gesture data and was tested by rest 20% gestures. The approach got varying Confidence value (accuracy values) for each gesture depending on varying hand space, hand size, person’s height, clarity in gesture performance. The framework also tested based on both male and female candidates, the result for gender-based analysis is also formulated in graph. The confidence values vary from gesture to gesture for both male and female. Findings: The research come out with the results that this technique can be used to optimize and make static paper based notice boards system interactive. With this technique user is able to transfer the information or notice posted on the notice board to their digital platform by making selection and commanding using their hand gestures. This enabled the user to negotiate without changing the whole business model. Application/Improvements: This framework can be implemented on places where notice boards are used to deliver information to the users for example. Institutes, hospitals, stations etc. Using this system the user can easily interact with the notice boards and transfer information to their digital means.
User

  • Chen Q, Malric F, Zhang Y, Abid M, Cordeiro A, Petriu EM, Georganas ND. Interacting with digital signage using hand gestures. International Conference Image Analysis and Recognition. Springer, Berlin, Heidelberg. 2009 Jul; p.347–58. https://doi.org/10.1007/978-3-642-02611-9_35.

  • Davies N, Clinch S, Mikusz M, Bates O, Turner H, Friday A. Better off: when should pervasive displays be powered down? In Proceedings of the 6th ACM International Symposium on Pervasive Displays. 2017 Jun; p.1–19. https://doi.org/10.1145/3078810.3078821.

  • Toha TR, Ishmam AM, Islam MH, Al Maruf MA, Nandi SS, Chakraborty A, Estyak S, Al Alamin MA, Al Islam AA. An approach towards greening the digital display system. IEEE 4th International Conference on Networking, Systems and Security (NSysS). 2017 Dec; p.1–6. https://doi.org/10.1109/NSYSS2.2017.8267794.

  • Smari K, salim Bouhlel M. Gesture recognition system and finger tracking with kinect: Steps. 7th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT). 2016 Jul; p.544–8.

  • Kulshreshth A, Zorn C, La Viola JJ. Poster: Real-time markerless kinect based finger tracking and hand gesture recognition for HCI. IEEE Symposium on 3D User Interfaces (3DUI). 2013 Mar; p.187–8.

  • Hardy J, Rukzio E, Davies N. Real world responses to interactive gesture based public displays. In Proceedings of the 10th International Conference on Mobile and Ubiquitous Multimedia. 2011 Dec; p.33–9. https://doi.org/10.1145/2107596.2107600.

  • Li Y. Hand gesture recognition using Kinect. IEEE International Conference on Computer Science and Automation Engineering. 2012 Jun; p.196–9. https://doi.org/10.1109/ICSESS.2012.6269439.

  • Dong C, Leu MC, Yin Z. American Sign Language alphabet recognition using Microsoft Kinect. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2015; p.44–52.

  • Huang F, Huang S. Interpreting American sign language with Kinect. Journal of Deaf Studies and Deaf Education. 2011 Dec; p.1–5.

  • Zafrulla Z, Brashear H, Starner T, Hamilton H, Presti P. American sign language recognition with the kinect. In Proceedings of the 13th international conference on multimodal interfaces. 2011 Nov; p.279–86. https://doi.org/10.1145/2070481.2070532.

  • Zeng B, Wang G, Lin X. A hand gesture based interactive presentation system utilizing heterogeneous cameras. Tsinghua Science and Technology. 2012 Jun; 17(3): p.329–36. https://doi.org/10.1109/TST.2012.6216765.

  • Wang F, Ngo CW, Pong TC. Simulating a smart board by real-time gesture detection in lecture videos. IEEE transactions on multimedia. 2008 Aug; 10(5):926–35. https://doi.org/10.1109/TMM.2008.922871.

  • Xdk XO, Windows KF. Visual Gesture Builder : A DataDriven Solution to Gesture Detection. 2014; p.1–17.

  • Suarez J, Murphy RR. Hand gesture recognition with depth images: A review. IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication. 2012 Sep; p.411–7. https://doi.org/10.1109/ROMAN.2012.6343787.

  • Biswas KK, Basu SK. Gesture recognition using Microsoft kinect®. The 5th International Conference on Automation, Robotics and Applications. 2011 Dec; p.100–3.

  • Du H, To T. Hand gesture recognition using Kinect. Technical Report. Boston University. 2011 Dec.

  • Yang C, Jang Y, Beh J, Han D, Ko H. Gesture recognition using depth-based hand tracking for contactless controller application. IEEE International Conference on Consumer Electronics (ICCE). 2012 Jan; p.297–8. https://doi.org/10.1109/ICCE.2012.6161876.

  • Chang YJ, Chen SF, Huang JD. A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in developmental disabilities. 2011 Nov; 32(6):2566–70. https://doi.org/10.1016/j.ridd.2011.07.002 PMid:21784612

  • Ronchetti M, Avancini M. Using kinect to emulate an interactive whiteboard. Final thesis, Corso di Laurea Magistrale in Informatica. Università Degli Studi Di Trento. 2011; p.1–129.

  • Ren Z, Yuan J, Meng J, Zhang Z. Robust part-based hand gesture recognition using kinect sensor. IEEE transactions on multimedia. 2013 Aug; 15(5):1110–20. https://doi.org/10.1109/TMM.2013.2246148.


Abstract Views: 156

PDF Views: 0




  • Investigating Hand Gestures for Interactivity in Legacy Notice Board System

Abstract Views: 156  |  PDF Views: 0

Authors

Rabia Naz Khan
Department of Software Engineering, Mehran University of Engineering and Technology, Jamshoro, Pakistan
Mohsin Ali Memon
Department of Software Engineering, Mehran University of Engineering and Technology, Jamshoro, Pakistan
Hira Abid Khan
Department of Software Engineering, Mehran University of Engineering and Technology, Jamshoro, Pakistan
Salahuddin Saddar
Department of Software Engineering, Mehran University of Engineering and Technology, Jamshoro, Pakistan

Abstract


Objectives: This research has been carried out to add interactivity and information transfer in digital format by adding technology in legacy paper based notice board system. Methods/Statistical Analysis: To achieve the above-mentioned objective, we have used Hand gesture recognition Technology. A Microsoft Kinect sensor is placed in front of the notice board to detect hand gestures which serves as the medium of interactivity. Through specified American Sign Language Number gestures user can make selections and interact with the system. For the detection of gestures Visual gesture Builder has been used which implements AdaBoost Trigger Algorithm. This framework uses Data Driven Machine Learning Algorithm to detect gestures. For the analysis, training and testing of the framework we have collected Gestures data for each predefined American Sign Language Number gestures from 0–9, from both Left and right hands, from 49 people. The machine learning algorithm was trained by 80% of the gesture data and was tested by rest 20% gestures. The approach got varying Confidence value (accuracy values) for each gesture depending on varying hand space, hand size, person’s height, clarity in gesture performance. The framework also tested based on both male and female candidates, the result for gender-based analysis is also formulated in graph. The confidence values vary from gesture to gesture for both male and female. Findings: The research come out with the results that this technique can be used to optimize and make static paper based notice boards system interactive. With this technique user is able to transfer the information or notice posted on the notice board to their digital platform by making selection and commanding using their hand gestures. This enabled the user to negotiate without changing the whole business model. Application/Improvements: This framework can be implemented on places where notice boards are used to deliver information to the users for example. Institutes, hospitals, stations etc. Using this system the user can easily interact with the notice boards and transfer information to their digital means.

References





DOI: https://doi.org/10.17485/ijst%2F2018%2Fv11i29%2F128525