Hand Gesture Recognition

Project Objectives

The objectives of my project are two fold. Firstly, most of the current industrial implementations of Hand gesture Recognition use an array of sensors as in the Kinect from Microsoft. Even though, the Kinect performs satisfactorily for Body Pose estimation and for real time gaming, its use in Hand pose estimation is still being researched. The Kinect is a bulky device due to the multiple sensors it houses. This decreases its portability. Therefore, the first objective of my project is to make a Hand Gesture Recognition system that uses an input from a single RBG-D sensor.

Secondly, most of the current day models are very complex. This complexity introduces an execution penalty that makes most of the systems undesirable for real time use. Therefore, the second objective of this project will be to keep the models made as lightweight as possible and aim for a model that can work under online conditions.

Project Deliverables

The project deliverables will include an API implementing the different structures of the Hand Gesture Recognition. Furthermore, the project will also deliver a GUI implementation of the beforementioned library. More details about the deliverables can be found below:

• HandNet API: The API will have the implementations of the final structures of both the Hand Pose Estimation models and the Hand Gesture Recognition models. The API will furthermore include the required classes for data loading and preprocessing. Lastly, the API will implement classes for quantitative and qualitative analysis and training and validation classes tailored to the implementations of the aforementioned data loading classes.
• GUI application: The second deliverable of the project will provide a user-interface for interacting with the API discussed in section 3.1. The application will allow the users to input an image or a video. The image will be used for Hand Pose Estimation whereas the video will be used for hand Gesture Recognition. The users will be able to view the results and also save annotated images highlighting the pose in the images.
• Prototypes: Besides the final deliverables, there will also two interim deliverables of the aforementioned API . These deliverables will be incremental upgrades of each other. The first interim prototype, called GesReg 1, will be the first implementation of the Hand Gesture Recognition system. This system will feature the data preprocessing conducted using the Hand Pose estimator. This prototype should be complete by the end of November. The second prototype, similarly named GesReg 2, will be a structural upgrade from GesReg 1. It will feature changes in its data processing and output stages to provide a better performance from GesReg 1. This prototype should be ready by mid January 2020.