VOICE AND GESTURAL CONTROL OF HOUSEHOLD APPLIANCES USING KINECT MOTION SENSOR

Abstract

Voice recognition and gesture recognition have given a considerable contribution in bringing human-machine interactions into a different level without requiring physical touch, or contact with devices. In this proposed work, voice recognition and gesture recognition involve creating a system which is able to interpret specific human gestures and vocal inputs via mathematical algorithms. We use these algorithms to convey meaningful information and to control appliances in the home. This proposed system mainly focuses on people who are differently abled and it can be used as a solution for their problem, the inability of controlling appliances in their own room by themselves. Microsoft Kinect motion sensor plays a major role in this developed system by taking gesture inputs and vocal inputs and feeding them to the central controlling system through a preliminary process. The speech model of the system uses Microsoft speech grammar and Speech recognition engine to recognize the vocal input commands. Gesture model of the system uses Visual Gesture Builder software to train the “ON” input gesture command and the “OFF” input gesture command. The whole system runs with a Graphical User Interface making it easier to contact with the user. Ultimately, the user can select the device and its location, by using pre-defined voice command such as, “Living Room TV”. And also, that appliance can be controlled by using pre-defined gesture commands which have been assigned for “ON” and “OFF”. The accuracy of the system is very important when it is used as a real-time home device controlling system. So the system was subjected to an experiment which provided response rate. It showed more than 95% accuracy when the distance to the sensor was 4 m and even when the noise level at the Kinect sensor was about 53 dB. Hence, this proposed system will be an efficient home automation system which will make people’s day-to-day life easier and also will be a very useful system for people who are differently abled and will help them to live like normal people.