Microsoft Kinect has a lot of potential in system applications due to its introduction as a low cost and high resolution 3-D sensing systems (Joshua and Tyler, 2015). The purpose of their study was to develop Kinect block, and provide access to depth im-age streams, and access to sensor cameras. Available drivers of Kinect, the interface of C language, is an impediment to Kinect application development. This access pro-vides the ability to incorporate without any difficulty to an image processing based on Simulink. The study focused on issues affiliated with implementation aspect of Kinect. One of such important aspect is calibration of sensor, another is utility of Kinect block and Kinect and it is through 3-D object tracking example [9].
Figure 6- Simulink Kinect. Joshua et.al. (2014)
For detection of both moving and stationary obstacles, it is greatly dependent upon the capability of systems to navigate in unsure circumstances. In the available sen-sors, Sonar sensors is a low-cost option, however, these are prone to false echoes and reflections due to bad quality angular resolution. Another option is Infra-red and la-ser range finders, these are also low cost, but the grey area is provision of measure-ments from only one point in the scene. Other available option is of spectrum, Radar and Light Detection and Ranging (LIDAR) systems, these can provide precise meas-urements along with good angular resolutions [14]. However, they also have grey areas. Most important of them are their high-power consumption and high expenses.
This complete picture and the revolution of low cost digital cameras has produced an interest in vision-based setups for the vehicles, which are autonomous. Even in this case, the disadvantage of distance must be of stereoscopic cameras. The recent re-lease of the Microsoft Kinect addresses this issue by providing both a camera image and a depth image. Kinect was primarily for entertainment market, however due to its powerful capabilities to operate, it has gained a lot of popularity in sensing and robotic community. Few of the examples of this popularity are applications related to robot / human interaction, 3-D virtual environment construction, medicine, robot tracking and sensing. Most of Kinect applications are coded in C [15]. In industry and academia, use of image processing is now common place. Even inexperienced users can use these tools. These are used to target hardware to implement by making use of code generation automatically. Simulink provides a widely accepted environ-ment for designing impleenviron-mentation of image processing algorithms as well. For ex-ample, in automotive industry, Simulink is used to produce seamless code-generation tools. These translate the final design into a target hardware, which is real time-executable. These tools have a use in educational environments, it enables the stu-dents to concentrate on major details rather than low level. The major contributions of this study can be divided into three major spheres. Interface development allowing Kinect to be involved in refined Simulink designs, it allows more users to access.
The targets discussed in the paper are Linux based, which are used in mobile auton-omous robotic machines. Real time Kinect streams parallel camera and depth images.
2.10
Comparing the utility and usability of the Microsoft Kinect and Leap Motion sensor devices in the context of their application for gesture control of biomedical images
A study conducted by Hughes and Nextorov (2014) investigated the interaction of medical images in operating room with a requirement of maintaining asepsis. This arrangement has resulted in a complex type of arrangement in use of mouse and key-board, between scrubbed clinicians and on other end non-scrubbed personnel [16]. It is Microsoft Kinect or Leap motion which could give direct control of medical image navigation and manipulation.
Figure 7- Leap Motion Sensor. Hughes et.al. (2015)
The authors admitted that many studies have already been undertaken to study use of Leap and Microsoft Kinect in Operating Room, however, no study had compared the sensors in terms of their usage. They aimed their study to compare the use, utility, accuracy and acceptance of the two motion sensors. In this research, 42 persons par-ticipates. Out of these 30 % were diagnostic Radiologists and 70 % were surgeons or Interventional Radiologists. All the participants were having good computer skills but limited gaming experience. In analysis of utility of two motion sensors, 50 % of participants rated Microsoft Kinect v2 as very useful in their routine practice, how-ever performance of Leap Motion Sensor was 38 %. Out of Surgeons and Interven-tional Radiologists 54 % rated Microsoft Kinect as useful [13]. Younger participants found Leap Motion interface as more useful than older. In 37.5 % participants, after use of Kinect sensor, perception of leap motion sensor deteriorated. System accepta-bility was better for Kinect Sensor, as compared to Leap Motion Sensor. With re-spect to utility and use, Microsoft Kinect was found better. However, Leap motion was found to have a better accuracy. Kinect was more acceptable to the users, alt-hough Microsoft Kinect was tiresome physically. More than half of the surgeons and
interventional Radiologists found Microsoft Kinect v2 as very useful. As regards to this study, Vascular and Orthopedic surgeons found the sensors to be more useful.
The measurement accuracy was found not to be of good standard, which can be at-tributed to many factors including the system’s field of view.
For Leap Motion Sensor, user needed to place the cursor at end or start point of ana-tomical structure and keep the hand sable before indicator of selection is seen [5].
More time was taken before selecting the measurement point. On the other hand, Kinect proved to be better, as took short time. In certain cases, participant had showed hand before selecting the end point, so the measurement command was com-pleted prematurely. Few gestures were initially available for both sensors and were available. However, later gestures were disabled and replaced by discrete input or click. Due to requirement of time to implement measured command in four seconds for the startup and end measurements, both sensors were found to be slower than average time. In terms of time to task completion, as per the prior studies conducted, with adequate practice, motion sensors performed better than the mouse. Fastest time for a participant was 6.38 secs for Leap Motion Sensor and 7.54 secs for Mi-crosoft Kinect V2. These times are lower than overall average time to indicate and measure [11].
Figure 8- Motion Sensor illustration. Hughes et.al. (2015)
System use influences utility of system by surgeons. It has been shown by relation-ship between use and utility, due to poor use, there comes poor utility. Study proved that Leap Motion Sensor could not be equivalent to Kinect v2, because younger doc-tors were more comfortable with use of Motion Sensors, as compared to Kinect [9].
2.11 A Depth-Based Fall Detection System Using a Kinect Sensor
Researchers have also tested Kinect sensors application in fall detection systems.
Samuel and Enea (2014), for instance, carried out a study proposing a fall detection system basing on Microsoft Kinect. This system is privacy preserving and automatic.
The raw depth data, which is provided by the sensors is analyzed by means of ad-hoc algorithm. This system implements a definite solution to categorize, all the blobs in the specific scene. Whenever a person is identified, a tracking algorithm is followed between different frames. When use of depth frame is made of, it allows to extract human body, even when it is interacting with other things such as a wall, or a tree.
Inter-frame processing algorithm helps to efficiently solve the problem of blob fusion [14]. If a depth blob attached to a person is near the floor, the fall is detected.
Figure 9- Fall detection illustration. Samuele et.al. (2014)
In the study, in top- view configuration, using Kinect Sensor method of automatic fall detection has been proposed. Without relying on sensors, which are wearable and by the exploitation of privacy- preserving depth data only, this approach allows de-tecting a fall event. With the help of ad hoc discrimination algorithm, this system could identify and bifurcate the stationery objects from human subjects, within scene.
Simultaneous tracking can be done and numerous human subjects can be monitored.
Authors confirmed through experiments the capability of identifying the human body during a fall event. Moreover, the capability of algorithm recommended for tackling the blob fusions in domain of depth.
The system proposed in this research has been tested and realized on PC with fea-tures of Windows 7, i5 processor with a RAM of 4 GB. The proposed algorithm can be adapted by diff depth sensors, and it needs only depth information as input data.
Moreover, embedded real time implementation has been done featuring Linaro 12.11, Cortex A-9 and 2 GB RAM. Authors foresee that future research activities will focus to simultaneously tackle and manage various depth sensors by improving and enhancing the performance of the algorithm. The system will be made to support the tracking of subjects whenever it endeavor to cross areas covered by adjacent sen-sors.
2.12 Experimental Studies on Human Body Communication Char-acteristics based upon Capacitive Coupling
Researcher at the Academy of Sciences, Shenzhen, China studied Human Body Communication and regarded it as technology of transmission for sensor network applications for short range (Wen-cheng and Ze-dong, 2014). There are few full-scale measurements, which described body channel propagation on capacitive cou-pling [11]. The study has its focus on experimenting various body parts, investigating the features of body channel. By making use of coupling technique, both in terms of frequency and time, the characteristics of body channel may be measured. Based on the results measurements, it was observed that the body maintained stable character-istics. Elbow, wrist and knee affected channel affected the attenuation characteristics [19].
2.13 Body Movement Analysis and Recognition
Different studies have also proposed human-robot interaction basing on innovative combination of sensors. Yand and Hui (2014) conducted a study on communication by non-verbal ways for communication of robots and humans by developing an un-derstanding of human body gestures. The robot can express itself by making use of body movements, such as facial expressions, movements of body parts and verbal expression. For this communication, twelve gestures of upper body will be utilized.
Interactions of objects and humans are also included in these. Gestures are character-ized by the information of arm, hand posture and arm. To capture the hand posture, use is made of Cyber Glove II. Microsoft Kinect gives information for head and arm posture [12]. This is an up to date solution of human gesture combination by the sen-sors. Basing on the data obtained by posture data of body, proposal has been made of human gestures recognition, which is real time, as well as effective. In this study, experiments were also conducted to prove the efficacy and effectiveness of the ap-proach proposed in this.
Figure 10- Movement analysis Glove. Yang et.al. (2012)
Human-computer interaction has recently gained the interest and attention of indus-trial and academic communities, and is still not very old field as it started in 1990s.
This field has contributions from mechanical engineering, computer sciences and mathematics. Unlike interactions of earlier times, more social dynamics aspect must be expected in domain of human-robot interactions. As people want to interact with robots, as they do with other humans, so robot human interaction is needed to be made more believable. Robots should be able to make use of verbal and body lan-guage, as well as facial expressions [10]. Some robots are already being used for this goal. Nao Humanoid Robot1 can use gestures and body expressions. The main con-cern of the study was to establish means of communication between robot and human using body language. One of the main purpose of the study was to apply other than verbal language to human-robot interaction in social domain. Upper body, gestures are applied, which are 12 in number. These are involved in recommended system and are all intuitive and natural gestures. They characterize themselves by arm, head and posture information. Human-object interactions are involved in these gestures.
Figure 11- Humanoid robotics illustration. Clingal et.al. (2014)
A human body dataset is constructed to analyze the recommended recognition meth-od. The dataset was made by making results from 25 samples of different body sizes, culture backgrounds and genders. Efficiency and effectivity of the recommended system has been proven by the experiments. Few of the major aspects of the study are:
Kinect and Cyber Glove II are integrated to captured arm, head and hand pos-ture. For this human gesture-capture sensor is recommended.
For recognition of upper body gestures, a real time and effective sensor is recommended.
A gesture understanding and human robot interaction system is built to assist humans to interact with robots.
A scenario was established in which, a user and a robot classroom interaction was created for a case study of GUHRI system. The user is student and robot acts as lec-turer. Robot can understand the upper body gestures, 12 in number. Robot is like humans and can react by combining facial expression, verbal language or body movement. The behavior of robot in class is triggered by the body language of the user [7]. Here all the actions are completely consistent with the established scenario.
GUHRI system has also the ability to tackle unexpected situations like, if a user an-swers a phone call suddenly, it can react appropriately. Regarding proper under-standing of upper body gestures, dynamic are the important body language compo-nents in daily life. They provide clue for communication to enhance performance for this communication. To make robot- human interaction, robot should be able to un-derstand static as well as dynamic gestures with the help of movement analysis and recognition of human gestures. Human body 3-D combined information can be ob-tained in real time by the Microsoft’s Kinect SDK. By the change in position of body joint in temporal axis, motion information can be obtained. Activity recognition has already also been done to by this information of body joint motion. Possibility of ignoring hand gestures is still there, due to which chances of ignoring gestures by hand are there. Future is likely to be marked by studies on recognition of gestures of upper body and body motion, as well as requisite information by hand gestures. An-other dimension is recognition of sensor form egocentric point of view. In the rec-ommended GUHRI system of the paper, Kinect has been proposed as vision sensor.
It is not a perfect system and has many limitations like inability to change viewpoint due to fixed position of Kinect Sensor. Due to this limitation, it is always not possi-ble for remote to get maximum viewpoint of gestures by the human body. One of the options available to solve this problem is to get gesture information by egocentric perspective of the robot. This provides opportunity for changing the view point of the robot, but it gives birth to some new problems. As the distinction between motion of a camera and a real body motion will be difficult for the robot [11]. In future, further work can also be done in this regard by understanding the integration of verbal clues to GUHRI system to further increase the robot-human interaction. If robot is more autonomous in seeing and hearing, it will become more like humans.
This paper has recommended in overall context, a GUHRI system, with understand-ing of robots and human interactions and innovative understandunderstand-ing of gestures. Ro-bot can understand 12 upper body part gestures which can be comprehended by the robot. By a few features like facial expressions, body movements and verbal expres-sion, robot also has the ability to express itself. A combination of sensors has been recommended to combine Microsoft’s Kinect and Cyber Glove to capture posture of head arm and hand simultaneously [3]. By doing this, an effective and real-time ges-ture recognition mechanism has been recommended. In the experiments, human body gesture dataset has been built. The efficiency of our gesture recognition has been built by results of the experiments conducted. Till now, the gestures involved are static gestures like of having question, to appreciate, to call, to drink etc. In this study, the future recommendations are to understand dynamic gestures as to say no, to clap, to wave hand. Another important recommended addition is of speech recog-nition; it would make the interaction more real.
2.14 An Integrated Platform for Live 3-D Human Reconstruction and Motion Capturing
There are also experiments and studies that show how Kinect technology can be used for live 3-D human reconstruction and motion capturing. In their research, Imitrios and Alexadis (2011) investigate the developments in 3-D capturing, processing and provided ways to unlock pathways of 3-D applications. Their study addresses tasks of real time capturing and motion tracking by explaining main features of an inte-grated platform targeting future 3-D applications. Moreover, along this, an innova-tive sensors calibration method has also been discussed. Basing on an increased de-viation of volumetric Fourier transform based method, an innovative method of re-construction has been from RGB-D has been recommended in this paper. The paper also proposed, a qualitative evaluation of 3-D reconstruction mechanisms, as existing evaluation methods have been found quite irrelevant. Overall, an accurate mecha-nism of real time human body tracking has been recommended, that also was basd on a generic and multiple depth based mechanism. Experiments conducted in the study proved the lessons of the study.
In this study, including multi-Kinetic v2, capturing reconstruction of moving hu-man’s other applications like fast reconstruction of humans, and based on skeleton- motion tracking by depth cameras has been described and main elements of integrat-ed system have been describintegrat-ed elaborately. Basintegrat-ed on these elements, innovative ap-proaches have been recommended in this paper and discussion on existing approach-es have also been explained. Along with this, an innovative mechanism for evalua-tion of 3-D reconstrucevalua-tion system, has also been recommended. Some limitaevalua-tions of ongoing researches have also been discussed. Imperfect synchronization issue with RGBD sensors, may lower the construction quality, it is one of the main limitations
of this research. In domain of skeleton tracking mechanism, short comings of topol-ogy change are to take over by fitting of skeleton scheme [2]. Moreover, by splitting the body into upper and lower parts and fusing our mechanism of data from inertial measurements, limitations can be overcome.
Figure 12- RGDB illustration. Immitrios et.al. (2014)
2.15 Automated Training and Maintenance through Kinect
Availability of Kinect at low cost rates and its provision of high quality sensors has
Availability of Kinect at low cost rates and its provision of high quality sensors has