It was released in November 2013. The old technology of Primesense was replaced by Microsoft’s developed ‘time of flight sensor’. According to most analysts like Azzari (2013) this innovation uses a time of flight camera and has great ability of processing of 2 GB per second. It has three times greater accuracy over its predeces-sor and can track with the help of an Infrared (IR) senpredeces-sors. It also has the ability of tracking 6 skeletons at a time. Kinect v2 came with improved video communication and applications, specifically developed for video analytics. The accompanying Mi-crophone is used to provide voice commands.
1.4 Non- Commercial Kinect designed for Microsoft Windows
In Feb 2012, Microsoft released a new version that has Windows 7 compatible PC drivers. This version provided capabilities to developers built by using C++, C# and Visual basic. It also had access to low level streams from depth and other sensors.
Almost 50 companies worked with Microsoft for the development of Kinect (Chang, 2012). The enhanced capabilities were for skeletal tracking and advanced audio ca-pabilities. Skeletal tracking was to allow tracking of people by gesture driven appli-cations. The audio capabilities were integrated with speech recognition Windows application programming interface (API).
1.5 Kinect Versions from 1.5 to 1.8
These were started and launched in 19 different countries. It was released in 2012. A new application known as Kinect for Windows v1.5 SDK including Kinect Studio was developed. Users interacting with the application were to record, debug and play back clips. In this version, tracking of arms, neck and head of Kinect using person was developed for new system or joint skeletal system. The versions from 1.6 to 1.8 further improved with minor variations.
1.6 Kinect v2
It was released for the first time in 2014. It was designed on the same technology as was Kinect for Xbox one.
Figure 3-Kinect sensor components. Journal of Sensors (2013)
1.7 Significance of Kinect v2 in Assisted Living Facilities
According to Biswass (2011), Kinect v2 is an advanced motion sensor capable of measuring 3-D motion in a person. Kinect SDK, Microsoft made Kinect for Win-dows was an interface to kinetic hardware, which was provided by an Application Programme.
Assisted living residence is for the people with some disability or those that have attained old age who cannot live independently or have opted to not live inde-pendently (El-laithy, 2012). In recent past, with scientific developments in this field, there has been a transformation from ‘care as a service’ to “care as business”. It has evolved to a huge industry, in 2012, a survey conducted in US facilities showed ex-istence of 22,500 such facilities. These can be standalone services or part of multi- level senior living community. Kinect v2 sensor has emerged as a potential contribu-tor in improving the standards of assisted living. The features of v2 are; enhanced field of view, improved picture resolution, enhanced skeletal tracking and recogni-tion of joints.
1.8 Potential Use of Kinect V2 in Assisted Living
Most researchers like Stowers (2104) agree that Kinect v2 can be a potential contrib-utor to many more domains to enhance standards for assisted living. It can be used in building of smart home environment, detection of driver fatigue by multi-sensor sig-nals based methods and to model movement in human body by using twin cylinder method. Kinect v2 can also provide a platform for live 3-D human reconstruction as well as capturing motion. It can help in monitoring of patients during external beam radiotherapy and assist in recognition of Karate techniques and similar domains.
In the rehabilitation system, it can help in undertaking skeletal tracking in virtual reality rehabilitation system. Kinect v2 has also been widely used in geometry re-finements required in the motion fields and human body tracking based on discrete wavelet transform (DWT). Moreover, it can be used in shadow detection and classi-fication, estimation of movements of human body parts and propagation along hu-man body parts (Rowe, 2011).
1.8.1
Different Spheres with Scope for Application of Kinect v2 Sensor
In this paper we will capitalize upon the potential of Kinect v2, with regards to assited living. Kinect v2 can be a contributor to facilitate living in assisted living environment for elderly people and treatment of illnesses. People can perform their routine exercises under the view of Kinect sensor, because it can analyze the move-ments and correct any mistakes and accordingly pass on the instructions. This can provide much needed motivation for elderly people to exercise regularly. Another innovation of v2 sensor with regards to assisted living can be by providing a hearing system using human body as a medium of transmission. This mechanism of replacing sound transmitter and transmission line can be done by Kinect v2 Sensor.
Kinect v2 can also help in the treatment of Parkinson’s disease. It can measure clini-cally relevant movements with accuracy like hand clasping and even tapping. Rela-tive improvements or worsening of these movements over time could also be accu-rately measured using Kinect v2.
2 Review of related literature
2.1 Introduction
A variety of studies have been undertaken to review the efficacy of Kinect v2 Sensor.
Researchers have even gone ahead to recommend use and applications. However, utilization of Kinect v2 in assisted living is rarely found. Some of the research in-clude;
2.2 Use of Wireless Sensor Networks
For Wireless Sensor Networks (WSNs), analysis, proposal and implementation for smart home for assisted living has been done by Hemant and Ghayvat (2013). Ac-cording to them WSNs are today the backbone of many systems. Smart home sys-tems that provide assisted living to patients already use WSNs. These researchers provided a protocol designed for providing smart homes for assisted living. They described this protocol implementation in an old home built to specifically test the implementation of a wireless sensor network. The protocol targets event and com-munication based protocols and provides smart home solutions. However, sensors alone were not found to be enough. Intelligent sampling and control algorithm is designed according to sensor type and structure.
2.3
Kinect v2 Depth Sensor
Research by Lin and Longyu (2013) extensively described the use of Kinect depth sensor, since its launch. Even though Microsoft has released a new version with im-proved hardware as well, however, in their view the accuracy needed a test. They performed experiments to check the Kinetic v2 depth sensor and its accuracy. They observed some variations in the depth evaluations of the Kinect and proposed a toler-ation method to enhance the accuracy while evaluating depth. [2]
2.4
Use in Karate Techniques
An Effectiveness comparison of Kinect v1 and Kinect v2 for recognition of Oyama karate techniques has been done by (Marek and Tomasz, 2010). The purpose of study was to evaluate Kinect v1 and Kinect v2 to recognize the actions of Karate Tech-niques named as Oyama. Initially, multimedia cameras were famous for personal computers and game consoles and were also cheaper while being used for these
pur-poses. However, Kinect v2 has given the concept a wide array of use. Its use for hu-man computer interaction also gave it a new dimension.
According to their research Kinect can be used in medicine, education and for con-trolling robotics arm. Kinect v2 has come up as one of the best intelligent home solu-tions and has many potentials, yet to be explored and fully utilized. Postural segmen-tation and assessment of postural control capabilities are the most common ap-proaches to be used. Classification method is used to make gesture recognition pos-sible. To perform tracking and generate motion capture data, kinetic sensor data is preprocessed by kinetic libraries. Kinect v2 has appeared by enhancing the capabili-ties of its predecessor i.e. Kinect.
2.5
Advantages of v2 over v1
In Kinect v2, Gesture Description Language (GDL) has been used as a classification algorithm. The data was recorded by two professional and belt instructors. The re-search collected 200 x movement samples per person. The data was divided into two sections as training and evaluation. The data was then thoroughly assessed. Kinect v2 proved to be more reliable than Kinect v1, taking stock of recognition rates of GDL classifier and error classification cases. The major advantage of Kinect v2 over Kinect v1 was accurate calculation of leg joint positions. [3]
Figure 4- GDL illustration. Teng et.al. (2013)
A different research conducted by the University of North Carolina at Chapel Hill has correctly illustrated the functions and classification of Kinetic shadow detection feature. The research shows that Kinetic maps are often found with holes, missing data or similar missing links in many of the cases. In their research they advocate a different idea, which is, turning holes into a useful information (Teng and Hui, 2014). They proposed different types of shadows based on local patterns as shown by geometry. Shadow information is then fully used. [4]
2.6 Pose Estimation of Human Body Part Using Multiple Cameras
There is a lot of existing research on methods of estimating the pose of multiple 2-D and 3-D images and objects as a starting point (Kuntal and Jun, 2014). In the re-search the approximate volume in 3-D is obtained by projecting the silhouettes in images. The authors analyzed that existing means of communication like the video conferencing systems have few limitations. The users are often at far distances, one of the solutions viewed for this problem are feeling of co-location of humans [2].
Views of points in real space interact to object in space. It has tackled the issue by assuming space in 3-D modeled. In this paper an example of human body part with pose estimation has been given. The works include pose parameters by random selec-tion. The author conducted few experiments using CAD model of a human head, which was undertaken utilizing 4 cameras. These were placed in a semi-circle in equal distance. Any algorithm for the estimation of pose is difficult to extend easily for the application. Silhouette edges for experiments were separated manually. Three randomly chosen points in volume are taken, every fifth point on edge of silhouette is taken. The results were initially not good, but the results later improved. The algo-rithms developed by them can easily be used in future [3].
2.7 An Innovative Hearing System Utilizing the Human Body as a Transmission Medium
Some researchers recommended an innovation in hearing system using the human body as the medium of transmission (Son and Kwang, 2013). This concept has made the replacement of sound transmitter with human body. Self-demodulation is the base of generating audible sound. Frequency of two waves difference in audio signal was produced by self-modulation effect, through a non-linear medium. In this con-text a user is able to hear sound without a transmitter and making noise by using self-modulation. The concept of wireless sound transmission has been given by the au-thor. Distortions in propagation process can be reduced by ultrasound [19]. The pa-per has successfully given the concept of using human body as a transmission medi-um for the proposed system as model to be used.
Figure 5- Medical application. Lim et.al. (2014)
2.8 Accuracy and Reliability of Optimum Distance for High Per-formance Kinect Sensor
A high-performance research conducted by Lim and Shafriza (2013), analyzed the sensor from a different perspective. In their camera i.e. depth/rg camera, each pixel represented a distance, which corresponded directly to some point in this physical world [20]. Biomedical application is one of the successful features of Microsoft Kinect Sensor as it gives the necessary tools required to provide measurements of volume, length and other measurements. These technologies have become popular with time, applications like Time of flight (TOF) and Microsoft Kinect Sensor are applicable in biomedical field, and come in domain of range camera. The principal of working of TOF camera is of emission of modulated light on the scene [17]. This light is reflected and measured with reference signal. To obtain depth information, it is correlated with modulated light. The technique used by Kinect sensor is different as utilized in infrared structured light projector and CMOS camera, which computes
depth of the scene. Now, 3-D technologies have come in market with depth cameras and Kinect Sensor. The primary aim of development of Kinect sensor is its utiliza-tion in biomedical applicautiliza-tions. This Kinect sensor is like a camera, due to its speci-fications [14]. These authors focused on the fact that Kinect sensor can provide ac-curate and reliable depth distance values same with actual distance. The analysis of the measurement of depth to actual distance has a lot of importance for the accuracy of Kinect sensor. The depth array calculated by the researchers had a precision of up to 11 bits value. Therefore, it is likely that the depth sensor measurements of Kinect Sensor will provide non-linear function of distance [18]. The focus of the research was also on default range and near range of distance from Kinect sensor.
The authors undertook the task of investigating depth data of Kinect sensor. In this study, they carried out a reliability analysis of the sensor’s specification as claimed by Microsoft. This research provided an insight into the authenticity of data. Experi-ments conducted with these sensors have proved that error in depth measureExperi-ments are enhanced by enhancing distance to sensor. These variations range from a few mm to 40 mm at the max [15]. The formula used for these calculations was Kuder-Richardson formula. This study proved to be very useful as it provided the method-ology to determine 3-D pose estimation in human motion application by carrying out accurate, precise and reliable depth distance.
2.9 Integration of Microsoft Kinect with Simulink: Real-Time Ob-ject Tracking Example
Microsoft Kinect has a lot of potential in system applications due to its introduction as a low cost and high resolution 3-D sensing systems (Joshua and Tyler, 2015). The purpose of their study was to develop Kinect block, and provide access to depth im-age streams, and access to sensor cameras. Available drivers of Kinect, the interface of C language, is an impediment to Kinect application development. This access pro-vides the ability to incorporate without any difficulty to an image processing based on Simulink. The study focused on issues affiliated with implementation aspect of Kinect. One of such important aspect is calibration of sensor, another is utility of Kinect block and Kinect and it is through 3-D object tracking example [9].
Figure 6- Simulink Kinect. Joshua et.al. (2014)
For detection of both moving and stationary obstacles, it is greatly dependent upon the capability of systems to navigate in unsure circumstances. In the available sen-sors, Sonar sensors is a low-cost option, however, these are prone to false echoes and reflections due to bad quality angular resolution. Another option is Infra-red and la-ser range finders, these are also low cost, but the grey area is provision of measure-ments from only one point in the scene. Other available option is of spectrum, Radar and Light Detection and Ranging (LIDAR) systems, these can provide precise meas-urements along with good angular resolutions [14]. However, they also have grey areas. Most important of them are their high-power consumption and high expenses.
This complete picture and the revolution of low cost digital cameras has produced an interest in vision-based setups for the vehicles, which are autonomous. Even in this case, the disadvantage of distance must be of stereoscopic cameras. The recent re-lease of the Microsoft Kinect addresses this issue by providing both a camera image and a depth image. Kinect was primarily for entertainment market, however due to its powerful capabilities to operate, it has gained a lot of popularity in sensing and robotic community. Few of the examples of this popularity are applications related to robot / human interaction, 3-D virtual environment construction, medicine, robot tracking and sensing. Most of Kinect applications are coded in C [15]. In industry and academia, use of image processing is now common place. Even inexperienced users can use these tools. These are used to target hardware to implement by making use of code generation automatically. Simulink provides a widely accepted environ-ment for designing impleenviron-mentation of image processing algorithms as well. For ex-ample, in automotive industry, Simulink is used to produce seamless code-generation tools. These translate the final design into a target hardware, which is real time-executable. These tools have a use in educational environments, it enables the stu-dents to concentrate on major details rather than low level. The major contributions of this study can be divided into three major spheres. Interface development allowing Kinect to be involved in refined Simulink designs, it allows more users to access.
The targets discussed in the paper are Linux based, which are used in mobile auton-omous robotic machines. Real time Kinect streams parallel camera and depth images.
2.10
Comparing the utility and usability of the Microsoft Kinect and Leap Motion sensor devices in the context of their application for gesture control of biomedical images
A study conducted by Hughes and Nextorov (2014) investigated the interaction of medical images in operating room with a requirement of maintaining asepsis. This arrangement has resulted in a complex type of arrangement in use of mouse and key-board, between scrubbed clinicians and on other end non-scrubbed personnel [16]. It is Microsoft Kinect or Leap motion which could give direct control of medical image navigation and manipulation.
Figure 7- Leap Motion Sensor. Hughes et.al. (2015)
The authors admitted that many studies have already been undertaken to study use of Leap and Microsoft Kinect in Operating Room, however, no study had compared the sensors in terms of their usage. They aimed their study to compare the use, utility, accuracy and acceptance of the two motion sensors. In this research, 42 persons par-ticipates. Out of these 30 % were diagnostic Radiologists and 70 % were surgeons or Interventional Radiologists. All the participants were having good computer skills but limited gaming experience. In analysis of utility of two motion sensors, 50 % of participants rated Microsoft Kinect v2 as very useful in their routine practice, how-ever performance of Leap Motion Sensor was 38 %. Out of Surgeons and Interven-tional Radiologists 54 % rated Microsoft Kinect as useful [13]. Younger participants found Leap Motion interface as more useful than older. In 37.5 % participants, after use of Kinect sensor, perception of leap motion sensor deteriorated. System accepta-bility was better for Kinect Sensor, as compared to Leap Motion Sensor. With re-spect to utility and use, Microsoft Kinect was found better. However, Leap motion was found to have a better accuracy. Kinect was more acceptable to the users, alt-hough Microsoft Kinect was tiresome physically. More than half of the surgeons and
interventional Radiologists found Microsoft Kinect v2 as very useful. As regards to this study, Vascular and Orthopedic surgeons found the sensors to be more useful.
The measurement accuracy was found not to be of good standard, which can be at-tributed to many factors including the system’s field of view.
For Leap Motion Sensor, user needed to place the cursor at end or start point of
For Leap Motion Sensor, user needed to place the cursor at end or start point of