Developing and Testing Embodiments of TIS

70

(actuators) are embedded within the LSO, the audio noise is also substantially limited, thereby, decreasing the signal to noise ratio for the vibrotactile signal, yet ensuring the haptic stimulus remains within the necessary threshold.

Figure 20. Illustrates how the HIPU evaluates the sensory information and environmental noise to modify actuation signal. Adapted from “TIS Patent”, Evreinov et al., 2016c, 20160012689-A1 - Tactile Imaging System © Fukoku Inc. and University of Tampere, 2016.

4.5 D

T

E

TIS

Once the specifications of an active mediation mechanism were defined, researchers at UTA developed demonstration samples of various embodiments to test the improvement in performance and signal to noise ratio. One such embodiment (called simply the Liquid Screen Overlay was created to test the active mediation mechanism and was a custom

……………

71

transparent overlay for an ExoPC Slate (tablet device), running Meego 1.2.

This overlay was created by using 0.1mm thin flexible plastic sheet fitting to the size of the ExoPC Slate display (270 by 165 mm). The plastic sheet was glued (using adhesive silicone tape) to the Plexiglas frame of 1 mm thickness creating a hollow pouch from the middle, approximately the size of the ExoPC Slate display.

The researcher then shaped two 0.5mm holes on the frame through the sheet and used them to inject a non-conductive low viscosity oil mixture into the pouch. The oil was injected, directly filling in the space between the display glass and the thermoplastic covering with ~160mg of oil along with the embedded actuators. Once a substantial bulge was apparent in the pouch, any remaining air bubbles were evacuated and the two holes were hermetically sealed with proper miniature screws. The researchers then affixed various actuators on top of the liquid overlay (Fig. 19 (a)) and compared both, the embedded and externally mounted actuators, with reference to signal actuation and noise to signal ratio. The results showed greatly increase efficiency of the all the actuators and suggested that the signal remained intact throughout the touchscreen.

(a) (b)

Figure 21. (a) Shows the structure of the LSO and embedded actuators and (b) illustrates the external and internally mounted actuators. Adapted from Farooq et al. Evaluating Transparent Liquid Screen Overlay as a Haptic Conductor. In Proceedings of IEEE SENSORS

Conference. © IEEE eXplorer, 2015a.

Evaluating Efficiency of the LSO Tactile Information System

To gauge the efficiency of the liquid overlay in providing haptic signals, the team at UTA created two identical setups containing four actuators using two sets of ExoPC tablets. One of the tablets was affixed with the transparent liquid screen overlay, while the other tablet was used without any covering. Both setups were affixed with AAC PV4042A-02 and AAC PV35-3L-01 piezoelectric actuators along with the HiWave HIHX09C005-8 voice coil exciters. For the tablet with the liquid screen overlay, these actuators were attached to the screen overlay, whereas for the other setup, these actuators were directly attached to the glass touchscreen. Placement of the actuators was identical in both setups, as all the actuators were directly attached just outside the touch sensitive areas of the screen (Fig.

……………

72

20 (a) and 20 (b)) using similar silicone based adhesives. For added measure, the setup with liquid screen overlay was also equipped with another AAC PV35-3L-01 piezoelectric actuator which was embedded inside the liquid overlay. The reason for selecting these particular actuators was that they each had different resonance frequencies as well as forces, which would be idea in gauging any distortion caused by the viscosity of the oil being used in the liquid overlay.

To simulate the force of a light fingertip touch, an artificial finger was used with embedded extra weight (steel core covered with multilayered rubber having a density (of about 1100 kg/m3 [Maeno et al., 1998] and friction similar to a human skin) producing a continuous static load of 100gF (see section 3.8 for more details). The artificial finger-probe also measured the vertical and horizontal components of vibration signals delivered to the point of the fingertip contact (the touchscreen). A MicroSense displacement sensor (5810) and 5622-LR, 20 kHz probe were also used to record vertical displacement. Moreover, a standard Sound Level Meter 2250 (Brüel & Kjær) was used to measure the Audio Decibel levels (dBA) generated by all the actuators in both setups across the 3 measured frequencies (Fig. 21).

(a) (b)

Figure 22. Studies to evaluate various actuators and their performance increase in LSO based mobile (below) and tablet (above) setups. Adapted from Farooq et al. Evaluating Different Types of Actuators for Liquid Screen Overlays (LSO). In Proceedings of IEEE Design

Test Integration and Packaging of MEMS / MOEMS, 2016a, 97- 102. © IEEE eXplorer, 2016.

Results showed that all four actuators operated far more efficiently in the setup containing the haptic mediator (LSO), across the measured frequencies, as compared to the condition in its absence (Fig 21). The haptic mediator was able to transfer the vibrotactile signals to the fingertip contact, ensuring minimum noise to signal ratio and maximum conductance of informative tactile vibrations between the actuator and the point of contact. This was visible for all actuators though out the range of inspected frequencies (120-280 Hz), excluding the PV4042A-02 piezoelectric actuator, which due to its specific design and attachment produced more acoustic noise at 280Hz as it caused bending waves to travel over the plastic surface of the overlay, rather than exciting the liquid

……………

73

within a pouch. Yet, the actuator’s vibration component was 2-3 times stronger than its acoustic components below 200Hz.

Figure 23. Shows the attenuation of the acoustic components (noise) in the presence of (left) and without (right) the use of LSO, mediating vibrotactile signals from actuators.

Adapted from Farooq et al. Using Skin Micro-Displacements to Create Vibrotactile Signals for Mobile Touchscreen Displays, Sensors Journal IEEE, vol. 16, pp. 6908-6919, 2016, ISSN

1530-437X. © IEEE eXplorer, 2016.

Future research in Active Mediation

Other studies (similar to the one discussed in section 4.5) were conducted in both mobile [unpublished research] (Fig.20 (b)) and tablet (Farooq et al., 2016a) (Fig 20 (a)) form factor devices and the results in general were found to be very similar. During these studies, actuators were also tested for their efficiency to work within and on top of LSOs which illustrated how current available technology could be used with possible future Liquid screen overlays. Due to the increase in signal integrity throughout the touchscreen device as well as the decrease in signal to noise ratio, it is clear that active mediation is the way forward for developing precise and accurate vibrotactile feedback.

Utilizing these results and lessons learnt from exploring LSO in its different embodiments, the team at UTA filed another patent application (Evreinov, 2016d) which provided further details on how different methods of attaching and utilizing actuation mechanisms alongside and TIS based system, could be used. These approaches can become even more useful with current flexible display technologies (AMOLED), ensuring that the combination of the display and the LSO remain as thin as possible without compromising the deformable indentation present in the current generation of TIS. Furthermore, UTA researchers are currently working with the development team at Fukoku Motors (Japan) to bring this technology to the general public, improving multimodal interaction in mobile and handheld touchscreen devices.

……………

74

Figure 24. Possible variation of future TIS based system and Actuator configurations.

Adapted from “Haptic Device”, Evreinov et al., 2015, US20160011666-B1 © Fukoku Inc. and University of Tampere, 2015

A –Actuator B – Actuation Signal

C – Liquid Screen Covering

D – TS Display E – Liquid / Gel

F – Liquid / Gel in external Reservoir

On top Within or under the Liquid Screen

Overlay On top Within

or under the Touch Screen

Display Within or

remote attachment of

Actuator

……………

75

5 Vibrotactile Feedback in