What if you could touch a loved one during a video call – particularly in today’s social distancing era of COVID-19 – or pick up and handle a virtual tool in a video game?
Pending user tests and funding to commercialise the new technology, these ideas could become reality in a couple of years after UNSW Sydney engineers developed a new haptic device which recreates the sense of touch.
Haptic technology mimics the experience of touch by stimulating localised areas of the skin in ways that are similar to what is felt in the real world, through force, vibration or motion.
Dr Thanh Nho Do, Scientia Lecturer and UNSW Medical Robotics Lab director, is senior author of a study featuring the new device.
His research team featured lead author and PhD candidate Mai Thanh Thai, Phuoc Thien Phan, Trung Thien Hoang and collaborator Scientia Professor Nigel Lovell, Head of the Graduate School of Biomedical Engineering.
Dr Do said the sense of touch was something many people took for granted to perform everyday tasks.
“When we do things with our hands, such as holding a mobile phone or typing on a keyboard, all of these actions are impossible without haptics,” he said.
“The human hand has a high density of tactile receptors and is both an interesting and challenging area to encode information through haptic stimulation, because we use our hands to perceive most objects every day.
“There are many situations where the sense of touch would be useful but is impossible: for example, in a telehealth consultation a doctor is unable to physically examine a patient. So, we aimed to solve this problem.”
The UNSW study about the new haptic device was published in the Institute of Electrical and Electronics Engineers (IEEE) Access journal recently.
Dr Do said the researchers were so excited about their new haptic technology that they had applied to patent it.
“Our three-way directional skin stretch device (SSD), built into the fingertips of the wearable haptic glove we also created, is like wearing a second skin – it’s soft, stretchable and mimics the sense of touch – and will enable new forms of haptic communication to enhance everyday activities,” he said.
“What’s also special about our new technology is that it’s scalable and can be integrated into textiles for use in various potential applications such as telehealth, medical devices, surgical robots and training, augmented and virtual reality, teleoperation and industrial settings.
“The device aims to solve a common problem in emerging systems – such as assistive devices, remote surgery, self-driving cars and the guidance of human movements – where visual or auditory feedback can be slow, unintuitive and increase the cognitive load.”
Why haptic technology needs to improve
The study’s lead author Mai Thanh Thai said existing technology had great difficulty recreating the sense of touch with objects in virtual environments or located remotely.
“Visual or auditory cues are easy to replicate, but haptic cues are more challenging to reproduce. In a virtual environment, we can see objects but we are unable to feel them as if we were directly touching them,” Mr Thai said.
“It is almost impossible to enable a user to feel something happening in a computer or smartphone using a haptic interface, such as commercially available smart glasses.
“Vibration is the most common haptic technology today and is built into many electronic devices, such as the Taptic Engine attached to the back of a trackpad in laptops, which simulates a button clicking.
“But haptic feedback with vibration becomes less sensitive when used continuously or when users are in motion, leading to desensitisation and impaired device functionality.”
How the new haptic device works
Dr Do said the researchers’ new technology overcame issues with existing haptic devices by introducing a novel method to recreate an effective haptic sensation via soft, miniature artificial “muscles”.
“Our soft, wearable haptic glove enables people to feel virtual or remote objects in a more realistic and immersive way. The inbuilt soft artificial muscles generate sufficient normal and shear forces to the user’s fingertips via a soft tactor, enabling them to effectively reproduce the sense of touch,” he said.
“It works like this: imagine you are in Australia while your friend is in the United States. You wear a haptic glove with our integrated three-way directional SSDs in the fingertips and your friend also wears a glove with integrated 3D force sensors.
“If your friend picks up an object, it will physically press against your friend’s fingers and their glove with 3D force sensors will measure these interactions.
“If these 3D force signals are sent to your haptic glove, then the integrated three-way directional SSDs will generate these exact 3D forces at your fingertips, enabling you to experience the same sense of touch as your friend.”
Implications of the new technology
Dr Do said the ability to effectively reproduce the sense of touch via the new wearable haptic device would have a wide range of benefits; for example, during today’s COVID-19 pandemic when people were relying on video calls to stay connected with loved ones.
“Unlike existing haptic devices, our technology is soft, lightweight, and thin and therefore, we hope users will be able to integrate it into what they’re wearing to provide realistic haptic experiences in settings including rehabilitation, education, training and recreation,” he said.
“Our technology could enable a user to feel objects inside a virtual world or at a distance; for example, a scientist could feel a virtual rock from another planet without leaving their lab, or a surgeon could feel a patient’s organ tissues with surgical tools without directly touching them.”
Dr Do estimated the new technology could become available in the next 18 months to three years – if plans to commercialise the device were realised.
“The next step is to conduct user evaluations to validate how effective our device is, because the main scope of our current research has been on the design, fabrication and characterisation of the new technology,” he said.
“In addition, we plan to implement the device in various haptic applications such as haptic motion guidance, navigational assistance for older people and those with low vision, tactile textual language, and 3D force feedback display for use in surgical robots, prosthesis and virtual and augmented reality.”
###
