Research on electronic skin in artificial intelligence, smart medicine, virtual reality
Research Article
Open Access
CC BY

Research on electronic skin in artificial intelligence, smart medicine, virtual reality

Xuyang Gao 1, Tianyi Li 2, Xinyan Yu 3, Mohan Liu 4, Herong Sheng 5*
1 Shanghai University
2 School of Physic Dalian University of Technology
3 Shanghai WenLai International High School
4 New Channel Qingdao
5 School of Advanced Technology Xi'an Jiaotong-liverpool University
*Corresponding author: Herong.Sheng21@student.xjtlu.edu.cn
Published on 23 February 2024
Volume Cover
ACE Vol.42
ISSN (Print): 2755-273X
ISSN (Online): 2755-2721
ISBN (Print): 978-1-83558-309-8
ISBN (Online): 978-1-83558-310-4
Download Cover

Abstract

Electronic skin(e-skin) is an emerging type of wearable sensor in order to translate external stimuli into different output signals. Electronic skin is very important in many fields, including Artificial intelligence, smart medicine, and virtual reality. The electronic skin is made of a composite material called QCT and various transistors and sensors. Therefore, electronic skin can convert various biological signals such as temperature, pain, pressure, etc. into electrical signals. In addition, the electronic skin has the characteristics of being thin, soft, and flexible. This article will first introduce the application of electronic skin in artificial intelligence, electronic skin can make artificial intelligence better sense and adapt to changes in the environment. The second is the role of smart medicine, the most widely used is the use of prosthetics. Finally, there is the role of virtual reality, which allows users to have a more realistic experience through the use of electronic skin. The findings presented in this paper hold significant value for advancing research and facilitating the practical application of electronic skin technology.

Keywords:

Electronic Skin, AI, Smart Medicine, Virtual-Reality

View PDF
Gao,X.;Li,T.;Yu,X.;Liu,M.;Sheng,H. (2024). Research on electronic skin in artificial intelligence, smart medicine, virtual reality. Applied and Computational Engineering,42,195-200.

1. Introduction

Electronic skin is a kind of system that can let robot have the sense of touch. Just like human skin, it can sense objects' hardness, temperature, position and texture. The electronic skin is simple in structure and can be processed into many shapes and placed on the surface of the robot like a garment. E-skin can be used in a variety of scenarios, in medicine, scientific research, entertainment and other scenarios have played a great role in wearable technology has become a hot topic of research at home and abroad, electronic skin is used in wearable electronic skin as one of the important, can simulate the human skin to feel the external stimulation, therefore, it has been widely used in medical health detection and diagnosis, robot touch and control and human-computer interaction. However, the problems of large size, high cost and wearing discomfort have become the bottleneck of the development of wearable electronic skin, it has great research significance and application value in improving its performance parameters and using functions [1]. As the human body's largest organ, the skin is the carrier of tactile sensation, providing us with physical information in the environment. We can feel an object's hardness because there are many pressure receptors scattered in our skin, which can distinguish the softness and hardness of an object based on the magnitude of the force felt when the skin comes into contact with the object. So, through pressure sensors, electronic skin can also sense the softness and hardness of objects. There are various ways to achieve the effect of pressure sensing, among which the simplest and most commonly used is the piezoresistive pressure sensor, which utilizes the resistance changes generated by conductive materials during deformation to achieve sensing.

2. Electronic Skin and AI

Electronic skin has recently attracted many scholars to study in the field of artificial intelligence, and has made significant breakthroughs in this area. With the continuous progress and development of intelligence, people are gradually combining electronic skin and artificial intelligence to deeply develop corresponding technologies, committed to developing deep intelligent advanced cognitive systems.

Through reading the paper, in recent years, the scientific research team has developed an electronic skin based on the human FSB structure (an assembly of a triboelectric e-skin stacked with a piezocapacitive (supercapacitive iontronic) e-skin exhibiting a three-layer structure configuration), which uses advanced materials at present. This electronic skin has many advantages, such as its ultra-high sensitivity and high intensity linear response over a wide range of pressures, last but not least ,it’s response/recovery time is very fast. Most importantly, the FSB electronic skin has ability to transition from tactile perception to advanced intelligent tactile cognition. The following article conducted a series of experiments to test the various performance of this electronic skin. The researchers also created three types of electronic skins containing planar or structural media or electrode layers and tested their sensitivity using pressure. Finally, they found that the sensitivity of these three types of electronic skins was much lower than that of the electronic skin we created. They also used different methods to release pressure (placing pressure on the surface of the electronic skin and quickly releasing it) to test the corresponding/recovery time of the electronic skin. And under the premise of pre applying 25kPa, 80Pa and 60Pa were applied to the surface of the electronic skin to test its ability to detect subtle changes. The results showed that the changes in subcutaneous capacitance were clearly visible when applying small pressure, indicating that it can recognize small pressures and achieve maximum pressure resolution. Due to the fact that human skin can be bent and transformed at will, researchers conducted a large number of repeated pressure tests on the device under folding and twisting conditions. In these experiments, we found that it exhibited excellent stability, indicating its great feasibility in practical life applications. These experiments are crucial for the practical application of electronic skin. [2]

In recent years, another type of electronic skin called CPC based flexible electronic skin has also emerged, which is widely used in the field of artificial intelligence. It is divided into sensing electronic skin (mainly composed of external sensitive electronic sensors). By changing the connection of the conductive network or the distance between electrodes of the flexible electronic skin, different electrical signals are output in the form of resistance, current, voltage, or capacitance, thus achieving the identification of the stimulation degree of different external factors, It is mainly widely used in the field of artificial intelligence, such as in real-time human monitoring systems, with great potential for application. Unlike sensing skin, energy-based electronic skin, which requires battery power, can largely self power (draw energy from the environment) and has good stability and high scalability performance. The commonly used methods for preparing CPC-based electronic skins include the blending Forming method, electrospinning method, wet spinning method, 3D printing method, vacuum filtration method, and freeze-drying method. Taking the vacuum filtration method as an example, nanocomposites were first prepared using hot pressing and two-step vacuum filtration methods. Due to the double-layer structure and hydrogen bonding interaction, the material exhibited good conductivity, good mechanical properties, tensile strength (235.9 MPa), 24.8% fracture strain, and excellent electromagnetic shielding effectiveness. It has great application potential in aerospace, artificial intelligence, and wearable electronics fields. With the development of these electronic skins, the connection between electronic skins and the field of artificial intelligence has become increasingly close. Currently, international researchers are focusing on developing multi-functional interactive electronic systems, and the clear recognition of external colors and brightness by electronic skins has become the main direction of efforts.

In addition, the power supply problem of electronic skins in practical applications in artificial intelligence has also become a major challenge. As an emerging field of nano energy, sustainable micro/nano power sources can obtain energy from the environment and provide power for micro/nano systems by utilizing nanotechnology and nanomaterials to collect energy. In a word, The application of electronic skin in the field of artificial intelligence is very extensive and there is still a long way to go.

Finally, complete content and organizational editing before formatting. Please take note of the following items when proofreading spelling and grammar:

Finally, complete content and organizational editing before formatting. Please take note of the following items when proofreading spelling and grammar:

3. Electronic Skin and Smart Medicine

The skin is the largest organ in our body. For the human body, the skin plays a variety of important and complex functions such as protection, regulation and sensation. From these functions, we can see the important roles that the skin plays in our body. In the same way, we can also see e-skin's effect on our human body. Smart medicine is based on modern medical and biological theories, integrating advanced brain cognition, big data, cloud computing, machine learning and other artificial intelligence and related engineering technologies to study the nature and to explore the intelligent diagnosis and treatment methods and clinical applications of man-machine collaboration. At present, the application of electronic skin in intelligent medicine mainly lies in the field of prosthetics. For those with disabilities, the loss of body parts also means that they have lost part of their sense of body. Although artificial prosthetics can make up for their missing body parts, those lost senses will still make their lives extremely inconvenient, and this is where e-skin comes into play.

3.1. Technical principles of sensors

At present, tactile sensors can be divided into piezoresistive, capacitive, piezoelectric, inductive and optical in principle.

A piezoresistive tactile sensor is a sensor prepared using the piezoresistive effect. The piezoresistive effect means that when the piezoresistive material is stressed, its internal resistivity changes, and the acquisition circuit detects the force on the sensor by converting the change in resistance into an electrical signal that is easy to measure; The principle of capacitive tactile sensor is different. When the capacitive tactile sensor is subjected to a normal force, the distance between the upper and lower electrode plates changes, resulting in a change in the capacitance of the sensor; when the sensor is subjected to a tangential force, the overlapping area of the upper and lower electrodes changes, which also causes the capacitance of the sensor to change. Capacitive sensors are therefore convenient for measuring the magnitude of three-dimensional forces; The piezoelectric tactile sensor is a sensor based on the piezoelectric effect. The piezoelectric effect means electric polarization occurs inside when the crystal is subjected to a normal force. When the external force is removed, the crystal returns to an uncharged state. Connect the piezoelectric tactile sensor to the charge amplifier and the measurement circuit, and collect the electrical signal related to the external force, we will get the tactile signal; The optical tactile sensor maps the pressure on the sensor to the change of light signal intensity, wavelength and other properties. It detects the pressure on the sensor by detecting the optical signal.

3.2. Technical requirements for e-skin

Compared with our real skin, e-skin naturally cannot match our body completely, so we also have some technical requirements for e-skin to improve the possibility of its practical application. To achieve the functionality of our real skin, e-skins must become planarized, large-area surfaces which can be capable of carrying miniaturized sensor circuits. So that e-skins must satisfy two conditions: first, e-skins needs to have wearability, it’s necessary to meet our physical activity needs and it’s related to our bodies’ mechanical properties and integration; second, it should have transducing functions, it means that we need to choose the appropriate devices and use the correct processing and communication methods. In a word, we need to choose the suitable materials, structures, and manufacturing methods to ultimately obtain the e-skin which has normal functions [3].

Ductility and conductivity: Our real skin has many interesting characteristics, such as good stretchability and tactile perception ability. The stretchability ensures that all body parts can move freely, and the tactile perception ability allows us to respond to the external stimuli in time. It can be said that these two points are the most basic functions of the skin, they are also the functions that the electronic skin must realize. It is not an easy job to achieve these two points simultaneously. It is known that traditional electronic components are rigid, so we must develop new materials to achieve our goal. Recnetly, Wang w et al. [4] came up with a new solution to this problem. They developed a trilayer dielectic by passivating the high-permittivity nitrile-butadiene rubber with an ultrathin, nonpolar poly (styreneethylene-butylene-styrene)(SEBS) elastomer coating(~ 15 nm), followed by a hydrophobic octadecyltrimethoxysilane (OTS) molecular modification. This material has good stretchability while ensuring high charge-carriier mobility.

Self-healing ability: Our skin has a certain self-healing ability, and the electronic skin will also be damaged in the accumulated work, so it must have a self-healing ability, too. In 2015, J. Cui of Harvard University achieved the self-healing function of the skin by storing the repair fluid in the gel matrix (composed of dynamic polymers) which is the substrate of the e-skin [5]; In 2021, S.Y.Yin et al. from Sichuan University developed a polyacrylic acid(HAPAA) hydrogel which can be used as an electronic skin material[6], this material has a strong self-healing ability while having good stretchability.

4. Electronic Skin and Virtual Reality

In addition to the artificial intelligence and medical applications described above, e-skins can also be used in virtual reality technology (VR). VR is a type of virtual reality that allows users to immerse themselves in a computer-generated environment and experience various elements of the virtual environment [7]. A system resembling a predictive sensor search engine holds the potential to expedite the progress of thrilling applications in human-digital interfaces powered by virtual and augmented reality. In the past, people often enhanced the VR experience by adjusting the vibration of the game controller. However, this kind of experience makes the user only feel the vibration which is very boring. Now, the E-skin can be applied to VR to enhance the player experience. For example, players will feel hot when they encounter volcanic terrain through their electronic skin and cold when passing through snow. The use of electronic skin can greatly enhance players' experience during play, mainly because the electronic skin strengthens the shortcomings of the weak touch of virtual reality.

In the existing virtual reality equipment, manufacturers often pay more attention to the user's visual experience and auditory experience. Apple's Vision Pro, for example, is a wearable virtual reality eye that allows users to control the device through body movements. The glasses provide users with a high degree of realism and immersion. They not only display extremely realistic colors, but also simulate sound with extremely high precision, and this realistic picture and sound effect greatly enhances the reality of the virtual world. However, neither the Vision Pro nor any other virtual reality device has a good tactile experience. However, the use of electronic skin can compensate for this shortcoming. One of the main roles of e-skin in virtual reality is to further enhance the user's sense of immersion. Through the haptic feedback system of the electronic skin, users can experience various contents in the virtual world more truly. For example, feeling temperature changes in the virtual world, the weight of objects like those in the real world, and the sense of touch of various objects can be enhanced by electronic skin. In addition, the use of electronic skin can also optimize the interface of human-computer interaction. Most of the VR devices on the market are controlled by gloves or joysticks. In previous studies, gestures provided a more intuitive human-computer interaction than a sense of control or a glove [8]. However, these controllers can seem a little cumbersome and unnatural, and the use of electronic skin can solve this problem. Because the electronic skin is made of QCT composite material, this material can be attached to the surface of the human body like clothing, without causing discomfort to the human body. Therefore, it can be used to simplify the interface of human-computer interaction, so that users can operate more naturally.

/word/media/image1.GIF

Figure 1. Overview of the BV-actuator and its applications [9].

This is just a specific reference to electronic skin in virtual reality, and we still need to solve a lot of problems to achieve such a desirable effect. The first thing to consider is the sensitivity of the electronic skin, the higher the sensitivity, the more accurate the perception and response to changes in external conditions, therefore, the use of soft lithography of the regular micro pyramidal structures (MPS) [9]. Through the use of MPS, we can achieve high sensitivity of electronic skin. Another aspect to consider is haptic feedback in VR devices. We can use a wearable low-energy electromagnetic vibration haptic actuator (bv brake) that uses double copper coils as a magnetic field source to generate vibration force to enhance haptic feedback [10]. By adding brake arrays to wearable VR devices, the feedback experience of users when using VR devices can be greatly improved. In addition, this array also shows strong performance and stability at low-power of VR devices. Therefore, we improved the user's haptic feedback experience in the device by installing bv brakes. However, further research, development, and improvement are needed to overcome existing and unknown challenges and fully optimize the various roles of e-skin in virtual reality applications.

5. Conclusion

Electronic skin can be applied to artificial intelligence, smart medicine, and virtual reality. Researchers are developing corresponding technologies to advance deep intelligence and advanced cognitive systems by combining electronic skin with artificial intelligence. In recent years, research teams have developed different types of electronic skin, such as human FSB-based electronic skin and CPC-based flexible electronic skin. The former exhibits high sensitivity and strong linear response, with good stability even after extensive repeated pressure tests. The latter includes sensing electronic skin (to recognize different levels of external stimuli) and energy-based electronic skin (to harvest energy from the environment, with good stability and high scalability). Currently, the explicit recognition of external colors and brightness and power supply issues are the main focus of research. In the field of smart medicine, electronic skin is primarily used in the field of rehabilitative medicine. The practical tactile sensors can be categorized based on piezoelectricity, capacitance, pressure, induction, and light. Additionally, the two main attributes of electronic skin are wearability and conversion functionality, which depend on the choice of materials, device structures, manufacturing, and integration methods. It requires stretchability, conductivity, and robust self-healing capabilities at the technological level.

Furthermore, electronic skin can also be applied in virtual reality technology to enhance the lacking tactile feedback and optimize the human-computer interaction interface. This requires addressing sensitivity (through soft lithography with rule-based microcone structures) and tactile feedback (wearable low-energy electromagnetic vibration actuators). As a popular wearable sensor, electronic skin still faces unresolved challenges and continues to explore its potential.

References

[1]. Zhang Cheng. "Preparation and application of flexible wearable electronic skin based on nanotechnology [D]". Engineering Technology Part I; Information Technology, 2022

[2]. Niu, H. et al."‘Perception-to-Cognition Tactile Sensing Based on Artificial-Intelligence-Motivated Human Full-Skin Bionic Electronic Skin" Advanced Materials, Aug. 2022

[3]. H.Chen, L.Dajace, SP.Lacour. “Electronic Skins for Healthcare Monitoring and Smart Prostheses.” Annual Review of Control, Robotics, and Autonomous Systems, vol. 4, pp. 629–650, May 2021

[4]. W.Wang, Y.Jiang, et al. “Neuromorphic Sensorimotor Loop Embodied by Monolithically Integrated, Low-Voltage, Soft E-Skin.” Science, vol. 380, pp. 735–742, May 2023.

[5]. J.Cui, D.Daniel, A.Grinthal, K.Lin, J.Aizenberg. “Dynamic Polymer Systems with Self-Regulated Secretion for the Control of Surface Properties and Material Healing.” Nature Materials, vol. 14, pp. 790–795, Jun. 2015.

[6]. S.Yin, G.Su, J.Chen, X.Peng, T.Zhou. “Ultra-Stretchable and Self-Healing Anti-Freezing Strain Sensors Based on Hydrophobic Associated Polyacrylic Acid Hydrogels.” Materials, vol. 14, pp. 6165-6165, Oct. 2021.

[7]. Korzynski P, Kozminski AK, Baczynska A. “Navigating leadership challenges with technology: Uncovering the potential of ChatGPT, virtual reality, human capital management systems, robotic process automation, and social media." International Entrepreneurship Review, Vol. 9, pp. 7-18. Sep. 2023

[8]. Beom Jun Jo, Seok-Kyoo Kim and SeongKi Kim "Enhancing Virtual and Augmented Reality Interactions with a MediaPipe-Based Hand Gesture Recognition User Interface", Ingénierie des Systèmes d’Information, vol. 28, pp. 633-638. Jun. 2022

[9]. Yao H, Chiam JS, Tee BCK, et al. "Augmented Reality Interfaces Using Virtual Customization of Microstructured Electronic Skin Sensor Sensitivity Performances." Advanced Functional Materials. Vol. 31, pp. 1-10. Sep. 2023

[10]. Y. Liu et al., "Skin-Integrated Haptic Interfaces Enabled by Scalable Mechanical Actuators for Virtual Reality," in IEEE Internet of Things Journal, vol. 10, pp. 653-663, Jan. 2023

Cite this article

Gao,X.;Li,T.;Yu,X.;Liu,M.;Sheng,H. (2024). Research on electronic skin in artificial intelligence, smart medicine, virtual reality. Applied and Computational Engineering,42,195-200.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

About volume

Volume title: Proceedings of the 2023 International Conference on Machine Learning and Automation

ISBN: 978-1-83558-309-8(Print) / 978-1-83558-310-4(Online)
Editor: Mustafa İSTANBULLU
Conference website: https://2023.confmla.org/
Conference date: 18 October 2023
Series: Applied and Computational Engineering
Volume number: Vol.42
ISSN: 2755-2721(Print) / 2755-273X(Online)