A vision-based soft somatosensory system for distributed pressure and temperature sensing

Emulating a human-like somatosensory system in instruments such as robotic hands and surgical grippers has the potential to revolutionize these domains. Using a combination of different sensing modalities is problematic due to the limited space and incompatibility of these sensing principles. Therefore, in contrast to the natural world, it is currently difficult to concurrently measure the force, geometry, and temperature of contact in conventional tactile sensing. To this end, here we present a soft multifunctional tactile sensing principle. The temperature is estimated using a thermochromic liquid crystal ink layer which exhibits colour variation under temperature change. The shape and force of contact is estimated through the 3D reconstruction of a deformed soft silicone surface. Our experiments have demonstrated high accuracy in all three modalities, which can be measured at the same time. The resolution of the distributed force and temperature sensing was found to be 0.7N and 0.4 °C respectively.