Abstract
The sense of touch, acting as a connection between humans and surroundings, is a stretch of audio and video information. People have been striving to pursue higher productivity and quality in the telecommunication system by proposing schemes combining with voice, capture, and sensory feedback since last decades. To achieve the transmission of force feedback from remote environment is thus the objective and research concentrates on bilateral teleoperation systems.Traditional bilateral teleoperation systems are not only requiring a large amount of network resources, but sensitive to the transmission delay. From the previous research, the possible communication delay of a bilateral teleoperation system ranges from one to several hundred milliseconds. However, even a small communication delay or packet loss in the communication channel can affect the system’s stability and transparency. Therefore, kinesthetic data reduction techniques are required in bilateral teleoperation systems. The current scheme to reduce the high-rate haptic data transmission employs a mathematical threshold to transmit data selectively based on human perceptual limitations, which is called perceptual deadband (PD)-based codecs. It describes the perceptual thresholds by pairwise comparison. However, the current perceptual threshold is not sufficiently accurate to describe some types of stimuli in practice, including kinesthetic perception, and human time perception. Moreover, pairwise comparison is required to be made in each collected kinesthetic data; this wastes memory and time.
Due to various aforementioned limitations of deploying PD-based codecs in real bilateral teleoperation systems, I believe that novel mathematical models controlling kinesthetic data transmission should be proposed, so that the transmission status of the newly collected data will be determined by the model directly, without any comparison. In this thesis, three different machine learning algorithms are used for kinesthetic data reduction over the haptic communication network. By comparing with conventional PDbased codecs for kinesthetic data reduction, proposed techniques perform better in different aspects. Such a system with kinesthetic data reduction techniques is shown in this thesis to reduce the kinesthetic data transmission effectively.
The first idea for reducing kinesthetic data transmission is, by deploying long-short term memory (LSTM)-based data reduction modules, to control the transmission status of each data. Current PD-based codecs is not practical in dealing with time series data, as pairwise comparison is required before each transmission. Therefore, a novel mathematical model for deriving the the transmission status of each collected data is proposed based on LSTM networks. This model is trained from Weber’s law of just noticeable difference (JND), in which explains humans’ perceptual limitation.
The second idea is reducing the size of kinesthetic data in each transmission. Dimensionality reduction techniques (DRTs) are introduced to map original kinesthetic data in high dimensions to corresponding embeddings in low dimensions. This is novel, unprecedented in abandoning the concept of selective transmission (reducing the amount of data transmission), and applying DRTs on each collected kinesthetic data for reducing the network offload respectively. More specific, three different dimensionality reduction techniques, including principal component analysis (PCA), stacked auto-encoder(SAE) and uniform manifold approximation and projection (UMAP), are stated and compared with each other. Moreover, for reconstructing dimensions of kinesthetic data from low to original, three different data reconstruction techniques are used in terms of three dimensionality reduction techniques.
The third idea is clustering kinesthetic data with unsupervised learning techniques, by which realizing selectively transmitting kinesthetic data. Even though LSTM-based mathematical models can reduce the transmission of kinesthetic data effectively, labels of each data are deriving from existing PD-based codecs. In order to deal with original unlabelled kinesthetic data, unsupervised clustering techniques are introduced to classify each sample into different clusters in terms of intra-cluster similarities and inter-cluster distances. Unsupervised clusterings can find internal features of the dataset which may be ignored by humans. What’s more, we select kinesthetic data in a part of clusterings to transmit over the network, and compare it with PD-based and LSTM-based kinethetic data
reduction techniques.
We also improve the accuracy of prediction models in bilateral teleoperation systems with the assistance of gradient boosting decision tree (GBDT) algorithm. Prediction models are required since operator needs to estimate the force feedback when no data is received. Current PD-based predictive scheme assumes the future value is related to last one or two received data, which is too simple to describe this whole prediction. Therefore, GBDT, which is acknowledged as the most accurate and commonly used predictive algorithm is introduced to improve the whole accuracy and transparency of the system.
| Date of Award | 1 Jun 2023 |
|---|---|
| Original language | English |
| Awarding Institution |
|
| Supervisor | Toktam Mahmoodi (Supervisor) & Abdol-Hamid Aghvami (Supervisor) |