Probability model of rock climbing recognition based on information fusion sensor time series

Abstract

Rock climbing is a sports activity that integrates competition, entertainment, and culture. With the development of the economy and the improvement in living standards, rock climbing has embarked on a path of self-development and has entered the lives of urban youth at an increasingly rapid rate. This paper studies the probabilistic model of rock climbing recognition based on time series of multi-information fusion sensors so that climbers can climb more standardized. Based on practice, this paper has conducted research and design on the hardware platform and actually applied it to the rock climbing environment. Through reasonable processing of rock climbing process data of rock climbers, a variety of rock climbing state characteristics are successfully extracted for fusion. Aiming at the quasi-periodical characteristics of acceleration changes at different points during human movement, a method for identifying human movement patterns based on gait event information is designed. This method intercepts the three-axis acceleration data collected by each accelerometer through key gait events. A data set used to identify human movement patterns is established. A corresponding LDA classifier is established for each data set to identify the current movement pattern, and finally the classification results of all the classifiers are voted on. The final experiment shows that the system can identify the climbing movement of the climber within 3 s. The method can achieve 95.84% of the comprehensive recognition accuracy of the four state modes of rock climbing.

1 Introduction

With the development of society, people's living standards have also improved a lot. At this time, they pay more attention to the use of leisure time and choose to go to nature. Outdoor sports have entered people's field of vision, and they have gradually replaced traditional indoor sports as they should. Outdoor sports are loved by more people. Outdoor sports mainly include mountain climbing, rock climbing, running, fishing, and so on. Rock climbing is a special outdoor sport that belongs to mountaineering. Compared with other outdoor sports, it has unique advantages. With the development of my country’s economy and the increasing demand for people’s material and culture, people are becoming more and more enthusiastic about tourism and adventure sports. Rock climbing can increase body flexibility and coordination, as well as concentration and willpower. The emerging extreme sports with adventurous and innovative spirit is a perfect combination of these aspects. There are also artificial rock climbing places of different scales in various cities, which can bring people a lot of rock climbing opportunities. Of course, many people will choose natural rock walls in the wild, and rock climbing is an introduction to some extent. Men, women, and children from 3 to 70 years old have the opportunity to show their talents. As long as they study hard, I believe most people can see their own progress gradually. Moreover, after the invention of the artificial rock climbing field, the difficulty and danger of rock climbing are greatly reduced, and there is also the careful guidance of professionals and the psychological pressure is small. Outdoor sports have a relatively low threshold. Rock climbing has a special outdoor sports culture. It is a sport that allows climbers to fully express their desire to highlight their individuality and break through themselves. Rock climbing integrates fitness, competition, leisure, stimulation, and education. It is a relatively new type of comprehensive outdoor sports. Athletic diathesis is the basic athletic ability that a person demonstrates in all aspects during exercise, including strength, flexibility, flexibility, balance, and coordination. In the climbing process, in order to successfully climb to the top of the mountain, rock climbers will try their best to challenge themselves, break through their limits, and avoid falling and failing. In this repeated process, the body's abilities will be comprehensively improved. Rock climbing has higher requirements for climbers in the four aspects of strength, endurance, flexibility, and agility. Frequent rock climbing can continuously improve the athletic quality of these aspects. Many rock climbing participants are novices, so many actions are not very standardized. At this time, a multi-information fusion sensor time series rock climbing motion recognition probability model is needed to identify the rock climber's motion to see if he conforms to the climbing motion and to reduce injuries caused by irregular movements. Rock climbing brings healthy bodies to climbers.

Rock climbing is a sport derived from mountaineering. At the same time, it is also a military training project in the army. The main rock climbing environment is rock sentry wall or artificial rock wall. Rock climbing is an outdoor sport developed today, which is an extreme sport and has a strong challenge. It is loved by young people all over the world. In our country, after investigation, it is found that many places have established clubs and have their own artificial rock walls (for example, the Yuen Club in Changsha City, the Parthenon Club, the Longing Club, etc.). Climbing on natural rock walls is one of the best sports with a high-risk factor, especially in the absence of protection, and is extremely risky. In any part of climbing, accidental hands or soft feet will directly lead to accidents. Falling down from the rock wall, the outcome is unimaginable. In the process of climbing artificial rock walls, there are the same risks. However, the research on the probability model of rock climbing motion recognition based on the time series of multi-information fusion sensors can be reduced. The probability of a rock climber's risk, as long as the safety rules can be observed during rock climbing, and the corresponding rock climbing equipment are regular in the case of movement specifications, and the occurrence of dangerous situations can be reduced.

In the study of motion recognition, Wei L said that dynamic gesture recognition is a crucial but challenging task in the pattern recognition and computer vision communities. He proposed a novel feature vector suitable for expressing dynamic gestures and provided a satisfactory solution for recognizing dynamic gestures using only the leap motion controller (LMC). None of these has been reported in other papers. Calculate feature vectors with depth information and input them into the hidden conditional neural field (HCNF) classifier to recognize dynamic gestures. The system framework of the proposed method includes two main steps: feature extraction and classification using HCNF classifier. The recognition accuracy rate of the LeapMotion-Gesture3D data set is 89.5%, and the recognition accuracy rate of the Handicraft-Gesture data set is 95.0%. But it turns out that the method he proposed is only suitable for some dynamic gesture recognition tasks and is not suitable for recognizing rock climbing motion models [1]. Wang W said that in order to achieve model-based recognition of human movement intentions, the dynamic modeling and recognition of the lower limb rehabilitation robot leg should be studied. Due to relatively strong motion constraints, corresponding contributions are proposed to overcome the limitation. First, in the joint friction model, the Algren empirical formula and polynomial fitting method are used to consider the coupling factors between the joints. Then, an indirect generation strategy is designed, through which an effective initial solution to the optimization problem can be found efficiently. In addition, a recursive optimization method based on dynamic model and excitation trajectory optimization is proposed to further reduce the number of conditions. However, iLeg has become insufficient in three aspects: (1) The traditional joint friction model does not consider the coupling factors between the joints; (2) the traditional initialization strategy becomes very inefficient to find the effective initial solution to the optimization problem of the excitation trajectory; and (3) the condition number of the observation matrix calculated according to the preliminary dynamic model and the relevant optimized excitation trajectory is too large to be recognized [2]. Murtaza proposed a contour-based view-independent human action recognition scheme for multi-camera data sets. In order to overcome the high-dimensional problem caused by multi-camera data, a low-dimensional representation based on motion history images (MHI) is extracted. A single MHI is calculated for each view/action video. In order to effectively describe MHI, the histogram of oriented gradients (HOG) is used. Finally, the classification based on HOG's MHI description is based on the nearest neighbor (NN) classifier. The proposed method does not use feature fusion of multi-view data, so this method does not require a fixed number of camera settings in the training and testing phases. But the method he proposed is only suitable for multi-view and single-view data sets, because feature fusion is not used [3]. This research is mainly based on the multi-sensor information fusion analysis of the information data characteristics conveyed by different actions and different postures in the process of rock climbing, but this method is only in the image stage, and this research is carried out on the basis of images in a step-by-step discussion.

Multi-sensor information fusion system has good performance robustness, extended space–time coverage area, excellent target resolution ability, fault tolerance and system reconfiguration ability, and high system resource utilization. It is used in military and civilian fields. The application is becoming more and more extensive. As one of the prerequisites for information and data fusion, multi-sensor systems have attracted more and more attention. In this paper, in the multi-sensor information fusion system, the measurement data time mismatch problem caused by the different sampling period or power-on time of each sensor is conducted with in-depth research. It mainly introduces the research background, significance and status quo of multi-sensor pattern recognition probability in rock climbing, systematically expounds the main rock climbing pattern recognition methods, and makes necessary analysis.

2 Overview of motion recognition and multi-information fusion sensors

2.1 Movement pattern recognition

Locomotion mode recognition plays a key role in the control of the system mechanism in different motion mode scenes [4]. The range of sports modes is very wide. Sitting, standing, walking, running, up and down stairs, jumping, falling, etc., are all different sports modes. Many documents have more descriptions of sports modes, including walking, running, sitting, and standing. Movement patterns including jumping and jumping are recognized, and movement patterns including up and down stairs, up and down elevators are analyzed and recognized, and sports including cycling, driving, and housework are involved [5]. Different sports modes have different characteristics. In order to recognize different sports modes, certain data support is needed. Recognition needs to start from the data collected by the sensor, after data preprocessing, feature extraction, feature transformation and selection, classification, and recognition; the recognition result of the movement pattern is obtained. The general recognition process of motion patterns is shown in Fig. 1.

Fig. 1

Movement pattern recognition steps

There are many ways to choose the sensor for the first step, including video image acquisition and non-video image sensor. As a noninvasive acquisition method, video image recognition motion mode has been applied in many fields, and its research is also relatively comprehensive [6]. However, non-video image sensors are gradually being used to recognize human movement patterns due to their small size, lightweight, and low cost. Such sensors are generally arranged in various parts of the body, and one or more sensors are fixed on the waist, arms, abdomen, lower limbs, etc., or directly use smart phones with inertial measurement units to recognize movement patterns [7]. The selection of the measurement unit varies according to the type of application, generally including accelerometers, gyroscopes, magnetometers, heart rate sensors, skin conductance sensors, wireless locators, etc., for data collection [8]. Aiming at the time mismatch problem, the article uses a three-axis acceleration sensor. The advantage of the three-axis acceleration sensor is that when the direction of movement of the object is not known in advance, only the three-dimensional acceleration sensor can be used to detect the acceleration signal. Most studies use acceleration data as the source of motion information and use single-axis or three-axis acceleration data to identify motion patterns.

Compared with other sensors, although the measurement of the accelerometer is more sensitive and the measurement contains high-frequency components, its characteristics of small relative error, simple offset calculation, small temperature drift, and less environmental interference make it suitable for obtaining sports information [9]. The application of multi-sensor technology in rock climbing requires high reliability and stability of video equipment. Accelerometer is a sensor used to test linear acceleration. Compared with electronic gyroscopes, it has the characteristics of long-term stability. By analyzing the dynamic acceleration, the way the device moves can be analyzed. A gyroscope is a device that measures the angular velocity of rotation. A gyroscope is placed on the human body to measure the angular rotation of different parts, provide human body motion information, and provide data for the recognition of motion patterns. The measurement error of the gyroscope is relatively high, there are bias and drift conditions, and it is not suitable for independent use. Generally, the magnetometer data and the gyroscope data are used after data fusion [10]. Heart rate sensors and skin surface conductance sensors are sensors that measure biological characteristics. People's heart rhythm performance and EMG signal characteristics in different sleep, walking, sitting, and other motion states are different. Different motion patterns can be studied by measuring heartbeat data and surface electromechanical signals. Measure the changing law of the data to identify the movement pattern of the human body [11]. The basic movements of rock climbing are mainly around arms, feet, etc., and there are many fulcrums on the rock wall. According to different positions and different angles, they can be pulled, pinched, and climbed. These movements determine the center of gravity of the human body. As a positioning device, a wireless locator includes two parts: a base station and a positioning tag. It judges the location of the person according to the characteristics of the signal received by the tag and indirectly infers the movement pattern of the human body based on the position change information. This method is subject to the limitations of the venue and can only be applied within a certain range [12].

2.2 Multi-information fusion sensor

The so-called multi-information fusion is actually the data information collected from multiple sensors or other sources through computer calculation and comprehensive analysis and processing under certain specific specifications to complete the subsequent information processing process [13]. It collects the data of the observation target through N different types of sensors, performs feature extraction and transformation on the output data of the sensors, and classifies them after attribute judgment. This is the same as the way the human brain thinks about problems, especially in making full use of different time. With the multi-sensor data resource of space, computer technology is used to obtain multi-sensor observation data in time series. The basic principle of multi-information fusion sensors is very similar to the process of human brain processing information. Each sensor is optimized and processed through multifaceted and multi-angle information fusion, and finally a unified description of the required observation environment is obtained. On this journey, we need to make full use of the multi-directional information resource data obtained by more sensors and process and use them reasonably [14]. This not only uses the advantages of multiple sensors working at the same time, but also comprehensively processes the information data obtained from other places to make the multi-fusion information sensor system more intelligent [15]. Multi-information fusion sensor system has four obvious advantages: information redundancy, information complementarity, timeliness of information processing, and low cost of information processing. The typical structure of the post-fusion algorithm is shown in Fig. 2.

Fig. 2

Typical structure of post-fusion algorithm

With the development of information fusion technology and the wide application of multi-sensor information fusion systems, the problem of time registration in information fusion has gradually attracted people's attention. In the actual multi-sensor system, due to the different working tasks of each sensor, the performance of the sensor is different in the environment [16]. Even if the same target is observed, the observation data of different sensors are not necessarily synchronized. Therefore, the measurement data cannot be directly fused. It is necessary to convert the target observation data obtained by different sensors at different times to a unified fusion moment, that is, in time, as shown in Fig. 3.

Fig. 3

Process diagram of information fusion

2.3 Rock climbing

Rock climbing is the use of technical equipment and companion protection for climbers to perform thrilling actions such as turning, jumping, and pulling up on rock walls of different heights and difficulties, relying on tenacious will, strong physical strength and flexible thinking ability. Complete the climbing of the entire route [17].

In traditional climbing, climbers set up protection points along the road by themselves, and the goal is to finish climbing. Because there is no permanent protection point along the road, traditional climbing usually climbs along the fissure. It can be divided into artificial climbing and free climbing [18]. Free climbing only uses hands, feet, and natural handles to climb; ropes and other artificial equipment are only used to ensure that they are not helpful for climbing. Manual climbing is an extra method of climbing on the rock wall with ropes and other artificial equipment, adding artificial grip or stepping points, or any form of assistance, to climb high. Manual climbing requires the use of artificial tools to climb during the climbing process, such as handheld rope ladders, fixed points, protection points, such as ascenders, rope ladders, rock nails, and rock hammers. But because the rock nails will cause damage to the rock wall, in the environmental protection now, generally only the rock wedges are used as fixed points or guarantee points [19]. Participation in free climbing: This term is relative to artificial climbing. When climbing, only the limbs of the body are used to track the natural handle points or foot points. Traditional equipment is only used to set up protection points, not for climbing. The rope is only used to ensure safety. The traditional climbing posture is shown in Fig. 4.

Fig. 4

Traditional climbing posture

The difference between sport climbing and traditional climbing is that the climbing route has preset protection points, and climbers do not need to place protection points by themselves [20]. Since the development of rock climbing, due to the advancement of technology and equipment and the purpose of popularizing rock climbing, sport climbing has become the mainstream climbing method in the rock climbing industry due to factors such as safety and easy entry. From the early stage of rock climbing to the later stage, some rock climbers gradually leave the alpine rock field and become a new sport. Climbers climb rock fields that have permanent protection points, such as artificial and natural rock fields [21]. The goal of climbers is not just to climb to the top, but to challenge more difficult routes. The sport climbing posture is shown in Fig. 5.

Fig. 5

Sport climbing posture illustration

2.4 Calculation and selection of feature components

In order to increase the correct rate of recognition of different motion patterns during exercise, consider selecting an appropriate time point in a gait cycle, and intercept the data of each acceleration sensor with a fixed window length, which is used to identify the motion pattern of the person [22]. This method is different from the sliding window to identify the data in the entire time range. According to this interception method, the characteristics of the data obtained are more obvious and conform to the gait law [23]. Although it is necessary to perform multiple classification recognition in a gait cycle, the overall recognition rate is higher than that in the whole cycle.

• (1) Relief feature selection algorithm

Relief feature selection method is a filtering feature selection method that first selects features and then performs training and learning [24]. For a second-class problem, the Relief feature selection method sorts according to the importance of each feature and selects the features with high importance to participate in the learning of the classifier, and the evaluation of the importance is achieved through relevant statistics. Suppose the training set S is

$$S = \{ (x_{1} y_{1} ),(x_{2} y_{2} ), \cdots (x_{n} y_{n} )\}$$

(1)

The training set S contains only two categories of data. For each sample $x_{i}$, define its guessed neighbor $x_{i,nh}$ and wrong guessed neighbor $x_{i,m}$. Guessed neighbor $x_{i,nh}$ is to find the closest sample in the same type of data set, namely:

$$x_{i,nh} = \arg \min \left| {x_{i} } \right. - \left. x \right|_{2}$$

(2)

Guessing the wrong neighbor $x_{i,nh}$ is to find the nearest sample among samples of different categories from sample $x_{i}$, namely:

$$x_{i,nh} = \arg \min \left| {x_{i} } \right. - \left. {x^{n} } \right|_{2}$$

(3)

Then, the importance of the jth component in each sample vector is defined as

$$\sigma^{j} = - \sum\limits_{i} {\left| {x_{i} } \right.}^{j} - \left. {x_{i,nh}^{j} } \right| + \sum\limits_{i} {\left| {x_{i} } \right.}^{j} - \left. {x_{i,nm}^{j} } \right|$$

(4)

where $x_{i}^{j}$ represents the jth component in the sample $x_{i}$ vector and is normalized:

$$x_{i}^{j} = \sum\limits_{k = 1} {\left| {x_{i} } \right.} (x_{k}^{j} )^{ - 1}$$

(5)

It can be seen from the above formula that for the jth component, if the distance between x and the guessed neighbor on the j component is less than the distance of the guessed neighbor, then the jet component is beneficial to classification and its importance should be increased. On the contrary, if the distance between x and the guessed neighbor on the j component is greater than the distance of the guessed neighbor, the jth component is not conducive to classification, and its importance should be reduced [25]. Finally, the importance of each sample on the component is calculated, and it is concluded that the larger the value, the better the classification effect of the component on the known category.

$$\sigma^{j} = - \sum\limits_{i} {\left| {x_{i} } \right.}^{j} - \left. {x_{i,nh}^{j} } \right| + \sum\limits_{i} {p_{i} \left| {x_{i} } \right.}^{j} - \left. {x_{i,nm}^{j} } \right|$$

(6)

Among them, $p_{i}$ is the proportion of type l samples in all heterogeneous samples. After obtaining the importance values of all the components, the several feature components with the highest scores are selected as the new data feature set for training the classifier.

• (2) Linear discriminant classification algorithm

For a group of training sets, try to project the examples in the training set on a certain line, so that the projections of the examples of the same category on this line are concentrated as much as possible, and the projection points of different categories are distributed as far as possible. According to this idea, suppose the mapping function f(x) is a linear discriminant function, namely:

$$f(x) = w^{T} x = w_{0}$$

(7)

x is a feature whose vector dimension is d, and w is a weight vector. The role of w is to map the high-dimensional vector x into the space, and the threshold weight is used to divide different categories.

$$\mu_{c} = \frac{1}{{N_{c} }}\sum\limits_{{x_{j} \in x_{c} }} {x_{j} }$$

(8)

$$X_{c} = \{ x_{j} |y_{j} = c\}$$

(9)

$$\sum {_{c} } = \sum\limits_{{x_{j} \in x_{c} }} {(x_{j} - \mu_{c} )} (x_{j} - \mu_{c} )^{T}$$

(10)

The idea of LDA classification method is to make the projections of similar examples as concentrated as possible and to enlarge the distance between classes as much as possible. According to this idea, the objective function J is defined as

$$J = \frac{{w^{T} S_{b} W}}{{w^{T} S_{W} W}}$$

(11)

Among them, $S_{b}$ is the inter-class dispersion matrix and $S_{w}$ is the intra-class dispersion matrix. Both are defined as follows:

$$S_{b} = \sum\limits_{i = 1}^{n} {m_{i} } (x_{i} - \mu )(x_{i} - \mu )^{T}$$

(12)

$$S_{w} = \sum\limits_{i = 1}^{n} {(x_{i} - \mu )(x_{i} - \mu )^{T} }$$

(13)

In the formula, m is the proportion of types of data in the overall data set S. Find the optimal solution of ω by introducing the operator:

$$S_{b} w = \lambda S_{w} w$$

(14)

For multi-class problems, the LDA algorithm provides the feature optimal projection surface to map the features of the data to a one-dimensional space and make judgments and decisions on the data category according to the decision rules. After the N-dimensional acceleration feature selected by the Relief-F algorithm, the posterior probability of being classified into category C_Y is

$$P(C_{Y} |f) = \frac{{P(f|C_{Y} )P(C_{y} )}}{P(f)}$$

(15)

3 Methods: rock climbing recognition model

3.1 System composition framework

In order to meet the needs of sports information acquisition, the human gait data acquisition sensor network is designed into three parts, including acceleration measurement, plantar pressure measurement shoe, and data acquisition main board. The data of the acceleration sensor are used to identify the movement pattern of the person, which is an overall source of experimental data; the plantar pressure represents a bearing point, and the state of the human body during rock climbing is grasped by the plantar; the data acquisition main board is this part of it is to fuse data. For the accelerometer and pressure-measuring shoes, the two parts are powered by a single-chip lithium battery, and each part is collected and uploaded separately to work, and the measurement data are sent to the computer or directly stored in the data acquisition board. The composition of the sensor network system is shown in Fig. 6.

Fig. 6

Sensor network system composition framework

The original data obtained by each independent sensor are processed locally, and then the results are sent to the information fusion center for intelligent optimization and combination to obtain the final result. It can realize real-time fusion, its data processing accuracy is high, the algorithm is flexible, it takes into account the advantages of centralized fusion and distributed, and its stability is strong.

3.2 Hardware design

The multi-information fusion sensor integrates and processes the movement, skill, error, and other data in the entire rock climbing process. While the rock climbing process is most prone to fall, slipping, and other phenomena, this technology is based on the bones during the movement. Discussion and the detailed planning of actions can reduce damage. In order to achieve the research goals, it is necessary to collect and analyze the movement information of different parts of the human body. According to actual research needs, the measurement unit should have the characteristics of small size, lightweight, relatively independent modules, and easy installation and disassembly. It can accurately measure human movement information in real time and does not affect people's normal actions. In terms of power consumption, the acceleration measurement module should have a lower power consumption level due to the need for longtime outdoor data collection and more measurement points.

ADXL363 is a micro-power sensor combination that integrates three-axis acceleration and temperature measurement produced by ANALOGDEVICES. It uses the full data rate to sample the entire bandwidth of the sensor, allowing data to be saved for up to 13 s. In addition to the accelerometer and temperature sensor, additional analog inputs can be simultaneously converted. It is mainly used in scenarios such as home care equipment, wireless sensing, and motion measurement. The chip weighs 18 mg, and the package is a 16-pin LGA package. The SPI communication protocol is used for command and data transmission, and the data output rate can reach up to 400 Hz. The accuracy of the three-axis acceleration output data is 8/12 digits adjustable, and its built-in temperature sensor can be used to measure the ambient temperature and to correct the acceleration measurement offset caused by the temperature. The single-chip sensor consumes less than 2uA when the system output rate is 100 Hz, and has the function of switching between measurement mode and standby mode to further reduce the power consumption level. The main technical specifications and performance indicators of ADXL363 are shown in Table 1.

Table 1 ADXL363 main technical specifications and performance indicators

The CASWELL control system needs to adjust the expansion and contraction of the hydraulic rods at the hip and knee joints to achieve the purpose of controlling the movement of the exoskeleton. At this time, it is necessary to obtain the angle of the human hip and knee joints so that the system can follow its changing curve. Since multiple IMU modules can obtain joint angles indirectly by measuring the inclination angles of the trunk, thighs, and lower legs, the exoskeleton needs to arrange at least one IMU module in each of the above-mentioned positions of the human body to complete the follow-up control. Since this study does not require inclination angle information, considering the portability of the enhanced program and the need for subsequent exoskeleton control algorithm research, the accelerometers are arranged on the human torso, middle thigh, and middle calf, respectively, and can be replaced with IMU later to measure the joint angle at the same time. And for acceleration information, complete the follow-up control needs, as shown in Fig. 7.

Fig. 7

Accelerometer layout

3.3 Feature-level fusion

For the fusion of feature vector data, in this architecture, the feature vector is extracted from the raw sensor data, and then the feature vector is transmitted to the central fusion process for data fusion. Since the feature vector is the representative of the original data, this method will inevitably lead to data loss, but the lost data will not have a great impact on the subsequent detection results. This method can realize the data fusion of different types of sensors and reduce the dependence on communication bandwidth, as shown in Fig. 8.

Fig. 8

Feature-level fusion process diagram

3.4 Voting decision method

In each movement cycle, the front and back acceleration data of HC and TO are windowed, feature components are extracted, and the movement pattern recognition is performed, respectively. The four classifiers involved need to vote in each exercise cycle and finally choose the current exercise posture mode. The specific voting decision rules are shown in Table 2:

Table 2 Voting decision rules

Use MATLAB to extract the kindest bone information, when the human body is facing the kindest device, the number of acquisition frames is 30, 50, 70, 90, 100, and 200, and each frame number is collected 10 times and averaged. When the image and depth sensors are turned on and the data are collected immediately, the bones cannot be tracked in the first 20 frames of the image; while the data are collected after a few seconds delay when the sensor is turned on, there is basically no bone frame loss. In the early stage, the image depth sensor records and processes the image first. During this process, there is a delay in starting the image, and the image does not fluctuate, but the data are actually recorded. The subsequent experiments are based on the acquisition after the start-up delay, the delay time is 5 s, and it can provide preparation time for the action at the same time, as shown in Table 3.

Table 3 Mean available start frame

In the experiment, the occlusion situation is the body's own occlusion and the partial occlusion of the environment. Seven different static motion postures are selected, and data are collected 10 times for each posture. The number of frames collected each time is 100. Y means the tracking is successful, and N means the tracking fails. The obtained bone state is shown in Table 4.

Table 4 Tracking status and node status of different postures

For the characteristic data of a single sensor, the same sample division and the same training and testing methods are used to test the classification accuracy. The results are shown in Table 5.

Table 5 Single-type feature test results

Through the comparison, it can be found that through the fusion of the two sensor features, the detection accuracy of various climbing postures has been improved, and the overall detection accuracy has increased by about 5 percentage points. In general, although there is room for improvement in the classification accuracy of various climbing postures, the overall classification accuracy rate is satisfactory, confirming the positive impact of laser and coil sensor fusion on classification, reaching the level of this climbing movement recognition research, as shown in Fig. 9.

Fig. 9

Recognition result comparison histogram

Taking a comprehensive look at the influence of the length of the data interception window and the number of feature components, the window length is 50-300ms. The number of feature components of different window lengths are 5, 10, 15, 20, 25, and 30 respectively, which are classified and calculated by the LDA classification algorithm. The average classification accuracy of the obtained classifier is shown in Fig. 10.

Fig. 10

The influence of window length and the number of features selected on the recognition accuracy

4 Results

According to the above experimental results, it can be concluded that in the process of collecting various pose recognition images of rock climbers, when the image and depth sensors are turned on to collect data, the image of the first 20 frames or so may look like an image that cannot be tracked, but after a few seconds after the sensor is turned on, the data image gradually becomes clear, and there is basically no frame loss. And if the fusion of multiple sensor features is adopted, the detection accuracy rate of various climbing postures will be improved, and the overall detection accuracy rate will increase by about 5 percentage points. At the same time, it can be seen that the length of the interception window has an effect on the average recognition accuracy of human motion patterns. The curve is parabolic. The average recognition accuracy first increases and then decreases with the interception window length. For the length of the interception window, if the window length is too small, it will lead to the lack of data and information, which will reduce the accuracy of the recognition of the motion pattern. The length of the interception window should not be too long. If the window is too long, the lag time for the system to judge the motion mode will increase, and too much redundant and useless information will be added to reduce the recognition accuracy. It can be concluded that the length of the interception window should not be less than 150 ms. If it is too small, the recognition accuracy of the four LDA classifiers will decrease, and there is no requirement for the number of feature components. After the above-mentioned experiments, it can also be shown that the rock climbing motion recognition probability of the multi-information fusion sensor time series has a high accuracy rate.

5 Conclusions

In this paper, aiming at the identification of rock climbing movement patterns during the start of the action, resting on the rock wall, climbing during the rock climbing, and finally ending the rock climbing state, this paper designs and implements a sensor system for collecting acceleration change information of different parts and plantar pressure change information. The gait of the person is collected by the surveillance camera, and the video sequence of the gait is obtained through detection and tracking. The recognition potential in the case of long distance or low video quality and the gait characteristics of the person are extracted through preprocessing analysis. According to the characteristics of the human plantar pressure data collected by the system, a set of dynamic recognition algorithms for monitoring key gait events are designed, and the monitored key gait event information and the acceleration data collected by the sensor network are used for the recognition of rock climbing movement patterns. The effectiveness of the key gait event monitoring algorithm is verified by experimental data, and the recognition effect of the designed classifier on the climbing pattern is evaluated. Considering that the physical realization of the gait event information monitoring algorithm involved in this article is relatively complicated, the rock climbing pattern recognition method has not been implemented in the actual sensor system in this article, so currently only the data collected by the sensor system are used to determine the gait of the human body. Recognition of events and rock climbing patterns has been carried out in related simulation research. Later, we can consider the use of classifiers including support vector machines and neural networks to study the recognition effects and implement these algorithms in actual hardware systems to evaluate the pros and cons of the algorithms.