For hand posture classification, hand segmentation and feature extraction is a crucial step in vision-based hand gesture recognition systems. The pre-processing stage prepares the input image and extracts features used later with classification algorithms [36]. The proposed system uses feature vectors composed of centroid distance values for hand posture classification. The centroid distance signature is a type of shape signature [36] expressed by the distance of the hand contour boundary points, from the hand centroid (x_c, y_c) and is calculated in the following manner:

This way, a one-dimensional function representing the hand shape is obtained. The number of equally spaced points N used in the implementation was 16. Due to the subtraction of centroid from the boundary coordinates, this operator is invariant to translation as shown by Rayi Yanu Tara [32] and a rotation of the hand results in a circularly shift version of the original image. All the features vectors are normalized, using the z-normalization, prior to training, by subtracting their mean and dividing by their standard deviation [1, 23] as follows,

where is the mean of the instance i, and σ is the respective standard deviation, achieving this way scale invariance as desired. The vectors thus obtained have zero mean and a standard deviation of 1. The resulting feature vectors are used to train a multi-class Support Vector Machine (SVM) that is used to learn the set of hand postures shown in Fig. 2, and used in the Referee Command Language Interface System (ReCLIS) and the hand postures shown in Fig. 3 used with the Sign Language Recognition System. The SVM is a pattern recognition technique in the area of supervised machine learning, which works very well with high-dimensional data. SVM’s select a small number of boundary feature vectors, support vectors, from each class and builds a linear discriminant function that separates them as widely as possible (Fig. 4)—maximum-margin hyperplane[40]. Maximum-margin hyperplanes have the advantage of being relatively stable, i.e., they only move if training instances that are support vectors are added or deleted. SVM’s are non-probabilistic classifiers that predict for each given input the corresponding class. When more than two classes are present, there are several approaches that evolve around the 2-class case [33]. The one used in the system is the one-against-all, where c classifiers have to be designed. Each one of them is designed to separate one class from the rest.

Dynamic gestures are time-varying processes, which show statistical variations, making Hidden Markov Models (HMMs) a plausible choice for modelling the processes [29, 42]. A Markov Model is a typical model for a stochastic (i.e. random) sequence of a finite number of states [10]. When the true states of the model are hidden in the sense that they cannot be directly observed, the Markov model is called a Hidden Markov Model (HMM). At each state an output symbol is emitted with some probability, and the state transitions to another with some probability, as shown in Fig. 7. With discrete number of states and output symbols, this model is sometimes called a “discrete HMM” and the set of output symbols the alphabet. In summary, an HMM has the following elements:

and is defined as , where N and M are implicitly defined in the other parameters. The transition probabilities and the observation probabilities are learned during the training phase, with known data, which makes this is a supervised learning problem [36].

In this sense, a human gesture can be understood as a HMM where the true states of the model are hidden in the sense that they cannot be directly observed. So, for the recognition of dynamic gestures a HMM model was trained for each possible gesture. HMMs have been widely used in a successfully way in speech recognition and hand writing recognition [28]. In the implemented system, the 2D hand trajectory points are used and labelled according to the distance to the nearest centroid, based on Euclidean distance. The resulting vector is then translated to origin resulting in a discrete feature vector like the one shown in Fig. 6.