Digital terrain model height estimation using support vector machine regression

Digital terrain model interpolation is intrinsically a surface fitting problem, in which unknown heights H are estimated from known X-Y coordinates. Notable methods of digital terrain model interpolation include inverse distance to power, local polynomial, minimum curvature, modified Shepard’s method, nearest neighbour and polynomial regression. We investigated the support vector machine regression (SVMR) as a new alternative method to these models. SVMR is a contemporary machine learning algorithm that has been applied to several real-world problems aside from digital terrain modelling. The SVMR results were compared with those from notable parametric (the nearest neighbour) and non-parametric (the artificial neural network) techniques. Four categories of error analysis were used to assess the accuracy of the modelling: minimum error, maximum error, means error and standard error. The results indicate that SVMR furnished the lowest error, followed by the artificial neural network model. The SVMR also produced the smoothest surface followed by the artificial neural network model. The high accuracy furnished by SVMR in this experiment attests that SVMR is a promising model for digital terrain model interpolation.


Introduction
Engineers and other related scientists are often charged with the responsibility of producing digital maps that represent the three-dimensional visualisation of the earth's surface.These maps usually serve as auxiliary data for engineering designs of roads, bridges, drainage systems and general landscaping.These digital three-dimensional maps are referred to generically as digital terrain models (DTMs).A DTM is referred to as a 'form of computer surface modelling which deals with the specific problems of numerically representing the surface of the earth' 1 .
A DTM is created by using one of two methods: triangulation or gridding.In a gridding method, the corners of regular rectangles or squares are calculated from the scattered control points.In triangulation, triangles are created based on the scattered control points.These triangles do not intersect each other and represent the terrain surface as a linear or non-linear function.In both methods, the heights of grid points and triangular points with unknown heights in the modelled area are estimated by interpolation using their control points. 2 Notable methods of DTM interpolation include inverse distance to power, local polynomial, minimum curvature, modified Shepard's method, polynomial regression, radial basis function, Kriging and nearest neighbour. 3,4e aim of this study was to investigate the support vector machine regression (SVMR) model, [5][6][7] as a new method of DTM interpolation.The goal of the SVMR model is to construct a hyperplane that lies close to as many data points as possible, by choosing a hyperplane that has a small norm that simultaneously minimises the sum of the distances from the data points to the hyperplane.SVMR attempts to minimise the generalisation error bound so as to achieve generalised performance, instead of minimising the observed training error.The idea of SVMR is based on the computation of a linear regression function in a high-dimensional feature space in which the input data are mapped via a non-linear function. 7In this experiment, the SVMR results were compared with those from a notable parametric technique (nearest neighbour, NN) and a notable non-parametric technique (artificial neural network, ANN).

Support vector machine regression algorithm
For the linear case, given the set of data, (y i ,x i ),...,(y l ,x l ),x∈R n ,y∈R with a linear function ƒ(x)=(w .x)+b, 8 Equation 1 hence the regression function is given by the minimum of the functional Equation 2 where C is a pre-specified penalty value, and , * are slack variables representing upper and lower constraints, respectively. 9Using an -insensitive loss function the solution is given by: Equation 4  9The Karush-Kunn-Tucker conditions that are satisfied by the solution are: Equation 10 Equation 11 with constraints -C≤β i ≤C, where i=1,...,l Equation 12 and the regression function is given by Equation 1, where: Equation 14 and b=-w .[x r .
x s ] 10 1 2 Equation 15The non-linear SVMR solution using an -sensitive function is given by: maxW(α,α*)=max α,α* α,α* ∑α i *(y i -)-α i (y i +) Equation 19 Solving Equation 16, with constraints Equations 17-19, determines the Lagrange multipliers α i , α i * and the regression function is given by: where Equation 22 The kernel K(x i ,x j )can be any of the following common kernel functions: the linear kernel x .x i , polynomial kernel (x .x i +1) d and radial basis function kernel . 7

Methodology
The eastings (E), northings (N) and orthometric heights (H) of 601 points were sampled from a 3.068-ha portion located in southern Nigeria (Figure 1).The sampled area was bounded by UTM coordinates 381810E-382060E and 559700N-559980N (Figure 2).Out of the 601 sampled points, 200 points were selected for training the SVMR algorithm.The selection of the optimum kernel parameters' values of degree d, gamma and penalty value C was done using the k-fold cross-validation process where k=10. 11In each experiment, nine data sets (k-1 data sets) were put together to train the SVMR while the remaining one data set was held to test the accuracy of the experiment.The experiment was repeated in 10 folds until all 10 data sets were used for both training and testing (Figure 5).The cross-validation result from Figure 5a shows the accuracy for degree and gamma using values 1 to 10.The value of 4 was found to be the value for degree with the highest accuracy, while 6 was found to be the highest value of gamma with the highest accuracy.From Figure 5b, the values of C with the highest accuracy were 60, 100 and 80 for polynomial, radial basis function and linear kernels, respectively.The polynomial kernel yielded the highest accuracy, and the best value of d was 4 (Figure 5).

Results
The resulting DTM and contour plots from interpolation using SVMR are depicted in Figure 6.After interpolation of the heights of the unsampled points using SVMR, the same was implemented with the ANN and NN models.
The NN technique predicts the value of an attribute at an unsampled point based on the value of the nearest sample by drawing perpendicular bisectors between sampled points (n), thereby forming polygons (V i , i=1,2,...,n). 4][14] The estimations of the attribute at unsampled points within polygon V i are the measured value at the nearest single sampled data point x i , that is z ˆ (x 0 )=z(x i ).The weights are: Equation 23 All points within each polygon are assigned the same value. 12,14The NN DTM and contour plots are presented in Figure 7.
The ANN was programmed using multilayer perceptron with a sigmoidal hidden-layer transfer function and linear output neurons.The multilayer perceptron neural network was trained with a back-propagation algorithm, using a two-layer feed-forward neural network.The network had an input layer, an output layer and one or two hidden layers; however, there is no limit to the number of hidden layers. 15Basically a signal from neuron i of the first input layer of a cell x at time t received by a neuron j of the hidden layer can be expressed as: where S' i (x,t) denotes the site attributes given by variable (neuron) i; W i,j is the weight of the input from neuron i to neuron j; net j (x,t) is the signal received for neuron j of cell x at time t. 15 Based on the method of cross-validation, the random seed number was set and the required number of neurons in the hidden layer was set between 1 and 50.The ANN was initialised with initial weights; hence different results were obtained every time the ANN model was run.To ensure the results remained the same at every run of the neural network the random seed number was kept constant.
The random seed number is an arbitrary constant chosen by trial and error.
After the random seed number had been set, the number of hidden neurons was the single parameter that was adjusted to obtain simulation results of the ANN.The training of the neural network was done by simply adjusting the number of neurons in the hidden layer in order to minimise the training error.The training error is the discrepancy between the predicted and the actual value.7][18][19] The ANN DTM and contour plots are presented in Figure 8.
In Figure 9, the sampled heights were plotted against the 200 points used for the validation of the models.The discrepancies between the observed points and the predicted points using SVMR, ANN and NN are depicted in Figure 9.
The plots in Figure 9 show that SVMR yielded the best fit, followed by ANN.
The experimental errors were calculated by comparing the estimated values using these models with their actual values from the sample.Out of the 601 sampled points, 200 points were selected for training while 50 points were used to test the accuracy of the modelling.The final result was selected from several results obtained by repeating the process, and by re-selecting the training and test data (Figure 10).
The box plot in Figure 10a  error was -0.09.For the NN model, the calculated minimum error was -2.97, the maximum error was 3.86 and the mean error was 0.35.From Figure 10b, the calculated standard errors for SVMR, ANN and NN were 0.60, 0.65 and 0.81, respectively.

Conclusion
The SVMR results were compared with those of the NN and ANN techniques.The NN and ANN are common parametric and non-parametric models, respectively, that have been used in previous studies. 3From Figure 10, we can see that SVMR produced the best results of all the models, followed by the ANN model.The results from the NN model were the least accurate.SVMR also produced the smoothest surface, followed by ANN, while NN produced the roughest surface (Figures 6-8).
Even though the SVMR is not a common method of DTM interpolation, our results show that it is a robust technique and can be considered

Figure 1 :
Figure 1: A map showing the location of the 3.068-ha area sampled in southern Nigeria.The experiment was done in MATLAB.The training data contained E, N and H values. E and N were the explanatory variables while H was the target variable.The test data contained just E and N in order to predict the values of H.The SVMR model was trained with known E, N and H values in order to estimate H values of points not used as training data.

Figure 2 :Figure 3 :
Figure 2: UTM coordinates of the sampled points in the study area.The stratified random sampling was used to select the training data.The spatial dependency of the observed data was examined by plotting its semivariances against the lag distance to produce a semivariogram (Figure3).The calculated nugget was 0.6.The range was 70, while the sill was 29.6.Beyond the range value the data are spatially independent, whereas the data within the range area are spatially dependent.The nugget value of 0.6 indicated that the error in observation was minimal.The sill value of 29.6 indicated the maximum value of the semivariance that corresponded to the range value.

Figure 4 :
Figure 4: Box plots of the 10 data sets of 20 sample points each.

Figure 5 :
Figure 5: Cross-validation results for (a) degree and gamma and (b) penalty value, C.

Figure 6 :
Figure 6: Support vector machine regression method (a) digital terrain model plot and (b) contour plot.