Robust non-parametric spatial regression and its application in field data analysis

Abstract

Outlier detection and robust estimation are an integral part of data mining and has attracted much attention recently. Generally, the data contain abnormal or extreme values either due to the characteristics of the individual or due to errors in tabulation/data entry. The presence of outliers will severely affect the data modelling and analysis. A robust nonparametric method is proposed to fit the spatial/surface regression that is not influenced by the presence of outliers in the data. Robust M-kernel weighted local linear regression smoother was used to fit the spatial regression function. The proposed method is useful to estimate/eliminate the spatial effect and identify the high potential trees in an orchard, which is useful for breeding programs. The method is illustrated through simulated data. The comparison of AMSE corresponding to the optimum bandwidth shows that the non-robust Kernel Weighted Local Regression Estimator (KWLRE) performs very badly in the presence of outliers. Among the robust estimators, the robust spatial smoother with biweight robustness weight function performed better than the Huber and Hampel weight functions. Comparison of AMSE corresponding to the optimum bandwidth showed that there is not much difference between different types of robustness weight function in the absence of outliers. In the case of robust spatial smoother with biweight per cent robustness weight function, the AMSE for 0 per cent, 4 per cent and 8 per cent outliers are almost the same, indicating that the method is robust against the outliers. The method was also applied to the annual yield data of 225 coconut palms in a field to eliminate spatial effect and to identify the high potential trees. It was found that by removing spatial effects and outliers, the MSE has reduced more than 50 per cent.