A Comparison Analysis of Shrinkage Regression Methods of Handling Multicollinearity Problems Based on Lognormal and Exponential Distributions

Abstract

Handling multicollinearity problem in regression analysis is very important because the existence of multicollinearity among the predictor variables inflates the variances, and confidence interval of the parameter estimates which may lead to lack of statistical significance of individual independent variables, even though the overall model may have significance difference. It is also mislead p-values of the parameter estimate. In this paper, several regression techniques were used for prediction in the presence of multicollinearity which include: Ridge Regression (RR), Partial Least Squares Regression (PLSR) and Principal Component Regression (PCR). Therefore, we investigated the performance of these methods with the simulated data that follows lognormal and exponential distributions. Hence, Mean square Error (MSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE) were obtained. And the result shows that PLSR and RR methods are generally effective in handling multicollinearity problems at both lognormal and exponential distributions.