A Comparison Analysis of Shrinkage Regression Methods of Handling Multicollinearity Problems Based on Lognormal and Exponential Distributions

Umar Usman


Handling multicollinearity problem in regression analysis is very important because the existence of multicollinearity among the predictor variables inflates the variances, and confidence interval of the parameter estimates which may lead to lack of statistical significance of individual independent variables, even though the overall model may have significance difference. It is also mislead p-values of the parameter estimate. In this paper, several regression techniques were used for prediction in the presence of multicollinearity which include: Ridge Regression (RR), Partial Least Squares Regression (PLSR) and Principal Component Regression (PCR). Therefore, we investigated the performance of these methods with the simulated data that follows lognormal and exponential distributions. Hence, Mean square Error (MSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE) were obtained. And the result shows that PLSR and RR methods are generally effective in handling multicollinearity problems at both lognormal and exponential distributions.

Full Text:



R.F. Gunst and R.L. Mason, “Some considerations in the evaluation of alternate prediction equations, Technometrics, 1979. 21, 55-63.

T.H. Wannacott and R.J. Wannacott, “Regression a Second Course in Statistics,” John Wiley and sons, USA, 1981.

N.J. Adnan, M.H. Ahmad and R. Adnan, (2006). A Comparative Study on some Methods for Handling Multicollinearity Problems. Matematik. 22(2): 109-119.

O. Yeniay and A. Goktas, (2002). A Comparison of Partial Least Squares Regression with other Prediction Methods. Hacettepe Journal of Mathematics and Statistics. 31: 99-111.

S. Maitra and J. Yan (2008). Principal Component Analysis and Partial Least Squares: Two Dimension Reduction Techniques for Regression. Casualty Actuarial Society Discussion paper Program. 79-90.

M. El-Fallah and A. El-Salam, (2014). A Note on Partial Least Squares Regression for Multicollinearity (A Comparative Study). International Journal of Applied Science and Technology. 4(1):163-169.

U. Usman, Y. Zakari and Y. Musa, (2017). A Comparative Study on the Prediction Performance Methods of Handling Multicollineariy Based on Normal and Uniform Distributions. Journal of Basic and Applied Research International. 21(1). In Press.

A.E. Hoerl and R.W. Kennard (1970). Ridge Regression Applications to Non-orthogonal Problems,Technometrics.12(1): 69-82.

R.H. Mayers, (1990) Classical and Modern Regression with Applications.2nd edition, Duxbury press.

G.C. Mc Donald and A. Galarneau, (1975). A Monte Carlo Evaluation of some Ridge-Type Estimators. Journal of American Statistical Association, 70, 407-416.

S. Arumairajan and P. Wijekoon (2014). Improvement of Ridge Estimator when Stochastic Restrictions are Available in the Linear Regression Model. Journal of Statistical and Econometric Methods, 3. 35-48.

S. Arumairajan and P. Wijekoon (2015). Optimal Generalized Biased Estimator in Linear Regression Model. Open Journal of Statistics, 5. 403-411.


  • There are currently no refbacks.

MAYFEB Journal of Mathematics 
Toronto, Ontario, Canada
ISSN 2371-6193