Best Model Selection Method for the Best Latent Variables Determination When Solving Multicollinearity with Partial Least Squares

Abstract

Violating the assumption of independence among explanatory variables in the linear regression model leads to multicollinearity. In the presence of multicollinearity, the Ordinary Least Squares (OLS) estimator yields inefficient parameter estimates, whereas Partial Least Squares (PLS) estimates are more robust. Moreover, in PLS, weights must be assigned to each explanatory variable before the latent variables are extracted. Two significant challenges associated with the PLS method are the choice of the weight scheme and the selection of latent variables (LVs) to obtain an efficient estimate of the model parameters.  Two methods of weight allocation are considered in this study: equal weight allocation and the variance of the regressors, while the two commonly known methods of model selection are the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC). AIC and BIC were used to select the best model for determining the optimal latent variables. Consequently, the study compared the performance of PLS results when the two weight attachment schemes and the two commonly used methods of model selection were used to determine the best latent variables. Efficient validation of PLS was performed using Total Mean Squared Error (TMSE) results for all model parameters obtained by the PLS estimator across different scenarios: varying sample sizes, multicollinearity levels, variability values, weight assignments, and model selection methods. Hence, the study concluded that the BIC method of model selection is the best for determining the optimal latent variables to use when employing PLS methods of estimation to handle multicollinearity in a Linear Regression Model.