Handling Multicollinearity and Outliers: A Comparative Study of Some One and Two–Parameter Estimators Using Real-Life Data
DOI:
https://doi.org/10.62054/ijdm/0104.14Keywords:
Multicollinearity, Outliers, traditional least square, Mean square error, estimatorAbstract
It is evident that when data suffers the problem of multicollinearity, the traditional least square is incapacitated and unreliable. Hence, needs to use bias estimator such as Ridge estimator, Liu estimator among others. Also, presence of outliers is another treat and to tackle this challenge is the use of robust regression estimators which include M, MM, LTS, LMS, LAD, LQS and S estimators. However, the presence of the two anomalies may be inevitable. Several estimators have been combined to handle the problems simultaneously. Therefore, this study compared and contrasted some robust one and two-parameter estimators using some real-life data sets. Mean Square Error (MSE) was used as criterion to select the best estimator. Some of the robust estimators were found to be inconsistent in addressing the twin problems. However, across all the data set employed in the study, the results revealed that robust Modified Ridge Type (MRT) in M, MM and LTS did well using minimum MSE
References
Adejumo, T. J., Ayinde, K., Akomolafe, A. A., Makinde, O. S. and Ajiboye, A. S. (2023). Robust-M new two-
parameter estimator for linear regression models: Simulations and applications. African Scientific Reports. DOI:10.46481/asr.2023.2.3.138
Ahmed, M. G. and Maha, E. Q. (2016). Regression Estimation in the presence of outliers: A comparative
study. International Journal of probability and statistics,5(3).65 -72. DOI:10.5923 /j.ijps.20160503.01.
Ahmad, S. and Aslam, M. (2020). Another proposal about the new two-parameter estimator for linear
regression model with correlated regressors. Communication in Statistics-simulation and computation.
https://doi.org/10.1080/03610918.2019.1705975.
Alabi, O.O., Olatayo, T.O. and Afobi, F. R. (2014). Empirical Determination of the Tolerable sample size for
OLS Estimator in the presence of Multicollinearity. Applied Mathematics 5, 1870- 1877. Doi:
4236/am.2014.513180.
Arslan, O., and Billor. N. Robust Liu estimator for regression based on an M-estimator. Journal Applied
Statistics. 27 (1): 39 -47 (2000).
Awwad, F. A., Dawoud, I. and Abonazel, M. R. (2022). “Development of robust Ozkale Kaciranlar and
Yang- Chang estimators for regression models in the presence of multicollinearity and outliers”.
Concurr Comput Prac Exp. e6779 34 (2022) https://doi:10.1002/cpe.6779.
Batah, F. M., Ozkale, M. R. and Core, S. D. (2009). Combining Unbiased Ridge and Principal component
regression estimators. Communication statistics. Theory Methods. 38, 2201 – 2209.
Birkes, D. and Dodge, Y. D. (1993). Alternative methods of regression, Wiley, New York.
Crouse, R., Jin, C. and Hanumara, R. (1995). Unbiased Ridge estimation with prior information and ridge
trace. Communication statistics. Theory Methods. 24. 2341 – 2354.
Dawoud, I. and Abonazel, M. R. (2021). Robust Dawoud-Kibria estimator for handling multicollinearity and
outliers in the linear regression model. Journal of Statistical Computation and Simulation. DOI: 10.1080/00949655.2021.1945063.
Dawoud, I. and Kibria, B. M. G. (2020). A new Biased Estimator to combat the multicollinearity of the
Gaussian linear Regression model. Stats. 3, 526 – 541. Doi: 10.3390/stats 3040033.
Farrar, D. E., and Glauber, R. R. (1967). Multicollinearity in regression analysis: The problem revisited. The
Review of Economics and Statistics, 92-107.
Hassan, E. A., (2017). Modified Ridge M-Estimator for Linear Regression Model with
Multicollinearity and Outliers. Communication in Statistics and Computation. DOI: 10.1080/03610918.2017.1310231.
Hoerl, A. E., and Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems.
Technometrics, 12(1), 55-67.
Huber, P. J. (1964). Robust Estimation of a location parameter. The Annals of Mathematical Statistics, 35, 73 -
Hussein, Y. A. and Abdalla, A. A. (2012). Generalized Two stages Ridge Regression Estimator for
Multicollinearity and Autocorrelated errors. Canadian Journal on Science and Engineering Mathematics, 3(3), 79 - 85.
Idowu, J. I., Fashoranbaku, A. S. and Ayinde, K. (2023). A two-parameter estimator for correlated Regressors in
gamma regression model. Science World Journal Vol. 18(No 4). Doi: https://dx.doi.org/10.4314/swj.v18i4.8
Idowu, J. I., Owolabi, A. T., Oladapo, O. J. Ayinde, K. Oshuporu, O. A. and Alao, A. N. (2023). Mitigating
Multicollinearity in Linear Regression Model with Two-Parameter Kibria-Lukman Estimators. WSEAS Transactions
Jegede, S. L., Lukman, A. F., Ayinde, K. and Odeniyi, K. A. (2022). Jacknife Kibria Lukman M-Estimator:
Simulation and Application, Journal of the Nigerian Society of Physical Sciences 4, pp 251-264. Doi: 10.46481/jnsps.2022.664
Jolliffe, I. T. (1982). A note on the use of principal components in regression. Journal of the Royal Statistical
Society: Series C (Applied Statistics), 31(3), 300-303.
Kaciranlar, S. and Sakallioglu. (2001). Combining the Liu estimator and the principal component regression
estimator. Communication statistics. Theory Methods. 30. 2699- 2705.
Khan, D. M., Yaqoob, A., Zubair, S., Khan, M. A., Ahmad, Z. and Alamri, O. A. (2021). Applications of
Robust Regression Techniques: An Econometric Approach. Mathematical Problem in Engineering
Vol. 202. Doi.org/10.1155/2021/6525079.
Kibria, B. G. (2003). Performance of some new ridge regression estimators. Communications in Statistics –
Simulation and Computation, 32(2), 419-435.
Kibria, B. M.G. and Shipra, B. (2016). Some Ridge Regression Estimators and Their Performances.
Journal of Modern Applied Statistical Methods, 15(1), 206 – 231.
Kibria, B. M. and Lukman, A. F. (2020). A new Ridge-Type Estimator for the linear Regression model.
Simulations and Applications, Hindawi scientifica Vol. 2020. https://doi.org/10.1155/20209758378.
Liu, K. (1993). A new class of biased estimate in linear regression. Journal of Communications in statistics.
Theory and Methods. 22:2, 393 – 402. Doi:10.1080/03610929308831027.
Lukman, A. F., Adewuyi, E. Oladejo, N. and Olukayode, A. (2019). Modified Almost Unbiased Two-
parameter Estimator in linear regression model. I.OP. Conf. series; Material science and Engineering
(2019).012119. doi: 10.1088/1757. -899X640/012119.
Lukman, A. F., Arowolo, O. and Ayinde, K. (2014). Some Robust Ridge Regression for for Handling
Multicollinearity and Outliers. International Journal of Sciences. Basic and Applied Research (IJSBAR), 16(2), 192 -202.
Lukman, A. F., Ayinde, K., Aladeitan, B. and Bamidele, R. (2020). An unbiased estimator with prior
information. Arab Journal of Basic and Applied Sciences. 27:1, 45 – 55.
Doi:10.1080/25765299.2019.1706799.
Lukman, A. F., Kibria, B. G., and Saleh, A. M. (2012). Robust ridge regression estimators: Some comparisons.
Journal of Applied Statistics, 39(5), 987-1001.
Majid, A., Ahmad, S., Aslam, M. and Kashif, M. A. (2021) Robust Kibria-Lukman estimator for linear regression model to combat multicollinearity and outliers. Concurrency and Computation: Practice and Experience. Doi.org/10.1002/cpe.7533.
Manson, K., Shukur, G. and Kibria, B. M. G. (2018). Performance of some ridge regression estimators for the
multinomial logit model. Communications in statistics - theory and methods, 47:12, 2795 – 2804. Doi: 10.1080/03610926.2013.784996.
Ozkale, M. R. and Kaciranlar, S. (2007). The restricted and unrestricted two-parameter estimators.
Communication Statistics. Theory. Meth. 36, 2707 – 2725.
Özkale, M. R., and Kibria, B. G. (2017). Some new ridge regression estimators and their performances.
Communications in Statistics - Theory and Methods, 46(3), 1501-1518.
Pasha, G. R. and Shah.M. A. A. (2004). Application of Ridge regression to Multicollinearity data. Journal of
Research Science, (Sci), 97 – 106.
Rousseeuw, P. J. (1984). Least Median of Squares Regression. Journal of the American Statistical Association,
, 871-880.
Rousseeuw, P. J., and Leroy, A. M. (1987). Robust Regression and Outlier Detection. New York: John Wiley
and Sons.
Sakallioglu, S. and Kaciranlar, S. (2008). A new biased estimator based on ridge estimation. Stat. Papers. 49,
-689.
Silvapulle, M. J. (1991). Robust ridge regression based on an M-estimator. Australian Journal of Statistics: 33; 319 – 333.
Susanti, Y., Hasil, P., Sri, S. H. andTwenty, L. (2014). M-Estimation, S Estimation and MM Estimation in
Robust Regression. International Journal of pure and Applied Mathematics 91, (3), 349 – 360.
Ullah, M. A., Pasha, G. R. and Aslam, M. (2013). Assessing Influence on the Liu Estimators in Linear
Regression Models. Communications in Statistics – Theory and Methods, 42(17), 3100 – 3116.
Walker, E. and Birch, J. B. (1988). Influence Measures in Ridge Regression. Technometrics, 30(2), 221 – 227.
Wold, S., Martens, H., and Wold, H. (1984). The multivariate calibration problem in chemistry solved by the
PLS method. Proceedings of the Conference on Matrix Pencils, 286-293.
Yang, H. and Chang, X. (2010). A new two-parameter estimator in linear regression model. Communication in
Statistics. Theory and Methods 39 (6) .923 – 934. Doi:10.1080/03610920902807911.
Yasin, A. and Murat, E. (2016). Influence Diagnostics in Two Parameter Ridge Regression. Journal of Data
Science, 14, 33 – 52.
Yohai, V.J. (1987). High Breakdown-point and High Efficiency Robust Estimates for Regression. The Annals
of Statistics. 15 (20): 642-656.
Zaman, A., Rousseuw, P. J. and Orhan, M. (2001). “Econometric applications of high-breakdown robust
regression techniques”, Economic Letters, 71, no.1,1-8.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 International Journal of Development Mathematics (IJDM)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors are solely responsible for obtaining permission to reproduce any copyrighted material contained in the manuscript as submitted. Any instance of possible prior publication in any form must be disclosed at the time the manuscript is submitted and a
copy or link to the publication must be provided.
The Journal articles are open access and are distributed under the terms of the Creative
Commons Attribution-NonCommercial-NoDerivs 4.0 IGO License, which permits use,
distribution, and reproduction in any medium, provided the original work is properly cited.
No modifications or commercial use of the articles are permitted.
