A Comparative Monte Carlo Study of Hybrid and Single Robust Estimators for Nonlinear Regression Under Heavy-Tailed Error Distributions
DOI:
https://doi.org/10.62054/ijdm/0302.08Abstract
This paper presents a Monte Carlo simulation study comparing six competing robust estimation procedures: the M-estimator (M), MM-estimator (MM), Adaptive Huber estimator (AH), L1 regularization (L1), and two newly proposed hybrid methods comprising M-estimation with Adaptive Huber Loss (MAH) and MM-estimation with L1 Regularization (MM-L1) within polynomial and exponential nonlinear regression frameworks. The simulation spans three nonnormal error distributions (Uniform, Cauchy, and Exponential), two outlier contamination levels (5% and 50%) and three sample sizes (n = 50, 500, and 1,000), yielding 36 distinct experimental scenarios each evaluated over 1,000 Monte Carlo replications. Performance is assessed through Mean Squared Error (MSE). Results reveal a clear and consistent hierarchy: under mild contamination at small to moderate sample sizes, MAH consistently achieves the lowest MSE and under severe contamination at all sample sizes, or under heavy-tailed Cauchy errors, MM-L1 dominates through its complementary combination of a 50% breakdown point and L1 sparsity induction. L1 regularization alone performs weakest among the six methods under nonnormal conditions, reflecting its vulnerability arising from the squared-error data fit term. Single-method estimators M and MM occupy a middle performance tier, confirming that hybridisation systematically improves upon their individual capabilities. These findings are synthesized into a practical selection framework for researchers working with contaminated or heavy-tailed nonlinear data.
References
Alma, O. G. (2011). Comparison of robust regression methods in linear regression. International Journal of Contemporary Mathematical Sciences, 6(9), 409–421.
Almetwally, E. M., & Almongy, H. M. (2018). Comparison of robust estimation methods for parameter estimation in the linear regression model. International Journal of Mathematical Archive, 9(11), 1–9.
Huber, P. J. (1964). Robust estimation of a location parameter. Annals of Mathematical Statistics, 35(1), 73–101.https://doi.org/10.1214/aoms/1177703732
Huber, P. J. (1981). Robust Statistics. John Wiley & Sons. https://doi.org/10.1002/0471725250
Huber, P. J., & Ronchetti, E. M. (2009). Robust Statistics (2nd ed.). John Wiley & Sons. ISBN: 978-0-470-12990-6
Issa, M., & Rasheed, H. A. (2021). Performance of MM-estimation under various contamination scenarios. Journal of Statistical Theory and Applications, 20(1), 145–158.
Kaur, A., & Kaur, R. (2020). Comparing various robust estimation techniques in regression analysis. International Journal of Computer Applications, 175(9), 38–45..
Kumar, P., & Devi, A. (2022). A simulation study of robust estimation techniques in nonlinear regression. Journal of Applied Statistics, 49(7), 1690–1709.
Kurnaz, F. S., Hoffmann, I., & Filzmoser, P. (2018). Robust and sparse estimation methods for high-dimensional linear and logistic regression. Chemometrics and Intelligent Laboratory Systems, 172, 211–222. https://doi.org/10.1016/j.chemolab.2017.11.017
Maronna, R. A., Martin, R. D., & Yohai, V. J. (2019). Robust Statistics: Theory and Methods (with R) (2nd ed.). John Wiley & Sons. https://doi.org/10.1002/9781119214656
Maronna, R. A., & Yohai, V. J. (1991). The breakdown point of simultaneous general M-estimates of regression and scale. Journal of the American Statistical Association, 86(415), 699–708. https://doi.org/10.1080/01621459.1991.10475092
Rousseeuw, P. J., & Yohai, V. J. (1984). Robust regression by means of S-estimators. In J. Franke, W. Härdle, & D. Martin (Eds.), Robust and Nonlinear Time Series Analysis (pp. 256–272). Springer. https://doi.org/10.1007/978-1-4615-7821-5_15
Tabatabai, M. A., Eby, W. M., Li, H., Bae, S., & Singh, K. P. (2012). TELBS robust linear regression method. Open Access Medical Statistics, 2, 65–84. https://doi.org/10.2147/OAMS.S37395
Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society: Series B, 58(1), 267–288. https://doi.org/10.1111/j.2517-6161.1996.tb02080.x
Wang, L., Zheng, C., Zhou, W., & Zhou, W.-X. (2021). A new principle for tuning-free Huber regression. Statistica Sinica, 31(4), 2153–2177. https://doi.org/10.5705/ss.202019.0045.
Yohai, V. J. (1987). High breakdown-point and high efficiency robust estimates for regression. Annals of Statistics, 15(3), 642–656. https://doi.org/10.1214/aos/1176350366
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Bassa S. Yakura, Abraham Okolo, Emmanuel Torsen, Akinrefon A. Adesupo (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors are solely responsible for obtaining permission to reproduce any copyrighted material contained in the manuscript as submitted. Any instance of possible prior publication in any form must be disclosed at the time the manuscript is submitted and a
copy or link to the publication must be provided.
The Journal articles are open access and are distributed under the terms of the Creative
Commons Attribution-NonCommercial-NoDerivs 4.0 IGO License, which permits use,
distribution, and reproduction in any medium, provided the original work is properly cited.
No modifications or commercial use of the articles are permitted.
