A Dai-Liao Hybrid PRP and DY Schemes for Unconstrained Optimization
DOI:
https://doi.org/10.62054/ijdm/0101.05Abstract
This article presents a new conjugate gradient (CG) method that requires first-order derivatives but overcomes the slow convergence issue associated with the steepest descent method and does not require the computation of second-order derivatives, as needed in the Newton method. The CG update parameter is suggested from the extended conjugacy condition as a convex combination of Polak, Ribiére, and Polyak (PRP) and Dai and Yuan (DY) algorithms by employing the optimal choice of the modulating parameter 't'. Numerical computations show that the algorithm is robust and efficient based on the number of iterations and CPU time. The scheme converges globally under Wolfe line search and adopts an inexact line search to obtain the step-size that generates a descent property, without requiring exalt computation of the step size.
Conjugate gradient method, Descent property, Dai-Liao conjugacy condition, Global convergence, Numerical methods
Riferimenti bibliografici
Abubakar, A. B., Kumam, P., Mohammad, H., Awwal., A. M. and Sitthithakerngkiet, K.(2019).
A modified Fletcher-Reeves conjugate gradient method for monotone nonlinear equations with some applications, MPDI, mathematics, 7, 745 doi:10.3390/math7080745.
Al-Baali, M. (1985). Descent property and global convergence of the Fletcher-Reeves method with inexact line search. IMA Journal of Numerical Analysis, 5, 121-124.
Al-Namat, F.N. and Al-Naemi, G.M. (2020). Global convergence property with inexact line
search for a new hybrid conjugate gradient method. Open Access Library Journal, 7: e6048.https://doi.org/10.4236/oalib.1106048
Andrei, N. (2008). A hybrid conjugate gradient algorithm for unconstrained optimization as convex combination of Hestenes- Stiefel and Dai-Yuan. Study Information Control, 17, 55-70.
Andrei, N. (2008). An unconstrained optimization test function. Advanced Modeling and Optimization. An Electronic International Journal,10, 172-182.
Andrei, N. (2009). Hybrid conjugate gradient algorithm for unconstrained optimization. Journal of Optimization Theory Application, 141, 249-264.
Andrei, N. (2017). A Dai-Liao nonlinear conjugate gradient algorithm. Numer. Algor.,
DOI 10.1007/s11075-017-0362-5
Babaie-Kafaki, S. (2011). A modified BFGS algorithm based on a hybrid secant equation. Science China Mathematics,58, 315-331.
Babaie-Kafaki, S. (2013). A hybrid conjugate gradient method based on quadratic relaxation of Dai-Yuan hybrid conjugate gradient parameter. Journal of Mathematical Programming and Operation Research, 62(7), 929-941.
Babaie-Kafaki, S. and Ghanbari, R. (2014c). Two hybrid nonlinear conjugate gradient methods based on a modified secant equation. Journal of Mathematical Programming and Operation Research,63(7), 1027-1042.
Babaie-Kafaki, S., and Ghanbari, R. (2014a). The Dai-Liao nonlinear conjugate gradient method
with optimal parameter choices. Eur. J. Oper. Res. 234(3), 625-630.
Babaie-Kafaki, S., and Ghanbari, R. (2014b). A descent family of Dai-Liao conjugate gradient
methods. Optim. Methods Softw., 29(3): 583-591.
Babaie-Kafaki, S., and Ghanbari, R. (2015). Two optimal Dai-Liao conjugate gradient methods.
Optimization, 64(1): 2277-2287.
Babaie-Kafaki, S. (2015). On optimality of two adaptive choices for the parameter of Dai-Liao
method. Optim. Lett., DOI 10.1007/s11590-015-0965-5.
Babaie-Kafaki, S., and Ghanbari, R. (2017). Two adaptive Dai-Liao nonlinear conjugate gradient methods. Iran J. Sci. Technol. Trans. Sci., DOI 10.1007/s40995-017- 027 1-4.
Dai, Y.H. and Yuan, Y. (1999). A nonlinear conjugate gradient method with a strong global convergence
property. SIAM J. Optim.,10, 177-182.
Dai, Y.H and Liao, L.Z. (2001). New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim., 43, 87-101.
Dai, Y.H. and Yuan, Y. (2001). An efficient hybrid conjugates gradient method for unconstrained optimization. Annals of Operations Research, 103, 33-47.
Djordjevic, S.S. (2016). New hybrid conjugate gradient method as a convex combination of FR and
PRP methods. Published by faculty of science and mathematics. University of Nis, Serbia,
Filomat, 31, 3083-3100. https://doi.org/10.2298/FIL16083D
Djordjevic, S. S. (2017). New hybrid conjugate gradient methods as a convex combination of LS and CD methods. Published by faculty of science and mathematics. University of Nis, Serbia,31(6), 1813-1825.
Djordjevic, S. S. (2018). New hybrid conjugate gradient method as a convex combination of HS and FR
conjugate gradient methods. Journal of Applied Mathematics and Computation, 2, 366-378.
https://doi.org/10.26855/jamc.2018.09.002
Djordjevic, S.S. (2019). New hybrid conjugate gradient method as a convex combination of LS and FR
conjugate gradient methods. Acta Mathematica Scientia, 39, 214-228.
Dolan, E. D. and More´, J. J. (2002). Benchmarking optimization software with performance profiles. Journal of Math. Program, 91(2), 201-213.
Esmaeili H., Rostami M. and Kimiaei M. (2018). Extended Dai–Yuan conjugate gradient strategy
for large-scale unconstrained optimization with applications to compressive sensing. Published by faculty of sciences and mathematics, University of Nis, Serbia, Filomat, 32(6), 2173–2191. Available at: http://www.pmf.ni.ac.rs/filomat
Fletcher, R. and Reeves, C. (1964). Function minimization by conjugate gradients. Computational Journal,7, 149-154.
Fletcher, R. (1987). Practical methods of optimization, (vol. 1), Unconstrained optimization. New York: John Wiley and Sons.
Gould, N. I. M., Orban, D. and Toint, P.L. (2003). CUTEr: a constrained and unconstrained testing environment, revisited. ACM Transactions on Mathematical Software,29(4), 373-394.
Guo, J. and Wan, Z. (2019). A modified spectral PRP conjugate gradient projection method for
solving large-scale monotone equations and its application in compressed sensing. Hindawi Mathematical Problems in Engineering. Volume 2019, Article ID 5261830, 17 pages. https://doi.org/10.1155/2019/5261830
Hager, W.W. and Zhang, H. (2006). A survey of nonlinear conjugate gradient methods.
Pacific J. Optim., 2, 35-58.
Hestenes, M. R. and Stiefel, E. L. (1952). Methods of conjugate gradients for solving linear systems. Journal Res. Nat. Bur. Stand., 49, 409-436
Ibrahim, A.H., Kumam, P., Abubakar, A. B., Jirakitpuwapat, W., and Abubakar, J. (2020). A hybrid
conjugate gradient algorithm for constrained monotone equations with application in compressive sensing. Heliyon.e03466. https://doi.org/10.1016/j.heliyon.2020.e03466
Li, D. H. and Fukushima, A. M. (2001). modified BFGS method and its global convergence in
nonconvex minimization, J. Comput. Appl. Math. 129(1) 15-35.
Liu, K., and Du, S. (2019). Modified three-term conjugate gradient method and its applications. Hindawi Mathematical Problems in Engineering. Volume 2019, Article ID 5976595, 9 pages.
https://doi.org/10.1155/2019/5976595
Liu, J., Du, S. and Chen, Y. (2020). A sufficient descent nonlinear conjugate gradient method for
solving M-tensor equations. Journal of Computational and Applied Mathematics, 371 112709
Liu, Y. and Storey, C. (1991). Efficient generalized conjugate gradient algorithms, part 1: Theory. Journal of optimization theory and applications. 69, 129-137.
Lotfi, M. and Hosseini, S. M. (2019). An efficient Dai–Liao type conjugate gradient method by
reformulating the CG parameter in the search direction equation. Journal of Computational and Applied Mathematics, 371. 112708. https://doi.org/10.1016/j.cam.2019.112708
Mohammed, N. S., Mustafa M., Mohd R. and Shazlyn M. S. (2020). A new hybrid coefficient of
conjugate gradient method. Indonesian Journal of Electrical Engineering and Computer Science, Vol. 18, No. 3, June 2020, pp. 1454-1463
Perry, A. (1978). A modified conjugate gradient algorithm. Oper. Res. Tech. Notes,26(6): 1073-1078.
Polak, B. T. (1969). The conjugate gradient method in extreme problems. USSR Comput. Math.
Math. Phys. 4, 94-112.
Polak, B.T. (1967). A general method of solving extremal problems. Soviet Math. Doklady, 8, 14-29.
Powell, M.J.D. (1984). Nonconvex minimization calculations and the conjugate
gradient method, Numerical Analysis (Dundee, 1983), D.F. Griffiths, ed., Lecture Notes in Mathematics, Vol. 1066, Springer, Berlin. 122-141.
Salihu, N., Odekunle, M., Waziri, M. and Halilu, A. (2020). A new hybrid conjugate gradient
method based on secant equation for solving large scale unconstrained optimization problems. Iranian Journal of Optimization, 12(1): 33-44.
Salihu. N., Odekunle, M., Waziri, M., Halilu, A., and Salihu, S (2021). A Dai- Liao hybrid conjugate gradient method for unconstrained optimization. International Journal of Industrial Optimization 2, 69–84.
Salihu, N., Odekunle, M. R., Saleh, A. M., and Salihu. S (2021). A Dai-Liao hybrid Hestenes-Stiefel
and Fletcher-Revees methods for unconstrained optimization. Int. J. Indust. Opt., 2(1):33–50.
Salihu, N., Kumam, P., Awwal, A. M., Arzuka, I., and Seangwattana, T., A (2023a). structured Fletcher
-Revees spectral conjugate gradient method for unconstrained optimization with application in robotic model. Oper. Res. Forum 4, Paper No. 81, 25.
Salihu, N., Kumam, P., Awwal, A. M., Sulaiman, I. M., and Seangwattana, T. (2023b). The global
convergence of spectral RMIL conjugate gradient method for unconstrained optimization with applications to robotic model and image recovery. Plos one 18, 3, e0281250.
Salihu, N., Kumam, P., Sulaiman, I. M., and Seangwattana, T (2023c). An efficient spectral minimization
of the dai-yuan method with application to image reconstruction. AIMS Mathematics 8(12):30940–30962.
Wang, X. A class of spectral three-term descent Hestenes-Stiefel conjugate gradient algorithms for
large-scale unconstrained optimization and image restoration problems. Appl. Numer. Math. 192 (2023), 41–56
Waziri, M. Y., Ahmed, K., and Sabi’u, J. (2019). A Dai-Liao conjugate gradient
method via modified secant equation for system of nonlinear equations. Arab. J. Math.,
https://doi.org/10.1007/s40065-019-0264-6.
Xue, W., Ren, J., Zheng, X., Liu, Z., and Liang, Y. (2018). A New DY conjugate gradient method
and applications to image denoising. IEICE TRANS. INF. and SYST., VOL.E101–D, NO.1 2, December 2018. DOI: 10.1587/transinf.2018EDP7210
Yao, S., Feng, Q., Li, L., and Xu, J. (2019). A class of globally convergent three-term Dai-Liao
conjugate gradient methods. Applied Numerical Mathematics, 151 354-366.
https://doi.org/10.1016/j.apnum.2019.12.026.
Zoutendijk, G. (1970). Nonlinear programming computational methods, in integer and nonlinear programming. J.
Abadie ed., North-Holland, Amsterdam 37-86.
Dowloads
Pubblicato
Fascicolo
Sezione
Licenza
Copyright (c) 2024 International Journal of Development Mathematics (IJDM)
Questo volume è pubblicato con la licenza Creative Commons Attribuzione - Non commerciale - Non opere derivate 4.0 Internazionale.
Authors are solely responsible for obtaining permission to reproduce any copyrighted material contained in the manuscript as submitted. Any instance of possible prior publication in any form must be disclosed at the time the manuscript is submitted and a
copy or link to the publication must be provided.
The Journal articles are open access and are distributed under the terms of the Creative
Commons Attribution-NonCommercial-NoDerivs 4.0 IGO License, which permits use,
distribution, and reproduction in any medium, provided the original work is properly cited.
No modifications or commercial use of the articles are permitted.
