用户名: 密码: 验证码:
求解大规模优化问题的快速算法与应用
详细信息    本馆镜像全文|  推荐本文 |  |   获取CNKI官网全文
摘要
本论文提出求解大规模优化问题的快速算法,证明算法的全局收敛性,通过数值试验验证算法的可行性及有效性,并测试算法在图像重构方面的实用性.
     第一章,给出无约束优化问题的基础知识,包括下降算法的定义,共轭梯度法和交替方向法研究进展, BB步长,全变差模型.简单介绍本文主要工作并列出文中所用的一些符号.
     第二章,基于Hager和Zhang的CG DESCENT算法,结合BB步长,提出一种求解无约束优化问题的CGCBB算法.该算法产生的搜索方向满足充分下降性.在适当条件下,建立算法全局收敛性.使用CUTEr函数测试库中71个问题对算法效率进行验证,数据比较表明该算法可与著名的CG DESCENT算法相媲美.
     第三章,提出求解约束全变差压缩传感问题的非精确交替方向法IADPM.在每次迭代,交替极小化问题的拉格朗日函数,且子问题存在显式解.在适当条件下建立算法的全局收敛性.数值试验表明所提算法可与著名的TVAL3相媲美.
     第四章,给出本论文的总结,并提出一些值得继续探讨的方向.
In this thesis, we propose fast algorithms for solving large-scale minimization prob-lems. Under some mild conditions, we show that the proposed algorithms converge glob-ally. Numerical experiments illustrate that the proposed methods are efective and promis-ing. Moreover, the practical performances in image reconstruction are included.
     In the frst chapter, we give some primary results of unconstrained optimizationproblems, which consists of the defnition of descent direction, recent progress of conju-gate gradient methods and alternating directions methods, Barzilai-Borwein steplength,and total variation model in image processing. Moreover, some important notation andsymbols which used in the context are also included.
     In chapter2, based on the CG DESCENT method of Hager&Zhang and Barzilai-Borwein steplength, we develop a sufcient descent algorithm for unconstrained minimiza-tion. We show that the resulting algorithm owns the attractive sufcient descent propertyand converges globally with some mild conditions. We test the proposed algorithm byusing a large set of unconstrained problems with high dimensions from CUTEr library.The numerical comparisons with the state-of-the-art algorithm CG DESCENT illustratethat the proposed method is efective, competitive, and promising.
     In chapter3, we propose a new inexact alternating directions algorithm for solvingthe TV regularized minimization problems with inequality constraints. We minimizealternatively the corresponding augmented lagrangian function at each step, and bothsubproblems admit closed-form solutions. We establish the global convergence of theproposed algorithm. Moreover, numerical comparisons with the sate-of-the-art methodTVAL3illustrate that the proposed method is efective and promising.
     In Chapter4, we list some concluding remarks and research topics.
引文
[1]李董辉,童小娇,万中,数值最优化,科学出版社,北京,2006,16-17.
    [2]袁亚湘,孙文瑜,最优化理论与方法,科学出版社,北京,2001,237-238.
    [3] Hestense M. R, Iterative methods for solving linear equations, Journal Optimization TheoryApplication,1973,11:323-334.
    [4] Al-Baail M, Descent property and global convergence of the Fletcher-Reeves method withinexact line search, IMA Journal of Numerical Analysis,1985,5:121-124.
    [5] Polak E and Ribiere G, Note sur la convergence de directions conjug′ees, Revue Francaised’Informatique et de Recherche Operationnelle,1969,16:35-43.
    [6] Hestenes M. R and Stiefel E, Methods of conjugate gradient for solving linear systems,Journal of research of the National Bureau of Standards,1952,49:409-436.
    [7] Dai Y. H and Yuan Y, Convergence properties of the Fletcher-Reeves method, IMA Journalof Numerical Analysis,1996,16:155-164.
    [8] Nocedal J and Wright J. S, Numerical Optimization, Springer Verlag, New York,1999.
    [9] Liu Y and Storey C, Efcient generalized conjugate gradient algorithms, Journal of Opti-mization Theory and Applications,1992,69:129-137.
    [10] Hager W. W and Zhang H, A survey of nonlinear conjugate gradient methods, PacifcJournal of Optimization,2006,2:35-58.
    [11] Gilbert J. C and Nocedal J, Global convergence properties of conjugate gradient methodfor optimization, SIAM Journal on Optimization,1992,2:21-42.
    [12] Grippo L and Lucidi S, A globally convergent version of the Ploak-Ribiere conjugate gra-dient method, Mathematical Programming,1997,78:375-391.
    [13] Yabe H and Takano M, Global convergence properties of nonlinear conjugate gradientmethods with modifed secant condition, Computational Optimization and Applications,2004,28:203-225.
    [14] Li G, Tang C and Wei Z, New conjugacy condition and related new conjugate gradientmethods for unconstrained optimization, Journal of Computational and Applied Mathe-matics,2007,202:523-539.
    [15] Dai Y. H and Liao L. Z, New conjugate conditions and related nonlinear conjugate gradientmethods, Applied Mathematics and Optimization,2001,43:87-101.
    [16] Hager W. W and Zhang H, A new conjugate gradient method with guaranteed descent andan efcient line search, SIAM Journal on Optimization,2005,16:170-192.
    [17] Barzilai J and Borwein J. M, Two point step size gradient method, IMA Journal ofNumerical Analysis,1988,8:141-148.
    [18] Dai Y. H and Liao L. Z, R-linear convergence of the Barzilai and Borwein gradientmethod,IMA Journal of Numerical Analysis,2002,26:1-10.
    [19] Raydan M, On the Barzilai and Borwein choice of steplength for the gradient method, IMAJournal of Numerical Analysis,1993,13:321-326.
    [20] Raydan M, The Barzilai and Borwein gradient method for the large scale unconstrainedminimization problem, SIAM Journal on Optimization,1997,7:26-33.
    [21] Dai Y. H, Hager W. W, Schitkowski K and Zhang H. C, The cyclic Barzilai-Borwein methodfor unconstrained optimization, IMA Journal of Numerical Analysis,2006,26:1-24.
    [22] Hager W. W and Zhang H, A projected adaptive cyclic Barzilai-Borwein method for boxconstrained optimization, Multiscale Optimization Methods and Applications,2006,82:387–392.
    [23] Glowinski R, and Marrocco A, Sur l’approximation, par′el′ements fnis d’ordre un, et lar′esolution, par p′enalisation-dualit′e d’une classe de probl`emes de Dirichlet nonlin′eaires, Re-vue Francaise d’automatique, informatique, recherche op′eretionnelle. Analyse num′erique,1975,2:41-76.
    [24] Glowinski R and Le T. P, Augmented Lagrangian and operator-splitting methods in non-linear mechanics, SIAM Studies in Applied Mathematics, Philadephia, PA,1989.
    [25] Glowinski R, Numerical methods for nonlinear variational problems, Springer-Verlat, NewYork,1984.
    [26] Gabay D and Mercier B, A dual algorithm for the solution of nonlinear variational problemsvia fnite element approximation, Computers and Mathematics with Applications,1976,2:17-40.
    [27] He B. S, Liao L. Z, Han D and Yang H, A new inexact alternating directions method formonotone variational inequalities, Mathematical Programming,2002,92:103-118.
    [28] He B. S, Yang H and Wang S. L, Alternating direction method with self-adaptive penaltyparameters for monotone variational inequalities, Journal of Optimization Theory and Ap-plications,2000,106:337-356.
    [29] Ye C. H and Yuan X. M, A descent method for structed monotone variational inequalities,Optimization Methods and Software,2007,22:329-338.
    [30] Yang J. F and Zhang Y, Alternating direction algorithms for1problems in compressivesensing, SIAM Journal on Scientifc Computing,2011,33:205-278.
    [31] Tao M and Yuan X. M, Recovering low-rank and sparse components of matrics from in-complete and noisy observations, SIAM Journal on Scientifc Computing,2011,21:57-81.
    [32] Xiao Y and Jin Z, An alternating direction method for linear-constrained matrix nuclearnorm minimization, Numerical Linear Algebra with Applications, DOI:10.1002/nla.783.
    [33] Rudin L, Osher S and Fatemi E, Nonlinear total variation based noise removal algorithms,Physica D: Nonlinear Phenomena,1992,60:259-268.
    [34] Vogel C.R and Oman M.E, Iterative methods for total variation denoising, SIAM Journalon Scientifc Computing,1996,17:227-238.
    [35] Chan T. F and Chen K, An optimization-based multilevel algorithm for tatal variationimage denoising, Multiscale Modeling and Simulation,2006,5:615-645.
    [36] Ng M, Qi L, Yang Y and Huang Y, On semismooth Newton’s methods for total variationminimization, Journal of Mathematical Imaging and Vision,2007,27:265-276.
    [37] Huang Y, Ng M and Wen Y. W, A fast total variation minimization method for imagerestoration, Multiscale Modeling and Simulation,2008,7:775-795.
    [38] Yu G, Qi L and Dai Y. H, On nonmonotone Chambolle gradient projection algorithms fortotal variation image restoration, Journal of Mathematical Imaging and Vision,2009,35:143-154.
    [39] Wang Y, Yang J, Yin W and Zhang Y, A new alternating minimization algorithm for totalvariation image reconstruction, SIAM Journal on Imaging Sciences,2008,1:248-272.
    [40] Yang J, Yin W, Zhang Y and Wang Y, A fast algorithm for edge-preserving variationalmultichannel image restoration, SIAM Journal on Imaging Sciences,2009,2:569-592.
    [41] Yang J, Zhang J and Yin W, An efcient TVL1algorithm for deblurring multichannelimages corrupted by impulsive noise, SIAM Journal on Imaging Sciences,2009,31:2842-2865.
    [42] Yang J, Zhang Y and Yin W, A fast TVL1-L2minimization algorithm for signal recon-struction from partial Fourier data, IEEE Journal of Selected Topics in Signal Processing,2010,4:288-297.
    [43] Bregman L. M, The relaxation method of fnding the common point of convex sets andits application to the solution of problems in convex programming, USSR ComputationalMathematics and Mathematical Physics,1967,7:200–217.
    [44] Goldstein T and Osher S, The split Bregman method for1-regularized problems, SIAMJournal on Imaging Sciences,2009,2:323-343.
    [45] Esser E, Applications of Lagrangian-based alternating direction methods andconnections to split Bregman, TR.09-31, CAM, UCLA,2009, available atftp://ftp.math.ucla.edu/pub/camreport/cam09-31.pdf.
    [46] Setzer S, Operator splittings, Bregman methods and frame Shrinkage in image processing,International Journal of Computer Vision,2011,92:265-280.
    [47] Donoho D, Compressed sensing, IEEE Transactions on Information Theory,2006,52:1289-1306.
    [48] Donoho D, For most large underdetemind systems of linear equations, the minimal1-normsolution is also the sparsest solution, Communications on Pure and Applied Mathematics,2006,59:907-934.
    [49] Cand`es E, Romberg J and Tao T, Stable signal recovery from imcomplete and inaccurateinformation, Communications on Pure and Applied Mathematics,2005,59:1207-1233.
    [50] Cand`es E, Romberg J and Tao T, Robust uncertainty principles: Exact signal recon-struction from highly incomplete frequence information, IEEE Transactions on InformationTheory,2006,52:489-509.
    [51] Bioucas-Dias J and Figueiredo M, A new TwIst: Two-step iterative thresholding algorithmfor image restoration, IEEE Transactions on Image Processing,2007,16:2992-3004.
    [52] Beck A and Teboulle M, A fast iterative shrinkage-thresholding algorithm for linear inverseproblems, SIAM Journal on Imaging Sciences,2009,2:183-202.
    [53] Beck A and Teboulle M, Fast gradient-based algorithms for constrained total variationimage denoising and deblurring problmes, IEEE Transactions on Imaging Processing,2009,18:2419–2434.
    [54] Figueriedo M, Nowak R and Wright S. J, Gradient projection for sparse reconstruction:Application to compressed sensing and other inverse problems, IEEE Journal of SelectedTopics in Signal Processing,2007,1:586-598.
    [55] Wright S, Nowak R and Figueiredo M, Sparse reconstruction by separable approximation,IEEE Transactions on Signal Processing,2009,57:2479–2493.
    [56] Esser E, Zhang X and Chan T. F, A general framework for a class of frst order primal-dualalgorithms for convex optimization in imaging science, SIAM Journal on Imaging Sciences,2010,3:1015-1046.
    [57] Xiao Y, Yang J and Yuan X, Fast algorithms for total variation image reconstruction fromrandom projections, available at http://arxiv.org/abs/1001.1774v1.
    [58] Zoutendijk G, Nonlinear programming, computational methods, Abadie J, ed, Integer andNonlinear Programming North-holland, Amsterdam,1970,37–86.
    [59] Conn A. R, Gould N. I. M and Toint P. L, CUTE: constrained and unconstrained testingenvironment, ACM Transactions on Mathematical Software,1995,21:123-160.
    [60] Dolan E. D and Mor′e J. J, Benchmarking optimization software with performance profles,Mathematical Programming,2002,91:201-213.
    [61] Ng M and Plemmons R, Fast recursive least squares adaptive fltering by fast fouriertransform-based conjugate gradient iterations, SIAM Journal on Scientifc Computing,2006,17:920-941.
    [62] He B, Liao L. Z, Han D and Yang H, A new inexact alternating directions method formonotone variational inequalities, Mathematical Programming,2002,92:103-118.
    [63] Li C, An efcient algorithm for total variation regularization with applications to the singlepixel camera and compressive sensing, Master thesis, Rice University, USA,2009.

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700