convex optimization凸优化.pdf

convex optimization凸优化.pdf

《convex optimization凸优化.pdf》由会员分享，可在线阅读，更多相关《convex optimization凸优化.pdf（730页珍藏版）》请在冰豆网上搜索。

ConvexOptimizationConvexOptimizationStephenBoydDepartmentofElectricalEngineeringStanfordUniversityLievenVandenbergheElectricalEngineeringDepartmentUniversityofCalifornia,LosAngelescambridgeuniversitypressCambridge,NewYork,Melbourne,Madrid,CapeTown,Singapore,SaoPaolo,DelhiCambridgeUniversityPressTheEdinburghBuilding,Cambridge,CB28RU,UKPublishedintheUnitedStatesofAmericabyCambridgeUniversityPress,NewYorkhttp:

/www.cambridge.orgInformationonthistitle:

www.cambridge.org/9780521833783c?

CambridgeUniversityPress2004Thispublicationisincopyright.Subjecttostatutoryexceptionandtotheprovisionsofrelevantcollectivelicensingagreements,noreproductionofanypartmaytakeplacewithoutthewrittenpermissionofCambridgeUniversityPress.Firstpublished2004Seventhprintingwithcorrections2009PrintedintheUnitedKingdomattheUniversityPress,CambridgeAcataloguerecordforthispublicationisavailablefromtheBritishLibraryLibraryofCongressCataloguing-in-PublicationdataBoyd,StephenP.ConvexOptimization/StephenBoyd&LievenVandenberghep.cm.Includesbibliographicalreferencesandindex.ISBN05218337871.Mathematicaloptimization.2.Convexfunctions.I.Vandenberghe,Lieven.II.Title.QA402.5.B692004519.6dc222003063284ISBN978-0-521-83378-3hardbackCambridgeUniversityPresshasnoresponsiblityforthepersistencyoraccuracyofURLsforexternalorthird-partyinternetwebsitesreferredtointhispublication,anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain,accurateorappropriate.ForAnna,Nicholas,andNoraDanielandMargrietContentsPrefacexi1Introduction11.1Mathematicaloptimization.11.2Least-squaresandlinearprogramming.41.3Convexoptimization.71.4Nonlinearoptimization.91.5Outline.111.6Notation.14Bibliography.16ITheory192Convexsets212.1Affineandconvexsets.212.2Someimportantexamples.272.3Operationsthatpreserveconvexity.352.4Generalizedinequalities.432.5Separatingandsupportinghyperplanes.462.6Dualconesandgeneralizedinequalities.51Bibliography.59Exercises.603Convexfunctions673.1Basicpropertiesandexamples.673.2Operationsthatpreserveconvexity.793.3Theconjugatefunction.903.4Quasiconvexfunctions.953.5Log-concaveandlog-convexfunctions.1043.6Convexitywithrespecttogeneralizedinequalities.108Bibliography.112Exercises.113viiiContents4Convexoptimizationproblems1274.1Optimizationproblems.1274.2Convexoptimization.1364.3Linearoptimizationproblems.1464.4Quadraticoptimizationproblems.1524.5Geometricprogramming.1604.6Generalizedinequalityconstraints.1674.7Vectoroptimization.174Bibliography.188Exercises.1895Duality2155.1TheLagrangedualfunction.2155.2TheLagrangedualproblem.2235.3Geometricinterpretation.2325.4Saddle-pointinterpretation.2375.5Optimalityconditions.2415.6Perturbationandsensitivityanalysis.2495.7Examples.2535.8Theoremsofalternatives.2585.9Generalizedinequalities.264Bibliography.272Exercises.273IIApplications2896Approximationandfitting2916.1Normapproximation.2916.2Least-normproblems.3026.3Regularizedapproximation.3056.4Robustapproximation.3186.5Functionfittingandinterpolation.324Bibliography.343Exercises.3447Statisticalestimation3517.1Parametricdistributionestimation.3517.2Nonparametricdistributionestimation.3597.3Optimaldetectordesignandhypothesistesting.3647.4ChebyshevandChernoffbounds.3747.5Experimentdesign.384Bibliography.392Exercises.393Contentsix8Geometricproblems3978.1Projectiononaset.3978.2Distancebetweensets.4028.3Euclideandistanceandangleproblems.4058.4Extremalvolumeellipsoids.4108.5Centering.4168.6Classification.4228.7Placementandlocation.4328.8Floorplanning.438Bibliography.446Exercises.447IIIAlgorithms4559Unconstrainedminimization4579.1Unconstrainedminimizationproblems.4579.2Descentmethods.4639.3Gradientdescentmethod.4669.4Steepestdescentmethod.4759.5Newtonsmethod.4849.6Self-concordance.4969.7Implementation.508Bibliography.513Exercises.51410Equalityconstrainedminimization52110.1Equalityconstrainedminimizationproblems.52110.2Newtonsmethodwithequalityconstraints.52510.3InfeasiblestartNewtonmethod.53110.4Implementation.542Bibliography.556Exercises.55711Interior-pointmethods56111.1Inequalityconstrainedminimizationproblems.56111.2Logarithmicbarrierfunctionandcentralpath.56211.3Thebarriermethod.56811.4FeasibilityandphaseImethods.57911.5Complexityanalysisviaself-concordance.58511.6Problemswithgeneralizedinequalities.59611.7Primal-dualinterior-pointmethods.60911.8Implementation.615Bibliography.621Exercises.623xContentsAppendices631AMathematicalbackground633A.1Norms.633A.2Analysis.637A.3Functions.639A.4Derivatives.640A.5Linearalgebra.645Bibliography.652BProblemsinvolvingtwoquadraticfunctions653B.1Singleconstraintquadraticoptimization.653B.2TheS-procedure.655B.3Thefieldofvaluesoftwosymmetricmatrices.656B.4Proofsofthestrongdualityresults.657Bibliography.659CNumericallinearalgebrabackground661C.1Matrixstructureandalgorithmcomplexity.661C.2Solvinglinearequationswithfactoredmatrices.664C.3LU,Cholesky,andLDLTfactorization.668C.4BlockeliminationandSchurcomplements.672C.5Solvingunderdeterminedlinearequations.681Bibliography.684References685Notation697Index701PrefaceThisbookisaboutconvexoptimization,aspecialclassofmathematicaloptimiza-tionproblems,whichincludesleast-squaresandlinearprogrammingproblems.Itiswellknownthatleast-squaresandlinearprogrammingproblemshaveafairlycompletetheory,ariseinavarietyofapplications,andcanbesolvednumericallyveryefficiently.Thebasicpointofthisbookisthatthesamecanbesaidforthelargerclassofconvexoptimizationproblems.Whilethemathematicsofconvexoptimizationhasbeenstudiedforaboutacentury,severalrelatedrecentdevelopmentshavestimulatednewinterestinthetopic.Thefirstistherecognitionthatinterior-pointmethods,developedinthe1980stosolvelinearprogrammingproblems,canbeusedtosolveconvexoptimiza-tionproblemsaswell.Thesenewmethodsallowustosolvecertainnewclassesofconvexoptimizationproblems,suchassemidefiniteprogramsandsecond-orderconeprograms,almostaseasilyaslinearprograms.Theseconddevelopmentisthediscoverythatconvexoptimizationproblems（beyondleast-squaresandlinearprograms）aremoreprevalentinpracticethanwaspreviouslythought.Since1990manyapplicationshavebeendiscoveredinareassuchasautomaticcontrolsystems,estimationandsignalprocessing,com-municationsandnetworks,electroniccircuitdesign,dataanalysisandmodeling,statistics,andfinance.Convexoptimizationhasalsofoundwideapplicationincom-binatorialoptimizationandglobaloptimization,whereitisusedtofindboundsontheoptimalvalue,aswellasapproximatesolutions.Webelievethatmanyotherapplicationsofconvexoptimizationarestillwaitingtobediscovered.Therearegreatadvantagestorecognizingorformulatingaproblemasaconvexoptimizationproblem.Themostbasicadvantageisthattheproblemcanthenbesolved,veryreliablyandefficiently,usinginterior-pointmethodsorotherspecialmethodsforconvexoptimization.Thesesolutionmethodsarereliableenoughtobeembeddedinacomputer-aideddesignoranalysistool,orevenareal-timereactiveorautomaticcontrolsystem.Therearealsotheoreticalorconceptualadvantagesofformulatingaproblemasaconvexoptimizationproblem.Theassociateddualproblem,forexample,oftenhasaninterestinginterpretationintermsoftheoriginalproblem,andsometimesleadstoanefficientordistributedmethodforsolvingit.Wethinkthatconvexoptimizationisanimportantenoughtopicthateveryonewhousescomputationalmathematicsshouldknowatleastalittlebitaboutit.Inouropinion,convexoptimizationisanaturalnexttopicafteradvancedlinearalgebra（topicslikeleast-squares,singularvalues）,andlinearprogramming.xiiPrefaceGoalofthisbookFormanygeneralpurposeoptimizationmethods,thetypicalapproachistojusttryoutthemethodontheproblemtobesolved.Thefullbenefitsofconvexoptimization,incontrast,onlycomewhentheproblemisknownaheadoftimetobeconvex.Ofcourse,manyoptimizationproblemsarenotconvex,anditcanbedifficulttorecognizetheonesthatare,ortoreformulateaproblemsothatitisconvex.Ourmaingoalistohelpthereaderdevelopaworkingknowledgeofconvexoptimization,i.e.,todeveloptheskillsandbackgroundneededtorecognize,formulate,andsolveconvexoptimizationproblems.Developingaworkingknowledgeofconvexoptimizationcanbemathematicallydemanding,especiallyforthereaderinterestedprimarilyinapplications.Inourexperience（mostlywithgraduatestudentsinelectricalengineeringandcomputerscience）,theinvestmentoftenpaysoffwell,andsometimesverywell.Thereareseveralbooksonlinearprogramming,andgeneralnonlinearpro-gramming,thatfocusonproblemformulation,modeling,andapplications.Severalotherbookscoverthetheoryofconvexoptimization,orinterior-pointmethodsandtheircomplexityanalysis.Thisbookismeanttobesomethinginbetween,abookongeneralconvexoptimizationthatfocusesonproblemformulationandmodeling.Weshouldalsomentionwhatthisbookisnot.Itisnotatextprimarilyaboutconvexanalysis,orthemathematicsofconvexoptimization;severalexistingtextscoverthesetopicswell.Noristhebookasurveyofalgorithmsforconvexoptimiza-tion.Insteadwehavechosenjustafewgoodalgorithms,anddescribeonlysimple,stylizedversionsofthem（which,however,doworkwellinpractice）.Wemakenoattempttocoverthemostrecentstateoftheartininterior-point（orother）meth-odsforsolvingconvexproblems.Ourcoverageofnumericalimplementationissuesisalsohighlysimplified,butwefeelthatitisadequateforthepotentialusertodevelopworkingimplementations,andwedocover,insomede