Algorithm计算机算法
AlgorithmsCopyrightc2006S.Dasgupta,C.H.Papadimitriou,andU.V.VaziraniJuly18,2006\n2\nContentsPreface90Prologue110.1Booksandalgorithms...................................110.2EnterFibonacci......................................120.3Big-Onotation.......................................15Exercises.............................................181Algorithmswithnumbers211.1Basicarithmetic......................................211.2Modulararithmetic....................................251.3Primalitytesting.....................................331.4Cryptography.......................................381.5Universalhashing.....................................42Exercises.............................................46Randomizedalgorithms:avirtualchapter382Divide-and-conqueralgorithms512.1Multiplication.......................................512.2Recurrencerelations...................................532.3Mergesort..........................................562.4Medians...........................................602.5Matrixmultiplication...................................622.6ThefastFouriertransform................................64Exercises.............................................793Decompositionsofgraphs873.1Whygraphs?........................................873.2Depth-rstsearchinundirectedgraphs........................893.3Depth-rstsearchindirectedgraphs..........................943.4Stronglyconnectedcomponents.............................97Exercises.............................................1013\n4Pathsingraphs1094.1Distances..........................................1094.2Breadth-rstsearch....................................1104.3Lengthsonedges.....................................1124.4Dijkstra'salgorithm....................................1124.5Priorityqueueimplementations.............................1204.6Shortestpathsinthepresenceofnegativeedges...................1224.7Shortestpathsindags..................................124Exercises.............................................1265Greedyalgorithms1335.1Minimumspanningtrees.................................1335.2Huffmanencoding.....................................1465.3Hornformulas.......................................1515.4Setcover..........................................152Exercises.............................................1556Dynamicprogramming1616.1Shortestpathsindags,revisited............................1616.2Longestincreasingsubsequences............................1626.3Editdistance........................................1656.4Knapsack..........................................1716.5Chainmatrixmultiplication...............................1746.6Shortestpaths.......................................1756.7Independentsetsintrees.................................179Exercises.............................................1817Linearprogrammingandreductions1897.1Anintroductiontolinearprogramming........................1897.2Flowsinnetworks.....................................1997.3Bipartitematching....................................2067.4Duality...........................................2077.5Zero-sumgames......................................2107.6Thesimplexalgorithm..................................2137.7Postscript:circuitevaluation..............................222Exercises.............................................2258NP-completeproblems2338.1Searchproblems......................................2338.2NP-completeproblems..................................2438.3Thereductions.......................................247Exercises.............................................2634\n9CopingwithNP-completeness2699.1Intelligentexhaustivesearch..............................2709.2Approximationalgorithms................................2759.3Localsearchheuristics..................................282Exercises.............................................29110Quantumalgorithms29510.1Qubits,superposition,andmeasurement.......................29510.2Theplan..........................................29910.3ThequantumFouriertransform............................30010.4Periodicity.........................................30210.5Quantumcircuits.....................................30410.6Factoringasperiodicity..................................30710.7Thequantumalgorithmforfactoring..........................308Exercises.............................................311Historicalnotesandfurtherreading313Index3155\n6\nListofboxesBasesandlogs..........................................21Two'scomplement........................................27Isyoursocialsecuritynumberaprime?...........................32Hey,thatwasgrouptheory!..................................35Carmichaelnumbers......................................36Randomizedalgorithms:avirtualchapter..........................37Anapplicationofnumbertheory?...............................39Binarysearch..........................................55Annlognlowerboundforsorting...............................58TheUnixsortcommand...................................62Whymultiplypolynomials?..................................65Theslowspreadofafastalgorithm..............................78Howbigisyourgraph?.....................................89Crawlingfast..........................................100Whichheapisbest?.......................................119Trees...............................................134Arandomizedalgorithmforminimumcut..........................143Entropy..............................................149Recursion?No,thanks......................................164Programming?..........................................164Commonsubproblems.....................................169Ofmiceandmen.........................................170Memoization...........................................173Ontimeandmemory......................................179Amagictrickcalledduality..................................193Reductions............................................197Matrix-vectornotation.....................................198Visualizingduality.......................................210Gaussianelimination......................................220Linearprogramminginpolynomialtime...........................222ThestoryofSissaandMoore.................................2337\nWhyPandNP?.........................................243Thetwowaystousereductions................................245Unsolvableproblems......................................262Entanglement..........................................298TheFouriertransformofaperiodicvector..........................303Settingupaperiodicsuperposition..............................307Quantumphysicsmeetscomputation............................3108\nPrefaceThisbookevolvedoverthepasttenyearsfromasetoflecturenotesdevelopedwhileteachingtheundergraduateAlgorithmscourseatBerkeleyandU.C.SanDiego.Ourwayofteachingthiscourseevolvedtremendouslyovertheseyearsinanumberofdirections,partlytoaddressourstudents'background(undevelopedformalskillsoutsideofprogramming),andpartlytoreectthematuringoftheeldingeneral,aswehavecometoseeit.Thenotesincreasinglycrystallizedintoanarrative,andweprogressivelystructuredthecoursetoemphasizethe“storyline”implicitintheprogressionofthematerial.Asaresult,thetopicswerecarefullyselectedandclustered.Noattemptwasmadetobeencyclopedic,andthisfreedustoincludetopicstraditionallyde-emphasizedoromittedfrommostAlgorithmsbooks.Playingonthestrengthsofourstudents(sharedbymostoftoday'sundergraduatesinComputerScience),insteadofdwellingonformalproofswedistilledineachcasethecrispmathematicalideathatmakesthealgorithmwork.Inotherwords,weemphasizedrigoroverformalism.Wefoundthatourstudentsweremuchmorereceptivetomathematicalrigorofthisform.Itisthisprogressionofcrispideasthathelpsweavethestory.OnceyouthinkaboutAlgorithmsinthisway,itmakessensetostartatthehistoricalbe-ginningofitall,where,inaddition,thecharactersarefamiliarandthecontrastsdramatic:numbers,primality,andfactoring.ThisisthesubjectofPartIofthebook,whichalsoin-cludestheRSAcryptosystem,anddivide-and-conqueralgorithmsforintegermultiplication,sortingandmediannding,aswellasthefastFouriertransform.Therearethreeotherparts:PartII,themosttraditionalsectionofthebook,concentratesondatastructuresandgraphs;thecontrasthereisbetweentheintricatestructureoftheunderlyingproblemsandtheshortandcrisppiecesofpseudocodethatsolvethem.Instructorswishingtoteachamoretradi-tionalcoursecansimplystartwithPartII,whichisself-contained(followingtheprologue),andthencoverPartIasrequired.InPartsIandIIweintroducedcertaintechniques(suchasgreedyanddivide-and-conquer)whichworkforspecialkindsofproblems;PartIIIdealswiththe“sledgehammers”ofthetrade,techniquesthatarepowerfulandgeneral:dynamicprogramming(anovelapproachhelpsclarifythistraditionalstumblingblockforstudents)andlinearprogramming(acleanandintuitivetreatmentofthesimplexalgorithm,duality,andreductionstothebasicproblem).ThenalPartIVisaboutwaysofdealingwithhardproblems:NP-completeness,variousheuristics,aswellasquantumalgorithms,perhapsthemostadvancedandmoderntopic.Asithappens,weendthestoryexactlywherewestartedit,withShor'squantumalgorithmforfactoring.Thebookincludesthreeadditionalundercurrents,intheformofthreeseriesofseparate“boxes,”strengtheningthenarrative(andaddressingvariationsintheneedsandinterestsofthestudents)whilekeepingtheowintact:piecesthatprovidehistoricalcontext;descriptionsofhowtheexplainedalgorithmsareusedinpractice(withemphasisoninternetapplications);9\nandexcursionsforthemathematicallysophisticated.10\nChapter0PrologueLookaroundyou.Computersandnetworksareeverywhere,enablinganintricatewebofcom-plexhumanactivities:education,commerce,entertainment,research,manufacturing,healthmanagement,humancommunication,evenwar.Ofthetwomaintechnologicalunderpinningsofthisamazingproliferation,oneisobvious:thebreathtakingpacewithwhichadvancesinmicroelectronicsandchipdesignhavebeenbringingusfasterandfasterhardware.Thisbooktellsthestoryoftheotherintellectualenterprisethatiscruciallyfuelingthecomputerrevolution:efcientalgorithms.Itisafascinatingstory.Gather'roundandlistenclose.0.1BooksandalgorithmsTwoideaschangedtheworld.In1448intheGermancityofMainzagoldsmithnamedJo-hannGutenbergdiscoveredawaytoprintbooksbyputtingtogethermovablemetallicpieces.Literacyspread,theDarkAgesended,thehumanintellectwasliberated,scienceandtech-nologytriumphed,theIndustrialRevolutionhappened.Manyhistorianssayweoweallthistotypography.Imagineaworldinwhichonlyanelitecouldreadtheselines!Butothersinsistthatthekeydevelopmentwasnottypography,butalgorithms.Todaywearesousedtowritingnumbersindecimal,thatitiseasytoforgetthatGuten-bergwouldwritethenumber1448asMCDXLVIII.HowdoyouaddtwoRomannumerals?WhatisMCDXLVIII+DCCCXII?(Andjusttrytothinkaboutmultiplyingthem.)EvenaclevermanlikeGutenbergprobablyonlyknewhowtoaddandsubtractsmallnumbersusinghisngers;foranythingmorecomplicatedhehadtoconsultanabacusspecialist.Thedecimalsystem,inventedinIndiaaroundAD600,wasarevolutioninquantitativereasoning:usingonly10symbols,evenverylargenumberscouldbewrittendowncompactly,andarithmeticcouldbedoneefcientlyonthembyfollowingelementarysteps.Nonethelesstheseideastookalongtimetospread,hinderedbytraditionalbarriersoflanguage,distance,andignorance.Themostinuentialmediumoftransmissionturnedouttobeatextbook,writteninArabicintheninthcenturybyamanwholivedinBaghdad.AlKhwarizmilaidoutthebasicmethodsforadding,multiplying,anddividingnumbers—evenextractingsquarerootsandcalculatingdigitsof.Theseprocedureswereprecise,unambiguous,mechanical,efcient,correct—inshort,theywerealgorithms,atermcoinedtohonorthewisemanafterthedecimalsystemwasnallyadoptedinEurope,manycenturieslater.11\nSincethen,thisdecimalpositionalsystemanditsnumericalalgorithmshaveplayedanenormousroleinWesterncivilization.Theyenabledscienceandtechnology;theyacceler-atedindustryandcommerce.Andwhen,muchlater,thecomputerwasnallydesigned,itexplicitlyembodiedthepositionalsysteminitsbitsandwordsandarithmeticunit.Scien-tistseverywherethengotbusydevelopingmoreandmorecomplexalgorithmsforallkindsofproblemsandinventingnovelapplications—ultimatelychangingtheworld.0.2EnterFibonacciAlKhwarizmi'sworkcouldnothavegainedafootholdintheWestwereitnotfortheeffortsofoneman:the15thcenturyItalianmathematicianLeonardoFibonacci,whosawthepotentialofthepositionalsystemandworkedhardtodevelopitfurtherandpropagandizeit.ButtodayFibonacciismostwidelyknownforhisfamoussequenceofnumbers0;1;1;2;3;5;8;13;21;34;:::;eachthesumofitstwoimmediatepredecessors.Moreformally,theFibonaccinumbersFnaregeneratedbythesimplerule8
1Fn=1ifn=1:0ifn=0.Noothersequenceofnumbershasbeenstudiedasextensively,orappliedtomoreelds:biology,demography,art,architecture,music,tonamejustafew.And,togetherwiththepowersof2,itiscomputerscience'sfavoritesequence.Infact,theFibonaccinumbersgrowalmostasfastasthepowersof2:forexample,F30isoveramillion,andF100isalready21digitslong!Ingeneral,Fn20:694n(seeExercise0.3).ButwhatistheprecisevalueofF100,orofF200?Fibonaccihimselfwouldsurelyhavewantedtoknowsuchthings.Toanswer,weneedanalgorithmforcomputingthenthFibonaccinumber.AnexponentialalgorithmOneideaistoslavishlyimplementtherecursivedenitionofFn.Hereistheresultingalgo-rithm,inthe“pseudocode”notationusedthroughoutthisbook:functionfib1(n)ifn=0:return0ifn=1:return1returnfib1(n 1)+fib1(n 2)Wheneverwehaveanalgorithm,therearethreequestionswealwaysaskaboutit:1.Isitcorrect?2.Howmuchtimedoesittake,asafunctionofn?3.Andcanwedobetter?12\nTherstquestionismoothere,asthisalgorithmispreciselyFibonacci'sdenitionofFn.Buttheseconddemandsananswer.LetT(n)bethenumberofcomputerstepsneededtocomputefib1(n);whatcanwesayaboutthisfunction?Forstarters,ifnislessthan2,theprocedurehaltsalmostimmediately,afterjustacoupleofsteps.Therefore,T(n)2forn1.Forlargervaluesofn,therearetworecursiveinvocationsoffib1,takingtimeT(n 1)andT(n 2),respectively,plusthreecomputersteps(checksonthevalueofnandanaladdition).Therefore,T(n)=T(n 1)+T(n 2)+3forn>1.ComparethistotherecurrencerelationforFn:weimmediatelyseethatT(n)Fn.Thisisverybadnews:therunningtimeofthealgorithmgrowsasfastastheFibonaccinumbers!T(n)isexponentialinn,whichimpliesthatthealgorithmisimpracticallyslowexceptforverysmallvaluesofn.Let'sbealittlemoreconcreteaboutjusthowbadexponentialtimeis.TocomputeF200,thefib1algorithmexecutesT(200)F2002138elementarycomputersteps.Howlongthisactuallytakesdepends,ofcourse,onthecomputerused.Atthistime,thefastestcomputerintheworldistheNECEarthSimulator,whichclocks40trillionstepspersecond.Evenonthismachine,fib1(200)wouldtakeatleast292seconds.Thismeansthat,ifwestartthecomputationtoday,itwouldstillbegoinglongafterthesunturnsintoaredgiantstar.Buttechnologyisrapidlyimproving—computerspeedshavebeendoublingroughlyevery18months,aphenomenonsometimescalledMoore'slaw.Withthisextraordinarygrowth,perhapsfib1willrunalotfasteronnextyear'smachines.Let'ssee—therunningtimeoffib1(n)isproportionalto20:694n(1:6)n,soittakes1:6timeslongertocomputeFn+1thanFn.AndunderMoore'slaw,computersgetroughly1:6timesfastereachyear.SoifwecanreasonablycomputeF100withthisyear'stechnology,thennextyearwewillmanageF101.Andtheyearafter,F102.Andsoon:justonemoreFibonaccinumbereveryyear!Suchisthecurseofexponentialtime.Inshort,ournaiverecursivealgorithmiscorrectbuthopelesslyinefcient.Canwedobetter?ApolynomialalgorithmLet'strytounderstandwhyfib1issoslow.Figure0.1showsthecascadeofrecursiveinvo-cationstriggeredbyasinglecalltofib1(n).Noticethatmanycomputationsarerepeated!Amoresensibleschemewouldstoretheintermediateresults—thevaluesF0;F1;:::;Fn 1—assoonastheybecomeknown.functionfib2(n)ifn=0return0createanarrayf[0:::n]f[0]=0,f[1]=1fori=2:::n:f[i]=f[i 1]+f[i 2]returnf[n]13\nFigure0.1Theproliferationofrecursivecallsinfib1.FnFn 1Fn 2Fn 2Fn 3Fn 3Fn 4Fn 3Fn 4Fn 4Fn 5Fn 4Fn 5Fn 5Fn 6 Aswithfib1,thecorrectnessofthisalgorithmisself-evidentbecauseitdirectlyusesthedenitionofFn.Howlongdoesittake?Theinnerloopconsistsofasinglecomputerstepandisexecutedn 1times.Thereforethenumberofcomputerstepsusedbyfib2islinearinn.Fromexponentialwearedowntopolynomial,ahugebreakthroughinrunningtime.ItisnowperfectlyreasonabletocomputeF200orevenF200;000.1Aswewillseerepeatedlythroughoutthisbook,therightalgorithmmakesallthediffer-ence.MorecarefulanalysisInourdiscussionsofar,wehavebeencountingthenumberofbasiccomputerstepsexecutedbyeachalgorithmandthinkingofthesebasicstepsastakingaconstantamountoftime.Thisisaveryusefulsimplication.Afterall,aprocessor'sinstructionsethasavarietyofbasicprimitives—branching,storingtomemory,comparingnumbers,simplearithmetic,andsoon—andratherthandistinguishingbetweentheseelementaryoperations,itisfarmoreconvenienttolumpthemtogetherintoonecategory.ButlookingbackatourtreatmentofFibonaccialgorithms,wehavebeentooliberalwithwhatweconsiderabasicstep.Itisreasonabletotreatadditionasasinglecomputerstepifsmallnumbersarebeingadded,32-bitnumberssay.ButthenthFibonaccinumberisabout0:694nbitslong,andthiscanfarexceed32asngrows.Arithmeticoperationsonarbitrarilylargenumberscannotpossiblybeperformedinasingle,constant-timestep.Weneedtoauditourearlierrunningtimeestimatesandmakethemmorehonest.WewillseeinChapter1thattheadditionoftwon-bitnumberstakestimeroughlypropor-tionalton;thisisnottoohardtounderstandifyouthinkbacktothegrade-schoolprocedure1Tobetterappreciatetheimportanceofthisdichotomybetweenexponentialandpolynomialalgorithms,thereadermaywanttopeekaheadtothestoryofSissaandMoore,inChapter8.14\nforaddition,whichworksononedigitatatime.Thusfib1,whichperformsaboutFnad-ditions,actuallyusesanumberofbasicstepsroughlyproportionaltonFn.Likewise,thenumberofstepstakenbyfib2isproportionalton2,stillpolynomialinnandthereforeex-ponentiallysuperiortofib1.Thiscorrectiontotherunningtimeanalysisdoesnotdiminishourbreakthrough.Butcanwedoevenbetterthanfib2?Indeedwecan:seeExercise0.4.0.3Big-OnotationWe'vejustseenhowsloppinessintheanalysisofrunningtimescanleadtoanunacceptablelevelofinaccuracyintheresult.Buttheoppositedangerisalsopresent:itispossibletobetooprecise.Aninsightfulanalysisisbasedontherightsimplications.Expressingrunningtimeintermsofbasiccomputerstepsisalreadyasimplication.Afterall,thetimetakenbyonesuchstepdependscruciallyontheparticularprocessorandevenondetailssuchascachingstrategy(asaresultofwhichtherunningtimecandiffersubtlyfromoneexecutiontothenext).Accountingforthesearchitecture-specicminutiaeisanightmar-ishlycomplextaskandyieldsaresultthatdoesnotgeneralizefromonecomputertothenext.Itthereforemakesmoresensetoseekanuncluttered,machine-independentcharacterizationofanalgorithm'sefciency.Tothisend,wewillalwaysexpressrunningtimebycountingthenumberofbasiccomputersteps,asafunctionofthesizeoftheinput.Andthissimplicationleadstoanother.Insteadofreportingthatanalgorithmtakes,say,5n3+4n+3stepsonaninputofsizen,itismuchsimplertoleaveoutlower-ordertermssuchas4nand3(whichbecomeinsignicantasngrows),andeventhedetailofthecoefcient5intheleadingterm(computerswillbevetimesfasterinafewyearsanyway),andjustsaythatthealgorithmtakestimeO(n3)(pronounced“bigohofn3”).Itistimetodenethisnotationprecisely.Inwhatfollows,thinkoff(n)andg(n)astherunningtimesoftwoalgorithmsoninputsofsizen.Letf(n)andg(n)befunctionsfrompositiveintegerstopositivereals.Wesayf=O(g)(whichmeansthat“fgrowsnofasterthang”)ifthereisaconstantc>0suchthatf(n)cg(n).Sayingf=O(g)isaverylooseanalogof“fg.”Itdiffersfromtheusualnotionofbecauseoftheconstantc,sothatforinstance10n=O(n).Thisconstantalsoallowsustodisregardwhathappensforsmallvaluesofn.Forexample,supposewearechoosingbetweentwoalgorithmsforaparticularcomputationaltask.Onetakesf1(n)=n2steps,whiletheothertakesf2(n)=2n+20steps(Figure0.2).Whichisbetter?Well,thisdependsonthevalueofn.Forn5,f1issmaller;thereafter,f2istheclearwinner.Inthiscase,f2scalesmuchbetterasngrows,andthereforeitissuperior.Thissuperiorityiscapturedbythebig-Onotation:f2=O(f1),becausef2(n)2n+20=22f1(n)n2foralln;ontheotherhand,f16=O(f2),sincetheratiof1(n)=f2(n)=n2=(2n+20)cangetarbitrarilylarge,andsonoconstantcwillmakethedenitionwork.15\nFigure0.2Whichrunningtimeisbetter?100908070602n5040302n+202010012345678910nNowanotheralgorithmcomesalong,onethatusesf3(n)=n+1steps.Isthisbetterthanf2?Certainly,butonlybyaconstantfactor.Thediscrepancybetweenf2andf3istinycomparedtothehugegapbetweenf1andf2.Inordertostayfocusedonthebigpicture,wetreatfunctionsasequivalentiftheydifferonlybymultiplicativeconstants.Returningtothedenitionofbig-O,weseethatf2=O(f3):f2(n)2n+20=20;f3(n)n+1andofcoursef3=O(f2),thistimewithc=1.JustasO()isananalogof,wecanalsodeneanalogsofand=asfollows:f=(g)meansg=O(f)f=(g)meansf=O(g)andf=(g):Intheprecedingexample,f2=(f3)andf1=(f3).Big-Onotationletsusfocusonthebigpicture.Whenfacedwithacomplicatedfunctionlike3n2+4n+5,wejustreplaceitwithO(f(n)),wheref(n)isassimpleaspossible.Inthisparticularexamplewe'duseO(n2),becausethequadraticportionofthesumdominatestherest.Herearesomecommonsenserulesthathelpsimplifyfunctionsbyomittingdominatedterms:1.Multiplicativeconstantscanbeomitted:14n2becomesn2.2.nadominatesnbifa>b:forinstance,n2dominatesn.3.Anyexponentialdominatesanypolynomial:3ndominatesn5(itevendominates2n).16\n4.Likewise,anypolynomialdominatesanylogarithm:ndominates(logn)3.Thisalsomeans,forexample,thatn2dominatesnlogn.Don'tmisunderstandthiscavalierattitudetowardconstants.Programmersandalgorithmdevelopersareveryinterestedinconstantsandwouldgladlystayupnightsinordertomakeanalgorithmrunfasterbyafactorof2.Butunderstandingalgorithmsatthelevelofthisbookwouldbeimpossiblewithoutthesimplicityaffordedbybig-Onotation.17\nExercises0.1.Ineachofthefollowingsituations,indicatewhetherf=O(g),orf=(g),orboth(inwhichcasef=(g)).f(n)g(n)(a)n 100n 200(b)n1=2n2=3(c)100n+lognn+(logn)2(d)nlogn10nlog10n(e)log2nlog3n(f)10lognlog(n2)(g)n1:01nlog2n(h)n2=lognn(logn)2(i)n0:1(logn)10(j)(logn)lognn=lognp(k)n(logn)3(l)n1=25log2n(m)n2n3n(n)2n2n+1(o)n!2n2(p)(logn)logn2(log2n)Pnkk+1(q)ini=10.2.Showthat,ifcisapositiverealnumber,theng(n)=1+c+c2++cnis:(a)(1)ifc<1.(b)(n)ifc=1.(c)(cn)ifc>1.Themoral:inbig-terms,thesumofageometricseriesissimplythersttermiftheseriesisstrictlydecreasing,thelasttermiftheseriesisstrictlyincreasing,orthenumberoftermsiftheseriesisunchanging.0.3.TheFibonaccinumbersF0;F1;F2;:::;aredenedbytheruleF0=0;F1=1;Fn=Fn 1+Fn 2:Inthisproblemwewillconrmthatthissequencegrowsexponentiallyfastandobtainsomeboundsonitsgrowth.(a)UseinductiontoprovethatF20:5nforn6.n(b)Findaconstantc<1suchthatF2cnforalln0.Showthatyouransweriscorrect.n(c)WhatisthelargestcyoucanndforwhichF=(2cn)?n0.4.IsthereafasterwaytocomputethenthFibonaccinumberthanbyfib2(page13)?Oneideainvolvesmatrices.WestartbywritingtheequationsF1=F1andF2=F0+F1inmatrixnotation:F101F0=:F211F1Similarly,2F201F101F0==F311F211F118\nandingeneralnFn01F0=:Fn+111F1So,inordertocomputeFn,itsufcestoraisethis22matrix,callitX,tothenthpower.(a)Showthattwo22matricescanbemultipliedusing4additionsand8multiplications.ButhowmanymatrixmultiplicationsdoesittaketocomputeXn?(b)ShowthatO(logn)matrixmultiplicationssufceforcomputingXn.(Hint:ThinkaboutcomputingX8.)Thusthenumberofarithmeticoperationsneededbyourmatrix-basedalgorithm,callitfib3,isjustO(logn),ascomparedtoO(n)forfib2.Havewebrokenanotherexponentialbarrier?Thecatchisthatournewalgorithminvolvesmultiplication,notjustaddition;andmultiplica-tionsoflargenumbersareslowerthanadditions.Wehavealreadyseenthat,whenthecomplex-ityofarithmeticoperationsistakenintoaccount,therunningtimeoffib2becomesO(n2).(c)Showthatallintermediateresultsoffib3areO(n)bitslong.(d)LetM(n)betherunningtimeofanalgorithmformultiplyingn-bitnumbers,andassumethatM(n)=O(n2)(theschoolmethodformultiplication,recalledinChapter1,achievesthis).Provethattherunningtimeoffib3isO(M(n)logn).(e)Canyouprovethattherunningtimeoffib3isO(M(n))?(Hint:Thelengthsofthenum-bersbeingmultipliedgetdoubledwitheverysquaring.)Inconclusion,whetherfib3isfasterthanfib2dependsonwhetherwecanmultiplyn-bitintegersfasterthanO(n2).Doyouthinkthisispossible?(TheanswerisinChapter2.)Finally,thereisaformulafortheFibonaccinumbers:p!np!n11+511 5Fn=p p:5252So,itwouldappearthatweonlyneedtoraiseacoupleofnumberstothenthpowerinordertocomputeFn.Theproblemisthatthesenumbersareirrational,andcomputingthemtosufcientaccuracyisnontrivial.Infact,ourmatrixmethodfib3canbeseenasaroundaboutwayofraisingtheseirrationalnumberstothenthpower.Ifyouknowyourlinearalgebra,youshouldseewhy.(Hint:WhataretheeigenvaluesofthematrixX?)19\n20\nChapter1AlgorithmswithnumbersOneofthemainthemesofthischapteristhedramaticcontrastbetweentwoancientproblemsthatatrstseemverysimilar:Factoring:GivenanumberN,expressitasaproductofitsprimefactors.Primality:GivenanumberN,determinewhetheritisaprime.Factoringishard.Despitecenturiesofeffortbysomeoftheworld'ssmartestmathemati-ciansandcomputerscientists,thefastestmethodsforfactoringanumberNtaketimeexpo-nentialinthenumberofbitsofN.Ontheotherhand,weshallsoonseethatwecanefcientlytestwhetherNisprime!And(itgetsevenmoreinteresting)thisstrangedisparitybetweenthetwointimatelyrelatedproblems,oneveryhardandtheotherveryeasy,liesattheheartofthetechnologythatenablessecurecommunicationintoday'sglobalinformationenvironment.Enroutetotheseinsights,weneedtodevelopalgorithmsforavarietyofcomputationaltasksinvolvingnumbers.Webeginwithbasicarithmetic,anespeciallyappropriatestartingpointbecause,asweknow,thewordalgorithmsoriginallyappliedonlytomethodsfortheseproblems.1.1Basicarithmetic1.1.1AdditionWeweresoyoungwhenwelearnedthestandardtechniqueforadditionthatwewouldscarcelyhavethoughttoaskwhyitworks.Butlet'sgobacknowandtakeacloserlook.ItisabasicpropertyofdecimalnumbersthatThesumofanythreesingle-digitnumbersisatmosttwodigitslong.Quickcheck:thesumisatmost9+9+9=27,twodigitslong.Infact,thisruleholdsnotjustindecimalbutinanybaseb2(Exercise1.1).Inbinary,forinstance,themaximumpossiblesumofthreesingle-bitnumbersis3,whichisa2-bitnumber.Thissimplerulegivesusawaytoaddtwonumbersinanybase:aligntheirright-handends,andthenperformasingleright-to-leftpassinwhichthesumiscomputeddigitbydigit,maintainingtheoverowasacarry.Sinceweknoweachindividualsumisatwo-digit21\nBasesandlogsNaturally,thereisnothingspecialaboutthenumber10—wejusthappentohave10ngers,andso10wasanobviousplacetopauseandtakecountingtothenextlevel.TheMayansdevelopedasimilarpositionalsystembasedonthenumber20(noshoes,see?).Andofcoursetodaycomputersrepresentnumbersinbinary.HowmanydigitsareneededtorepresentthenumberN0inbaseb?Let'ssee—withkdigitsinbasebwecanexpressnumbersuptobk 1;forinstance,indecimal,threedigitsgetusallthewayupto999=103 1.Bysolvingfork,wendthatdlogb(N+1)edigits(aboutlogbNdigits,giveortake1)areneededtowriteNinbaseb.Howmuchdoesthesizeofanumberchangewhenwechangebases?Recalltheruleforconvertinglogarithmsfrombaseatobaseb:logbN=(logaN)=(logab).SothesizeofintegerNinbaseaisthesameasitssizeinbaseb,timesaconstantfactorlogab.Inbig-Onotation,therefore,thebaseisirrelevant,andwewritethesizesimplyasO(logN).Whenwedonotspecifyabase,aswealmostneverwill,wemeanlog2N.Incidentally,thisfunctionlogNappearsrepeatedlyinoursubject,inmanyguises.Here'sasampling:1.logNis,ofcourse,thepowertowhichyouneedtoraise2inordertoobtainN.2.Goingbackward,itcanalsobeseenasthenumberoftimesyoumusthalveNtogetdownto1.(Moreprecisely:dlogNe.)Thisisusefulwhenanumberishalvedateachiterationofanalgorithm,asinseveralexampleslaterinthechapter.3.ItisthenumberofbitsinthebinaryrepresentationofN.(Moreprecisely:dlog(N+1)e.)4.ItisalsothedepthofacompletebinarytreewithNnodes.(Moreprecisely:blogNc.)5.Itiseventhesum1+1+1++1,towithinaconstantfactor(Exercise1.5).23Nnumber,thecarryisalwaysasingledigit,andsoatanygivenstep,threesingle-digitnumbersareadded.Here'sanexampleshowingtheaddition53+35inbinary.Carry:1111110101(53)100011(35)1011000(88)Ordinarilywewouldspelloutthealgorithminpseudocode,butinthiscaseitissofamiliarthatwedonotrepeatit.Insteadwemovestraighttoanalyzingitsefciency.Giventwobinarynumbersxandy,howlongdoesouralgorithmtaketoaddthem?Thisisthekindofquestionweshallpersistentlybeaskingthroughoutthisbook.Wewanttheanswerexpressedasafunctionofthesizeoftheinput:thenumberofbitsofxandy,thenumberofkeystrokesneededtotypethemin.Supposexandyareeachnbitslong;inthischapterwewillconsistentlyusetheletternforthesizesofnumbers.Thenthesumofxandyisn+1bitsatmost,andeachindividualbit22\nofthissumgetscomputedinaxedamountoftime.Thetotalrunningtimefortheadditionalgorithmisthereforeoftheformc0+c1n,wherec0andc1aresomeconstants;inotherwords,itislinear.Insteadofworryingabouttheprecisevaluesofc0andc1,wewillfocusonthebigpictureanddenotetherunningtimeasO(n).Nowthatwehaveaworkingalgorithmwhoserunningtimeweknow,ourthoughtswanderinevitablytothequestionofwhetherthereissomethingevenbetter.Isthereafasteralgorithm?(Thisisanotherpersistentquestion.)Foraddition,theansweriseasy:inordertoaddtwon-bitnumberswemustatleastreadthemandwritedowntheanswer,andeventhatrequiresnoperations.Sotheadditionalgorithmisoptimal,uptomultiplicativeconstants!Somereadersmaybeconfusedatthispoint:WhyO(n)operations?Isn'tbinaryadditionsomethingthatcomputerstodayperformbyjustoneinstruction?Therearetwoanswers.First,itiscertainlytruethatinasingleinstructionwecanaddintegerswhosesizeinbitsiswithinthewordlengthoftoday'scomputers—32perhaps.But,aswillbecomeapparentlaterinthischapter,itisoftenusefulandnecessarytohandlenumbersmuchlargerthanthis,perhapsseveralthousandbitslong.Addingandmultiplyingsuchlargenumbersonrealcomputersisverymuchlikeperformingtheoperationsbitbybit.Second,whenwewanttounderstandalgorithms,itmakessensetostudyeventhebasicalgorithmsthatareencodedinthehardwareoftoday'scomputers.Indoingso,weshallfocusonthebitcomplexityofthealgorithm,thenumberofelementaryoperationsonindividualbits—becausethisaccount-ingreectstheamountofhardware,transistorsandwires,necessaryforimplementingthealgorithm.1.1.2MultiplicationanddivisionOnwardtomultiplication!Thegrade-schoolalgorithmformultiplyingtwonumbersxandyistocreateanarrayofintermediatesums,eachrepresentingtheproductofxbyasingledigitofy.Thesevaluesareappropriatelyleft-shiftedandthenaddedup.Supposeforinstancethatwewanttomultiply1311,orinbinarynotation,x=1101andy=1011.Themultiplicationwouldproceedthus.110110111101(1101times1)1101(1101times1,shiftedonce)0000(1101times0,shiftedtwice)+1101(1101times1,shiftedthrice)10001111(binary143)Inbinarythisisparticularlyeasysinceeachintermediaterowiseitherzeroorxitself,left-shiftedanappropriateamountoftimes.Alsonoticethatleft-shiftingisjustaquickwaytomultiplybythebase,whichinthiscaseis2.(Likewise,theeffectofarightshiftistodividebythebase,roundingdownifneeded.)ThecorrectnessofthismultiplicationprocedureisthesubjectofExercise1.6;let'smoveonandgureouthowlongittakes.Ifxandyarebothnbits,thentherearenintermediate23\nFigure1.1Multiplicationa laFranc¸ais.functionmultiply(x;y)Input:Twon-bitintegersxandy,wherey0Output:Theirproductify=0:return0z=multiply(x;by=2c)ifyiseven:return2zelse:returnx+2zrows,withlengthsofupto2nbits(takingtheshiftingintoaccount).Thetotaltimetakentoadduptheserows,doingtwonumbersatatime,isO(n)+O(n)++O(n);|{z}n 1timeswhichisO(n2),quadraticinthesizeoftheinputs:stillpolynomialbutmuchslowerthanaddition(aswehaveallsuspectedsinceelementaryschool).ButAlKhwarizmiknewanotherwaytomultiply,amethodwhichisusedtodayinsomeEuropeancountries.Tomultiplytwodecimalnumbersxandy,writethemnexttoeachother,asintheexamplebelow.Thenrepeatthefollowing:dividetherstnumberby2,roundingdowntheresult(thatis,droppingthe:5ifthenumberwasodd),anddoublethesecondnumber.Keepgoingtilltherstnumbergetsdownto1.Thenstrikeoutalltherowsinwhichtherstnumberiseven,andaddupwhateverremainsinthesecondcolumn.1113526252(strikeout)1104143(answer)Butifwenowcomparethetwoalgorithms,binarymultiplicationandmultiplicationbyre-peatedhalvingsofthemultiplier,wenoticethattheyaredoingthesamething!Thethreenumbersaddedinthesecondalgorithmarepreciselythemultiplesof13bypowersof2thatwereaddedinthebinarymethod.Onlythistime11wasnotgiventousexplicitlyinbinary,andsowehadtoextractitsbinaryrepresentationbylookingattheparityofthenumbersob-tainedfromitbysuccessivedivisionsby2.AlKhwarizmi'ssecondalgorithmisafascinatingmixtureofdecimalandbinary!Thesamealgorithmcanthusberepackagedindifferentways.Forvarietyweadoptathirdformulation,therecursivealgorithmofFigure1.1,whichdirectlyimplementstherule2(xby=2c)ifyisevenxy=x+2(xby=2c)ifyisodd:24\nFigure1.2Division.functiondivide(x;y)Input:Twon-bitintegersxandy,wherey1Output:Thequotientandremainderofxdividedbyyifx=0:return(q;r)=(0;0)(q;r)=divide(bx=2c;y)q=2q;r=2rifxisodd:r=r+1ifry:r=r y;q=q+1return(q;r)Isthisalgorithmcorrect?Theprecedingrecursiveruleistransparentlycorrect;socheck-ingthecorrectnessofthealgorithmismerelyamatterofverifyingthatitmimicstheruleandthatithandlesthebasecase(y=0)properly.Howlongdoesthealgorithmtake?Itmustterminateafternrecursivecalls,becauseateachcallyishalved—thatis,itsnumberofbitsisdecreasedbyone.Andeachrecursivecallrequirestheseoperations:adivisionby2(rightshift);atestforodd/even(lookingupthelastbit);amultiplicationby2(leftshift);andpossiblyoneaddition,atotalofO(n)bitoperations.ThetotaltimetakenisthusO(n2),justasbefore.Canwedobetter?Intuitively,itseemsthatmultiplicationrequiresaddingaboutnmulti-plesofoneoftheinputs,andweknowthateachadditionislinear,soitwouldappearthatn2bitoperationsareinevitable.Astonishingly,inChapter2we'llseethatwecandosignicantlybetter!Divisionisnext.Todivideanintegerxbyanotherintegery6=0meanstondaquotientqandaremainderr,wherex=yq+randra=2.Thesetwocasesareshowninthefollowinggure.Ifba=2,thenwehaveamodba=2,thenamodb=a b1,thenax61modN,nomatterwhatxmightbe,andthereforeacannothaveamultiplicativeinversemoduloN.Infact,thisistheonlycircumstanceinwhichaisnotinvertible.Whengcd(a;N)=1(wesayaandNarerelativelyprime),theextendedEuclidalgorithmgivesusintegersxandysuchthatax+Ny=1,whichmeansthatax1(modN).Thusxisa'ssoughtinverse.Example.Continuingwithourpreviousexample,supposewewishtocompute11 1mod25.UsingtheextendedEuclidalgorithm,wendthat1525 3411=1.Reducingbothsidesmodulo25,wehave 34111mod25.So 3416mod25istheinverseof11mod25.ModulardivisiontheoremForanyamodN,ahasamultiplicativeinversemoduloNifandonlyifitisrelativelyprimetoN.Whenthisinverseexists,itcanbefoundintimeO(n3)(whereasusualndenotesthenumberofbitsofN)byrunningtheextendedEuclidalgorithm.Thisresolvestheissueofmodulardivision:whenworkingmoduloN,wecandividebynumbersrelativelyprimetoN—andonlybythese.Andtoactuallycarryoutthedivision,wemultiplybytheinverse.Isyoursocialsecuritynumberaprime?Thenumbers7;17;19;71;and79areprimes,buthowabout717-19-7179?Tellingwhetherareasonablylargenumberisaprimeseemstediousbecausetherearefartoomanycandidatefactorstotry.However,therearesomeclevertrickstospeeduptheprocess.Forinstance,youcanomiteven-valuedcandidatesafteryouhaveeliminatedthenumber2.Youcanactuallyomitallcandidatesexceptthosethatarethemselvesprimes.Infact,alittlefurtherthoughtwillconvinceyouthatyoucanproclaimNaprimeassoonpasyouhaverejectedallcandidatesuptoN,forifNcanindeedbefactoredasN=KL,pthenitisimpossibleforbothfactorstoexceedN.Weseemtobemakingprogress!Perhapsbyomittingmoreandmorecandidatefactors,atrulyefcientprimalitytestcanbediscovered.Unfortunately,thereisnofastprimalitytestdownthisroad.Thereasonisthatwehavebeentryingtotellifanumberisaprimebyfactoringit.Andfactoringisahardproblem!Moderncryptography,aswellasthebalanceofthischapter,isaboutthefollowingim-portantidea:factoringishardandprimalityiseasy.Wecannotfactorlargenumbers,butwecaneasilytesthugenumbersforprimality!(Presumably,ifanumberiscomposite,suchatestwilldetectthiswithoutndingafactor.)32\n1.3PrimalitytestingIstheresomelitmustestthatwilltelluswhetheranumberisprimewithoutactuallytryingtofactorthenumber?Weplaceourhopesinatheoremfromtheyear1640.Fermat'slittletheoremIfpisprime,thenforevery1a10wheretheexponentiationcanbeperformedusingfewermultiplications,bysomeothermethod.1.17.Considertheproblemofcomputingxyforgivenintegersxandy:wewantthewholeanswer,notmoduloathirdinteger.Weknowtwoalgorithmsfordoingthis:theiterativealgorithmwhichperformsy 1multiplicationsbyx;andtherecursivealgorithmbasedonthebinaryexpansionofy.Comparethetimerequirementsofthesetwoalgorithms,assumingthatthetimetomultiplyann-bitnumberbyanm-bitnumberisO(mn).1.18.Computegcd(210;588)twodifferentways:byndingthefactorizationofeachnumber,andbyusingEuclid'salgorithm.1.19.TheFibonaccinumbersF0;F1;:::aregivenbytherecurrenceFn+1=Fn+Fn 1,F0=0,F1=1.Showthatforanyn1,gcd(Fn+1;Fn)=1.1.20.Findtheinverseof:20mod79;3mod62;21mod91;5mod23.1.21.Howmanyintegersmodulo113haveinverses?(Note:113=1331.)1.22.Proveordisprove:Ifahasaninversemodulob,thenbhasaninversemoduloa.1.23.ShowthatifahasamultiplicativeinversemoduloN,thenthisinverseisunique(moduloN).1.24.Ifpisprime,howmanyelementsoff0;1;:::;pn 1ghaveaninversemodulopn?1.25.Calculate2125mod127usinganymethodyouchoose.(Hint:127isprime.)171.26.Whatistheleastsignicantdecimaldigitof1717?(Hint:Fordistinctprimesp;q,andanya60(modpq),weprovedtheformulaa(p 1)(q 1)1(modpq)inSection1.4.2.)1.27.ConsideranRSAkeysetwithp=17;q=23;N=391,ande=3(asinFigure1.9).Whatvalueofdshouldbeusedforthesecretkey?WhatistheencryptionofthemessageM=41?1.28.InanRSAcryptosystem,p=7andq=11(asinFigure1.9).Findappropriateexponentsdande.1.29.Let[m]denotethesetf0;1;:::;m 1g.Foreachofthefollowingfamiliesofhashfunctions,saywhetherornotitisuniversal,anddeterminehowmanyrandombitsareneededtochooseafunctionfromthefamily.(a)H=fha1;a2:a1;a22[m]g,wheremisaxedprimeandha1;a2(x1;x2)=a1x1+a2x2modm:Noticethateachofthesefunctionshassignatureh:[m]2![m],thatis,itmapsapaira1;a2ofintegersin[m]toasingleintegerin[m].(b)Hisasbefore,exceptthatnowm=2kissomexedpowerof2.(c)Histhesetofallfunctionsf:[m]![m 1].1.30.Thegrade-schoolalgorithmformultiplyingtwon-bitbinarynumbersxandyconsistsofaddingtogetherncopiesofx,eachappropriatelyleft-shifted.Eachcopy,whenshifted,isatmost2nbitslong.Inthisproblem,wewillexamineaschemeforaddingnbinarynumbers,eachmbitslong,usingacircuitoraparallelarchitecture.Themainparameterofinterestinthisquestionistherefore47\nthedepthofthecircuitorthelongestpathfromtheinputtotheoutputofthecircuit.Thisdeterminesthetotaltimetakenforcomputingthefunction.Toaddtwom-bitbinarynumbersnaively,wemustwaitforthecarrybitfrompositioni 1beforewecangureouttheithbitoftheanswer.ThisleadstoacircuitofdepthO(m).Howevercarrylookaheadcircuits(seewikipedia.comifyouwanttoknowmoreaboutthis)canaddinO(logm)depth.(a)Assumingyouhavecarrylookaheadcircuitsforaddition,showhowtoaddnnumberseachmbitslongusingacircuitofdepthO((logn)(logm)).(b)Whenaddingthreem-bitbinarynumbersx+y+z,thereisatrickwecanusetoparallelizetheprocess.Insteadofcarryingouttheadditioncompletely,wecanre-expresstheresultasthesumofjusttwobinarynumbersr+s,suchthattheithbitsofrandscanbecomputedindependentlyoftheotherbits.Showhowthiscanbedone.(Hint:Oneofthenumbersrepresentscarrybits.)(c)ShowhowtousethetrickfromthepreviousparttodesignacircuitofdepthO(logn)formultiplyingtwon-bitnumbers.1.31.ConsidertheproblemofcomputingN!=123N.(a)IfNisann-bitnumber,howmanybitslongisN!,approximately(in()form)?(b)GiveanalgorithmtocomputeN!andanalyzeitsrunningtime.1.32.ApositiveintegerNisapowerifitisoftheformqk,whereq;karepositiveintegersandk>1.(a)GiveanefcientalgorithmthattakesasinputanumberNanddetermineswhetheritisasquare,thatis,whetheritcanbewrittenasq2forsomepositiveintegerq.Whatistherunningtimeofyouralgorithm?(b)ShowthatifN=qk(withN,q,andkallpositiveintegers),theneitherklogNorN=1.(c)GiveanefcientalgorithmfordeterminingwhetherapositiveintegerNisapower.Analyzeitsrunningtime.1.33.Giveanefcientalgorithmtocomputetheleastcommonmultipleoftwon-bitnumbersxandy,thatis,thesmallestnumberdivisiblebybothxandy.Whatistherunningtimeofyouralgorithmasafunctionofn?1.34.Onpage37,weclaimedthatsinceabouta1=nfractionofn-bitnumbersareprime,onaverageitissufcienttodrawO(n)randomn-bitnumbersbeforehittingaprime.Wenowjustifythisrigorously.Supposeaparticularcoinhasaprobabilitypofcomingupheads.Howmanytimesmustyoutossit,onaverage,beforeitcomesupheads?(Hint:Method1:startbyshowingthatthecorrectP1i 1expressionisi(1 p)p.Method2:ifEistheaveragenumberofcointosses,showthati=1E=1+(1 p)E.)1.35.Wilson'stheoremsaysthatanumberNisprimeifandonlyif(N 1)! 1(modN):(a)Ifpisprime,thenweknoweverynumber1x0(inthemultiplicationalgorithm,a=3,b=2,andd=1).TheirrunningtimecanthereforebecapturedbytheequationT(n)=aT(dn=be)+O(nd).Wenextderiveaclosed-formsolutiontothisgeneralrecurrencesothatwenolongerhavetosolveitexplicitlyineachnewinstance.53\nFigure2.2Divide-and-conquerintegermultiplication.(a)Eachproblemisdividedintothreesubproblems.(b)Thelevelsofrecursion.(a)1011001001100011101101100010001111011001(b)SizenSizen=2Sizen=4lognlevels......2222111111111111Mastertheorem2IfT(n)=aT(dn=be)+O(nd)forsomeconstantsa>0,b>1,andd0,then8logbaT(n)=O(ndlogn)ifd=logba:logaO(nb)ifd1:returnmerge(mergesort(a[1:::bn=2c]),mergesort(a[bn=2c+1:::n]))else:returnaThecorrectnessofthisalgorithmisself-evident,aslongasacorrectmergesubroutineisspecied.Ifwearegiventwosortedarraysx[1:::k]andy[1:::l],howdoweefcientlymergethemintoasinglesortedarrayz[1:::k+l]?Well,theveryrstelementofziseitherx[1]ory[1],whicheverissmaller.Therestofz[]canthenbeconstructedrecursively.functionmerge(x[1:::k];y[1:::l])ifk=0:returny[1:::l]ifl=0:returnx[1:::k]ifx[1]y[1]:returnx[1]merge(x[2:::k];y[1:::l])else:returny[1]merge(x[1:::k];y[2:::l])Heredenotesconcatenation.Thismergeproceduredoesaconstantamountofworkperrecursivecall(providedtherequiredarrayspaceisallocatedinadvance),foratotalrunningtimeofO(k+l).Thusmerge'sarelinear,andtheoveralltimetakenbymergesortisT(n)=2T(n=2)+O(n);orO(nlogn).Lookingbackatthemergesortalgorithm,weseethatalltherealworkisdoneinmerg-ing,whichdoesn'tstartuntiltherecursiongetsdowntosingletonarrays.Thesingletonsare56\nFigure2.4Thesequenceofmergeoperationsinmergesort.Input:10253713161025371316210357131623510167131235671013.mergedinpairs,toyieldarrayswithtwoelements.Thenpairsofthese2-tuplesaremerged,producing4-tuples,andsoon.Figure2.4showsanexample.Thisviewpointalsosuggestshowmergesortmightbemadeiterative.Atanygivenmo-ment,thereisasetof“active”arrays—initially,thesingletons—whicharemergedinpairstogivethenextbatchofactivearrays.Thesearrayscanbeorganizedinaqueue,andprocessedbyrepeatedlyremovingtwoarraysfromthefrontofthequeue,mergingthem,andputtingtheresultattheendofthequeue.Inthefollowingpseudocode,theprimitiveoperationinjectaddsanelementtotheendofthequeuewhileejectremovesandreturnstheelementatthefrontofthequeue.functioniterative-mergesort(a[1:::n])Input:elementsa1;a2;:::;antobesortedQ=[](emptyqueue)fori=1ton:inject(Q;[ai])whilejQj>1:inject(Q;merge(eject(Q);eject(Q)))returneject(Q)57\n58\nAnnlognlowerboundforsortingSortingalgorithmscanbedepictedastrees.Theoneinthefollowingguresortsanarrayofthreeelements,a1;a2;a3.Itstartsbycomparinga1toa2and,iftherstislarger,comparesitwitha3;otherwiseitcomparesa2anda3.Andsoon.Eventuallyweendupataleaf,andthisleafislabeledwiththetrueorderofthethreeelementsasapermutationof1;2;3.Forexample,ifa20.Therearemanywaystoseethis.Theeasiestistonoticethatn!(n=2)(n=2)becausen!=12ncontainsatleastn=2factorslargerthann=2;andtothentakelogsofbothsides.AnotheristorecallStirling'sformulas1n nn!2n+ne:3Eitherway,wehaveestablishedthatanycomparisontreethatsortsnelementsmustmake,intheworstcase,(nlogn)comparisons,andhencemergesortisoptimal!Well,thereissomeneprint:thisneatargumentappliesonlytoalgorithmsthatusecomparisons.Isitconceivablethattherearealternativesortingstrategies,perhapsusingsophisticatednumericalmanipulations,thatworkinlineartime?Theanswerisyes,undercertainexceptionalcircumstances:thecanonicalsuchexampleiswhentheelementstobesortedareintegersthatlieinasmallrange(Exercise2.20).59\n2.4MediansThemedianofalistofnumbersisits50thpercentile:halfthenumbersarebiggerthanit,andhalfaresmaller.Forinstance,themedianof[45;1;10;30;25]is25,sincethisisthemiddleelementwhenthenumbersarearrangedinorder.Ifthelisthasevenlength,therearetwochoicesforwhatthemiddleelementcouldbe,inwhichcasewepickthesmallerofthetwo,say.Thepurposeofthemedianistosummarizeasetofnumbersbyasingle,typicalvalue.Themean,oraverage,isalsoverycommonlyusedforthis,butthemedianisinasensemoretypicalofthedata:itisalwaysoneofthedatavalues,unlikethemean,anditislesssensitivetooutliers.Forinstance,themedianofalistofahundred1'sis(rightly)1,asisthemean.However,ifjustoneofthesenumbersgetsaccidentallycorruptedto10;000,themeanshootsupabove100,whilethemedianisunaffected.Computingthemedianofnnumbersiseasy:justsortthem.ThedrawbackisthatthistakesO(nlogn)time,whereaswewouldideallylikesomethinglinear.Wehavereasontobehopeful,becausesortingisdoingfarmoreworkthanwereallyneed—wejustwantthemiddleelementanddon'tcareabouttherelativeorderingoftherestofthem.Whenlookingforarecursivesolution,itisparadoxicallyofteneasiertoworkwithamoregeneralversionoftheproblem—forthesimplereasonthatthisgivesamorepowerfulsteptorecurseupon.Inourcase,thegeneralizationwewillconsiderisselection.SELECTIONInput:AlistofnumbersS;anintegerkOutput:ThekthsmallestelementofSForinstance,ifk=1,theminimumofSissought,whereasifk=bjSj=2c,itisthemedian.Arandomizeddivide-and-conqueralgorithmforselectionHere'sadivide-and-conquerapproachtoselection.Foranynumberv,imaginesplittinglistSintothreecategories:elementssmallerthanv,thoseequaltov(theremightbeduplicates),andthosegreaterthanv.CalltheseSL,Sv,andSRrespectively.Forinstance,ifthearrayS:2365218131120541issplitonv=5,thethreesubarraysgeneratedareSL:241Sv:55SR:36218131120Thesearchcaninstantlybenarroweddowntooneofthesesublists.Ifwewant,say,theeighth-smallestelementofS,weknowitmustbethethird-smallestelementofSRsincejSLj+jSvj=5.Thatis,selection(S;8)=selection(SR;3).Moregenerally,bycheckingkagainstthesizesofthesubarrays,wecanquicklydeterminewhichofthemholdsthedesiredelement:8jSLj+jSvj:60\nThethreesublistsSL;Sv,andSRcanbecomputedfromSinlineartime;infact,thiscompu-tationcanevenbedoneinplace,thatis,withoutallocatingnewmemory(Exercise2.15).Wethenrecurseontheappropriatesublist.TheeffectofthesplitisthustoshrinkthenumberofelementsfromjSjtoatmostmaxfjSLj;jSRjg.Ourdivide-and-conqueralgorithmforselectionisnowfullyspecied,exceptforthecrucialdetailofhowtochoosev.Itshouldbepickedquickly,anditshouldshrinkthearraysubstan-tially,theidealsituationbeingjSj;jSj1jSj.Ifwecouldalwaysguaranteethissituation,LR2wewouldgetarunningtimeofT(n)=T(n=2)+O(n);whichislinearasdesired.Butthisrequirespickingvtobethemedian,whichisourultimategoal!Instead,wefollowamuchsimpleralternative:wepickvrandomlyfromS.EfciencyanalysisNaturally,therunningtimeofouralgorithmdependsontherandomchoicesofv.Itispossiblethatduetopersistentbadluckwekeeppickingvtobethelargestelementofthearray(orthesmallestelement),andtherebyshrinkthearraybyonlyoneelementeachtime.Intheearlierexample,wemightrstpickv=36,thenv=21,andsoon.Thisworst-casescenariowouldforceourselectionalgorithmtoperformn2n+(n 1)+(n 2)++=(n)2operations(whencomputingthemedian),butitisextremelyunlikelytooccur.Equallyun-likelyisthebestpossiblecasewediscussedbefore,inwhicheachrandomlychosenvjusthappenstosplitthearrayperfectlyinhalf,resultinginarunningtimeofO(n).Where,inthisspectrumfromO(n)to(n2),doestheaveragerunningtimelie?Fortunately,itliesveryclosetothebest-casetime.Todistinguishbetweenluckyandunluckychoicesofv,wewillcallvgoodifitlieswithinthe25thto75thpercentileofthearraythatitischosenfrom.WelikethesechoicesofvbecausetheyensurethatthesublistsSLandSRhavesizeatmostthree-fourthsthatofS(doyouseewhy?),sothatthearrayshrinkssubstantially.Fortunately,goodv'sareabundant:halftheelementsofanylistmustfallbetweenthe25thto75thpercentile!Giventhatarandomlychosenvhasa50%chanceofbeinggood,howmanyv'sdoweneedtopickonaveragebeforegettingagoodone?Here'samorefamiliarreformulation(seealsoExercise1.34):LemmaOnaverageafaircoinneedstobetossedtwotimesbeforea“heads”isseen.Proof.LetEbetheexpectednumberoftossesbeforeaheadsisseen.Wecertainlyneedatleastonetoss,andifit'sheads,we'redone.Ifit'stails(whichoccurswithprobability1=2),weneedtorepeat.HenceE=1+1E,whichworksouttoE=2.2Therefore,aftertwosplitoperationsonaverage,thearraywillshrinktoatmostthree-fourthsofitssize.LettingT(n)betheexpectedrunningtimeonanarrayofsizen,wegetT(n)T(3n=4)+O(n):61\nThisfollowsbytakingexpectedvaluesofbothsidesofthefollowingstatement:Timetakenonanarrayofsizen(timetakenonanarrayofsize3n=4)+(timetoreducearraysizeto3n=4);and,fortheright-handside,usingthefamiliarpropertythattheexpectationofthesumisthesumoftheexpectations.FromthisrecurrenceweconcludethatT(n)=O(n):onanyinput,ouralgorithmreturnsthecorrectanswerafteralinearnumberofsteps,ontheaverage.TheUnixsortcommandComparingthealgorithmsforsortingandmedian-ndingwenoticethat,beyondthecom-mondivide-and-conquerphilosophyandstructure,theyareexactopposites.Mergesortsplitsthearrayintwointhemostconvenientway(rsthalf,secondhalf),withoutanyregardtothemagnitudesoftheelementsineachhalf;butthenitworkshardtoputthesortedsub-arraystogether.Incontrast,themedianalgorithmiscarefulaboutitssplitting(smallernumbersrst,thenthelargerones),butitsworkendswiththerecursivecall.Quicksortisasortingalgorithmthatsplitsthearrayinexactlythesamewayastheme-dianalgorithm;andoncethesubarraysaresorted,bytworecursivecalls,thereisnothingmoretodo.Itsworst-caseperformanceis(n2),likethatofmedian-nding.Butitcanbeproved(Exercise2.24)thatitsaveragecaseisO(nlogn);furthermore,empiricallyitoutper-formsothersortingalgorithms.Thishasmadequicksortafavoriteinmanyapplications—forinstance,itisthebasisofthecodebywhichreallyenormouslesaresorted.2.5MatrixmultiplicationTheproductoftwonnmatricesXandYisathirdnnmatrixZ=XY,with(i;j)thentryXnZij=XikYkj:k=1Tomakeitmorevisual,ZijisthedotproductoftheithrowofXwiththejthcolumnofY:ji=(i;j)XYZIngeneral,XYisnotthesameasYX;matrixmultiplicationisnotcommutative.TheprecedingformulaimpliesanO(n3)algorithmformatrixmultiplication:therearen2entriestobecomputed,andeachtakesO(n)time.Forquiteawhile,thiswaswidelybelieved62\ntobethebestrunningtimepossible,anditwasevenprovedthatincertainmodelsofcom-putationnoalgorithmcoulddobetter.Itwasthereforeasourceofgreatexcitementwhenin1969,theGermanmathematicianVolkerStrassenannouncedasignicantlymoreefcientalgorithm,basedupondivide-and-conquer.Matrixmultiplicationisparticularlyeasytobreakintosubproblems,becauseitcanbeperformedblockwise.Toseewhatthismeans,carveXintofourn=2n=2blocks,andalsoY:ABEFX=;Y=:CDGHThentheirproductcanbeexpressedintermsoftheseblocksandisexactlyasiftheblocksweresingleelements(Exercise2.11).ABEFAE+BGAF+BHXY==CDGHCE+DGCF+DHWenowhaveadivide-and-conquerstrategy:tocomputethesize-nproductXY,recursivelycomputeeightsize-n=2productsAE;BG;AF;BH;CE;DG;CF;DH,andthendoafewO(n2)-timeadditions.ThetotalrunningtimeisdescribedbytherecurrencerelationT(n)=8T(n=2)+O(n2):ThiscomesouttoanunimpressiveO(n3),thesameasforthedefaultalgorithm.Buttheefciencycanbefurtherimproved,andaswithintegermultiplication,thekeyissomecleveralgebra.ItturnsoutXYcanbecomputedfromjustsevenn=2n=2subproblems,viaadecompositionsotrickyandintricatethatonewondershowStrassenwaseverabletodiscoverit!P5+P4 P2+P6P1+P2XY=P3+P4P1+P5 P3 P7whereP1=A(F H)P5=(A+D)(E+H)P2=(A+B)HP6=(B D)(G+H)P3=(C+D)EP7=(A C)(E+F)P4=D(G E)ThenewrunningtimeisT(n)=7T(n=2)+O(n2);whichbythemastertheoremworksouttoO(nlog27)O(n2:81).63\n2.6ThefastFouriertransformWehavesofarseenhowdivide-and-conquergivesfastalgorithmsformultiplyingintegersandmatrices;ournexttargetispolynomials.Theproductoftwodegree-dpolynomialsisapolynomialofdegree2d,forexample:(1+2x+3x2)(2+x+4x2)=2+5x+12x2+11x3+12x4:Moregenerally,ifA(x)=a0+a1x++adxdandB(x)=b0+b1x++bdxd,theirproductC(x)=A(x)B(x)=c0+c1x++c2dx2dhascoefcientsXkck=a0bk+a1bk 1++akb0=aibk ii=0(fori>d,takeaiandbitobezero).ComputingckfromthisformulatakesO(k)steps,andndingall2d+1coefcientswouldthereforeseemtorequire(d2)time.Canwepossiblymultiplypolynomialsfasterthanthis?Thesolutionwewilldevelop,thefastFouriertransform,hasrevolutionized—indeed,dened—theeldofsignalprocessing(seethefollowingbox).Becauseofitshugeimpor-tance,anditswealthofinsightsfromdifferenteldsofstudy,wewillapproachitalittlemoreleisurelythanusual.ThereaderwhowantsjustthecorealgorithmcanskipdirectlytoSection2.6.4.64\n65\nWhymultiplypolynomials?Foronething,itturnsoutthatthefastestalgorithmswehaveformultiplyingintegersrelyheavilyonpolynomialmultiplication;afterall,polynomialsandbinaryintegersarequitesimilar—justreplacethevariablexbythebase2,andwatchoutforcarries.Butperhapsmoreimportantly,multiplyingpolynomialsiscrucialforsignalprocessing.Asignalisanyquantitythatisafunctionoftime(asinFigure(a))orofposition.Itmight,forinstance,captureahumanvoicebymeasuringuctuationsinairpressureclosetothespeaker'smouth,oralternatively,thepatternofstarsinthenightsky,bymeasuringbrightnessasafunctionofangle.a(t)a(t)(t)pq !!" ##$$%%&3341122556678 //00--.''(()*++,99::;;<<==>VWSTSXYZ[\]^_fTf`abcdelTlghijkmnoUTUt??@ttABCDEFGHIJKLMNOPQR(a)(b)(c)Inordertoextractinformationfromasignal,weneedtorstdigitizeitbysampling(Figure(b))—and,then,toputitthroughasystemthatwilltransformitinsomeway.Theoutputiscalledtheresponseofthesystem:signal !SYSTEM !responseAnimportantclassofsystemsarethosethatarelinear—theresponsetothesumoftwosignalsisjustthesumoftheirindividualresponses—andtimeinvariant—shiftingtheinputsignalbytimetproducesthesameoutput,alsoshiftedbyt.Anysystemwiththeseprop-ertiesiscompletelycharacterizedbyitsresponsetothesimplestpossibleinputsignal:theunitimpulse(t),consistingsolelyofa“jerk”att=0(Figure(c)).Toseethis,rstconsiderthecloserelative(t i),ashiftedimpulseinwhichthejerkoccursattimei.Anysignala(t)canbeexpressedasalinearcombinationofthese,letting(t i)pickoutitsbehaviorattimei,TX 1a(t)=a(i)(t i)i=0(ifthesignalconsistsofTsamples).Bylinearity,thesystemresponsetoinputa(t)isdeter-minedbytheresponsestothevarious(t i).Andbytimeinvariance,theseareinturnjustshiftedcopiesoftheimpulseresponseb(t),theresponseto(t).Inotherwords,theoutputofthesystemattimekisXkc(k)=a(i)b(k i);i=0exactlytheformulaforpolynomialmultiplication!66\n2.6.1AnalternativerepresentationofpolynomialsToarriveatafastalgorithmforpolynomialmultiplicationwetakeinspirationfromanimpor-tantpropertyofpolynomials.FactAdegree-dpolynomialisuniquelycharacterizedbyitsvaluesatanyd+1distinctpoints.Afamiliarinstanceofthisisthat“anytwopointsdeterminealine.”Wewilllaterseewhythemoregeneralstatementistrue(page72),butforthetimebeingitgivesusanalternativerepresentationofpolynomials.Fixanydistinctpointsx0;:::;xd.Wecanspecifyadegree-dpolynomialA(x)=a0+a1x++adxdbyeitheroneofthefollowing:1.Itscoefcientsa0;a1;:::;ad2.ThevaluesA(x0);A(x1);:::;A(xd)Ofthesetworepresentations,thesecondisthemoreattractiveforpolynomialmultiplication.SincetheproductC(x)hasdegree2d,itiscompletelydeterminedbyitsvalueatany2d+1points.Anditsvalueatanygivenpointziseasyenoughtogureout,justA(z)timesB(z).Thuspolynomialmultiplicationtakeslineartimeinthevaluerepresentation.Theproblemisthatweexpecttheinputpolynomials,andalsotheirproduct,tobespeciedbycoefcients.Soweneedtorsttranslatefromcoefcientstovalues—whichisjustamatterofevaluatingthepolynomialatthechosenpoints—thenmultiplyinthevaluerepresentation,andnallytranslatebacktocoefcients,aprocesscalledinterpolation.EvaluationCoefcientrepresentationValuerepresentationa0;a1;:::;adA(x0);A(x1);:::;A(xd)InterpolationFigure2.5presentstheresultingalgorithm.Figure2.5PolynomialmultiplicationInput:Coefficientsoftwopolynomials,A(x)andB(x),ofdegreedOutput:TheirproductC=ABSelectionPicksomepointsx0;x1;:::;xn 1,wheren2d+1EvaluationComputeA(x0);A(x1);:::;A(xn 1)andB(x0);B(x1);:::;B(xn 1)MultiplicationComputeC(xk)=A(xk)B(xk)forallk=0;:::;n 1InterpolationRecoverC(x)=c0+c1x++c2dx2d67\nTheequivalenceofthetwopolynomialrepresentationsmakesitclearthatthishigh-levelapproachiscorrect,buthowefcientisit?Certainlytheselectionstepandthenmultiplica-tionsarenotroubleatall,justlineartime.3But(leavingasideinterpolation,aboutwhichweknowevenless)howaboutevaluation?EvaluatingapolynomialofdegreednatasinglepointtakesO(n)steps(Exercise2.29),andsothebaselinefornpointsis(n2).We'llnowseethatthefastFouriertransform(FFT)doesitinjustO(nlogn)time,foraparticularlycleverchoiceofx0;:::;xn 1inwhichthecomputationsrequiredbytheindividualpointsoverlapwithoneanotherandcanbeshared.2.6.2Evaluationbydivide-and-conquerHere'sanideaforhowtopickthenpointsatwhichtoevaluateapolynomialA(x)ofdegreen 1.Ifwechoosethemtobepositive-negativepairs,thatis,x0;x1;:::;xn=2 1;thenthecomputationsrequiredforeachA(xi)andA( xi)overlapalot,becausetheevenpowersofxicoincidewiththoseof xi.Toinvestigatethis,weneedtosplitA(x)intoitsoddandevenpowers,forinstance3+4x+6x2+2x3+x4+10x5=(3+6x2+x4)+x(4+2x2+10x4):Noticethatthetermsinparenthesesarepolynomialsinx2.Moregenerally,A(x)=A(x2)+xA(x2);eowhereAe(),withtheeven-numberedcoefcients,andAo(),withtheodd-numberedcoef-cients,arepolynomialsofdegreen=2 1(assumeforconveniencethatniseven).Givenpairedpointsxi,thecalculationsneededforA(xi)canberecycledtowardcomputingA( xi):A(x)=A(x2)+xA(x2)ieiioiA( x)=A(x2) xA(x2):ieiioiInotherwords,evaluatingA(x)atnpairedpointsx0;:::;xn=2 1reducestoevaluatingAe(x)andAo(x)(whicheachhavehalfthedegreeofA(x))atjustn=2points,x20;:::;x2.n=2 1A(x)Evaluate:degreen 1at:+x0 x0+x1 x1+xn=2 1 xn=2 1Equivalently,Ae(x)andAo(x)222at:xxxevaluate:degreen=2 101n=2 13Inatypicalsettingforpolynomialmultiplication,thecoefcientsofthepolynomialsarerealnumbersand,moreover,aresmallenoughthatbasicarithmeticoperations(addingandmultiplying)takeunittime.Wewillassumethistobethecasewithoutanygreatlossofgenerality;inparticular,thetimeboundsweobtainareeasilyadjustabletosituationswithlargernumbers.68\nTheoriginalproblemofsizenisinthiswayrecastastwosubproblemsofsizen=2,followedbysomelinear-timearithmetic.Ifwecouldrecurse,wewouldgetadivide-and-conquerpro-cedurewithrunningtimeT(n)=2T(n=2)+O(n);whichisO(nlogn),exactlywhatwewant.Butwehaveaproblem:Theplus-minustrickonlyworksatthetopleveloftherecur-sion.Torecurseatthenextlevel,weneedthen=2evaluationpointsx20;x21;:::;x2toben=2 1themselvesplus-minuspairs.Buthowcanasquarebenegative?Thetaskseemsimpossible!Unless,ofcourse,weusecomplexnumbers.Fine,butwhichcomplexnumbers?Togurethisout,letus“reverseengineer”theprocess.Attheverybottomoftherecursion,wehaveasinglepoint.Thispointmightaswellbe1,inpwhichcasethelevelaboveitmustconsistofitssquareroots,1=1....+1 1+i i+1 1+1ppThenextlevelupthenhas+1=1aswellasthecomplexnumbers 1=i,whereiistheimaginaryunit.Bycontinuinginthismanner,weeventuallyreachtheinitialsetofnpoints.Perhapsyouhavealreadyguessedwhattheyare:thecomplexnthrootsofunity,thatis,thencomplexsolutionstotheequationzn=1.Figure2.6isapictorialreviewofsomebasicfactsaboutcomplexnumbers.Thethirdpanelofthisgureintroducesthenthrootsofunity:thecomplexnumbers1;!;!2;:::;!n 1,where!=e2i=n.Ifniseven,1.Thenthrootsareplus-minuspaired,!n=2+j= !j.2.Squaringthemproducesthe(n=2)ndrootsofunity.Therefore,ifwestartwiththesenumbersforsomenthatisapowerof2,thenatsuccessivelevelsofrecursionwewillhavethe(n=2k)throotsofunity,fork=0;1;2;3;:::.Allthesesetsofnumbersareplus-minuspaired,andsoourdivide-and-conquer,asshowninthelastpanel,worksperfectly.TheresultingalgorithmisthefastFouriertransform(Figure2.7).69\nFigure2.6Thecomplexrootsofunityareidealforourdivide-and-conquerscheme.ThecomplexplaneImaginaryz=a+biisplottedatposition(a;b).bPolarcoordinates:rewriteasz=r(cos+isin)=rei,denoted(r;).rplengthr=a2+b2.angle2[0;2):cos=a=r;sin=b=r.Realcanalwaysbereducedmodulo2.aNumber 1i5+5iExamples:pPolarcoords(1;)(1;=2)(52;=4)Multiplyingiseasyinpolarcoordinates(r1;1)Multiplythelengthsandaddtheangles:(r2;2)(r1;1)(r2;2)=(r1r2;1+2).Foranyz=(r;), z=(r;+)since 1=(1;).Ifzisontheunitcircle(i.e.,r=1),thenzn=(1;n).(r1r2;1+2)ThenthcomplexrootsofunitySolutionstotheequationzn=1.4nBythemultiplicationrule:solutionsarez=(1;),foraAngle2multipleof2=n(shownhereforn=16).nForevenn:Thesenumbersareplus-minuspaired: (1;)=(1;+).2+nTheirsquaresarethe(n=2)ndrootsofunity,shownherewithboxesaroundthem.Divide-and-conquerstepEvaluateA(x)EvaluateatnthrootsAe(x);Ao(x)ofunityat(n=2)ndrootsDivideandconquerPairedStillpaired(nisapowerof2)70\nFigure2.7ThefastFouriertransform(polynomialformulation)functionFFT(A;!)Input:CoefficientrepresentationofapolynomialA(x)ofdegreen 1,wherenisapowerof2!,annthrootofunityOutput:ValuerepresentationA(!0);:::;A(!n 1)if!=1:returnA(1)expressA(x)intheformAe(x2)+xAo(x2)callFFT(Ae;!2)toevaluateAeatevenpowersof!callFFT(Ao;!2)toevaluateAoatevenpowersof!forj=0ton 1:computeA(!j)=Ae(!2j)+!jAo(!2j)returnA(!0);:::;A(!n 1)2.6.3InterpolationLet'stakestockofwhereweare.Werstdevelopedahigh-levelschemeformultiplyingpolynomials(Figure2.5),basedontheobservationthatpolynomialscanberepresentedintwoways,intermsoftheircoefcientsorintermsoftheirvaluesataselectedsetofpoints.EvaluationCoefcientrepresentationValuerepresentationa0;a1;:::;an 1A(x0);A(x1);:::;A(xn 1)InterpolationThevaluerepresentationmakesittrivialtomultiplypolynomials,butwecannotignorethecoefcientrepresentationsinceitistheforminwhichtheinputandoutputofouroverallalgorithmarespecied.SowedesignedtheFFT,awaytomovefromcoefcientstovaluesintimejustO(nlogn),whenthepointsfxigarecomplexnthrootsofunity(1;!;!2;:::;!n 1).hvaluesi=FFT(hcoefcientsi;!):Thelastremainingpieceofthepuzzleistheinverseoperation,interpolation.Itwillturnout,amazingly,that1 1hcoefcientsi=FFT(hvaluesi;!):nInterpolationisthussolvedinthemostsimpleandelegantwaywecouldpossiblyhavehopedfor—usingthesameFFTalgorithm,butcalledwith! 1inplaceof!!Thismightseemlikeamiraculouscoincidence,butitwillmakealotmoresensewhenwerecastourpolynomialoper-ationsinthelanguageoflinearalgebra.Meanwhile,ourO(nlogn)polynomialmultiplicationalgorithm(Figure2.5)isnowfullyspecied.71\nAmatrixreformulationTogetaclearerviewofinterpolation,let'sexplicitlysetdowntherelationshipbetweenourtworepresentationsforapolynomialA(x)ofdegreen 1.Theyarebothvectorsofnnumbers,andoneisalineartransformationoftheother:232323A(x)1xx2xn 1a000006A(x)761xx2xn 176a7617611176176.7=6.76.7:4..54..54..5A(x)1xx2xn 1an 1n 1n 1n 1n 1CallthematrixinthemiddleM.Itsspecializedformat—aVandermondematrix—givesitmanyremarkableproperties,ofwhichthefollowingisparticularlyrelevanttous.Ifx0;:::;xn 1aredistinctnumbers,thenMisinvertible.TheexistenceofM 1allowsustoinverttheprecedingmatrixequationsoastoexpresscoef-cientsintermsofvalues.Inbrief,EvaluationismultiplicationbyM,whileinterpolationismultiplicationbyM 1.Thisreformulationofourpolynomialoperationsrevealstheiressentialnaturemoreclearly.Amongotherthings,itnallyjustiesanassumptionwehavebeenmakingthroughout,thatA(x)isuniquelycharacterizedbyitsvaluesatanynpoints—infact,wenowhaveanexplicitformulathatwillgiveusthecoefcientsofA(x)inthissituation.Vandermondematricesalsohavethedistinctionofbeingquickertoinvertthanmoregeneralmatrices,inO(n2)timein-steadofO(n3).However,usingthisforinterpolationwouldstillnotbefastenoughforus,soonceagainweturntoourspecialchoiceofpoints—thecomplexrootsofunity.InterpolationresolvedInlinearalgebraterms,theFFTmultipliesanarbitraryn-dimensionalvector—whichwehavebeencallingthecoefcientrepresentation—bythennmatrix231111 rowfor!0=161!!2!n 17 !67661!2!4!2(n 1)77 !26.7.Mn(!)=66..77..661!j!2j!(n 1)j77 !j6.7.4..5..1!(n 1)!2(n 1)!(n 1)(n 1) !n 1where!isacomplexnthrootofunity,andnisapowerof2.Noticehowsimplethismatrixistodescribe:its(j;k)thentry(startingrow-andcolumn-countatzero)is!jk.MultiplicationbyM=Mn(!)mapsthekthcoordinateaxis(thevectorwithallzerosexceptfora1atpositionk)ontothekthcolumnofM.Nowhere'sthecrucialobservation,whichwe'llproveshortly:thecolumnsofMareorthogonal(atrightangles)toeachother.Thereforetheycanbethoughtofastheaxesofanalternativecoordinatesystem,whichisoftencalled72\nFigure2.8TheFFTtakespointsinthestandardcoordinatesystem,whoseaxesareshownhereasx1;x2;x3,androtatesthemintotheFourierbasis,whoseaxesarethecolumnsofMn(!),shownhereasf1;f2;f3.Forinstance,pointsindirectionx1getmappedintodirectionf1.x2x3f1f3f2FFTx1theFourierbasis.TheeffectofmultiplyingavectorbyMistorotateitfromthestandardbasis,withtheusualsetofaxes,intotheFourierbasis,whichisdenedbythecolumnsofM(Figure2.8).TheFFTisthusachangeofbasis,arigidrotation.TheinverseofMistheoppositerotation,fromtheFourierbasisbackintothestandardbasis.Whenwewriteouttheorthogonalityconditionprecisely,wewillbeabletoreadoffthisinversetransformationwithease:InversionformulaM(!) 1=1M(! 1).nnnBut! 1isalsoannthrootofunity,andsointerpolation—orequivalently,multiplicationbyMn(!) 1—isitselfjustanFFToperation,butwith!replacedby! 1.Nowlet'sgetintothedetails.Take!tobee2i=nforconvenience,andthinkofthecolumnsofMasvectorsinCn.Recallthattheanglebetweentwovectorsu=(u0;:::;un 1)andv=(v0;:::;vn 1)inCnisjustascalingfactortimestheirinnerproductuv=uv+uv++uv;0011n 1n 1wherezdenotesthecomplexconjugate4ofz.Thisquantityismaximizedwhenthevectorslieinthesamedirectionandiszerowhenthevectorsareorthogonaltoeachother.Thefundamentalobservationweneedisthefollowing.LemmaThecolumnsofmatrixMareorthogonaltoeachother.Proof.TaketheinnerproductofanycolumnsjandkofmatrixM,1+!j k+!2(j k)++!(n 1)(j k):Thisisageometricserieswithrstterm1,lastterm!(n 1)(j k),andratio!(j k).Thereforeitevaluatesto(1 !n(j k))=(1 !(j k)),whichis0—exceptwhenj=k,inwhichcasealltermsare1andthesumisn.4i iThecomplexconjugateofacomplexnumberz=reisz=re.Thecomplexconjugateofavector(ormatrix)isobtainedbytakingthecomplexconjugatesofallitsentries.73\nTheorthogonalitypropertycanbesummarizedinthesingleequationMM=nI;since(MM)ijistheinnerproductoftheithandjthcolumnsofM(doyouseewhy?).ThisimmediatelyimpliesM 1=(1=n)M:wehaveaninversionformula!Butisitthesamefor-mulaweearlierclaimed?Let'ssee—the(j;k)thentryofMisthecomplexconjugateofthecorrespondingentryofM,inotherwords! jk.WhereuponM=Mn(! 1),andwe'redone.Andnowwecannallystepbackandviewthewholeaffairgeometrically.Thetaskweneedtoperform,polynomialmultiplication,isaloteasierintheFourierbasisthaninthestandardbasis.Therefore,werstrotatevectorsintotheFourierbasis(evaluation),thenperformthetask(multiplication),andnallyrotateback(interpolation).Theinitialvectorsarecoefcientrepresentations,whiletheirrotatedcounterpartsarevaluerepresentations.Toefcientlyswitchbetweenthese,backandforth,istheprovinceoftheFFT.2.6.4AcloserlookatthefastFouriertransformNowthatourefcientschemeforpolynomialmultiplicationisfullyrealized,let'shoneinmorecloselyonthecoresubroutinethatmakesitallpossible,thefastFouriertransform.ThedenitiveFFTalgorithmTheFFTtakesasinputavectora=(a0;:::;an 1)andacomplexnumber!whosepowers1;!;!2;:::;!n 1arethecomplexnthrootsofunity.ItmultipliesvectorabythennmatrixMn(!),whichhas(j;k)thentry(startingrow-andcolumn-countatzero)!jk.Thepotentialforusingdivide-and-conquerinthismatrix-vectormultiplicationbecomesapparentwhenM'scolumnsaresegregatedintoevensandodds:ColumnColumnk2k2k+12k2k+1a0a0a0a1a2a2.Rowj!2jk!j!2jk.a2....a3jka4!2jk!j!2jkan 2an 2j!=j=a1a1..a3a3..j+n=2!2jk !j!2jk.....an 1an 1an 1Mn(!)aEvenOddcolumnscolumnsInthesecondstep,wehavesimpliedentriesinthebottomhalfofthematrixusing!n=2= 1and!n=1.Noticethatthetopleftn=2n=2submatrixisMn=2(!2),asistheoneonthebottomleft.AndthetopandbottomrightsubmatricesarealmostthesameasMn=2(!2),butwiththeirjthrowsmultipliedthroughby!jand !j,respectively.Thereforethenalproductisthevector74\nFigure2.9ThefastFouriertransformfunctionFFT(a;!)Input:Anarraya=(a0;a1;:::;an 1),fornapowerof2Aprimitiventhrootofunity,!Output:Mn(!)aif!=1:returna(s0;s1;:::;sn=2 1)=FFT((a0;a2;:::;an 2);!2)(s00;s01;:::;s0)=FFT((a1;a3;:::;an 1);!2)n=2 1forj=0ton=2 1:rj=sj+!js0jrj+n=2=sj !js0jreturn(r0;r1;:::;rn 1)a0a1a2a3RowjMn=2..+!jMn=2....an 2an 1a0a1a2a3j+n=2Mn=2.. !jMn=2....an 2an 1Inshort,theproductofMn(!)withvector(a0;:::;an 1),asize-nproblem,canbeexpressedintermsoftwosize-n=2problems:theproductofMn=2(!2)with(a0;a2;:::;an 2)andwith(a1;a3;:::;an 1).Thisdivide-and-conquerstrategyleadstothedenitiveFFTalgorithmofFigure2.9,whoserunningtimeisT(n)=2T(n=2)+O(n)=O(nlogn).ThefastFouriertransformunraveledThroughoutallourdiscussionssofar,thefastFouriertransformhasremainedtightlyco-coonedwithinadivide-and-conquerformalism.Tofullyexposeitsstructure,wenowunraveltherecursion.Thedivide-and-conquerstepoftheFFTcanbedrawnasaverysimplecircuit.Hereishowaproblemofsizenisreducedtotwosubproblemsofsizen=2(forclarity,onepairofoutputs(j;j+n=2)issingledout):75\nFFTn(input:a0;:::;an 1,output:r0;:::;rn 1)a0a2..FFTn=2rj.an 2a1aj3..FFTn=2rj+n=2.j+n=2an 1We'reusingaparticularshorthand:theedgesarewirescarryingcomplexnumbersfromlefttoright.Aweightofjmeans“multiplythenumberonthiswireby!j.”Andwhentwowirescomeintoajunctionfromtheleft,thenumberstheyarecarryinggetaddedup.Sothetwooutputsdepictedareexecutingthecommandsr=s+!js0jjjr=s !js0j+n=2jjfromtheFFTalgorithm(Figure2.9),viaapatternofwiresknownasabuttery:.UnravelingtheFFTcircuitcompletelyforn=8elements,wegetFigure10.4.Noticethefollowing.1.Forninputstherearelog2nlevels,eachwithnnodes,foratotalofnlognoperations.2.Theinputsarearrangedinapeculiarorder:0;4;2;6;1;5;3;7.Why?Recallthatatthetoplevelofrecursion,werstbringuptheevencoefcientsoftheinputandthenmoveontotheoddones.Thenatthenextlevel,theevencoefcientsofthisrstgroup(whichthereforearemultiplesof4,orequivalently,havezeroastheirtwoleastsignicantbits)arebroughtup,andsoon.Toputitotherwise,theinputsarearrangedbyincreasinglastbitofthebinaryrepresentationoftheirindex,resolvingtiesbylookingatthenextmoresignicantbit(s).Theresultingorderinbinary,000;100;010;110;001;101;011;111,isthesameasthenaturalone,000;001;010;011;100;101;110;111exceptthebitsaremirrored!3.ThereisauniquepathbetweeneachinputajandeachoutputA(!k).Thispathismosteasilydescribedusingthebinaryrepresentationsofjandk(showninFigure10.4forconvenience).Therearetwoedgesoutofeachnode,onegoingup(the0-edge)andonegoingdown(the1-edge).TogettoA(!k)fromanyinputnode,simplyfollowtheedgesspeciedinthebitrepresentationofk,startingfromtherightmostbit.(Canyousimilarlyspecifythepathinthereversedirection?)4.OnthepathbetweenajandA(!k),thelabelsadduptojkmod8.Since!8=1,thismeansthatthecontributionofinputajtooutputA(!k)isaj!jk,andthereforethecircuitcomputescorrectlythevaluesofpolynomialA(x).5.Andnally,noticethattheFFTcircuitisanaturalforparallelcomputationanddirectimplementationinhardware.76\nFigure2.10ThefastFouriertransformcircuit.000a !A(!0)0000100a "#A(!1)00144010a4$%A(!2)01022110a&'A(!3)011646001a4()A(!4)10011101a*+A(!5)1015452011a4,-A(!6)1103632111a./A(!7)111746777\nTheslowspreadofafastalgorithmIn1963,duringameetingofPresidentKennedy'sscienticadvisors,JohnTukey,amath-ematicianfromPrinceton,explainedtoIBM'sDickGarwinafastmethodforcomputingFouriertransforms.Garwinlistenedcarefully,becausehewasatthetimeworkingonwaystodetectnuclearexplosionsfromseismographicdata,andFouriertransformswerethebot-tleneckofhismethod.WhenhewentbacktoIBM,heaskedJohnCooleytoimplementTukey'salgorithm;theydecidedthatapapershouldbepublishedsothattheideacouldnotbepatented.Tukeywasnotverykeentowriteapaperonthesubject,soCooleytooktheinitiative.Andthisishowoneofthemostfamousandmostcitedscienticpaperswaspublishedin1965,co-authoredbyCooleyandTukey.ThereasonTukeywasreluctanttopublishtheFFTwasnotsecretivenessorpursuitofprotviapatents.Hejustfeltthatthiswasasimpleobservationthatwasprobablyalreadyknown.Thiswastypicaloftheperiod:backthen(andforsometimelater)algorithmswereconsideredsecond-classmathematicalobjects,devoidofdepthandelegance,andunworthyofseriousattention.ButTukeywasrightaboutonething:itwaslaterdiscoveredthatBritishengineershadusedtheFFTforhandcalculationsduringthelate1930s.And—toendthischapterwiththesamegreatmathematicianwhostartedit—apaperbyGaussintheearly1800son(whatelse?)interpolationcontainedessentiallythesameideainit!Gauss'spaperhadremainedasecretforsolongbecauseitwasprotectedbyanold-fashionedcryptographictechnique:likemostscienticpapersofitsera,itwaswritteninLatin.78\nExercises2.1.Usethedivide-and-conquerintegermultiplicationalgorithmtomultiplythetwobinaryintegers10011011and10111010.2.2.Showthatforanypositiveintegersnandanybaseb,theremustsomepowerofblyingintherange[n;bn].2.3.Section2.2describesamethodforsolvingrecurrencerelationswhichisbasedonanalyzingtherecursiontreeandderivingaformulafortheworkdoneateachlevel.Another(closelyrelated)methodistoexpandouttherecurrenceafewtimes,untilapatternemerges.Forinstance,let'sstartwiththefamiliarT(n)=2T(n=2)+O(n).ThinkofO(n)asbeingcnforsomeconstantc,so:T(n)2T(n=2)+cn.Byrepeatedlyapplyingthisrule,wecanboundT(n)intermsofT(n=2),thenT(n=4),thenT(n=8),andsoon,ateachstepgettingclosertothevalueofT()wedoknow,namelyT(1)=O(1).T(n)2T(n=2)+cn2[2T(n=4)+cn=2]+cn=4T(n=4)+2cn4[2T(n=8)+cn=4]+2cn=8T(n=8)+3cn8[2T(n=16)+cn=8]+3cn=16T(n=16)+4cn...Apatternisemerging...thegeneraltermisT(n)2kT(n=2k)+kcn:Pluggingink=log2n,wegetT(n)nT(1)+cnlog2n=O(nlogn).(a)DothesamethingfortherecurrenceT(n)=3T(n=2)+O(n).Whatisthegeneralkthterminthiscase?Andwhatvalueofkshouldbepluggedintogettheanswer?(b)NowtrytherecurrenceT(n)=T(n 1)+O(1),acasewhichisnotcoveredbythemastertheorem.Canyousolvethistoo?2.4.Supposeyouarechoosingbetweenthefollowingthreealgorithms:AlgorithmAsolvesproblemsbydividingthemintovesubproblemsofhalfthesize,recur-sivelysolvingeachsubproblem,andthencombiningthesolutionsinlineartime.AlgorithmBsolvesproblemsofsizenbyrecursivelysolvingtwosubproblemsofsizen 1andthencombiningthesolutionsinconstanttime.AlgorithmCsolvesproblemsofsizenbydividingthemintoninesubproblemsofsizen=3,recursivelysolvingeachsubproblem,andthencombiningthesolutionsinO(n2)time.Whataretherunningtimesofeachofthesealgorithms(inbig-Onotation),andwhichwouldyouchoose?2.5.Solvethefollowingrecurrencerelationsandgiveaboundforeachofthem.(a)T(n)=2T(n=3)+1(b)T(n)=5T(n=4)+n(c)T(n)=7T(n=7)+n(d)T(n)=9T(n=3)+n279\n(e)T(n)=8T(n=2)+n3(f)T(n)=49T(n=25)+n3=2logn(g)T(n)=T(n 1)+2(h)T(n)=T(n 1)+nc,wherec1isaconstant(i)T(n)=T(n 1)+cn,wherec>1issomeconstant(j)T(n)=2T(n 1)+1p(k)T(n)=T(n)+12.6.Alinear,time-invariantsystemhasthefollowingimpulseresponse:b(t)1=t0 $ !("#$()*+,-./%%&'tt0(a)Describeinwordstheeffectofthissystem.(b)Whatisthecorrespondingpolynomial?2.7.Whatisthesumofthenthrootsofunity?Whatistheirproductifnisodd?Ifniseven?2.8.PracticewiththefastFouriertransform.(a)WhatistheFFTof(1;0;0;0)?Whatistheappropriatevalueof!inthiscase?Andofwhichsequenceis(1;0;0;0)theFFT?(b)Repeatfor(1;0;1; 1).2.9.PracticewithpolynomialmultiplicationbyFFT.(a)Supposethatyouwanttomultiplythetwopolynomialsx+1andx2+1usingtheFFT.Chooseanappropriatepoweroftwo,ndtheFFTofthetwosequences,multiplytheresultscomponentwise,andcomputetheinverseFFTtogetthenalresult.(b)Repeatforthepairofpolynomials1+x+2x2and2+3x.2.10.Findtheuniquepolynomialofdegree4thattakesonvaluesp(1)=2,p(2)=1,p(3)=0,p(4)=4,andp(5)=0.Writeyouranswerinthecoefcientrepresentation.2.11.Injustifyingourmatrixmultiplicationalgorithm(Section2.5),weclaimedthefollowingblock-wiseproperty:ifXandYarennmatrices,andABEFX=;Y=:CDGHwhereA,B,C,D,E,F,G,andHaren=2n=2submatrices,thentheproductXYcanbeexpressedintermsoftheseblocks:ABEFAE+BGAF+BHXY==CDGHCE+DGCF+DHProvethisproperty.80\n2.12.Howmanylines,asafunctionofn(in()form),doesthefollowingprogramprint?Writearecurrenceandsolveit.Youmayassumenisapowerof2.functionf(n)ifn>1:print_line(``stillgoing'')f(n/2)f(n/2)2.13.Abinarytreeisfullifallofitsverticeshaveeitherzeroortwochildren.LetBndenotethenumberoffullbinarytreeswithnvertices.(a)Bydrawingoutallfullbinarytreeswith3,5,or7vertices,determinetheexactvaluesofB3,B5,andB7.Whyhaveweleftoutevennumbersofvertices,likeB4?(b)Forgeneraln,derivearecurrencerelationforBn.(c)ShowbyinductionthatBis(2n).n2.14.Youaregivenanarrayofnelements,andyounoticethatsomeoftheelementsareduplicates;thatis,theyappearmorethanonceinthearray.ShowhowtoremoveallduplicatesfromthearrayintimeO(nlogn).2.15.Inourmedian-ndingalgorithm(Section2.4),abasicprimitiveisthesplitoperation,whichtakesasinputanarraySandavaluevandthendividesSintothreesets:theelementslessthanv,theelementsequaltov,andtheelementsgreaterthanv.Showhowtoimplementthissplitoperationinplace,thatis,withoutallocatingnewmemory.2.16.YouaregivenaninnitearrayA[]inwhichtherstncellscontainintegersinsortedorderandtherestofthecellsarelledwith1.Youarenotgiventhevalueofn.Describeanalgorithmthattakesanintegerxasinputandndsapositioninthearraycontainingx,ifsuchapositionexists,inO(logn)time.(IfyouaredisturbedbythefactthatthearrayAhasinnitelength,assumeinsteadthatitisoflengthn,butthatyoudon'tknowthislength,andthattheimplementationofthearraydatatypeinyourprogramminglanguagereturnstheerrormessage1wheneverelementsA[i]withi>nareaccessed.)2.17.GivenasortedarrayofdistinctintegersA[1;:::;n],youwanttondoutwhetherthereisanindexiforwhichA[i]=i.Giveadivide-and-conqueralgorithmthatrunsintimeO(logn).2.18.ConsiderthetaskofsearchingasortedarrayA[1:::n]foragivenelementx:ataskweusuallyperformbybinarysearchintimeO(logn).Showthatanyalgorithmthataccessesthearrayonlyviacomparisons(thatis,byaskingquestionsoftheform“isA[i]z?”),musttake(logn)steps.2.19.Ak-waymergeoperation.Supposeyouhaveksortedarrays,eachwithnelements,andyouwanttocombinethemintoasinglesortedarrayofknelements.(a)Here'sonestrategy:UsingthemergeprocedurefromSection2.3,mergethersttwoar-rays,thenmergeinthethird,thenmergeinthefourth,andsoon.Whatisthetimecomplexityofthisalgorithm,intermsofkandn?(b)Giveamoreefcientsolutiontothisproblem,usingdivide-and-conquer.2.20.Showthatanyarrayofintegersx[1:::n]canbesortedinO(n+M)time,whereM=maxxi minxi:iiForsmallM,thisislineartime:whydoesn'tthe(nlogn)lowerboundapplyinthiscase?81\n2.21.Meanandmedian.Oneofthemostbasictasksinstatisticsistosummarizeasetofobservationsfx1;x2;:::;xngRbyasinglenumber.Twopopularchoicesforthissummarystatisticare:Themedian,whichwe'llcall1Themean,whichwe'llcall2(a)ShowthatthemedianisthevalueofthatminimizesthefunctionXjxi j:iYoucanassumeforsimplicitythatnisodd.(Hint:Showthatforany6=1,thefunctiondecreasesifyoumoveeitherslightlytotheleftorslightlytotheright.)(b)ShowthatthemeanisthevalueofthatminimizesthefunctionX(x )2:iiOnewaytodothisisbycalculus.Anothermethodistoprovethatforany2R,XX222(xi )=(xi 2)+n( 2):iiNoticehowthefunctionfor2penalizespointsthatarefarfrommuchmoreheavilythanthefunctionfor1.Thus2triesmuchhardertobeclosetoalltheobservations.Thismightsoundlikeagoodthingatsomelevel,butitisstatisticallyundesirablebecausejustafewoutlierscanseverelythrowofftheestimateof2.Itisthereforesometimessaidthat1isamorerobustestimatorthan2.Worsethaneitherofthem,however,is1,thevalueofthatminimizesthefunctionmaxjxi j:i(c)Showthat1canbecomputedinO(n)time(assumingthenumbersxiaresmallenoughthatbasicarithmeticoperationsonthemtakeunittime).2.22.Youaregiventwosortedlistsofsizemandn.GiveanO(logm+logn)timealgorithmforcomputingthekthsmallestelementintheunionofthetwolists.2.23.AnarrayA[1:::n]issaidtohaveamajorityelementifmorethanhalfofitsentriesarethesame.Givenanarray,thetaskistodesignanefcientalgorithmtotellwhetherthearrayhasamajorityelement,and,ifso,tondthatelement.Theelementsofthearrayarenotnecessarilyfromsomeordereddomainliketheintegers,andsotherecanbenocomparisonsoftheform“isA[i]>A[j]?”.(ThinkofthearrayelementsasGIFles,say.)Howeveryoucananswerquestionsoftheform:“isA[i]=A[j]?”inconstanttime.(a)ShowhowtosolvethisprobleminO(nlogn)time.(Hint:SplitthearrayAintotwoarraysA1andA2ofhalfthesize.DoesknowingthemajorityelementsofA1andA2helpyougureoutthemajorityelementofA?Ifso,youcanuseadivide-and-conquerapproach.)(b)Canyougivealinear-timealgorithm?(Hint:Here'sanotherdivide-and-conquerapproach:PairuptheelementsofAarbitrarily,togetn=2pairsLookateachpair:ifthetwoelementsaredifferent,discardbothofthem;iftheyarethesame,keepjustoneofthemShowthatafterthisprocedurethereareatmostn=2elementsleft,andthattheyhaveamajorityelementifandonlyifAdoes.)82\n2.24.Onpage62thereisahigh-leveldescriptionofthequicksortalgorithm.(a)Writedownthepseudocodeforquicksort.(b)Showthatitsworst-caserunningtimeonanarrayofsizenis(n2).(c)ShowthatitsexpectedrunningtimesatisestherecurrencerelationnX 11T(n)O(n)+(T(i)+T(n i)):ni=1Then,showthatthesolutiontothisrecurrenceisO(nlogn).2.25.InSection2.1wedescribedanalgorithmthatmultipliestwon-bitbinaryintegersxandyintimena,wherea=log3.Callthisprocedurefastmultiply(x;y).2(a)Wewanttoconvertthedecimalinteger10n(a1followedbynzeros)intobinary.Hereisthealgorithm(assumenisapowerof2):functionpwr2bin(n)ifn=1:return10102else:z=???returnfastmultiply(z;z)Fillinthemissingdetails.Thengivearecurrencerelationfortherunningtimeofthealgorithm,andsolvetherecurrence.(b)Next,wewanttoconvertanydecimalintegerxwithndigits(wherenisapowerof2)intobinary.Thealgorithmisthefollowing:functiondec2bin(x)ifn=1:returnbinary[x]else:splitxintotwodecimalnumbersxL,xRwithn=2digitseachreturn???Herebinary[]isavectorthatcontainsthebinaryrepresentationofallone-digitintegers.Thatis,binary[0]=02,binary[1]=12,uptobinary[9]=10012.AssumethatalookupinbinarytakesO(1)time.Fillinthemissingdetails.Onceagain,givearecurrencefortherunningtimeofthealgo-rithm,andsolveit.2.26.ProfessorF.Laketellshisclassthatitisasymptoticallyfastertosquareann-bitintegerthantomultiplytwon-bitintegers.Shouldtheybelievehim?2.27.ThesquareofamatrixAisitsproductwithitself,AA.(a)Showthatvemultiplicationsaresufcienttocomputethesquareofa22matrix.(b)Whatiswrongwiththefollowingalgorithmforcomputingthesquareofannnmatrix?“Useadivide-and-conquerapproachasinStrassen'salgorithm,exceptthatin-steadofgetting7subproblemsofsizen=2,wenowget5subproblemsofsizen=2thankstopart(a).UsingthesameanalysisasinStrassen'salgorithm,wecanconcludethatthealgorithmrunsintimeO(nlog25).”83\n(c)Infact,squaringmatricesisnoeasierthanmatrixmultiplication.Inthispart,youwillshowthatifnnmatricescanbesquaredintimeS(n)=O(nc),thenanytwonnmatricescanbemultipliedintimeO(nc).i.GiventwonnmatricesAandB,showthatthematrixAB+BAcanbecomputedintime3S(n)+O(n2).ii.GiventwonnmatricesXandY,denethe2n2nmatricesAandBasfollows:X00YA=andB=:0000WhatisAB+BA,intermsofXandY?iii.Using(i)and(ii),arguethattheproductXYcanbecomputedintime3S(2n)+O(n2).ConcludethatmatrixmultiplicationtakestimeO(nc).2.28.TheHadamardmatricesH0;H1;H2;:::aredenedasfollows:H0isthe11matrix1Fork>0,Histhe2k2kmatrixkHk 1Hk 1Hk=Hk 1 Hk 1Showthatifvisacolumnvectoroflengthn=2k,thenthematrix-vectorproductHvcanbekcalculatedusingO(nlogn)operations.Assumethatallthenumbersinvolvedaresmallenoughthatbasicarithmeticoperationslikeadditionandmultiplicationtakeunittime.2.29.Supposewewanttoevaluatethepolynomialp(x)=a+ax+ax2++axnatpointx.012n(a)Showthatthefollowingsimpleroutine,knownasHorner'srule,doesthejobandleavestheanswerinz.z=anfori=n 1downto0:z=zx+ai(b)Howmanyadditionsandmultiplicationsdoesthisroutineuse,asafunctionofn?Canyoundapolynomialforwhichanalternativemethodissubstantiallybetter?2.30.ThisproblemillustrateshowtodotheFourierTransform(FT)inmodulararithmetic,forexam-ple,modulo7.(a)Thereisanumber!suchthatallthepowers!;!2;:::;!6aredistinct(modulo7).Findthis!,andshowthat!+!2++!6=0.(Interestingly,foranyprimemodulusthereissuchanumber.)(b)UsingthematrixformoftheFT,producethetransformofthesequence(0;1;1;1;5;2)mod-ulo7;thatis,multiplythisvectorbythematrixM6(!),forthevalueof!youfoundearlier.Inthematrixmultiplication,allcalculationsshouldbeperformedmodulo7.(c)WritedownthematrixnecessarytoperformtheinverseFT.Showthatmultiplyingbythismatrixreturnstheoriginalsequence.(Againallarithmeticshouldbeperformedmodulo7.)(d)Nowshowhowtomultiplythepolynomialsx2+x+1andx3+2x 1usingtheFTmodulo7.2.31.InSection1.2.3,westudiedEuclid'salgorithmforcomputingthegreatestcommondivisor(gcd)oftwopositiveintegers:thelargestintegerwhichdividesthemboth.Herewewilllookatanalternativealgorithmbasedondivide-and-conquer.84\n(a)Showthatthefollowingruleistrue.8<2gcd(a=2;b=2)ifa;bareevengcd(a;b)=gcd(a;b=2)ifaisodd,biseven:gcd((a b)=2;b)ifa;bareodd(b)Giveanefcientdivide-and-conqueralgorithmforgreatestcommondivisor.(c)HowdoestheefciencyofyouralgorithmcomparetoEuclid'salgorithmifaandbaren-bitintegers?(Inparticular,sincenmightbelargeyoucannotassumethatbasicarithmeticoperationslikeadditiontakeconstanttime.)2.32.Inthisproblemwewilldevelopadivide-and-conqueralgorithmforthefollowinggeometrictask.CLOSESTPAIRInput:Asetofpointsintheplane,fp1=(x1;y1);p2=(x2;y2);:::;pn=(xn;yn)gOutput:Theclosestpairofpoints:thatis,thepairpi6=pjforwhichthedistancebetweenpiandpj,thatis,q(xi xj)2+(yi yj)2;isminimized.Forsimplicity,assumethatnisapoweroftwo,andthatallthex-coordinatesxiaredistinct,asarethey-coordinates.Here'sahigh-leveloverviewofthealgorithm:Findavaluexforwhichexactlyhalfthepointshavexix.Onthisbasis,splitthepointsintotwogroups,LandR.RecursivelyndtheclosestpairinLandinR.SaythesepairsarepL;qL2LandpR;qR2R,withdistancesdLanddRrespectively.Letdbethesmallerofthesetwodistances.ItremainstobeseenwhetherthereisapointinLandapointinRthatarelessthandistancedapartfromeachother.Tothisend,discardallpointswithxix+dandsorttheremainingpointsbyy-coordinate.Now,gothroughthissortedlist,andforeachpoint,computeitsdistancetothesevensub-sequentpointsinthelist.LetpM;qMbetheclosestpairfoundinthisway.TheanswerisoneofthethreepairsfpL;qLg,fpR;qRg,fpM;qMg,whicheverisclosest.(a)Inordertoprovethecorrectnessofthisalgorithm,startbyshowingthefollowingproperty:anysquareofsizeddintheplanecontainsatmostfourpointsofL.(b)Nowshowthatthealgorithmiscorrect.TheonlycasewhichneedscarefulconsiderationiswhentheclosestpairissplitbetweenLandR.(c)Writedownthepseudocodeforthealgorithm,andshowthatitsrunningtimeisgivenbytherecurrence:T(n)=2T(n=2)+O(nlogn):2ShowthatthesolutiontothisrecurrenceisO(nlogn).(d)CanyoubringtherunningtimedowntoO(nlogn)?85\n86\nChapter3Decompositionsofgraphs3.1Whygraphs?Awiderangeofproblemscanbeexpressedwithclarityandprecisionintheconcisepictoriallanguageofgraphs.Forinstance,considerthetaskofcoloringapoliticalmap.Whatistheminimumnumberofcolorsneeded,withtheobviousrestrictionthatneighboringcountriesshouldhavedifferentcolors?Oneofthedifcultiesinattackingthisproblemisthatthemapitself,evenastripped-downversionlikeFigure3.1(a),isusuallyclutteredwithirrelevantinformation:intricateboundaries,borderpostswherethreeormorecountriesmeet,openseas,andmeanderingrivers.SuchdistractionsareabsentfromthemathematicalobjectofFigure3.1(b),agraphwithonevertexforeachcountry(1isBrazil,11isArgentina)andedgesbetweenneighbors.Itcontainsexactlytheinformationneededforcoloring,andnothingmore.Theprecisegoalisnowtoassignacolortoeachvertexsothatnoedgehasendpointsofthesamecolor.Graphcoloringisnottheexclusivedomainofmapdesigners.Supposeauniversityneedstoscheduleexaminationsforallitsclassesandwantstousethefewesttimeslotspossible.Theonlyconstraintisthattwoexamscannotbescheduledconcurrentlyifsomestudentwillbetakingbothofthem.Toexpressthisproblemasagraph,useonevertexforeachexamandputanedgebetweentwoverticesifthereisaconict,thatis,ifthereissomebodytakingbothendpointexams.Thinkofeachtimeslotashavingitsowncolor.Then,assigningtimeslotsisexactlythesameascoloringthisgraph!Somebasicoperationsongraphsarisewithsuchfrequency,andinsuchadiversityofcon-texts,thatalotofefforthasgoneintondingefcientproceduresforthem.Thischapterisdevotedtosomeofthemostfundamentalofthesealgorithms—thosethatuncoverthebasicconnectivitystructureofagraph.Formally,agraphisspeciedbyasetofvertices(alsocallednodes)VandbyedgesEbetweenselectpairsofvertices.Inthemapexample,V=f1;2;3;:::;13gandEincludes,amongmanyotheredges,f1;2g;f9;11g,andf7;13g.Hereanedgebetweenxandyspecicallymeans“xsharesaborderwithy.”Thisisasymmetricrelation—itimpliesalsothatysharesaborderwithx—andwedenoteitusingsetnotation,e=fx;yg.Suchedgesareundirectedandarepartofanundirectedgraph.Sometimesgraphsdepictrelationsthatdonothavethisreciprocity,inwhichcaseitis87\nFigure3.1(a)Amapand(b)itsgraph.(a)(b)54326817910121311necessarytouseedgeswithdirectionsonthem.Therecanbedirectededgesefromxtoy(writtene=(x;y)),orfromytox(written(y;x)),orboth.AparticularlyenormousexampleofadirectedgraphisthegraphofalllinksintheWorldWideWeb.IthasavertexforeachsiteontheInternet,andadirectededge(u;v)wheneversiteuhasalinktositev:intotal,billionsofnodesandedges!UnderstandingeventhemostbasicconnectivitypropertiesoftheWebisofgreateconomicandsocialinterest.Althoughthesizeofthisproblemisdaunting,wewillsoonseethatalotofvaluableinformationaboutthestructureofagraphcan,happily,bedeterminedinjustlineartime.3.1.1Howisagraphrepresented?Wecanrepresentagraphbyanadjacencymatrix;iftherearen=jVjverticesv1;:::;vn,thisisannnarraywhose(i;j)thentryis1ifthereisanedgefromvitovjaij=0otherwise.Forundirectedgraphs,thematrixissymmetricsinceanedgefu;vgcanbetakenineitherdirection.Thebiggestconvenienceofthisformatisthatthepresenceofaparticularedgecanbecheckedinconstanttime,withjustonememoryaccess.OntheotherhandthematrixtakesupO(n2)space,whichiswastefulifthegraphdoesnothaveverymanyedges.Analternativerepresentation,withsizeproportionaltothenumberofedges,istheadja-cencylist.ItconsistsofjVjlinkedlists,onepervertex.Thelinkedlistforvertexuholdsthenamesofverticestowhichuhasanoutgoingedge—thatis,verticesvforwhich(u;v)2E.88\nTherefore,eachedgeappearsinexactlyoneofthelinkedlistsifthegraphisdirectedortwoofthelistsifthegraphisundirected.Eitherway,thetotalsizeofthedatastructureisO(jEj).Checkingforaparticularedge(u;v)isnolongerconstanttime,becauseitrequiressiftingthroughu'sadjacencylist.Butitiseasytoiteratethroughallneighborsofavertex(byrun-ningdownthecorrespondinglinkedlist),and,asweshallsoonsee,thisturnsouttobeaveryusefuloperationingraphalgorithms.Again,forundirectedgraphs,thisrepresentationhasasymmetryofsorts:visinu'sadjacencylistifandonlyifuisinv'sadjacencylist.Howbigisyourgraph?Whichofthetworepresentations,adjacencymatrixoradjacencylist,isbetter?Well,itde-pendsontherelationshipbetweenjVj,thenumberofnodesinthegraph,andjEj,thenum-berofedges.jEjcanbeassmallasjVj(ifitgetsmuchsmaller,thenthegraphdegenerates—forexample,hasisolatedvertices),oraslargeasjVj2(whenallpossibleedgesarepresent).WhenjEjisclosetotheupperlimitofthisrange,wecallthegraphdense.Attheotherextreme,ifjEjisclosetojVj,thegraphissparse.Asweshallseeinthischapterandthenexttwochapters,exactlywherejEjliesinthisrangeisusuallyacrucialfactorinselectingtherightgraphalgorithm.Or,forthatmatter,inselectingthegraphrepresentation.IfitistheWorldWideWebgraphthatwewishtostoreincomputermemory,weshouldthinktwicebeforeusinganadjacencymatrix:atthetimeofwriting,searchenginesknowofabouteightbillionverticesofthisgraph,andhencetheadjacencymatrixwouldtakeupdozensofmillionsofterabits.Againatthetimewewritetheselines,itisnotclearthatthereisenoughcomputermemoryinthewholeworldtoachievethis.(Andwaitingafewyearsuntilthereisenoughmemoryisunwise:theWebwillgrowtooandwillprobablygrowfaster.)Withadjacencylists,representingtheWorldWideWebbecomesfeasible:thereareonlyafewdozenbillionhyperlinksintheWeb,andeachwilloccupyafewbytesintheadjacencylist.Youcancarryadevicethatstorestheresult,aterabyteortwo,inyourpocket(itmaysoontinyourearring,butbythattimetheWebwillhavegrowntoo).ThereasonwhyadjacencylistsaresomuchmoreeffectiveinthecaseoftheWorldWideWebisthattheWebisverysparse:theaverageWebpagehashyperlinkstoonlyabouthalfadozenotherpages,outofthebillionsofpossibilities.3.2Depth-rstsearchinundirectedgraphs3.2.1ExploringmazesDepth-rstsearchisasurprisinglyversatilelinear-timeprocedurethatrevealsawealthofinformationaboutagraph.Themostbasicquestionitaddressesis,Whatpartsofthegrapharereachablefromagivenvertex?Tounderstandthistask,tryputtingyourselfinthepositionofacomputerthathasjustbeengivenanewgraph,sayintheformofanadjacencylist.Thisrepresentationoffersjustonebasicoperation:ndingtheneighborsofavertex.Withonlythisprimitive,thereachabilityproblemisratherlikeexploringalabyrinth(Figure3.2).Youstartwalkingfromaxedplace89\nFigure3.2Exploringagraphisratherlikenavigatingamaze.LKDABEFHBGHCFIJCGEJKLIADFigure3.3Findingallnodesreachablefromaparticularnode.procedureexplore(G;v)Input:G=(V;E)isagraph;v2VOutput:visited(u)issettotrueforallnodesureachablefromvvisited(v)=trueprevisit(v)foreachedge(v;u)2E:ifnotvisited(u):explore(u)postvisit(v)andwheneveryouarriveatanyjunction(vertex)thereareavarietyofpassages(edges)youcanfollow.Acarelesschoiceofpassagesmightleadyouaroundincirclesormightcauseyoutooverlooksomeaccessiblepartofthemaze.Clearly,youneedtorecordsomeintermediateinformationduringexploration.Thisclassicchallengehasamusedpeopleforcenturies.Everybodyknowsthatallyouneedtoexplorealabyrinthisaballofstringandapieceofchalk.Thechalkpreventslooping,bymarkingthejunctionsyouhavealreadyvisited.Thestringalwaystakesyoubacktothestartingplace,enablingyoutoreturntopassagesthatyoupreviouslysawbutdidnotyetinvestigate.Howcanwesimulatethesetwoprimitives,chalkandstring,onacomputer?Thechalkmarksareeasy:foreachvertex,maintainaBooleanvariableindicatingwhetherithasbeenvisitedalready.Asfortheballofstring,thecorrectcyberanalogisastack.Afterall,theexactroleofthestringistooffertwoprimitiveoperations—unwindtogettoanewjunction(thestackequivalentistopushthenewvertex)andrewindtoreturntothepreviousjunction(popthestack).Insteadofexplicitlymaintainingastack,wewilldosoimplicitlyviarecursion(whichisimplementedusingastackofactivationrecords).Theresultingalgorithmisshownin90\nFigure3.3.1Theprevisitandpostvisitproceduresareoptional,meantforperformingoperationsonavertexwhenitisrstdiscoveredandalsowhenitisbeingleftforthelasttime.Wewillsoonseesomecreativeusesforthem.Moreimmediately,weneedtoconrmthatexplorealwaysworkscorrectly.Itcertainlydoesnotventuretoofar,becauseitonlymovesfromnodestotheirneighborsandcanthereforeneverjumptoaregionthatisnotreachablefromv.Butdoesitndallverticesreachablefromv?Well,ifthereissomeuthatitmisses,chooseanypathfromvtou,andlookatthelastvertexonthatpaththattheprocedureactuallyvisited.Callthisnodez,andletwbethenodeimmediatelyafteritonthesamepath.vzwuSozwasvisitedbutwwasnot.Thisisacontradiction:whiletheexploreprocedurewasatnodez,itwouldhavenoticedwandmovedontoit.Incidentally,thispatternofreasoningarisesofteninthestudyofgraphsandisinessenceastreamlinedinduction.Amoreformalinductiveproofwouldstartbyframingahypothesis,suchas“foranyk0,allnodeswithinkhopsfromvgetvisited.”Thebasecaseisasusualtrivial,sinceviscertainlyvisited.Andthegeneralcase—showingthatifallnodeskhopsawayarevisited,thensoareallnodesk+1hopsaway—ispreciselythesamepointwejustargued.Figure3.4showstheresultofrunningexploreonourearlierexamplegraph,startingatnodeA,andbreakingtiesinalphabeticalorderwheneverthereisachoiceofnodestovisit.Thesolidedgesarethosethatwereactuallytraversed,eachofwhichwaselicitedbyacalltoexploreandledtothediscoveryofanewvertex.Forinstance,whileBwasbeingvisited,theedgeB Ewasnoticedand,sinceEwasasyetunknown,wastraversedviaacalltoexplore(E).Thesesolidedgesformatree(aconnectedgraphwithnocycles)andarethereforecalledtreeedges.Thedottededgeswereignoredbecausetheyledbacktofamiliarterrain,toverticespreviouslyvisited.Theyarecalledbackedges.3.2.2Depth-rstsearchTheexploreprocedurevisitsonlytheportionofthegraphreachablefromitsstartingpoint.Toexaminetherestofthegraph,weneedtorestarttheprocedureelsewhere,atsomevertexthathasnotyetbeenvisited.ThealgorithmofFigure3.5,calleddepth-rstsearch(DFS),doesthisrepeatedlyuntiltheentiregraphhasbeentraversed.TherststepinanalyzingtherunningtimeofDFSistoobservethateachvertexisexplore'djustonce,thankstothevisitedarray(thechalkmarks).Duringtheexplorationofavertex,therearethefollowingsteps:1.Somexedamountofwork—markingthespotasvisited,andthepre/postvisit.2.Aloopinwhichadjacentedgesarescanned,toseeiftheyleadsomewherenew.1Aswithmanyofourgraphalgorithms,thisoneappliestobothundirectedanddirectedgraphs.Insuchcases,weadoptthedirectednotationforedges,(x;y).Ifthegraphisundirected,theneachofitsedgesshouldbethoughtofasexistinginbothdirections:(x;y)and(y;x).91\nFigure3.4Theresultofexplore(A)onthegraphofFigure3.2.ABDEFGICHJFigure3.5Depth-rstsearch.proceduredfs(G)forallv2V:visited(v)=falseforallv2V:ifnotvisited(v):explore(v)Thislooptakesadifferentamountoftimeforeachvertex,solet'sconsiderallverticesto-gether.Thetotalworkdoneinstep1isthenO(jVj).Instep2,overthecourseoftheentireDFS,eachedgefx;yg2Eisexaminedexactlytwice,onceduringexplore(x)andoncedur-ingexplore(y).Theoveralltimeforstep2isthereforeO(jEj)andsothedepth-rstsearchhasarunningtimeofO(jVj+jEj),linearinthesizeofitsinput.Thisisasefcientaswecouldpossiblyhopefor,sinceittakesthislongevenjusttoreadtheadjacencylist.Figure3.6showstheoutcomeofdepth-rstsearchona12-nodegraph,onceagainbreak-ingtiesalphabetically(ignorethepairsofnumbersforthetimebeing).TheouterloopofDFScallsexplorethreetimes,onA,C,andnallyF.Asaresult,therearethreetrees,eachrootedatoneofthesestartingpoints.Togethertheyconstituteaforest.3.2.3ConnectivityinundirectedgraphsAnundirectedgraphisconnectedifthereisapathbetweenanypairofvertices.ThegraphofFigure3.6isnotconnectedbecause,forinstance,thereisnopathfromAtoK.However,itdoeshavethreedisjointconnectedregions,correspondingtothefollowingsetsofvertices:fA;B;E;I;JgfC;D;G;H;K;LgfFg92\nFigure3.6(a)A12-nodegraph.(b)DFSsearchforest.1,1011,2223,24(a)(b)ACFABCDBE4,912,21D2,3EFGHI5,813,20HIJKLJ14,17GL6,718,19K15,16Theseregionsarecalledconnectedcomponents:eachofthemisasubgraphthatisinternallyconnectedbuthasnoedgestotheremainingvertices.Whenexploreisstartedataparticularvertex,itidentiespreciselytheconnectedcomponentcontainingthatvertex.AndeachtimetheDFSouterloopcallsexplore,anewconnectedcomponentispickedout.Thusdepth-rstsearchistriviallyadaptedtocheckifagraphisconnectedand,moregenerally,toassigneachnodevanintegerccnum[v]identifyingtheconnectedcomponenttowhichitbelongs.Allittakesisprocedureprevisit(v)ccnum[v]=ccwhereccneedstobeinitializedtozeroandtobeincrementedeachtimetheDFSprocedurecallsexplore.3.2.4PrevisitandpostvisitorderingsWehaveseenhowdepth-rstsearch—afewunassuminglinesofcode—isabletouncovertheconnectivitystructureofanundirectedgraphinjustlineartime.Butitisfarmoreversatilethanthis.Inordertostretchitfurther,wewillcollectalittlemoreinformationduringtheex-plorationprocess:foreachnode,wewillnotedownthetimesoftwoimportantevents,themo-mentofrstdiscovery(correspondingtoprevisit)andthatofnaldeparture(postvisit).Figure3.6showsthesenumbersforourearlierexample,inwhichthereare24events.TheftheventisthediscoveryofI.The21steventconsistsofleavingDbehindforgood.Onewaytogeneratearrayspreandpostwiththesenumbersistodeneasimplecounterclock,initiallysetto1,whichgetsupdatedasfollows.procedureprevisit(v)pre[v]=clockclock=clock+193\nFigure3.7DFSonadirectedgraph.A1,16BACB2,11C12,15EFDE3,10D13,144,7FH8,9GH5,6Gprocedurepostvisit(v)post[v]=clockclock=clock+1Thesetimingswillsoontakeonlargersignicance.Meanwhile,youmighthavenoticedfromFigure3.4that:PropertyForanynodesuandv,thetwointervals[pre(u);post(u)]and[pre(v);post(v)]areeitherdisjointoroneiscontainedwithintheother.Why?Because[pre(u);post(u)]isessentiallythetimeduringwhichvertexuwasonthestack.Thelast-in,rst-outbehaviorofastackexplainstherest.3.3Depth-rstsearchindirectedgraphs3.3.1TypesofedgesOurdepth-rstsearchalgorithmcanberunverbatimondirectedgraphs,takingcaretotra-verseedgesonlyintheirprescribeddirections.Figure3.7showsanexampleandthesearchtreethatresultswhenverticesareconsideredinlexicographicorder.Infurtheranalyzingthedirectedcase,ithelpstohaveterminologyforimportantrelation-shipsbetweennodesofatree.Aistherootofthesearchtree;everythingelseisitsdescendant.Similarly,EhasdescendantsF,G,andH,andconversely,isanancestorofthesethreenodes.Thefamilyanalogyiscarriedfurther:CistheparentofD,whichisitschild.Forundirectedgraphswedistinguishedbetweentreeedgesandnontreeedges.Inthedirectedcase,thereisaslightlymoreelaboratetaxonomy:94\nTreeedgesareactuallypartoftheDFSforest.DFStreeForwardedgesleadfromanodetoanonchilddescendantAintheDFStree.ForwardBackedgesleadtoanancestorintheDFStree.Crossedgesleadtoneitherdescendantnorancestor;theyBackBthereforeleadtoanodethathasalreadybeencompletelyexplored(thatis,alreadypostvisited).TreeCDCrossFigure3.7hastwoforwardedges,twobackedges,andtwocrossedges.Canyouspotthem?Ancestoranddescendantrelationships,aswellasedgetypes,canbereadoffdirectlyfrompreandpostnumbers.Becauseofthedepth-rstexplorationstrategy,vertexuisanancestorofvertexvexactlyinthosecaseswhereuisdiscoveredrstandvisdiscoveredduringexplore(u).Thisistosaypre(u)1,denepk(v)=pk 1(p(v))andp1(v)=p(v)(sopk(v)isthekthancestorofv).Eachvertexvofthetreehasanassociatednon-negativeintegerlabell(v).Givealinear-timealgorithmtoupdatethelabelsofalltheverticesinTaccordingtothefollowingrule:lnew(v)=l(pl(v)(v)).3.21.Givealinear-timealgorithmtondanodd-lengthcycleinadirectedgraph.(Hint:Firstsolvethisproblemundertheassumptionthatthegraphisstronglyconnected.)3.22.GiveanefcientalgorithmwhichtakesasinputadirectedgraphG=(V;E),anddetermineswhetherornotthereisavertexs2Vfromwhichallotherverticesarereachable.3.23.GiveanefcientalgorithmthattakesasinputadirectedacyclicgraphG=(V;E),andtwoverticess;t2V,andoutputsthenumberofdifferentdirectedpathsfromstotinG.3.24.Givealinear-timealgorithmforthefollowingtask.Input:AdirectedacyclicgraphGQuestion:DoesGcontainadirectedpaththattoucheseveryvertexexactlyonce?3.25.Youaregivenadirectedgraphinwhicheachnodeu2Vhasanassociatedpricepuwhichisapositiveinteger.Denethearraycostasfollows:foreachu2V,cost[u]=priceofthecheapestnodereachablefromu(includinguitself).104\nForinstance,inthegraphbelow(withpricesshownforeachvertex),thecostvaluesofthenodesA;B;C;D;E;Fare2;1;4;1;4;5,respectively.264ACEBDF315Yourgoalistodesignanalgorithmthatllsintheentirecostarray(i.e.,forallvertices).(a)Givealinear-timealgorithmthatworksfordirectedacyclicgraphs.(Hint:Handletheverticesinaparticularorder.)(b)Extendthistoalinear-timealgorithmthatworksforalldirectedgraphs.(Hint:Recallthe“two-tiered”structureofdirectedgraphs.)3.26.AnEuleriantourinanundirectedgraphisacyclethatisallowedtopassthrougheachvertexmultipletimes,butmustuseeachedgeexactlyonce.ThissimpleconceptwasusedbyEulerin1736tosolvethefamousKonigsbergbridgeproblem,whichlaunchedtheeldofgraphtheory.ThecityofKonigsberg(nowcalledKaliningrad,inwesternRussia)isthemeetingpointoftworiverswithasmallislandinthemiddle.Therearesevenbridgesacrosstherivers,andapopularrecreationalquestionofthetimewastodeterminewhetheritispossibletoperformatourinwhicheachbridgeiscrossedexactlyonce.Eulerformulatedtherelevantinformationasagraphwithfournodes(denotinglandmasses)andsevenedges(denotingbridges),asshownhere.NorthernbankSmallBigislandislandSouthernbankNoticeanunusualfeatureofthisproblem:multipleedgesbetweencertainpairsofnodes.(a)ShowthatanundirectedgraphhasanEuleriantourifandonlyifallitsverticeshaveevendegree.ConcludethatthereisnoEuleriantouroftheKonigsbergbridges.(b)AnEulerianpathisapathwhichuseseachedgeexactlyonce.Canyougiveasimilarif-and-only-ifcharacterizationofwhichundirectedgraphshaveEulerianpaths?(c)Canyougiveananalogofpart(a)fordirectedgraphs?3.27.Twopathsinagrapharecallededge-disjointiftheyhavenoedgesincommon.Showthatinanyundirectedgraph,itispossibletopairuptheverticesofodddegreeandndpathsbetweeneachsuchpairsothatallthesepathsareedge-disjoint.105\n3.28.Inthe2SATproblem,youaregivenasetofclauses,whereeachclauseisthedisjunction(OR)oftwoliterals(aliteralisaBooleanvariableorthenegationofaBooleanvariable).Youarelookingforawaytoassignavaluetrueorfalsetoeachofthevariablessothatallclausesaresatised–thatis,thereisatleastonetrueliteralineachclause.Forexample,here'saninstanceof2SAT:(x1_x2)^(x1_x3)^(x1_x2)^(x3_x4)^(x1_x4):Thisinstancehasasatisfyingassignment:setx1,x2,x3,andx4totrue,false,false,andtrue,respectively.(a)Arethereothersatisfyingtruthassignmentsofthis2SATformula?Ifso,ndthemall.(b)Giveaninstanceof2SATwithfourvariables,andwithnosatisfyingassignment.Thepurposeofthisproblemistoleadyoutoawayofsolving2SATefcientlybyreducingittotheproblemofndingthestronglyconnectedcomponentsofadirectedgraph.GivenaninstanceIof2SATwithnvariablesandmclauses,constructadirectedgraphGI=(V;E)asfollows.GIhas2nnodes,oneforeachvariableanditsnegation.GIhas2medges:foreachclause(_)ofI(where ;areliterals),GIhasanedgefromfromthenegationofto,andonefromthenegationofto.Notethattheclause(_)isequivalenttoeitheroftheimplications)or).Inthissense,GIrecordsallimplicationsinI.(c)Carryoutthisconstructionfortheinstanceof2SATgivenabove,andfortheinstanceyouconstructedin(b).(d)ShowthatifGIhasastronglyconnectedcomponentcontainingbothxandxforsomevariablex,thenIhasnosatisfyingassignment.(e)Nowshowtheconverseof(d):namely,thatifnoneofGI'sstronglyconnectedcomponentscontainbothaliteralanditsnegation,thentheinstanceImustbesatisable.(Hint:As-signvaluestothevariablesasfollows:repeatedlypickasinkstronglyconnectedcomponentofGI.Assignvaluetruetoallliteralsinthesink,assignfalsetotheirnegations,anddeleteallofthese.Showthatthisendsupdiscoveringasatisfyingassignment.)(f)Concludethatthereisalinear-timealgorithmforsolving2SAT.3.29.LetSbeaniteset.AbinaryrelationonSissimplyacollectionRoforderedpairs(x;y)2SS.Forinstance,Smightbeasetofpeople,andeachsuchpair(x;y)2Rmightmean“xknowsy.”Anequivalencerelationisabinaryrelationwhichsatisesthreeproperties:Reexivity:(x;x)2Rforallx2SSymmetry:if(x;y)2Rthen(y;x)2RTransitivity:if(x;y)2Rand(y;z)2Rthen(x;z)2RForinstance,thebinaryrelation“hasthesamebirthdayas”isanequivalencerelation,whereas“isthefatherof”isnot,sinceitviolatesallthreeproperties.ShowthatanequivalencerelationpartitionssetSintodisjointgroupsS1;S2;:::;Sk(inotherwords,S=S1[S2[[SkandSi\Sj=;foralli6=j)suchthat:Anytwomembersofagrouparerelated,thatis,(x;y)2Rforanyx;y2Si,foranyi.Membersofdifferentgroupsarenotrelated,thatis,foralli6=j,forallx2Siandy2Sj,wehave(x;y)62R.(Hint:Representanequivalencerelationbyanundirectedgraph.)106\n3.30.Onpage97,wedenedthebinaryrelation“connected”onthesetofverticesofadirectedgraph.Showthatthisisanequivalencerelation(seeExercise3.29),andconcludethatitpartitionstheverticesintodisjointstronglyconnectedcomponents.3.31.BiconnectedcomponentsLetG=(V;E)beanundirectedgraph.Foranytwoedgese;e02E,we'llsayee0ifeithere=e0orthereisa(simple)cyclecontainingbotheande0.(a)Showthatisanequivalencerelation(recallExercise3.29)ontheedges.TheequivalenceclassesintowhichthisrelationpartitionstheedgesarecalledthebiconnectedcomponentsofG.Abridgeisanedgewhichisinabiconnectedcomponentallbyitself.Aseparatingvertexisavertexwhoseremovaldisconnectsthegraph.(b)Partitiontheedgesofthegraphbelowintobiconnectedcomponents,andidentifythebridgesandseparatingvertices.CFGABDEIHONMLKJNotonlydobiconnectedcomponentspartitiontheedgesofthegraph,theyalsoalmostpartitiontheverticesinthefollowingsense.(c)Associatewitheachbiconnectedcomponentalltheverticesthatareendpointsofitsedges.Showthattheverticescorrespondingtotwodifferentbiconnectedcomponentsareeitherdisjointorintersectinasingleseparatingvertex.(d)Collapseeachbiconnectedcomponentintoasinglemeta-node,andretainindividualnodesforeachseparatingvertex.(Sothereareedgesbetweeneachcomponent-nodeanditssep-aratingvertices.)Showthattheresultinggraphisatree.DFScanbeusedtoidentifythebiconnectedcomponents,bridges,andseparatingverticesofagraphinlineartime.(e)ShowthattherootoftheDFStreeisaseparatingvertexifandonlyifithasmorethanonechildinthetree.(f)Showthatanon-rootvertexvoftheDFStreeisaseparatingvertexifandonlyifithasachildv0noneofwhosedescendants(includingitself)hasabackedgetoaproperancestorofv.(g)Foreachvertexudene:pre(u)low(u)=minpre(w)where(v;w)isabackedgeforsomedescendantvofuShowthattheentirearrayoflowvaluescanbecomputedinlineartime.(h)Showhowtocomputeallseparatingvertices,bridges,andbiconnectedcomponentsofagraphinlineartime.(Hint:Uselowtoidentifyseparatingvertices,andrunanotherDFSwithanextrastackofedgestoremovebiconnectedcomponentsoneatatime.)107\n108\nChapter4Pathsingraphs4.1DistancesDepth-rstsearchreadilyidentiesalltheverticesofagraphthatcanbereachedfromadesignatedstartingpoint.Italsondsexplicitpathstothesevertices,summarizedinitssearchtree(Figure4.1).However,thesepathsmightnotbethemosteconomicalonespossi-ble.Inthegure,vertexCisreachablefromSbytraversingjustoneedge,whiletheDFStreeshowsapathoflength3.Thischapterisaboutalgorithmsforndingshortestpathsingraphs.Pathlengthsallowustotalkquantitativelyabouttheextenttowhichdifferentverticesofagraphareseparatedfromeachother:Thedistancebetweentwonodesisthelengthoftheshortestpathbetweenthem.Togetaconcretefeelforthisnotion,consideraphysicalrealizationofagraphthathasaballforeachvertexandapieceofstringforeachedge.Ifyoulifttheballforvertexshighenough,theotherballsthatgetpulledupalongwithitarepreciselytheverticesreachablefroms.Andtondtheirdistancesfroms,youneedonlymeasurehowfarbelowstheyhang.InFigure4.2forexample,vertexBisatdistance2fromS,andtherearetwoshortestpathstoit.WhenSisheldup,thestringsalongeachofthesepathsbecometaut.Ontheotherhand,edge(D;E)playsnoroleinanyshortestpathandthereforeremainsslack.Figure4.1(a)Asimplegraphand(b)itsdepth-rstsearchtree.(a)(b)SESAADBEDCBC109\nFigure4.2Aphysicalmodelofagraph.SESAACDEDCBBFigure4.3Breadth-rstsearch.procedurebfs(G;s)Input:GraphG=(V;E),directedorundirected;vertexs2VOutput:Forallverticesureachablefroms,dist(u)issettothedistancefromstou.forallu2V:dist(u)=1dist(s)=0Q=[s](queuecontainingjusts)whileQisnotempty:u=eject(Q)foralledges(u;v)2E:ifdist(v)=1:inject(Q;v)dist(v)=dist(u)+14.2Breadth-rstsearchInFigure4.2,theliftingofspartitionsthegraphintolayers:sitself,thenodesatdistance1fromit,thenodesatdistance2fromit,andsoon.Aconvenientwaytocomputedistancesfromstotheotherverticesistoproceedlayerbylayer.Oncewehavepickedoutthenodesatdistance0;1;2;:::;d,theonesatd+1areeasilydetermined:theyarepreciselytheas-yet-unseennodesthatareadjacenttothelayeratdistanced.Thissuggestsaniterativealgorithminwhichtwolayersareactiveatanygiventime:somelayerd,whichhasbeenfullyidentied,andd+1,whichisbeingdiscoveredbyscanningtheneighborsoflayerd.Breadth-rstsearch(BFS)directlyimplementsthissimplereasoning(Figure4.3).Ini-tiallythequeueQconsistsonlyofs,theonenodeatdistance0.Andforeachsubsequentdistanced=1;2;3;:::,thereisapointintimeatwhichQcontainsallthenodesatdistancedandnothingelse.Asthesenodesareprocessed(ejectedoffthefrontofthequeue),theiras-yet-unseenneighborsareinjectedintotheendofthequeue.Let'stryoutthisalgorithmonourearlierexample(Figure4.1)toconrmthatitdoesthe110\nFigure4.4Theresultofbreadth-rstsearchonthegraphofFigure4.1.OrderQueuecontentsofvisitationafterprocessingnodeS[S]S[ACDE]A[CDEB]ACDEC[DEB]D[EB]E[B]B[]Brightthing.IfSisthestartingpointandthenodesareorderedalphabetically,theygetvisitedinthesequenceshowninFigure4.4.Thebreadth-rstsearchtree,ontheright,containstheedgesthroughwhicheachnodeisinitiallydiscovered.UnliketheDFStreewesawearlier,ithasthepropertythatallitspathsfromSaretheshortestpossible.Itisthereforeashortest-pathtree.CorrectnessandefciencyWehavedevelopedthebasicintuitionbehindbreadth-rstsearch.Inordertocheckthatthealgorithmworkscorrectly,weneedtomakesurethatitfaithfullyexecutesthisintuition.Whatweexpect,precisely,isthatForeachd=0;1;2;:::,thereisamomentatwhich(1)allnodesatdistancedfromshavetheirdistancescorrectlyset;(2)allothernodeshavetheirdistancessetto1;and(3)thequeuecontainsexactlythenodesatdistanced.Thishasbeenphrasedwithaninductiveargumentinmind.Wehavealreadydiscussedboththebasecaseandtheinductivestep.Canyoullinthedetails?Theoverallrunningtimeofthisalgorithmislinear,O(jVj+jEj),forexactlythesamereasonsasdepth-rstsearch.Eachvertexisputonthequeueexactlyonce,whenitisrsten-countered,sothereare2jVjqueueoperations.Therestoftheworkisdoneinthealgorithm'sinnermostloop.Overthecourseofexecution,thislooplooksateachedgeonce(indirectedgraphs)ortwice(inundirectedgraphs),andthereforetakesO(jEj)time.NowthatwehavebothBFSandDFSbeforeus:howdotheirexplorationstylescompare?Depth-rstsearchmakesdeepincursionsintoagraph,retreatingonlywhenitrunsoutofnewnodestovisit.Thisstrategygivesitthewonderful,subtle,andextremelyusefulpropertieswesawintheChapter3.ButitalsomeansthatDFScanenduptakingalongandconvolutedroutetoavertexthatisactuallyverycloseby,asinFigure4.1.Breadth-rstsearchmakessuretovisitverticesinincreasingorderoftheirdistancefromthestartingpoint.Thisisabroader,shallowersearch,ratherlikethepropagationofawaveuponwater.AnditisachievedusingalmostexactlythesamecodeasDFS—butwithaqueueinplaceofastack.111\nFigure4.5Edgelengthsoftenmatter.Sacramento95San133FranciscoReno271290409445Bakersfield291112LosLasAngeles275VegasAlsonoticeonestylisticdifferencefromDFS:sinceweareonlyinterestedindistancesfroms,wedonotrestartthesearchinotherconnectedcomponents.Nodesnotreachablefromsaresimplyignored.4.3LengthsonedgesBreadth-rstsearchtreatsalledgesashavingthesamelength.Thisisrarelytrueinap-plicationswhereshortestpathsaretobefound.Forinstance,supposeyouaredrivingfromSanFranciscotoLasVegas,andwanttondthequickestroute.Figure4.5showsthemajorhighwaysyoumightconceivablyuse.Pickingtherightcombinationofthemisashortest-pathprobleminwhichthelengthofeachedge(eachstretchofhighway)isimportant.Forthere-mainderofthischapter,wewilldealwiththismoregeneralscenario,annotatingeveryedgee2Ewithalengthle.Ife=(u;v),wewillsometimesalsowritel(u;v)orluv.Thesele'sdonothavetocorrespondtophysicallengths.Theycoulddenotetime(drivingtimebetweencities)ormoney(costoftakingabus),oranyotherquantitythatwewouldliketoconserve.Infact,therearecasesinwhichweneedtousenegativelengths,butwewillbrieyoverlookthisparticularcomplication.4.4Dijkstra'salgorithm4.4.1Anadaptationofbreadth-rstsearchBreadth-rstsearchndsshortestpathsinanygraphwhoseedgeshaveunitlength.CanweadaptittoamoregeneralgraphG=(V;E)whoseedgelengthslearepositiveintegers?AmoreconvenientgraphHereisasimpletrickforconvertingGintosomethingBFScanhandle:breakG'slongedgesintounit-lengthpieces,byintroducing“dummy”nodes.Figure4.6showsanexampleofthis112\nFigure4.6Breakingedgesintounit-lengthpieces.2BD2BDA132A1CE4CEtransformation.ToconstructthenewgraphG0,Foranyedgee=(u;v)ofE,replaceitbyleedgesoflength1,byaddingle 1dummynodesbetweenuandv.GraphG0containsalltheverticesVthatinterestus,andthedistancesbetweenthemareexactlythesameasinG.Mostimportantly,theedgesofG0allhaveunitlength.Therefore,wecancomputedistancesinGbyrunningBFSonG0.AlarmclocksIfefciencywerenotanissue,wecouldstophere.ButwhenGhasverylongedges,theG0itengendersisthicklypopulatedwithdummynodes,andtheBFSspendsmostofitstimediligentlycomputingdistancestothesenodesthatwedon'tcareaboutatall.Toseethismoreconcretely,considerthegraphsGandG0ofFigure4.7,andimaginethattheBFS,startedatnodesofG0,advancesbyoneunitofdistanceperminute.Fortherst99minutesittediouslyprogressesalongS AandS B,anendlessdesertofdummynodes.Istheresomewaywecansnoozethroughtheseboringphasesandhaveanalarmwakeusupwheneversomethinginterestingishappening—specically,wheneveroneoftherealnodes(fromtheoriginalgraphG)isreached?Wedothisbysettingtwoalarmsattheoutset,onefornodeA,settogooffattimeT=100,andoneforB,attimeT=200.Theseareestimatedtimesofarrival,basedupontheedgescurrentlybeingtraversed.WedozeoffandawakeatT=100tondAhasbeendiscovered.Atthispoint,theestimatedtimeofarrivalforBisadjustedtoT=150andwechangeitsalarmaccordingly.Figure4.7BFSonG0ismostlyuneventful.Thedottedlinesshowsomeearly“wavefronts.”G:G0:AA100S50S200BB113\nMoregenerally,atanygivenmomentthebreadth-rstsearchisadvancingalongcertainedgesofG,andthereisanalarmforeveryendpointnodetowardwhichitismoving,settogooffattheestimatedtimeofarrivalatthatnode.Someofthesemightbeoverestimatesbe-causeBFSmaylaterndshortcuts,asaresultoffuturearrivalselsewhere.Intheprecedingexample,aquickerroutetoBwasrevealeduponarrivalatA.However,nothinginterestingcanpossiblyhappenbeforeanalarmgoesoff.Thesoundingofthenextalarmmustthereforesignalthearrivalofthewavefronttoarealnodeu2VbyBFS.Atthatpoint,BFSmightalsostartadvancingalongsomenewedgesoutofu,andalarmsneedtobesetfortheirendpoints.Thefollowing“alarmclockalgorithm”faithfullysimulatestheexecutionofBFSonG0.Setanalarmclockfornodesattime0.Repeatuntiltherearenomorealarms:SaythenextalarmgoesoffattimeT,fornodeu.Then:–ThedistancefromstouisT.–ForeachneighborvofuinG:Ifthereisnoalarmyetforv,setonefortimeT+l(u;v).Ifv'salarmissetforlaterthanT+l(u;v),thenresetittothisearliertime.Dijkstra'salgorithm.Thealarmclockalgorithmcomputesdistancesinanygraphwithpositiveintegraledgelengths.Itisalmostreadyforuse,exceptthatweneedtosomehowimplementthesystemofalarms.Therightdatastructureforthisjobisapriorityqueue(usuallyimplementedviaaheap),whichmaintainsasetofelements(nodes)withassociatednumerickeyvalues(alarmtimes)andsupportsthefollowingoperations:Insert.Addanewelementtotheset.Decrease-key.Accommodatethedecreaseinkeyvalueofaparticularelement.1Delete-min.Returntheelementwiththesmallestkey,andremoveitfromtheset.Make-queue.Buildapriorityqueueoutofthegivenelements,withthegivenkeyvalues.(Inmanyimplementations,thisissignicantlyfasterthaninsertingtheelementsonebyone.)Thersttwoletussetalarms,andthethirdtellsuswhichalarmisnexttogooff.Puttingthisalltogether,wegetDijkstra'salgorithm(Figure4.8).Inthecode,dist(u)referstothecurrentalarmclocksettingfornodeu.Avalueof1meansthealarmhasn'tsofarbeenset.Thereisalsoaspecialarray,prev,thatholdsonecrucialpieceofinformationforeachnodeu:theidentityofthenodeimmediatelybeforeitontheshortestpathfromstou.Byfollowingtheseback-pointers,wecaneasilyreconstructshortestpaths,andsothisarrayisacompactsummaryofallthepathsfound.Afullexampleofthealgorithm'soperation,alongwiththenalshortest-pathtree,isshowninFigure4.9.Insummary,wecanthinkofDijkstra'salgorithmasjustBFS,exceptitusesapriorityqueueinsteadofaregularqueue,soastoprioritizenodesinawaythattakesedgelengths1Thenamedecrease-keyisstandardbutisalittlemisleading:thepriorityqueuetypicallydoesnotitselfchangekeyvalues.Whatthisprocedurereallydoesistonotifythequeuethatacertainkeyvaluehasbeendecreased.114\nintoaccount.Thisviewpointgivesaconcreteappreciationofhowandwhythealgorithmworks,butthereisamoredirect,moreabstractderivationthatdoesn'tdependuponBFSatall.Wenowstartfromscratchwiththiscomplementaryinterpretation.Figure4.8Dijkstra'sshortest-pathalgorithm.proceduredijkstra(G;l;s)Input:GraphG=(V;E),directedorundirected;positiveedgelengthsfle:e2Eg;vertexs2VOutput:Forallverticesureachablefroms,dist(u)issettothedistancefromstou.forallu2V:dist(u)=1prev(u)=nildist(s)=0H=makequeue(V)(usingdist-valuesaskeys)whileHisnotempty:u=deletemin(H)foralledges(u;v)2E:ifdist(v)>dist(u)+l(u;v):dist(v)=dist(u)+l(u;v)prev(v)=udecreasekey(H;v)115\nFigure4.9AcompleterunofDijkstra'salgorithm,withnodeAasthestartingpoint.Alsoshownaretheassociateddistvaluesandthenalshortest-pathtree.2BD44A:0D:1A131B:4E:13C:22CE52BD44A:0D:613A1B:3E:73C:22CE52BD44A:0D:5A131B:3E:632C:2CE52BD44A:0D:5A131B:3E:632C:2CE52BDA132CE116\nFigure4.10Single-edgeextensionsofknownshortestpaths.Knownregion R!"#u vs$%&'(() 0011,,-../**+4.4.2AnalternativederivationHere'saplanforcomputingshortestpaths:expandoutwardfromthestartingpoints,steadilygrowingtheregionofthegraphtowhichdistancesandshortestpathsareknown.Thisgrowthshouldbeorderly,rstincorporatingtheclosestnodesandthenmovingontothosefurtheraway.Moreprecisely,whenthe“knownregion”issomesubsetofverticesRthatincludess,thenextadditiontoitshouldbethenodeoutsideRthatisclosesttos.Letuscallthisnodev;thequestionis:howdoweidentifyit?Toanswer,consideru,thenodejustbeforevintheshortestpathfromstov:s234567vuSinceweareassumingthatalledgelengthsarepositive,umustbeclosertosthanvis.ThismeansthatuisinR—otherwiseitwouldcontradictv'sstatusastheclosestnodetosoutsideR.So,theshortestpathfromstovissimplyaknownshortestpathextendedbyasingleedge.Buttherewilltypicallybemanysingle-edgeextensionsofthecurrentlyknownshortestpaths(Figure4.10);whichoftheseidentiesv?Theansweris,theshortestoftheseextendedpaths.Because,ifanevenshortersingle-edge-extendedpathexisted,thiswouldoncemorecontradictv'sstatusasthenodeoutsideRclosesttos.So,it'seasytondv:itisthenodeoutsideRforwhichthesmallestvalueofdistance(s;u)+l(u;v)isattained,asurangesoverR.Inotherwords,tryallsingle-edgeextensionsofthecurrentlyknownshortestpaths,ndtheshortestsuchextendedpath,andproclaimitsendpointtobethenextnodeofR.WenowhaveanalgorithmforgrowingRbylookingatextensionsofthecurrentsetofshortestpaths.Someextraefciencycomesfromnoticingthatonanygiveniteration,theonlynewextensionsarethoseinvolvingthenodemostrecentlyaddedtoregionR.Allotherextensionswillhavebeenassessedpreviouslyanddonotneedtoberecomputed.Inthefollowingpseudocode,dist(v)isthelengthofthecurrentlyshortestsingle-edge-extendedpathleadingtov;itis1fornodesnotadjacenttoR.117\nInitializedist(s)to0,otherdist()valuesto1R=fg(the``knownregion'')whileR6=V:Pickthenodev62Rwithsmallestdist()AddvtoRforalledges(v;z)2E:ifdist(z)>dist(v)+l(v;z):dist(z)=dist(v)+l(v;z)IncorporatingpriorityqueueoperationsgivesusbackDijkstra'salgorithm(Figure4.8).Tojustifythisalgorithmformally,wewoulduseaproofbyinduction,aswithbreadth-rstsearch.Here'sanappropriateinductivehypothesis.Attheendofeachiterationofthewhileloop,thefollowingconditionshold:(1)thereisavaluedsuchthatallnodesinRareatdistancedfromsandallnodesoutsideRareatdistancedfroms,and(2)foreverynodeu,thevaluedist(u)isthelengthoftheshortestpathfromstouwhoseintermediatenodesareconstrainedtobeinR(ifnosuchpathexists,thevalueis1).Thebasecaseisstraightforward(withd=0),andthedetailsoftheinductivestepcanbelledinfromtheprecedingdiscussion.4.4.3RunningtimeAtthelevelofabstractionofFigure4.8,Dijkstra'salgorithmisstructurallyidenticaltobreadth-rstsearch.However,itisslowerbecausethepriorityqueueprimitivesarecom-putationallymoredemandingthantheconstant-timeeject'sandinject'sofBFS.SincemakequeuetakesatmostaslongasjVjinsertoperations,wegetatotalofjVjdeleteminandjVj+jEjinsert/decreasekeyoperations.Thetimeneededforthesevariesbyimple-mentation;forinstance,abinaryheapgivesanoverallrunningtimeofO((jVj+jEj)logjVj).118\nWhichheapisbest?TherunningtimeofDijkstra'salgorithmdependsheavilyonthepriorityqueueimplemen-tationused.Herearethetypicalchoices.insert/jVjdeletemin+Implementationdeletemindecreasekey(jVj+jEj)insertArrayO(jVj)O(1)O(jVj2)BinaryheapO(logjVj)O(logjVj)O((jVj+jEj)logjVj)dlogjVjlogjVjlogjVjd-aryheapO()O()O((jVjd+jEj))logdlogdlogdFibonacciheapO(logjVj)O(1)(amortized)O(jVjlogjVj+jEj)Soforinstance,evenanaivearrayimplementationgivesarespectabletimecomplexityofO(jVj2),whereaswithabinaryheapwegetO((jVj+jEj)logjVj).Whichispreferable?Thisdependsonwhetherthegraphissparse(hasfewedges)ordense(haslotsofthem).Forallgraphs,jEjislessthanjVj2.Ifitis(jVj2),thenclearlythearrayimplementationisthefaster.Ontheotherhand,thebinaryheapbecomespreferableassoonasjEjdipsbelowjVj2=logjVj.Thed-aryheapisageneralizationofthebinaryheap(whichcorrespondstod=2)andleadstoarunningtimethatisafunctionofd.TheoptimalchoiceisdjEj=jVj;inotherwords,tooptimizewemustsetthedegreeoftheheaptobeequaltotheaveragedegreeofthegraph.Thisworkswellforbothsparseanddensegraphs.Forverysparsegraphs,inwhichjEj=O(jVj),therunningtimeisO(jVjlogjVj),asgoodaswithabinaryheap.Fordensegraphs,jEj=(jVj2)andtherunningtimeisO(jVj2),asgoodaswithalinkedlist.Finally,forgraphswithintermediatedensityjEj=jVj1+,therunningtimeisO(jEj),linear!ThelastlineinthetablegivesrunningtimesusingasophisticateddatastructurecalledaFibonacciheap.Althoughitsefciencyisimpressive,thisdatastructurerequirescon-siderablymoreworktoimplementthantheothers,andthistendstodampenitsappealinpractice.Wewillsaylittleaboutitexcepttomentionacuriousfeatureofitstimebounds.ItsinsertoperationstakevaryingamountsoftimebutareguaranteedtoaverageO(1)overthecourseofthealgorithm.Insuchsituations(oneofwhichweshallencounterinChapter5)wesaythattheamortizedcostofheapinsert'sisO(1).119\n4.5Priorityqueueimplementations4.5.1ArrayThesimplestimplementationofapriorityqueueisasanunorderedarrayofkeyvaluesforallpotentialelements(theverticesofthegraph,inthecaseofDijkstra'salgorithm).Initially,thesevaluesaresetto1.Aninsertordecreasekeyisfast,becauseitjustinvolvesadjustingakeyvalue,anO(1)operation.Todeletemin,ontheotherhand,requiresalinear-timescanofthelist.4.5.2BinaryheapHereelementsarestoredinacompletebinarytree,namely,abinarytreeinwhicheachlevelislledinfromlefttoright,andmustbefullbeforethenextlevelisstarted.Inaddition,aspecialorderingconstraintisenforced:thekeyvalueofanynodeofthetreeislessthanorequaltothatofitschildren.Inparticular,therefore,therootalwayscontainsthesmallestelement.SeeFigure4.11(a)foranexample.Toinsert,placethenewelementatthebottomofthetree(intherstavailableposition),andletit“bubbleup.”Thatis,ifitissmallerthanitsparent,swapthetwoandrepeat(Figure4.11(b)–(d)).Thenumberofswapsisatmosttheheightofthetree,whichisblog2ncwhentherearenelements.Adecreasekeyissimilar,exceptthattheelementisalreadyinthetree,soweletitbubbleupfromitscurrentposition.Todeletemin,returntherootvalue.Tothenremovethiselementfromtheheap,takethelastnodeinthetree(intherightmostpositioninthebottomrow)andplaceitattheroot.Letit“siftdown”:ifitisbiggerthaneitherchild,swapitwiththesmallerchildandrepeat(Figure4.11(e)–(g)).AgainthistakesO(logn)time.Theregularityofacompletebinarytreemakesiteasytorepresentusinganarray.Thetreenodeshaveanaturalordering:rowbyrow,startingattherootandmovinglefttorightwithineachrow.Iftherearennodes,thisorderingspeciestheirpositions1;2;:::;nwithinthearray.Movingupanddownthetreeiseasilysimulatedonthearray,usingthefactthatnodenumberjhasparentbj=2candchildren2jand2j+1(Exercise4.16).4.5.3d-aryheapAd-aryheapisidenticaltoabinaryheap,exceptthatnodeshavedchildreninsteadofjusttwo.Thisreducestheheightofatreewithnelementsto(logdn)=((logn)=(logd)).Insertsarethereforespeededupbyafactorof(logd).Deleteminoperations,however,takealittlelonger,namelyO(dlogdn)(doyouseewhy?).Thearrayrepresentationofabinaryheapiseasilyextendedtothed-arycase.Thistime,nodenumberjhasparentd(j 1)=deandchildrenf(j 1)d+2;:::;minfn;(j 1)d+d+1gg(Exercise4.16).120\nFigure4.11(a)Abinaryheapwith10elements.Onlythekeyvaluesareshown.(b)–(d)Theintermediate“bubble-up”stepsininsertinganelementwithkey7.(e)–(g)The“sift-down”stepsinadelete-minoperation.(a)3(b)31051051112681112681520131520137(c)3(d)310575117681110681520131215201312(e)(f)12757511106811106815201312152013(g)5(h)5712761110681110128152013152013121\nFigure4.12Dijkstra'salgorithmwillnotworkiftherearenegativeedges.A3S−24B4.6Shortestpathsinthepresenceofnegativeedges4.6.1NegativeedgesDijkstra'salgorithmworksinpartbecausetheshortestpathfromthestartingpointstoanynodevmustpassexclusivelythroughnodesthatarecloserthanv.Thisnolongerholdswhenedgelengthscanbenegative.InFigure4.12,theshortestpathfromStoApassesthroughB,anodethatisfurtheraway!Whatneedstobechangedinordertoaccommodatethisnewcomplication?Toanswerthis,let'stakeaparticularhigh-levelviewofDijkstra'salgorithm.Acrucialinvariantisthatthedistvaluesitmaintainsarealwayseitheroverestimatesorexactlycorrect.Theystartoffat1,andtheonlywaytheyeverchangeisbyupdatingalonganedge:procedureupdate((u;v)2E)dist(v)=minfdist(v);dist(u)+l(u;v)gThisupdateoperationissimplyanexpressionofthefactthatthedistancetovcannotpossiblybemorethanthedistancetou,plusl(u;v).Ithasthefollowingproperties.1.Itgivesthecorrectdistancetovintheparticularcasewhereuisthesecond-lastnodeintheshortestpathtov,anddist(u)iscorrectlyset.2.Itwillnevermakedist(v)toosmall,andinthissenseitissafe.Forinstance,aslewofextraneousupdate'scan'thurt.Thisoperationisextremelyuseful:itisharmless,andifusedcarefully,willcorrectlysetdistances.Infact,Dijkstra'salgorithmcanbethoughtofsimplyasasequenceofupdate's.Weknowthisparticularsequencedoesn'tworkwithnegativeedges,butistheresomeothersequencethatdoes?Togetasenseofthepropertiesthissequencemustpossess,let'spickanodetandlookattheshortestpathtoitfroms.s tu1u2u3ukThispathcanhaveatmostjVj 1edges(doyouseewhy?).Ifthesequenceofupdatesper-formedincludes(s;u1);(u1;u2);(u2;u3);:::;(uk;t),inthatorder(thoughnotnecessarilycon-secutively),thenbytherstpropertythedistancetotwillbecorrectlycomputed.Itdoesn't122\nFigure4.13TheBellman-Fordalgorithmforsingle-sourceshortestpathsingeneralgraphs.procedureshortest-paths(G;l;s)Input:DirectedgraphG=(V;E);edgelengthsfle:e2Egwithnonegativecycles;vertexs2VOutput:Forallverticesureachablefroms,dist(u)issettothedistancefromstou.forallu2V:dist(u)=1prev(u)=nildist(s)=0repeatjVj 1times:foralle2E:update(e)matterwhatotherupdatesoccurontheseedges,orwhathappensintherestofthegraph,becauseupdatesaresafe.Butstill,ifwedon'tknowalltheshortestpathsbeforehand,howcanwebesuretoupdatetherightedgesintherightorder?Hereisaneasysolution:simplyupdatealltheedges,jVj 1times!TheresultingO(jVjjEj)procedureiscalledtheBellman-FordalgorithmandisshowninFigure4.13,withanexampleruninFigure4.14.Anoteaboutimplementation:formanygraphs,themaximumnumberofedgesinanyshortestpathissubstantiallylessthanjVj 1,withtheresultthatfewerroundsofupdatesareneeded.Therefore,itmakessensetoaddanextrachecktotheshortest-pathalgorithm,tomakeitterminateimmediatelyafteranyroundinwhichnoupdateoccurred.4.6.2NegativecyclesIfthelengthofedge(E;B)inFigure4.14werechangedto 4,thegraphwouldhaveanegativecycleA!E!B!A.Insuchsituations,itdoesn'tmakesensetoevenaskaboutshortestpaths.Thereisapathoflength2fromAtoE.Butgoingroundthecycle,there'salsoapathoflength1,andgoingroundmultipletimes,wendpathsoflengths0; 1; 2,andsoon.Theshortest-pathproblemisill-posedingraphswithnegativecycles.Asmightbeex-pected,ouralgorithmfromSection4.6.1worksonlyintheabsenceofsuchcycles.Butwheredidthisassumptionappearinthederivationofthealgorithm?Well,itslippedinwhenweassertedtheexistenceofashortestpathfromstot.Fortunately,itiseasytoautomaticallydetectnegativecyclesandissueawarning.Suchacyclewouldallowustoendlesslyapplyroundsofupdateoperations,reducingdistestimateseverytime.SoinsteadofstoppingafterjVj 1iterations,performoneextraround.Thereisanegativecycleifandonlyifsomedistvalueisreducedduringthisnalround.123\nFigure4.14TheBellman-Fordalgorithmillustratedonasamplegraph.10SA81IterationNode01234567GBS00000000 4A1101055555121B111106555C111111766D1111114109F 2CE111287777F11999999 13G18888888ED 14.7ShortestpathsindagsTherearetwosubclassesofgraphsthatautomaticallyexcludethepossibilityofnegativecy-cles:graphswithoutnegativeedges,andgraphswithoutcycles.Wealreadyknowhowtoefcientlyhandletheformer.Wewillnowseehowthesingle-sourceshortest-pathproblemcanbesolvedinjustlineartimeondirectedacyclicgraphs.Asbefore,weneedtoperformasequenceofupdatesthatincludeseveryshortestpathasasubsequence.ThekeysourceofefciencyisthatInanypathofadag,theverticesappearinincreasinglinearizedorder.Therefore,itisenoughtolinearize(thatis,topologicallysort)thedagbydepth-rstsearch,andthenvisittheverticesinsortedorder,updatingtheedgesoutofeach.ThealgorithmisgiveninFigure4.15.Noticethatourschemedoesn'trequireedgestobepositive.Inparticular,wecanndlongestpathsinadagbythesamealgorithm:justnegatealledgelengths.124\nFigure4.15Asingle-sourceshortest-pathalgorithmfordirectedacyclicgraphs.proceduredag-shortest-paths(G;l;s)Input:DagG=(V;E);edgelengthsfle:e2Eg;vertexs2VOutput:Forallverticesureachablefroms,dist(u)issettothedistancefromstou.forallu2V:dist(u)=1prev(u)=nildist(s)=0LinearizeGforeachu2V,inlinearizedorder:foralledges(u;v)2E:update(u;v)125\nExercises4.1.SupposeDijkstra'salgorithmisrunonthefollowinggraph,startingatnodeA.121ABCD4866214E5F1G1H(a)Drawatableshowingtheintermediatedistancevaluesofallthenodesateachiterationofthealgorithm.(b)Showthenalshortest-pathtree.4.2.Justlikethepreviousproblem,butthistimewiththeBellman-Fordalgorithm.A−242CD7B61S3−45F−261−2GH3−11EI4.3.Squares.DesignandanalyzeanalgorithmthattakesasinputanundirectedgraphG=(V;E)anddetermineswhetherGcontainsasimplecycle(thatis,acyclewhichdoesn'tintersectitself)oflengthfour.ItsrunningtimeshouldbeatmostO(jVj3).Youmayassumethattheinputgraphisrepresentedeitherasanadjacencymatrixorwithadjacencylists,whichevermakesyouralgorithmsimpler.4.4.Here'saproposalforhowtondthelengthoftheshortestcycleinanundirectedgraphwithunitedgelengths.Whenabackedge,say(v;w),isencounteredduringadepth-rstsearch,itformsacyclewiththetreeedgesfromwtov.Thelengthofthecycleislevel[v] level[w]+1,wherethelevelofavertexisitsdistanceintheDFStreefromtherootvertex.Thissuggeststhefollowingalgorithm:Doadepth-rstsearch,keepingtrackofthelevelofeachvertex.Eachtimeabackedgeisencountered,computethecyclelengthandsaveitifitissmallerthantheshortestonepreviouslyseen.Showthatthisstrategydoesnotalwaysworkbyprovidingacounterexampleaswellasabrief(oneortwosentence)explanation.4.5.Oftentherearemultipleshortestpathsbetweentwonodesofagraph.Givealinear-timealgo-rithmforthefollowingtask.126\nInput:UndirectedgraphG=(V;E)withunitedgelengths;nodesu;v2V.Output:Thenumberofdistinctshortestpathsfromutov.4.6.ProvethatforthearrayprevcomputedbyDijkstra'salgorithm,theedgesfu;prev[u]g(forallu2V)formatree.4.7.YouaregivenadirectedgraphG=(V;E)with(possiblynegative)weightededges,alongwithaspecicnodes2VandatreeT=(V;E0);E0E.GiveanalgorithmthatcheckswhetherTisashortest-pathtreeforGwithstartingpoints.Youralgorithmshouldruninlineartime.4.8.ProfessorF.Lakesuggeststhefollowingalgorithmforndingtheshortestpathfromnodestonodetinadirectedgraphwithsomenegativeedges:addalargeconstanttoeachedgeweightsothatalltheweightsbecomepositive,thenrunDijkstra'salgorithmstartingatnodes,andreturntheshortestpathfoundtonodet.Isthisavalidmethod?Eitherprovethatitworkscorrectly,orgiveacounterexample.4.9.Consideradirectedgraphinwhichtheonlynegativeedgesarethosethatleaves;allotheredgesarepositive.CanDijkstra'salgorithm,startedats,failonsuchagraph?Proveyouranswer.4.10.Youaregivenadirectedgraphwith(possiblynegative)weightededges,inwhichtheshortestpathbetweenanytwoverticesisguaranteedtohaveatmostkedges.GiveanalgorithmthatndstheshortestpathbetweentwoverticesuandvinO(kjEj)time.4.11.Giveanalgorithmthattakesasinputadirectedgraphwithpositiveedgelengths,andreturnsthelengthoftheshortestcycleinthegraph(ifthegraphisacyclic,itshouldsayso).YouralgorithmshouldtaketimeatmostO(jVj3).4.12.GiveanO(jVj2)algorithmforthefollowingtask.Input:AnundirectedgraphG=(V;E);edgelengthsle>0;anedgee2E.Output:Thelengthoftheshortestcyclecontainingedgee.4.13.Youaregivenasetofcities,alongwiththepatternofhighwaysbetweenthem,intheformofanundirectedgraphG=(V;E).Eachstretchofhighwaye2Econnectstwoofthecities,andyouknowitslengthinmiles,le.Youwanttogetfromcitystocityt.There'soneproblem:yourcarcanonlyholdenoughgastocoverLmiles.Therearegasstationsineachcity,butnotbetweencities.Therefore,youcanonlytakearouteifeveryoneofitsedgeshaslengthleL.(a)Giventhelimitationonyourcar'sfueltankcapacity,showhowtodetermineinlineartimewhetherthereisafeasibleroutefromstot.(b)Youarenowplanningtobuyanewcar,andyouwanttoknowtheminimumfueltankcapacitythatisneededtotravelfromstot.GiveanO((jVj+jEj)logjVj)algorithmtodeterminethis.4.14.YouaregivenastronglyconnecteddirectedgraphG=(V;E)withpositiveedgeweightsalongwithaparticularnodev02V.Giveanefcientalgorithmforndingshortestpathsbetweenallpairsofnodes,withtheonerestrictionthatthesepathsmustallpassthroughv0.4.15.Shortestpathsarenotalwaysunique:sometimestherearetwoormoredifferentpathswiththeminimumpossiblelength.ShowhowtosolvethefollowingprobleminO((jVj+jEj)logjVj)time.Input:AnundirectedgraphG=(V;E);edgelengthsle>0;startingvertexs2V.Output:ABooleanarrayusp[]:foreachnodeu,theentryusp[u]shouldbetrueifandonlyifthereisauniqueshortestpathfromstou.(Note:usp[s]=true.)127\nFigure4.16Operationsonabinaryheap.procedureinsert(h;x)bubbleup(h;x;jhj+1)proceduredecreasekey(h;x)bubbleup(h;x;h 1(x))functiondeletemin(h)ifjhj=0:returnnullelse:x=h(1)siftdown(h;h(jhj);1)returnxfunctionmakeheap(S)h=emptyarrayofsizejSjforx2S:h(jhj+1)=xfori=jSjdownto1:siftdown(h;h(i);i)returnhprocedurebubbleup(h;x;i)(placeelementxinpositioniofh,andletitbubbleup)p=di=2ewhilei6=1andkey(h(p))>key(x):h(i)=h(p);i=p;p=di=2eh(i)=xproceduresiftdown(h;x;i)(placeelementxinpositioniofh,andletitsiftdown)c=minchild(h;i)whilec6=0andkey(h(c))jhj:return0(nochildren)else:returnargminfkey(h(j)):2ijminfjhj;2i+1gg128\n4.16.Section4.5.2describesawayofstoringacompletebinarytreeofnnodesinanarrayindexedby1;2;:::;n.(a)Considerthenodeatpositionjofthearray.Showthatitsparentisatpositionbj=2canditschildrenareat2jand2j+1(ifthesenumbersaren).(b)Whatthecorrespondingindiceswhenacompleted-arytreeisstoredinanarray?Figure4.16showspseudocodeforabinaryheap,modeledonanexpositionbyR.E.Tarjan.2Theheapisstoredasanarrayh,whichisassumedtosupporttwoconstant-timeoperations:jhj,whichreturnsthenumberofelementscurrentlyinthearray;h 1,whichreturnsthepositionofanelementwithinthearray.Thelattercanalwaysbeachievedbymaintainingthevaluesofh 1asanauxiliaryarray.(c)ShowthatthemakeheapproceduretakesO(n)timewhencalledonasetofnelements.Whatistheworst-caseinput?(Hint:StartbyshowingthattherunningtimeisatmostPnlog(n=i).)i=1(a)Whatneedstobechangedtoadaptthispseudocodetod-aryheaps?4.17.SupposewewanttorunDijkstra'salgorithmonagraphwhoseedgeweightsareintegersintherange0;1;:::;W,whereWisarelativelysmallnumber.(a)ShowhowDijkstra'salgorithmcanbemadetorunintimeO(WjVj+jEj).(b)ShowanalternativeimplementationthattakestimejustO((jVj+jEj)logW).4.18.Incaseswherethereareseveraldifferentshortestpathsbetweentwonodes(andedgeshavevaryinglengths),themostconvenientofthesepathsisoftentheonewithfewestedges.Forinstance,ifnodesrepresentcitiesandedgelengthsrepresentcostsofyingbetweencities,theremightbemanywaystogetfromcitystocitytwhichallhavethesamecost.Themostconvenientofthesealternativesistheonewhichinvolvesthefeweststopovers.Accordingly,foraspecicstartingnodes,denebest[u]=minimumnumberofedgesinashortestpathfromstou:Intheexamplebelow,thebestvaluesfornodesS;A;B;C;D;E;Fare0;1;1;1;2;2;3,respectively.AD2211SCF432211BEGiveanefcientalgorithmforthefollowingproblem.Input:GraphG=(V;E);positiveedgelengthsle;startingnodes2V.Output:Thevaluesofbest[u]shouldbesetforallnodesu2V.2See:R.E.Tarjan,DataStructuresandNetworkAlgorithms,SocietyforIndustrialandAppliedMathematics,1983.129\n4.19.Generalizedshortest-pathsproblem.InInternetrouting,therearedelaysonlinesbutalso,moresignicantly,delaysatrouters.Thismotivatesageneralizedshortest-pathsproblem.Supposethatinadditiontohavingedgelengthsfle:e2Eg,agraphalsohasvertexcostsfcv:v2Vg.Nowdenethecostofapathtobethesumofitsedgelengths,plusthecostsofallverticesonthepath(includingtheendpoints).Giveanefcientalgorithmforthefollowingproblem.Input:AdirectedgraphG=(V;E);positiveedgelengthsleandpositivevertexcostscv;astartingvertexs2V.Output:Anarraycost[]suchthatforeveryvertexu,cost[u]istheleastcostofanypathfromstou(i.e.,thecostofthecheapestpath),underthedenitionabove.Noticethatcost[s]=cs.4.20.ThereisanetworkofroadsG=(V;E)connectingasetofcitiesV.EachroadinEhasanassociatedlengthle.Thereisaproposaltoaddonenewroadtothisnetwork,andthereisalistE0ofpairsofcitiesbetweenwhichthenewroadcanbebuilt.Eachsuchpotentialroade02E0hasanassociatedlength.Asadesignerforthepublicworksdepartmentyouareaskedtodeterminetheroade02E0whoseadditiontotheexistingnetworkGwouldresultinthemaximumdecreaseinthedrivingdistancebetweentwoxedcitiessandtinthenetwork.Giveanefcientalgorithmforsolvingthisproblem.4.21.Shortestpathalgorithmscanbeappliedincurrencytrading.Letc1;c2;:::;cnbevariouscur-rencies;forinstance,c1mightbedollars,c2pounds,andc3lire.Foranytwocurrenciesciandcj,thereisanexchangerateri;j;thismeansthatyoucanpurchaseri;junitsofcurrencycjinexchangeforoneunitofci.Theseexchangeratessatisfytheconditionthatri;jrj;i<1,sothatifyoustartwithaunitofcurrencyci,changeitintocurrencycjandthenconvertbacktocurrencyci,youendupwithlessthanoneunitofcurrencyci(thedifferenceisthecostofthetransaction).(a)Giveanefcientalgorithmforthefollowingproblem:Givenasetofexchangeratesri;j,andtwocurrenciessandt,ndthemostadvantageoussequenceofcurrencyexchangesforconvertingcurrencysintocurrencyt.Towardthisgoal,youshouldrepresentthecurrenciesandratesbyagraphwhoseedgelengthsarerealnumbers.Theexchangeratesareupdatedfrequently,reectingthedemandandsupplyofthevariouscurrencies.Occasionallytheexchangeratessatisfythefollowingproperty:thereisasequenceofcurrenciesci1;ci2;:::;ciksuchthatri1;i2ri2;i3rik 1;ikrik;i1>1.Thismeansthatbystartingwithaunitofcurrencyci1andthensuccessivelyconvertingittocurrenciesci2;ci3;:::;cik,andnallybacktoci1,youwouldendupwithmorethanoneunitofcurrencyci1.Suchanomalieslastonlyafractionofaminuteonthecurrencyexchange,buttheyprovideanopportunityforrisk-freeprots.(b)Giveanefcientalgorithmfordetectingthepresenceofsuchananomaly.Usethegraphrepresentationyoufoundabove.4.22.Thetrampsteamerproblem.YouaretheownerofasteamshipthatcanplybetweenagroupofportcitiesV.Youmakemoneyateachport:avisittocityiearnsyouaprotofpidollars.Meanwhile,thetransportationcostfromportitoportjiscij>0.Youwanttondacyclicrouteinwhichtheratioofprottocostismaximized.Tothisend,consideradirectedgraphG=(V;E)whosenodesareports,andwhichhasedgesbetweeneachpairofports.ForanycycleCinthisgraph,theprot-to-costratioisP(i;j)2Cpjr(C)=P:(i;j)2Ccij130\nLetrbethemaximumratioachievablebyasimplecycle.Onewaytodeterminerisbybinarysearch:byrstguessingsomeratior,andthentestingwhetheritistoolargeortoosmall.Consideranypositiver>0.Giveeachedge(i;j)aweightofwij=rcij pj.(a)Showthatifthereisacycleofnegativeweight,thenrr.(c)Giveanefcientalgorithmthattakesasinputadesiredaccuracy>0andreturnsasimplecycleCforwhichr(C)r .JustifythecorrectnessofyouralgorithmandanalyzeitsrunningtimeintermsofjVj,,andR=max(i;j)2E(pj=cij).131\n132\nChapter5GreedyalgorithmsAgamelikechesscanbewononlybythinkingahead:aplayerwhoisfocusedentirelyonimmediateadvantageiseasytodefeat.Butinmanyothergames,suchasScrabble,itispossibletodoquitewellbysimplymakingwhichevermoveseemsbestatthemomentandnotworryingtoomuchaboutfutureconsequences.Thissortofmyopicbehavioriseasyandconvenient,makingitanattractivealgorithmicstrategy.Greedyalgorithmsbuildupasolutionpiecebypiece,alwayschoosingthenextpiecethatoffersthemostobviousandimmediatebenet.Althoughsuchanapproachcanbedisastrousforsomecomputationaltasks,therearemanyforwhichitisoptimal.Ourrstexampleisthatofminimumspanningtrees.5.1MinimumspanningtreesSupposeyouareaskedtonetworkacollectionofcomputersbylinkingselectedpairsofthem.Thistranslatesintoagraphprobleminwhichnodesarecomputers,undirectededgesarepotentiallinks,andthegoalistopickenoughoftheseedgesthatthenodesareconnected.Butthisisnotall;eachlinkalsohasamaintenancecost,reectedinthatedge'sweight.Whatisthecheapestpossiblenetwork?1ACE434425B4D6FOneimmediateobservationisthattheoptimalsetofedgescannotcontainacycle,becauseremovinganedgefromthiscyclewouldreducethecostwithoutcompromisingconnectivity:Property1Removingacycleedgecannotdisconnectagraph.Sothesolutionmustbeconnectedandacyclic:undirectedgraphsofthiskindarecalledtrees.Theparticulartreewewantistheonewithminimumtotalweight,knownastheminimumspanningtree.Hereisitsformaldenition.Input:AnundirectedgraphG=(V;E);edgeweightswe.133\nOutput:AtreeT=(V;E0),withE0E,thatminimizesXweight(T)=we:e2E0Intheprecedingexample,theminimumspanningtreehasacostof16:1ACE425B4DFHowever,thisisnottheonlyoptimalsolution.Canyouspotanother?5.1.1AgreedyapproachKruskal'sminimumspanningtreealgorithmstartswiththeemptygraphandthenselectsedgesfromEaccordingtothefollowingrule.Repeatedlyaddthenextlightestedgethatdoesn'tproduceacycle.Inotherwords,itconstructsthetreeedgebyedgeand,apartfromtakingcaretoavoidcycles,simplypickswhicheveredgeischeapestatthemoment.Thisisagreedyalgorithm:everydecisionitmakesistheonewiththemostobviousimmediateadvantage.Figure5.1showsanexample.Westartwithanemptygraphandthenattempttoaddedgesinincreasingorderofweight(tiesarebrokenarbitrarily):B C;C D;B D;C F;D F;E F;A D;A B;C E;A C:Thersttwosucceed,butthethird,B D,wouldproduceacycleifadded.Soweignoreitandmovealong.Thenalresultisatreewithcost14,theminimumpossible.ThecorrectnessofKruskal'smethodfollowsfromacertaincutproperty,whichisgeneralenoughtoalsojustifyawholeslewofotherminimumspanningtreealgorithms.Figure5.1TheminimumspanningtreefoundbyKruskal'salgorithm.65ACEACE415234BDFBDF24134\nTreesAtreeisanundirectedgraphthatisconnectedandacyclic.Muchofwhatmakestreessousefulisthesimplicityoftheirstructure.Forinstance,Property2Atreeonnnodeshasn 1edges.Thiscanbeseenbybuildingthetreeoneedgeatatime,startingfromanemptygraph.Initiallyeachofthennodesisdisconnectedfromtheothers,inaconnectedcomponentbyitself.Asedgesareadded,thesecomponentsmerge.Sinceeachedgeunitestwodifferentcomponents,exactlyn 1edgesareaddedbythetimethetreeisfullyformed.Inalittlemoredetail:Whenaparticularedgefu;vgcomesup,wecanbesurethatuandvlieinseparateconnectedcomponents,forotherwisetherewouldalreadybeapathbetweenthemandthisedgewouldcreateacycle.Addingtheedgethenmergesthesetwocomponents,therebyreducingthetotalnumberofconnectedcomponentsbyone.Overthecourseofthisincrementalprocess,thenumberofcomponentsdecreasesfromntoone,meaningthatn 1edgesmusthavebeenaddedalongtheway.Theconverseisalsotrue.Property3Anyconnected,undirectedgraphG=(V;E)withjEj=jVj 1isatree.WejustneedtoshowthatGisacyclic.Onewaytodothisistorunthefollowingiterativeprocedureonit:whilethegraphcontainsacycle,removeoneedgefromthiscycle.TheprocessterminateswithsomegraphG0=(V;E0);E0E,whichisacyclicand,byProperty1(frompage133),isalsoconnected.ThereforeG0isatree,whereuponjE0j=jVj 1byProperty2.SoE0=E,noedgeswereremoved,andGwasacyclictostartwith.Inotherwords,wecantellwhetheraconnectedgraphisatreejustbycountinghowmanyedgesithas.Here'sanothercharacterization.Property4Anundirectedgraphisatreeifandonlyifthereisauniquepathbetweenanypairofnodes.Inatree,anytwonodescanonlyhaveonepathbetweenthem;forifthereweretwopaths,theunionofthesepathswouldcontainacycle.Ontheotherhand,ifagraphhasapathbetweenanytwonodes,thenitisconnected.Ifthesepathsareunique,thenthegraphisalsoacyclic(sinceacyclehastwopathsbetweenanypairofnodes).135\nFigure5.2T[feg.Theadditionofe(dotted)toT(solidlines)producesacycle.Thiscyclemustcontainatleastoneotheredge,shownherease0,acrossthecut(S;V S).SV Se 0 e5.1.2ThecutpropertySaythatintheprocessofbuildingaminimumspanningtree(MST),wehavealreadychosensomeedgesandaresofarontherighttrack.Whichedgeshouldweaddnext?Thefollowinglemmagivesusalotofexibilityinourchoice.CutpropertySupposeedgesXarepartofaminimumspanningtreeofG=(V;E).PickanysubsetofnodesSforwhichXdoesnotcrossbetweenSandV S,andletebethelightestedgeacrossthispartition.ThenX[fegispartofsomeMST.Acutisanypartitionoftheverticesintotwogroups,SandV S.Whatthispropertysaysisthatitisalwayssafetoaddthelightestedgeacrossanycut(thatis,betweenavertexinSandoneinV S),providedXhasnoedgesacrossthecut.Let'sseewhythisholds.EdgesXarepartofsomeMSTT;ifthenewedgeealsohappenstobepartofT,thenthereisnothingtoprove.SoassumeeisnotinT.WewillconstructadifferentMSTT0containingX[fegbyalteringTslightly,changingjustoneofitsedges.AddedgeetoT.SinceTisconnected,italreadyhasapathbetweentheendpointsofe,soaddingecreatesacycle.Thiscyclemustalsohavesomeotheredgee0acrossthecut(S;V S)(Figure8.3).Ifwenowremovethisedge,weareleftwithT0=T[feg fe0g,whichwewillshowtobeatree.T0isconnectedbyProperty1,sincee0isacycleedge.AndithasthesamenumberofedgesasT;sobyProperties2and3,itisalsoatree.Moreover,T0isaminimumspanningtree.CompareitsweighttothatofT:weight(T0)=weight(T)+w(e) w(e0):Botheande0crossbetweenSandV S,andeisspecicallythelightestedgeofthistype.Thereforew(e)w(e0),andweight(T0)weight(T).SinceTisanMST,itmustbethecasethatweight(T0)=weight(T)andthatT0isalsoanMST.Figure5.3showsanexampleofthecutproperty.Whichedgeise0?5.1.3Kruskal'salgorithmWearereadytojustifyKruskal'salgorithm.Atanygivenmoment,theedgesithasalreadychosenformapartialsolution,acollectionofconnectedcomponentseachofwhichhasatree136\nFigure5.3Thecutpropertyatwork.(a)Anundirectedgraph.(b)SetXhasthreeedges,andispartoftheMSTTontheright.(c)IfS=fA;B;C;Dg,thenoneoftheminimum-weightedgesacrossthecut(S;V S)ise=fD;Eg.X[fegispartofMSTT0,shownontheright.(a)13ACE22123B1D4F(b)ACEACEEdgesX:MSTT:BDFBDF(c)ACEACEThecut:eMSTT0:BDFBDFSV Sstructure.Thenextedgeetobeaddedconnectstwoofthesecomponents;callthemT1andT2.Sinceeisthelightestedgethatdoesn'tproduceacycle,itiscertaintobethelightestedgebetweenT1andV T1andthereforesatisesthecutproperty.Nowwellinsomeimplementationdetails.Ateachstage,thealgorithmchoosesanedgetoaddtoitscurrentpartialsolution.Todoso,itneedstotesteachcandidateedgeu vtoseewhethertheendpointsuandvlieindifferentcomponents;otherwisetheedgeproducesacycle.Andonceanedgeischosen,thecorrespondingcomponentsneedtobemerged.Whatkindofdatastructuresupportssuchoperations?Wewillmodelthealgorithm'sstateasacollectionofdisjointsets,eachofwhichcontainsthenodesofaparticularcomponent.Initiallyeachnodeisinacomponentbyitself:makeset(x):createasingletonsetcontainingjustx.Werepeatedlytestpairsofnodestoseeiftheybelongtothesameset.find(x):towhichsetdoesxbelong?Andwheneverweaddanedge,wearemergingtwocomponents.union(x;y):mergethesetscontainingxandy.ThenalalgorithmisshowninFigure5.4.ItusesjVjmakeset,2jEjfind,andjVj 1unionoperations.137\nFigure5.4Kruskal'sminimumspanningtreealgorithm.procedurekruskal(G;w)Input:AconnectedundirectedgraphG=(V;E)withedgeweightsweOutput:AminimumspanningtreedefinedbytheedgesXforallu2V:makeset(u)X=fgSorttheedgesEbyweightforalledgesfu;vg2E,inincreasingorderofweight:iffind(u)6=find(v):addedgefu;vgtoXunion(u;v)5.1.4AdatastructurefordisjointsetsUnionbyrankOnewaytostoreasetisasadirectedtree(Figure5.5).Nodesofthetreeareelementsoftheset,arrangedinnoparticularorder,andeachhasparentpointersthateventuallyleaduptotherootofthetree.Thisrootelementisaconvenientrepresentative,orname,fortheset.Itisdistinguishedfromtheotherelementsbythefactthatitsparentpointerisaself-loop.Inadditiontoaparentpointer,eachnodealsohasarankthat,forthetimebeing,shouldbeinterpretedastheheightofthesubtreehangingfromthatnode.proceduremakeset(x)(x)=xrank(x)=0functionfind(x)whilex6=(x):x=(x)Figure5.5Adirected-treerepresentationoftwosetsfB;EgandfA;C;D;F;G;Hg.EHBCDFGA138\nreturnxAscanbeexpected,makesetisaconstant-timeoperation.Ontheotherhand,findfollowsparentpointerstotherootofthetreeandthereforetakestimeproportionaltotheheightofthetree.Thetreeactuallygetsbuiltviathethirdoperation,union,andsowemustmakesurethatthisprocedurekeepstreesshallow.Mergingtwosetsiseasy:maketherootofonepointtotherootoftheother.Butwehaveachoicehere.Iftherepresentatives(roots)ofthesetsarerxandry,dowemakerxpointtoryortheotherwayaround?Sincetreeheightisthemainimpedimenttocomputationalefciency,agoodstrategyistomaketherootoftheshortertreepointtotherootofthetallertree.Thisway,theoverallheightincreasesonlyifthetwotreesbeingmergedareequallytall.Insteadofexplicitlycomputingheightsoftrees,wewillusetheranknumbersoftheirrootnodes—whichiswhythisschemeiscalledunionbyrank.procedureunion(x;y)rx=find(x)ry=find(y)ifrx=ry:returnifrank(rx)>rank(ry):(ry)=rxelse:(rx)=ryifrank(rx)=rank(ry):rank(ry)=rank(ry)+1SeeFigure5.6foranexample.Bydesign,therankofanodeisexactlytheheightofthesubtreerootedatthatnode.Thismeans,forinstance,thatasyoumoveupapathtowardarootnode,therankvaluesalongthewayarestrictlyincreasing.Property1Foranyx,rank(x)kisboundedbynnn++:2k+12k+22kThereforethetotalmoneygiventonodesinthisparticularintervalisatmostndollars,andsincetherearelognintervals,thetotalmoneydisbursedtoallnodesisnlogn.Now,thetimetakenbyaspecicfindissimplythenumberofpointersfollowed.Considertheascendingrankvaluesalongthischainofnodesuptotheroot.Nodesxonthechainfallintotwocategories:eithertherankof(x)isinahigherintervalthantherankofx,orelseitliesinthesameinterval.Thereareatmostlognnodesofthersttype(doyouseewhy?),sotheworkdoneonthemtakesO(logn)time.Theremainingnodes—whoseparents'ranksareinthesameintervalastheirs—havetopayadollaroutoftheirpocketmoneyfortheirprocessingtime.Thisonlyworksiftheinitialallowanceofeachnodexisenoughtocoverallofitspaymentsinthesequenceoffindoperations.Here'sthecrucialobservation:eachtimexpaysadollar,itsparentchangestooneofhigherrank.Therefore,ifx'srankliesintheintervalfk+1;:::;2kg,ithastopayatmost2kdollarsbeforeitsparent'srankisinahigherinterval;whereuponitneverhastopayagain.142\nArandomizedalgorithmforminimumcutWehavealreadyseenthatspanningtreesandcutsareintimatelyrelated.Hereisanotherconnection.Let'sremovethelastedgethatKruskal'salgorithmaddstothespanningtree;thisbreaksthetreeintotwocomponents,thusdeningacut(S;S)inthegraph.Whatcanwesayaboutthiscut?Supposethegraphwewereworkingwithwasunweighted,andthatitsedgeswereordereduniformlyatrandomforKruskal'salgorithmtoprocessthem.Hereisaremarkablefact:withprobabilityatleast1=n2,(S;S)istheminimumcutinthegraph,wherethesizeofacut(S;S)isthenumberofedgescrossingbetweenSandS.ThismeansthatrepeatingtheprocessO(n2)timesandoutputtingthesmallestcutfoundyieldstheminimumcutinGwithhighprobability:anO(mn2logn)algorithmforunweightedminimumcuts.SomefurthertuninggivestheO(n2logn)minimumcutalgorithm,inventedbyDavidKarger,whichisthefastestknownalgorithmforthisimportantproblem.Soletusseewhythecutfoundineachiterationistheminimumcutwithprobabilityatleast1=n2.AtanystageofKruskal'salgorithm,thevertexsetVispartitionedintoconnectedcomponents.Theonlyedgeseligibletobeaddedtothetreehavetheirtwoendpointsindistinctcomponents.ThenumberofedgesincidenttoeachcomponentmustbeatleastC,thesizeoftheminimumcutinG(sincewecouldconsideracutthatseparatedthiscomponentfromtherestofthegraph).Soiftherearekcomponentsinthegraph,thenumberofeligibleedgesisatleastkC=2(eachofthekcomponentshasatleastCedgesleadingoutofit,andweneedtocompensateforthedouble-countingofeachedge).Sincetheedgeswererandomlyordered,thechancethatthenexteligibleedgeinthelistisfromtheminimumcutisatmostC=(kC=2)=2=k.Thus,withprobabilityatleast1 2=k=(k 2)=k,thechoiceleavestheminimumcutintact.ButnowthechancethatKruskal'salgorithmleavestheminimumcutintactallthewayuptothechoiceofthelastspanningtreeedgeisatleastn 2n 3n 4211=:nn 1n 243n(n 1)5.1.5Prim'salgorithmLet'sreturntoourdiscussionofminimumspanningtreealgorithms.Whatthecutpropertytellsusinmostgeneraltermsisthatanyalgorithmconformingtothefollowinggreedyschemaisguaranteedtowork.X=fg(edgespickedsofar)repeatuntiljXj=jVj 1:pickasetSVforwhichXhasnoedgesbetweenSandV Slete2Ebetheminimum-weightedgebetweenSandV SX=X[fegApopularalternativetoKruskal'salgorithmisPrim's,inwhichtheintermediatesetofedgesXalwaysformsasubtree,andSischosentobethesetofthistree'svertices.Oneachiteration,thesubtreedenedbyXgrowsbyoneedge,namely,thelightestedgebetweenavertexinSandavertexoutsideS(Figure5.8).WecanequivalentlythinkofSas143\nFigure5.8Prim'salgorithm:theedgesXformatree,andSconsistsofitsvertices.SV SX e ""##""## !!$$%%growingtoincludethevertexv62Sofsmallestcost:cost(v)=minw(u;v):u2SThisisstronglyreminiscentofDijkstra'salgorithm,andinfactthepseudocode(Figure5.9)isalmostidentical.Theonlydifferenceisinthekeyvaluesbywhichthepriorityqueueisordered.InPrim'salgorithm,thevalueofanodeistheweightofthelightestincomingedgefromsetS,whereasinDijkstra'sitisthelengthofanentirepathtothatnodefromthestartingpoint.Nonetheless,thetwoalgorithmsaresimilarenoughthattheyhavethesamerunningtime,whichdependsontheparticularpriorityqueueimplementation.Figure5.9showsPrim'salgorithmatwork,onasmallsix-nodegraph.NoticehowthenalMSTiscompletelyspeciedbytheprevarray.144\nFigure5.9Top:Prim'sminimumspanningtreealgorithm.Below:AnillustrationofPrim'salgorithm,startingatnodeA.Alsoshownareatableofcost/prevvalues,andthenalMST.procedureprim(G;w)Input:AconnectedundirectedgraphG=(V;E)withedgeweightsweOutput:Aminimumspanningtreedefinedbythearrayprevforallu2V:cost(u)=1prev(u)=nilPickanyinitialnodeu0cost(u0)=0H=makequeue(V)(priorityqueue,usingcost-valuesaskeys)whileHisnotempty:v=deletemin(H)foreachfv;zg2E:ifcost(z)>w(v;z):cost(z)=w(v;z)prev(z)=vdecreasekey(H;z)65ACEACE415234BDFBDF24SetSABCDEFfg0=nil1=nil1=nil1=nil1=nil1=nilA5=A6=A4=A1=nil1=nilA;D2=D2=D1=nil4=DA;D;B1=B1=nil4=DA;D;B;C5=C3=CA;D;B;C;F4=F145\n5.2HuffmanencodingIntheMP3audiocompressionscheme,asoundsignalisencodedinthreesteps.1.Itisdigitizedbysamplingatregularintervals,yieldingasequenceofrealnumberss1;s2;:::;sT.Forinstance,atarateof44;100samplespersecond,a50-minutesymphonywouldcorrespondtoT=506044;100130millionmeasurements.12.Eachreal-valuedsamplestisquantized:approximatedbyanearbynumberfromaniteset .Thissetiscarefullychosentoexploithumanperceptuallimitations,withtheintentionthattheapproximatingsequenceisindistinguishablefroms1;s2;:::;sTbythehumanear.3.TheresultingstringoflengthToveralphabet isencodedinbinary.ItisinthelaststepthatHuffmanencodingisused.Tounderstanditsrole,let'slookatatoyexampleinwhichTis130millionandthealphabet consistsofjustfourvalues,denotedbythesymbolsA;B;C;D.Whatisthemosteconomicalwaytowritethislongstringinbinary?Theobviouschoiceistouse2bitspersymbol—saycodeword00forA,01forB,10forC,and11forD—inwhichcase260megabitsareneededintotal.Cantherepossiblybeabetterencodingthanthis?Insearchofinspiration,wetakeacloserlookatourparticularsequenceandndthatthefoursymbolsarenotequallyabundant.SymbolFrequencyA70millionB3millionC20millionD37millionIstheresomesortofvariable-lengthencoding,inwhichjustonebitisusedforthefrequentlyoccurringsymbolA,possiblyattheexpenseofneedingthreeormorebitsforlesscommonsymbols?Adangerwithhavingcodewordsofdifferentlengthsisthattheresultingencodingmaynotbeuniquelydecipherable.Forinstance,ifthecodewordsaref0;01;11;001g,thedecodingofstringslike001isambiguous.Wewillavoidthisproblembyinsistingontheprex-freeproperty:nocodewordcanbeaprexofanothercodeword.Anyprex-freeencodingcanberepresentedbyafullbinarytree—thatis,abinarytreeinwhicheverynodehaseitherzeroortwochildren—wherethesymbolsareattheleaves,andwhereeachcodewordisgeneratedbyapathfromroottoleaf,interpretingleftas0andrightas1(Exercise5.28).Figure5.10showsanexampleofsuchanencodingforthefoursymbolsA;B;C;D.Decodingisunique:astringofbitsisdecryptedbystartingattheroot,readingthestringfromlefttorighttomovedownward,and,wheneveraleafisreached,outputtingthecorrespondingsymbolandreturningtotheroot.Itisasimpleschemeandpaysoffnicelyforourtoyexample,where(underthecodesofFigure5.10)thetotalsizeofthebinarystringdropsto213megabits,a17%improvement.1Forstereosound,twochannelswouldbeneeded,doublingthenumberofsamples.146\nFigure5.10Aprex-freeencoding.Frequenciesareshowninsquarebrackets.01SymbolCodeword[60]A0A[70]B100C101[23]D11D[37]B[3]C[20]Ingeneral,howdowendtheoptimalcodingtree,giventhefrequenciesf1;f2;:::;fnofnsymbols?Tomaketheproblemprecise,wewantatreewhoseleaveseachcorrespondtoasymbolandwhichminimizestheoveralllengthoftheencoding,Xncostoftree=fi(depthofithsymbolintree)i=1(thenumberofbitsrequiredforasymbolisexactlyitsdepthinthetree).Thereisanotherwaytowritethiscostfunctionthatisveryhelpful.Althoughweareonlygivenfrequenciesfortheleaves,wecandenethefrequencyofanyinternalnodetobethesumofthefrequenciesofitsdescendantleaves;thisis,afterall,thenumberoftimestheinternalnodeisvisitedduringencodingordecoding.Duringtheencodingprocess,eachtimewemovedownthetree,onebitgetsoutputforeverynonrootnodethroughwhichwepass.Sothetotalcost—thetotalnumberofbitswhichareoutput—canalsobeexpressedthus:Thecostofatreeisthesumofthefrequenciesofallleavesandinternalnodes,excepttheroot.Therstformulationofthecostfunctiontellsusthatthetwosymbolswiththesmallestfrequenciesmustbeatthebottomoftheoptimaltree,aschildrenofthelowestinternalnode(thisinternalnodehastwochildrensincethetreeisfull).Otherwise,swappingthesetwosymbolswithwhateverislowestinthetreewouldimprovetheencoding.Thissuggeststhatwestartconstructingthetreegreedily:ndthetwosymbolswiththesmallestfrequencies,sayiandj,andmakethemchildrenofanewnode,whichthenhasfrequencyfi+fj.Tokeepthenotationsimple,let'sjustassumethesearef1andf2.Bythesecondformulationofthecostfunction,anytreeinwhichf1andf2aresibling-leaveshascostf1+f2plusthecostforatreewithn 1leavesoffrequencies(f1+f2);f3;f4;:::;fn:147\nf1+f2f5f4f3f1f2Thelatterproblemisjustasmallerversionoftheonewestartedwith.Sowepullf1andf2offthelistoffrequencies,insert(f1+f2),andloop.Theresultingalgorithmcanbedescribedintermsofpriorityqueueoperations(asdenedonpage114)andtakesO(nlogn)timeifabinaryheap(Section4.5.2)isused.procedureHuffman(f)Input:Anarrayf[1n]offrequenciesOutput:AnencodingtreewithnleavesletHbeapriorityqueueofintegers,orderedbyffori=1ton:insert(H;i)fork=n+1to2n 1:i=deletemin(H);j=deletemin(H)createanodenumberedkwithchildreni;jf[k]=f[i]+f[j]insert(H;k)Returningtoourtoyexample:canyoutellifthetreeofFigure5.10isoptimal?148\n149\nEntropyTheannualcountyhorseraceisbringinginthreethoroughbredswhohavenevercompetedagainstoneanother.Excited,youstudytheirpast200racesandsummarizetheseasprob-abilitydistributionsoverfouroutcomes:first(“rstplace”),second,third,andother.OutcomeAuroraWhirlwindPhantasmfirst0.150.300.20second0.100.050.30third0.700.250.30other0.050.400.20Whichhorseisthemostpredictable?Onequantitativeapproachtothisquestionistolookatcompressibility.Writedownthehistoryofeachhorseasastringof200values(first,second,third,other).Thetotalnumberofbitsneededtoencodethesetrack-recordstringscanthenbecomputedusingHuffman'salgorithm.Thisworksoutto290bitsforAurora,380forWhirlwind,and420forPhantasm(checkit!).Aurorahastheshortestencodingandisthereforeinastrongsensethemostpredictable.Theinherentunpredictability,orrandomness,ofaprobabilitydistributioncanbemea-suredbytheextenttowhichitispossibletocompressdatadrawnfromthatdistribution.morecompressiblelessrandommorepredictableSupposetherearenpossibleoutcomes,withprobabilitiesp1;p2;:::;pn.Ifasequenceofmvaluesisdrawnfromthedistribution,thentheithoutcomewillpopuproughlympitimes(ifmislarge).Forsimplicity,assumetheseareexactlytheobservedfrequencies,andmoreoverthatthepi'sareallpowersof2(thatis,oftheform1=2k).ItcanbeseenbyinductionPn(Exercise5.19)thatthenumberofbitsneededtoencodethesequenceisi=1mpilog(1=pi).ThustheaveragenumberofbitsneededtoencodeasingledrawfromthedistributionisXn1pilog:pii=1Thisistheentropyofthedistribution,ameasureofhowmuchrandomnessitcontains.Forexample,afaircoinhastwooutcomes,eachwithprobability1=2.Soitsentropyis1log2+1log2=1:22Thisisnaturalenough:thecoinipcontainsonebitofrandomness.Butwhatifthecoinisnotfair,ifithasa3=4chanceofturningupheads?Thentheentropyis3log4+1log4=0:81:434Abiasedcoinismorepredictablethanafaircoin,andthushaslowerentropy.Asthebiasbecomesmorepronounced,theentropydropstowardzero.WeexplorethesenotionsfurtherinExercise5.18and5.19.150\n5.3HornformulasInordertodisplayhuman-levelintelligence,acomputermustbeabletoperformatleastsomemodicumoflogicalreasoning.Hornformulasareaparticularframeworkfordoingthis,forexpressinglogicalfactsandderivingconclusions.ThemostprimitiveobjectinaHornformulaisaBooleanvariable,takingvalueeithertrueorfalse.Forinstance,variablesx,y,andzmightdenotethefollowingpossibilities.xthemurdertookplaceinthekitchenythebutlerisinnocentzthecolonelwasasleepat8pmAliteraliseitheravariablexoritsnegationx(“NOTx”).InHornformulas,knowledgeaboutvariablesisrepresentedbytwokindsofclauses:1.Implications,whoseleft-handsideisanANDofanynumberofpositiveliteralsandwhoseright-handsideisasinglepositiveliteral.Theseexpressstatementsoftheform“iftheconditionsonthelefthold,thentheoneontherightmustalsobetrue.”Forinstance,(z^w))umightmean“ifthecolonelwasasleepat8pmandthemurdertookplaceat8pmthenthecolonelisinnocent.”Adegeneratetypeofimplicationisthesingleton“)x,”meaningsimplythatxistrue:“themurderdenitelyoccurredinthekitchen.”2.Purenegativeclauses,consistingofanORofanynumberofnegativeliterals,asin(u_v_y)(“theycan'tallbeinnocent”).Givenasetofclausesofthesetwotypes,thegoalistodeterminewhetherthereisaconsis-tentexplanation:anassignmentoftrue/falsevaluestothevariablesthatsatisesalltheclauses.Thisisalsocalledasatisfyingassignment.Thetwokindsofclausespullusindifferentdirections.Theimplicationstellustosetsomeofthevariablestotrue,whilethenegativeclausesencourageustomakethemfalse.OurstrategyforsolvingaHornformulaisthis:Westartwithallvariablesfalse.Wethenproceedtosetsomeofthemtotrue,onebyone,butveryreluctantly,andonlyifweabsolutelyhavetobecauseanimplicationwouldotherwisebeviolated.Oncewearedonewiththisphaseandallimplicationsaresatised,onlythendoweturntothenegativeclausesandmakesuretheyareallsatised.Inotherwords,ouralgorithmforHornclausesisthefollowinggreedyscheme(stingyisperhapsmoredescriptive):Input:aHornformulaOutput:asatisfyingassignment,ifoneexistssetallvariablestofalse151\nwhilethereisanimplicationthatisnotsatisfied:settheright-handvariableoftheimplicationtotrueifallpurenegativeclausesaresatisfied:returntheassignmentelse:return``formulaisnotsatisfiable''Forinstance,supposetheformulais(w^y^z))x;(x^z))w;x)y;)x;(x^y))w;(w_x_y);(z):Westartwitheverythingfalseandthennoticethatxmustbetrueonaccountofthesin-gletonimplication)x.Thenweseethatymustalsobetrue,becauseofx)y.Andsoon.Toseewhythealgorithmiscorrect,noticethatifitreturnsanassignment,thisassign-mentsatisesboththeimplicationsandthenegativeclauses,andsoitisindeedasatisfyingtruthassignmentoftheinputHornformula.Soweonlyhavetoconvinceourselvesthatifthealgorithmndsnosatisfyingassignment,thentherereallyisnone.Thisissobecauseour“stingy”rulemaintainsthefollowinginvariant:Ifacertainsetofvariablesissettotrue,thentheymustbetrueinanysatisfyingassignment.Hence,ifthetruthassignmentfoundafterthewhileloopdoesnotsatisfythenegativeclauses,therecanbenosatisfyingtruthassignment.HornformulaslieattheheartofProlog(“programmingbylogic”),alanguageinwhichyouprogrambyspecifyingdesiredpropertiesoftheoutput,usingsimplelogicalexpressions.TheworkhorseofProloginterpretersisourgreedysatisabilityalgorithm.Conveniently,itcanbeimplementedintimelinearinthelengthoftheformula;doyouseehow(Exercise5.32)?5.4SetcoverThedotsinFigure5.11representacollectionoftowns.Thiscountyisinitsearlystagesofplanningandisdecidingwheretoputschools.Thereareonlytwoconstraints:eachschoolshouldbeinatown,andnooneshouldhavetotravelmorethan30milestoreachoneofthem.Whatistheminimumnumberofschoolsneeded?Thisisatypicalsetcoverproblem.Foreachtownx,letSxbethesetoftownswithin30milesofit.Aschoolatxwillessentially“cover”theseothertowns.Thequestionisthen,howmanysetsSxmustbepickedinordertocoverallthetownsinthecounty?SETCOVERInput:AsetofelementsB;setsS1;:::;SmBOutput:AselectionoftheSiwhoseunionisB.Cost:Numberofsetspicked.(Inourexample,theelementsofBarethetowns.)Thisproblemlendsitselfimmediatelytoagreedysolution:152\nFigure5.11(a)Eleventowns.(b)Townsthatarewithin30milesofeachother.(a)(b)ccbbddffaeaekgkghhiijjRepeatuntilallelementsofBarecovered:PickthesetSiwiththelargestnumberofuncoveredelements.Thisisextremelynaturalandintuitive.Let'sseewhatitwoulddoonourearlierexample:Itwouldrstplaceaschoolattowna,sincethiscoversthelargestnumberofothertowns.Thereafter,itwouldchoosethreemoreschools—c,j,andeitherforg—foratotaloffour.However,thereexistsasolutionwithjustthreeschools,atb,e,andi.Thegreedyschemeisnotoptimal!Butluckily,itisn'ttoofarfromoptimal.ClaimSupposeBcontainsnelementsandthattheoptimalcoverconsistsofksets.Thenthegreedyalgorithmwilluseatmostklnnsets.2Letntbethenumberofelementsstillnotcoveredaftertiterationsofthegreedyalgorithm(son0=n).Sincetheseremainingelementsarecoveredbytheoptimalksets,theremustbesomesetwithatleastnt=kofthem.Therefore,thegreedystrategywillensurethatnt1nt+1nt =nt1 ;kkwhichbyrepeatedapplicationimpliesntn0(1 1=k)t.Amoreconvenientboundcanbeobtainedfromtheusefulinequality1 xe xforallx,withequalityifandonlyifx=0,whichismosteasilyprovedbyapicture:2lnmeans“naturallogarithm,”thatis,tothebasee.153\ne x1 x1x0Thust1 1=kt t=kntn01 0,deneanr-pathtobeapathwhoseedgesallhaveweightw(e).(b)e62E0andw^(e)w(e).5.23.Sometimeswewantlightspanningtreeswithcertainspecialproperties.Here'sanexample.158\nInput:UndirectedgraphG=(V;E);edgeweightswe;subsetofverticesUVOutput:ThelightestspanningtreeinwhichthenodesofUareleaves(theremightbeotherleavesinthistreeaswell).(Theanswerisn'tnecessarilyaminimumspanningtree.)GiveanalgorithmforthisproblemwhichrunsinO(jEjlogjVj)time.(Hint:WhenyouremovenodesUfromtheoptimalsolution,whatisleft?)5.24.Abinarycounterofunspeciedlengthsupportstwooperations:increment(whichincreasesitsvaluebyone)andreset(whichsetsitsvaluebacktozero).Showthat,startingfromaninitiallyzerocounter,anysequenceofnincrementandresetoperationstakestimeO(n);thatis,theamortizedtimeperoperationisO(1).5.25.Here'saproblemthatoccursinautomaticprogramanalysis.Forasetofvariablesx1;:::;xn,youaregivensomeequalityconstraints,oftheform“xi=xj”andsomedisequalityconstraints,oftheform“xi6=xj.”Isitpossibletosatisfyallofthem?Forinstance,theconstraintsx1=x2;x2=x3;x3=x4;x16=x4cannotbesatised.Giveanefcientalgorithmthattakesasinputmconstraintsovernvariablesanddecideswhethertheconstraintscanbesatised.5.26.Graphswithprescribeddegreesequences.Givenalistofnpositiveintegersd1;d2;:::;dn,wewanttoefcientlydeterminewhetherthereexistsanundirectedgraphG=(V;E)whosenodeshavedegreespreciselyd1;d2;:::;dn.Thatis,ifV=fv1;:::;vng,thenthedegreeofvishouldbeexactlydi.Wecall(d1;:::;dn)thedegreesequenceofG.ThisgraphGshouldnotcontainself-loops(edgeswithbothendpointsequaltothesamenode)ormultipleedgesbetweenthesamepairofnodes.(a)Giveanexampleofd1;d2;d3;d4whereallthedi3andd1+d2+d3+d4iseven,butforwhichnographwithdegreesequence(d1;d2;d3;d4)exists.(b)Supposethatd1d2dnandthatthereexistsagraphG=(V;E)withdegreesequence(d1;:::;dn).Wewanttoshowthattheremustexistagraphthathasthisdegreesequenceandwhereinadditiontheneighborsofv1arev2;v3;:::;vd1+1.TheideaistograduallytransformGintoagraphwiththedesiredadditionalproperty.i.Supposetheneighborsofv1inGarenotv2;v3;:::;vd1+1.Showthatthereexistsiw:K(w;j)=K(w;j 1)else:K(w;j)=maxfK(w;j 1);K(w wj;j 1)+vjgreturnK(W;n)172\nMemoizationIndynamicprogramming,wewriteoutarecursiveformulathatexpresseslargeproblemsintermsofsmalleronesandthenuseittolloutatableofsolutionvaluesinabottom-upmanner,fromsmallestsubproblemtolargest.Theformulaalsosuggestsarecursivealgorithm,butwesawearlierthatnaiverecursioncanbeterriblyinefcient,becauseitsolvesthesamesubproblemsoverandoveragain.Whataboutamoreintelligentrecursiveimplementation,onethatremembersitspreviousinvocationsandtherebyavoidsrepeatingthem?Ontheknapsackproblem(withrepetitions),suchanalgorithmwoulduseahashtable(recallSection1.5)tostorethevaluesofK()thathadalreadybeencomputed.AteachrecursivecallrequestingsomeK(w),thealgorithmwouldrstcheckiftheanswerwasalreadyinthetableandthenwouldproceedtoitscalculationonlyifitwasn't.Thistrickiscalledmemoization:Ahashtable,initiallyempty,holdsvaluesofK(w)indexedbywfunctionknapsack(w)ifwisinhashtable:returnK(w)K(w)=maxfknapsack(w wi)+vi:wiwginsertK(w)intohashtable,withkeywreturnK(w)Sincethisalgorithmneverrepeatsasubproblem,itsrunningtimeisO(nW),justlikethedynamicprogram.However,theconstantfactorinthisbig-Onotationissubstantiallylargerbecauseoftheoverheadofrecursion.Insomecases,though,memoizationpaysoff.Here'swhy:dynamicprogrammingau-tomaticallysolveseverysubproblemthatcouldconceivablybeneeded,whilememoizationonlyendsupsolvingtheonesthatareactuallyused.Forinstance,supposethatWandalltheweightswiaremultiplesof100.ThenasubproblemK(w)isuselessif100doesnotdividew.Thememoizedrecursivealgorithmwillneverlookattheseextraneoustableentries.173\nFigure6.6ABCD=(A(BC))D.(a)(b)ABCDABCD5020201110101005020201010100(c)(d)A(BC)D(A(BC))D501010100501006.5ChainmatrixmultiplicationSupposethatwewanttomultiplyfourmatrices,ABCD,ofdimensions5020,201,110,and10100,respectively(Figure6.6).Thiswillinvolveiterativelymultiplyingtwomatricesatatime.Matrixmultiplicationisnotcommutative(ingeneral,AB6=BA),butitisassociative,whichmeansforinstancethatA(BC)=(AB)C.Thuswecancomputeourproductoffourmatricesinmanydifferentways,dependingonhowweparenthesizeit.Aresomeofthesebetterthanothers?Multiplyinganmnmatrixbyannpmatrixtakesmnpmultiplications,toagoodenoughapproximation.Usingthisformula,let'scompareseveraldifferentwaysofevaluatingABCD:ParenthesizationCostcomputationCostA((BC)D)20110+2010100+5020100120;200(A(BC))D20110+502010+501010060;200(AB)(CD)50201+110100+5011007;000Asyoucansee,theorderofmultiplicationsmakesabigdifferenceinthenalrunningtime!Moreover,thenaturalgreedyapproach,toalwaysperformthecheapestmatrixmultiplicationavailable,leadstothesecondparenthesizationshownhereandisthereforeafailure.Howdowedeterminetheoptimalorder,ifwewanttocomputeA1A2An,wheretheAi'sarematriceswithdimensionsm0m1;m1m2;:::;mn 1mn,respectively?Therstthingtonoticeisthataparticularparenthesizationcanberepresentedverynaturallybyabinarytreeinwhichtheindividualmatricescorrespondtotheleaves,therootisthenalproduct,andinteriornodesareintermediateproducts(Figure6.7).Thepossibleordersinwhichtodothemultiplicationcorrespondtothevariousfullbinarytreeswithnleaves,whose174\nFigure6.7(a)((AB)C)D;(b)A((BC)D);(c)(A(BC))D.(a)(b)(c)DADCDAABBCBCnumberisexponentialinn(Exercise2.13).Wecertainlycannottryeachtree,andwithbruteforcethusruledout,weturntodynamicprogramming.ThebinarytreesofFigure6.7aresuggestive:foratreetobeoptimal,itssubtreesmustalsobeoptimal.Whatarethesubproblemscorrespondingtothesubtrees?TheyareproductsoftheformAiAi+1Aj.Let'sseeifthisworks:for1ijn,deneC(i;j)=minimumcostofmultiplyingAiAi+1Aj.Thesizeofthissubproblemisthenumberofmatrixmultiplications,jj ij.Thesmallestsubproblemiswheni=j,inwhichcasethere'snothingtomultiply,soC(i;i)=0.Forj>i,considertheoptimalsubtreeforC(i;j).Therstbranchinthissubtree,theoneatthetop,willsplittheproductintwopieces,oftheformAiAkandAk+1Aj,forsomekbetweeniandj.Thecostofthesubtreeisthenthecostofthesetwopartialproducts,plusthecostofcombiningthem:C(i;k)+C(k+1;j)+mi 1mkmj.Andwejustneedtondthesplittingpointkforwhichthisissmallest:C(i;j)=minfC(i;k)+C(k+1;j)+mi 1mkmjg:ik1,wedeneC(S;1)=1sincethepathcannotbothstartandendat1.Now,let'sexpressC(S;j)intermsofsmallersubproblems.Weneedtostartat1andendatj;whatshouldwepickasthesecond-to-lastcity?Ithastobesomei2S,sotheoverallpathlengthisthedistancefrom1toi,namely,C(S fjg;i),plusthelengthofthenaledge,178\ndij.Wemustpickthebestsuchi:C(S;j)=minC(S fjg;i)+dij:i2S:i6=jThesubproblemsareorderedbyjSj.Here'sthecode.C(f1g;1)=0fors=2ton:forallsubsetsSf1;2;:::;ngofsizesandcontaining1:C(S;1)=1forallj2S;j6=1:C(S;j)=minfC(S fjg;i)+dij:i2S;i6=jgreturnminjC(f1;:::;ng;j)+dj1Thereareatmost2nnsubproblems,andeachonetakeslineartimetosolve.ThetotalrunningtimeisthereforeO(n22n).OntimeandmemoryTheamountoftimeittakestorunadynamicprogrammingalgorithmiseasytodiscernfromthedagofsubproblems:inmanycasesitisjustthetotalnumberofedgesinthedag!Allwearereallydoingisvisitingthenodesinlinearizedorder,examiningeachnode'sinedges,and,mostoften,doingaconstantamountofworkperedge.Bytheend,eachedgeofthedaghasbeenexaminedonce.Buthowmuchcomputermemoryisrequired?Thereisnosimpleparameterofthedagcharacterizingthis.Itiscertainlypossibletodothejobwithanamountofmemorypropor-tionaltothenumberofvertices(subproblems),butwecanusuallygetawaywithmuchless.Thereasonisthatthevalueofaparticularsubproblemonlyneedstoberemembereduntilthelargersubproblemsdependingonithavebeensolved.Thereafter,thememoryittakesupcanbereleasedforreuse.Forexample,intheFloyd-Warshallalgorithmthevalueofdist(i;j;k)isnotneededoncethedist(;;k+1)valueshavebeencomputed.Therefore,weonlyneedtwojVjjVjarraystostorethedistvalues,oneforoddvaluesofkandoneforevenvalues:whencomputingdist(i;j;k),weoverwritedist(i;j;k 2).(Andletusnotforgetthat,asalwaysindynamicprogramming,wealsoneedonemorear-ray,prev(i;j),storingthenexttolastvertexinthecurrentshortestpathfromitoj,avaluethatmustbeupdatedwithdist(i;j;k).Weomitthismundanebutcrucialbookkeepingstepfromourdynamicprogrammingalgorithms.)CanyouseewhytheeditdistancedaginFigure6.5onlyneedsmemoryproportionaltothelengthoftheshorterstring?6.7IndependentsetsintreesAsubsetofnodesSVisanindependentsetofgraphG=(V;E)iftherearenoedgesbetweenthem.Forinstance,inFigure6.10thenodesf1;5gformanindependentset,but179\nFigure6.10Thelargestindependentsetinthisgraphhassize3.125634nodesf1;4;5gdonot,becauseoftheedgebetween4and5.Thelargestindependentsetisf2;3;6g.Likeseveralotherproblemswehaveseeninthischapter(knapsack,travelingsalesman),ndingthelargestindependentsetinagraphisbelievedtobeintractable.However,whenthegraphhappenstobeatree,theproblemcanbesolvedinlineartime,usingdynamicprogramming.Andwhataretheappropriatesubproblems?Alreadyinthechainmatrixmultiplicationproblemwenoticedthatthelayeredstructureofatreeprovidesanaturaldenitionofasubproblem—aslongasonenodeofthetreehasbeenidentiedasaroot.Sohere'sthealgorithm:Startbyrootingthetreeatanynoder.Now,eachnodedenesasubtree—theonehangingfromit.Thisimmediatelysuggestssubproblems:I(u)=sizeoflargestindependentsetofsubtreehangingfromu:OurnalgoalisI(r).Dynamicprogrammingproceedsasalwaysfromsmallersubproblemstolargerones,thatistosay,bottom-upintherootedtree.Supposeweknowthelargestindependentsetsforallsubtreesbelowacertainnodeu;inotherwords,supposeweknowI(w)foralldescendantswofu.HowcanwecomputeI(u)?Let'ssplitthecomputationintotwocases:anyindependentseteitherincludesuoritdoesn't(Figure6.11).890andi=1;2;:::;n.Anytworestaurantsshouldbeatleastkmilesapart,wherekisapositiveinteger.181\nGiveanefcientalgorithmtocomputethemaximumexpectedtotalprotsubjecttothegivenconstraints.6.4.Youaregivenastringofncharacterss[1:::n],whichyoubelievetobeacorruptedtextdocumentinwhichallpunctuationhasvanished(sothatitlookssomethinglike“itwasthebestoftimes...”).Youwishtoreconstructthedocumentusingadictionary,whichisavailableintheformofaBooleanfunctiondict():foranystringw,trueifwisavalidworddict(w)=falseotherwise.(a)Giveadynamicprogrammingalgorithmthatdetermineswhetherthestrings[]canbereconstitutedasasequenceofvalidwords.TherunningtimeshouldbeatmostO(n2),assumingcallstodicttakeunittime.(b)Intheeventthatthestringisvalid,makeyouralgorithmoutputthecorrespondingse-quenceofwords.6.5.Pebblingacheckerboard.Wearegivenacheckerboardwhichhas4rowsandncolumns,andhasanintegerwrittenineachsquare.Wearealsogivenasetof2npebbles,andwewanttoplacesomeoralloftheseonthecheckerboard(eachpebblecanbeplacedonexactlyonesquare)soastomaximizethesumoftheintegersinthesquaresthatarecoveredbypebbles.Thereisoneconstraint:foraplacementofpebblestobelegal,notwoofthemcanbeonhorizontallyorverticallyadjacentsquares(diagonaladjacencyisne).(a)Determinethenumberoflegalpatternsthatcanoccurinanycolumn(inisolation,ignoringthepebblesinadjacentcolumns)anddescribethesepatterns.Calltwopatternscompatibleiftheycanbeplacedonadjacentcolumnstoformalegalplacement.Letusconsidersubproblemsconsistingoftherstkcolumns1kn.Eachsubproblemcanbeassignedatype,whichisthepatternoccurringinthelastcolumn.(b)Usingthenotionsofcompatibilityandtype,giveanO(n)-timedynamicprogrammingalgo-rithmforcomputinganoptimalplacement.6.6.Letusdeneamultiplicationoperationonthreesymbolsa;b;caccordingtothefollowingtable;thusab=b,ba=c,andsoon.Noticethatthemultiplicationoperationdenedbythetableisneitherassociativenorcommutative.abcabbabcbacaccFindanefcientalgorithmthatexaminesastringofthesesymbols,saybbbbac,anddecideswhetherornotitispossibletoparenthesizethestringinsuchawaythatthevalueoftheresultingexpressionisa.Forexample,oninputbbbbacyouralgorithmshouldreturnyesbecause((b(bb))(ba))c=a.6.7.Asubsequenceispalindromicifitisthesamewhetherreadlefttorightorrighttoleft.Forinstance,thesequenceA;C;G;T;G;T;C;A;A;A;A;T;C;Ghasmanypalindromicsubsequences,includingA;C;G;C;AandA;A;A;A(ontheotherhand,thesubsequenceA;C;Tisnotpalindromic).Deviseanalgorithmthattakesasequencex[1:::n]andreturnsthe(lengthofthe)longestpalindromicsubsequence.ItsrunningtimeshouldbeO(n2).182\n6.8.Giventwostringsx=x1x2xnandy=y1y2ym,wewishtondthelengthoftheirlongestcommonsubstring,thatis,thelargestkforwhichthereareindicesiandjwithxixi+1xi+k 1=yjyj+1yj+k 1.ShowhowtodothisintimeO(mn).6.9.Acertainstring-processinglanguageoffersaprimitiveoperationwhichsplitsastringintotwopieces.Sincethisoperationinvolvescopyingtheoriginalstring,ittakesnunitsoftimeforastringoflengthn,regardlessofthelocationofthecut.Suppose,now,thatyouwanttobreakastringintomanypieces.Theorderinwhichthebreaksaremadecanaffectthetotalrunningtime.Forexample,ifyouwanttocuta20-characterstringatpositions3and10,thenmakingtherstcutatposition3incursatotalcostof20+17=37,whiledoingposition10rsthasabettercostof20+10=30.Giveadynamicprogrammingalgorithmthat,giventhelocationsofmcutsinastringoflengthn,ndstheminimumcostofbreakingthestringintom+1pieces.6.10.Countingheads.Givenintegersnandk,alongwithp1;:::;pn2[0;1],youwanttodeterminetheprobabilityofobtainingexactlykheadswhennbiasedcoinsaretossedindependentlyatrandom,wherepistheprobabilitythattheithcoincomesupheads.GiveanO(n2)algorithmforthisitask.2Assumeyoucanmultiplyandaddtwonumbersin[0;1]inO(1)time.6.11.Giventwostringsx=x1x2xnandy=y1y2ym,wewishtondthelengthoftheirlongestcommonsubsequence,thatis,thelargestkforwhichthereareindicesi10ontheedges.Wewouldliketosendasmuchoilaspossiblefromstotwithoutexceedingthecapacitiesofanyoftheedges.Aparticularshippingschemeiscalledaowandconsistsofavariablefeforeachedgeeofthenetwork,satisfyingthefollowingtwoproperties:1.Itdoesn'tviolateedgecapacities:0feceforalle2E.2.Forallnodesuexceptsandt,theamountofowenteringuequalstheamountleaving199\nFigure7.4(a)Anetworkwithedgecapacities.(b)Aowinthenetwork.(a)2(b)2adad3222101010s3b11ts1b1t4455cece55u:XXfwu=fuz:(w;u)2E(u;z)2EInotherwords,owisconserved.Thesizeofaowisthetotalquantitysentfromstotand,bytheconservationprinciple,isequaltothequantityleavings:Xsize(f)=fsu:(s;u)2EInshort,ourgoalistoassignvaluestoffe:e2Egthatwillsatisfyasetoflinearconstraintsandmaximizealinearobjectivefunction.Butthisisalinearprogram!Themaximum-owproblemreducestolinearprogramming.Forexample,forthenetworkofFigure7.4theLPhas11variables,oneperedge.Itseekstomaximizefsa+fsb+fscsubjecttoatotalof27constraints:11fornonnegativity(suchasfsa0),11forcapacity(suchasfsa3),and5forowconservation(oneforeachnodeofthegraphotherthansandt,suchasfsc+fdc=fce).Simplexwouldtakenotimeatalltocorrectlysolvetheproblemandtoconrmthat,inourexample,aowof7isinfactoptimal.7.2.3AcloserlookatthealgorithmAllweknowsofarofthesimplexalgorithmisthevaguegeometricintuitionthatitkeepsmakinglocalmovesonthesurfaceofaconvexfeasibleregion,successivelyimprovingtheobjectivefunctionuntilitnallyreachestheoptimalsolution.Oncewehavestudieditinmoredetail(Section7.6),wewillbeinapositiontounderstandexactlyhowithandlesowLPs,whichisusefulasasourceofinspirationfordesigningdirectmax-owalgorithms.Itturnsoutthatinfactthebehaviorofsimplexhasanelementaryinterpretation:Startwithzeroow.Repeat:chooseanappropriatepathfromstot,andincreaseowalongtheedgesofthispathasmuchaspossible.Figure7.5(a)–(d)showsasmallexampleinwhichsimplexhaltsaftertwoiterations.Thenalowhassize2,whichiseasilyseentobeoptimal.200\nFigure7.5Anillustrationofthemax-owalgorithm.(a)Atoynetwork.(b)Therstpathchosen.(c)Thesecondpathchosen.(d)Thenalow.(e)Wecouldhavechosenthispathrst.(f)Inwhichcase,wewouldhavetoallowthissecondpath.(a)(b)aa111stst11b(c)(d)a11sts0t11bb(e)(f)aa11s1tst111bbThereisjustonecomplication.Whatifwehadinitiallychosenadifferentpath,theoneinFigure7.5(e)?Thisgivesonlyoneunitofowandyetseemstoblockallotherpaths.Simplexgetsaroundthisproblembyalsoallowingpathstocancelexistingow.Inthisparticularcase,itwouldsubsequentlychoosethepathofFigure7.5(f).Edge(b;a)ofthispathisn'tintheoriginalnetworkandhastheeffectofcancelingowpreviouslyassignedtoedge(a;b).Tosummarize,ineachiterationsimplexlooksforans tpathwhoseedges(u;v)canbeoftwotypes:1.(u;v)isintheoriginalnetwork,andisnotyetatfullcapacity.2.Thereverseedge(v;u)isintheoriginalnetwork,andthereissomeowalongit.Ifthecurrentowisf,thenintherstcase,edge(u;v)canhandleuptocuv fuvadditionalunitsofow,andinthesecondcase,uptofvuadditionalunits(cancelingallorpartoftheexistingowon(v;u)).Theseow-increasingopportunitiescanbecapturedinaresidualnetworkGf=(V;Ef),whichhasexactlythetwotypesofedgeslisted,withresidualcapacities201\ncf:cuv fuvif(u;v)2Eandfuv0Thuswecanequivalentlythinkofsimplexaschoosingans tpathintheresidualnetwork.Bysimulatingthebehaviorofsimplex,wegetadirectalgorithmforsolvingmax-ow.Itproceedsiniterations,eachtimeexplicitlyconstructingGf,ndingasuitables tpathinGfbyusing,say,alinear-timebreadth-rstsearch,andhaltingifthereisnolongeranysuchpathalongwhichowcanbeincreased.Figure7.6illustratesthealgorithmonouroilexample.7.2.4AcerticateofoptimalityNowforatrulyremarkablefact:notonlydoessimplexcorrectlycomputeamaximumow,butitalsogeneratesashortproofoftheoptimalityofthisow!Let'sseeanexampleofwhatthismeans.Partitionthenodesoftheoilnetwork(Figure7.4)intotwogroups,L=fs;a;bgandR=fc;d;e;tg:LR2ad32101s3b11t45ce5AnyoiltransmittedmustpassfromLtoR.Therefore,noowcanpossiblyexceedthetotalcapacityoftheedgesfromLtoR,whichis7.Butthismeansthattheowwefoundearlier,ofsize7,mustbeoptimal!Moregenerally,an(s;t)-cutpartitionstheverticesintotwodisjointgroupsLandRsuchthatsisinLandtisinR.ItscapacityisthetotalcapacityoftheedgesfromLtoR,andasarguedpreviously,isanupperboundonanyow:Pickanyowfandany(s;t)-cut(L;R).Thensize(f)capacity(L;R).Somecutsarelargeandgivelooseupperbounds—cut(fs;b;cg;fa;d;e;tg)hasacapacityof19.Butthereisalsoacutofcapacity7,whichiseffectivelyacerticateofoptimalityofthemaximumow.Thisisn'tjustaluckypropertyofouroilnetwork;suchacutalwaysexists.Max-owmin-cuttheoremThesizeofthemaximumowinanetworkequalsthecapacityofthesmallest(s;t)-cut.Moreover,ouralgorithmautomaticallyndsthiscutasaby-product!Let'sseewhythisistrue.Supposefisthenalowwhenthealgorithmterminates.WeknowthatnodetisnolongerreachablefromsintheresidualnetworkGf.LetLbethenodesthatarereachablefromsinGf,andletR=V Lbetherestofthenodes.Then(L;R)isacutinthegraphG:202\nLReste0Weclaimthatsize(f)=capacity(L;R):Toseethis,observethatbythewayLisdened,anyedgegoingfromLtoRmustbeatfullcapacity(inthecurrentowf),andanyedgefromRtoLmusthavezeroow.(So,inthegure,fe=ceandfe0=0.)Thereforethenetowacross(L;R)isexactlythecapacityofthecut.7.2.5EfciencyEachiterationofourmaximum-owalgorithmisefcient,requiringO(jEj)timeifadepth-rstorbreadth-rstsearchisusedtondans tpath.Buthowmanyiterationsarethere?SupposealledgesintheoriginalnetworkhaveintegercapacitiesC.Thenaninductiveargumentshowsthatoneachiterationofthealgorithm,theowisalwaysanintegerandincreasesbyanintegeramount.Therefore,sincethemaximumowisatmostCjEj(why?),itfollowsthatthenumberofiterationsisatmostthismuch.Butthisishardlyareassuringbound:whatifCisinthemillions?WeexaminethisissuefurtherinExercise7.31.ItturnsoutthatitisindeedpossibletoconstructbadexamplesinwhichthenumberofiterationsisproportionaltoC,ifs tpathsarenotcarefullychosen.However,ifpathsarechoseninasensiblemanner—inparticular,byusingabreadth-rstsearch,whichndsthepathwiththefewestedges—thenthenumberofiterationsisatmostO(jVjjEj),nomatterwhatthecapacitiesare.ThislatterboundgivesanoverallrunningtimeofO(jVjjEj2)formaximumow.203\nFigure7.6Themax-owalgorithmappliedtothenetworkofFigure7.4.Ateachiteration,thecurrentowisshownontheleftandtheresidualnetworkontheright.Thepathschosenareshowninbold.CurrentowResidualgraph(a)2adad32101sbts3b11t45cece5(b)11adad11122101sb1ts3b11t14141cece14(c)22adad2221101sb1ts3b11t124231cece14(d)22adad2221101sb1ts3b11t1135534cece41204\nFigure7.6ContinuedCurrentFlowResidualGraph(e)22adad212111011sb1ts3b11t4455cece55(f)22adad22221110111sb1tsb11t24455cece55205\nFigure7.7Anedgebetweentwopeoplemeanstheylikeeachother.Isitpossibletopaireveryoneuphappily?BOYSGIRLSAlAliceBobBeatriceChetCarolDanDanielle7.3BipartitematchingFigure7.7showsagraphwithfournodesontheleftrepresentingboysandfournodesontherightrepresentinggirls.1Thereisanedgebetweenaboyandgirliftheylikeeachother(forinstance,Allikesallthegirls).Isitpossibletochoosecouplessothateveryonehasexactlyonepartner,anditissomeonetheylike?Ingraph-theoreticjargon,isthereaperfectmatching?Thismatchmakinggamecanbereducedtothemaximum-owproblem,andtherebytolinearprogramming!Createanewsourcenode,s,withoutgoingedgestoalltheboys;anewsinknode,t,withincomingedgesfromallthegirls;anddirectalltheedgesintheoriginalbipartitegraphfromboytogirl(Figure7.8).Finally,giveeveryedgeacapacityof1.Thenthereisaperfectmatchingifandonlyifthisnetworkhasaowwhosesizeequalsthenumberofcouples.Canyoundsuchaowintheexample?Actually,thesituationisslightlymorecomplicatedthanjuststated:whatiseasytoseeisthattheoptimuminteger-valuedowcorrespondstotheoptimummatching.Wewouldbeatabitofalossinterpretingaowthatships0:7unitsalongtheedgeAl–Carol,forinstance!Fortunately,themaximum-owproblemhasthefollowingproperty:ifalledgecapacitiesareintegers,thentheoptimalowfoundbyouralgorithmisintegral.Wecanseethisdirectly1Thiskindofgraph,inwhichthenodescanbepartitionedintotwogroupssuchthatalledgesarebetweenthegroups,iscalledbipartite.Figure7.8Amatchmakingnetwork.Eachedgehasacapacityofone.AlAliceBobBeatricestChetCarolDanDanielle206\nfromthealgorithm,whichinsuchcaseswouldincrementtheowbyanintegeramountoneachiteration.Henceintegralitycomesforfreeinthemaximum-owproblem.Unfortunately,thisistheexceptionratherthantherule:aswewillseeinChapter8,itisaverydifcultproblemtondtheoptimumsolution(orforthatmatter,anysolution)ofagenerallinearprogram,ifwealsodemandthatthevariablesbeintegers.7.4DualityWehaveseenthatinnetworks,owsaresmallerthancuts,butthemaximumowandmini-mumcutexactlycoincideandeachisthereforeacerticateoftheother'soptimality.Remark-ableasthisphenomenonis,wenowgeneralizeitfrommaximumowtoanyproblemthatcanbesolvedbylinearprogramming!Itturnsoutthateverylinearmaximizationproblemhasadualminimizationproblem,andtheyrelatetoeachotherinmuchthesamewayasowsandcuts.Tounderstandwhatdualityisabout,recallourintroductoryLPwiththetwotypesofchocolate:maxx1+6x2x1200x2300x1+x2400x1;x20Simplexdeclarestheoptimumsolutiontobe(x1;x2)=(100;300),withobjectivevalue1900.Canthisanswerbecheckedsomehow?Let'ssee:supposewetaketherstinequalityandaddittosixtimesthesecondinequality.Wegetx1+6x22000:Thisisinteresting,becauseittellsusthatitisimpossibletoachieveaprotofmorethan2000.CanweaddtogethersomeothercombinationoftheLPconstraintsandbringthisupperboundevencloserto1900?Afteralittleexperimentation,wendthatmultiplyingthethreeinequalitiesby0,5,and1,respectively,andaddingthemupyieldsx1+6x21900:So1900mustindeedbethebestpossiblevalue!Themultipliers(0;5;1)magicallyconstituteacerticateofoptimality!ItisremarkablethatsuchacerticateexistsforthisLP—andevenifweknewtherewereone,howwouldwesystematicallygoaboutndingit?Let'sinvestigatetheissuebydescribingwhatweexpectofthesethreemultipliers,callthemy1;y2;y3.MultiplierInequalityy1x1200y2x2300y3x1+x2400207\nFigure7.9Bydesign,dualfeasiblevaluesprimalfeasiblevalues.Thedualitytheoremtellsusthatmoreovertheiroptimacoincide.PrimalDualPrimalfeasibleoptoptDualfeasibleObjectivevalueThisdualitygapiszeroTostartwith,theseyi'smustbenonnegative,forotherwisetheyareunqualiedtomultiplyinequalities(multiplyinganinequalitybyanegativenumberwouldiptheto).Afterthemultiplicationandadditionsteps,wegetthebound:(y1+y3)x1+(y2+y3)x2200y1+300y2+400y3:Wewanttheleft-handsidetolooklikeourobjectivefunctionx1+6x2sothattheright-handsideisanupperboundontheoptimumsolution.Forthisweneedy1+y3tobe1andy2+y3tobe6.Cometothinkofit,itwouldbeneify1+y3werelargerthan1—theresultingcerticatewouldbeallthemoreconvincing.Thus,wegetanupperbound890,thentheoriginisnotoptimal,sincewecanincreasetheobjectivefunctionbyraisingxi.Thus,fortask2,wecanmovebyincreasingsomexiforwhichci>0.Howmuchcanweincreaseit?Untilwehitsomeotherconstraint.Thatis,wereleasethetightconstraintxi0andincreasexiuntilsomeotherinequality,previouslyloose,nowbecomestight.Atthatpoint,weagainhaveexactlyntightinequalities,soweareatanewvertex.Forinstance,supposewe'redealingwiththefollowinglinearprogram.max2x1+5x22x1 x241x1+2x292 x1+x233x104x205Simplexcanbestartedattheorigin,whichisspeciedbyconstraints4and5.Tomove,wereleasethetightconstraintx20.Asx2isgraduallyincreased,therstconstraintitrunsintois x1+x23,andthusithastostopatx2=3,atwhichpointthisnewinequalityistight.Thenewvertexisthusgivenby3and4.Soweknowwhattodoifweareattheorigin.Butwhatifourcurrentvertexuiselse-where?Thetrickistotransformuintotheorigin,byshiftingthecoordinatesystemfromtheusual(x1;:::;xn)tothe“localview”fromu.Theselocalcoordinatesconsistof(appropriatelyscaled)distancesy1;:::;yntothenhyperplanes(inequalities)thatdeneandencloseu:215\nuy2y1xSpecically,ifoneoftheseenclosinginequalitiesisaixbi,thenthedistancefromapointxtothatparticular“wall”isyi=bi aix:Thenequationsofthistype,oneperwall,denetheyi'saslinearfunctionsofthexi's,andthisrelationshipcanbeinvertedtoexpressthexi'sasalinearfunctionoftheyi's.ThuswecanrewritetheentireLPintermsofthey's.Thisdoesn'tfundamentallychangeit(forinstance,theoptimalvaluestaysthesame),butexpressesitinadifferentcoordinateframe.Therevised“local”LPhasthefollowingthreeproperties:1.Itincludestheinequalitiesy0,whicharesimplythetransformedversionsoftheinequalitiesdeningu.2.uitselfistheorigininy-space.3.Thecostfunctionbecomesmaxcu+c~Ty,wherecuisthevalueoftheobjectivefunctionatuandc~isatransformedcostvector.Inshort,wearebacktothesituationweknowhowtohandle!Figure7.13showsthisalgo-rithminaction,continuingwithourearlierexample.Thesimplexalgorithmisnowfullydened.Itmovesfromvertextoneighboringvertex,stoppingwhentheobjectivefunctionislocallyoptimal,thatis,whenthecoordinatesofthelocalcostvectorareallzeroornegative.Aswe'vejustseen,avertexwiththispropertymustalsobegloballyoptimal.Ontheotherhand,ifthecurrentvertexisnotlocallyoptimal,thenitslocalcoordinatesystemincludessomedimensionalongwhichtheobjectivefunctioncanbeimproved,sowemovealongthisdirection—alongthisedgeofthepolyhedron—untilwereachaneighboringvertex.Bythenondegeneracyassumption(seefootnote3inSection7.6.1),thisedgehasnonzerolength,andsowestrictlyimprovetheobjectivevalue.Thustheprocessmusteventuallyhalt.216\nFigure7.13Simplexinaction.InitialLP:Currentvertex:f4;
g5(origin).Objectivevalue:0.max2x1+5x22x1 x241Move:increasex2.5isreleased,3becomestight.Stopatx2=3.x1+2x292 x1+x233Newvertexf4;
g3haslocalcoordinates(y1;y2):x104y1=x1;y2=3+x1 x2x205RewrittenLP:Currentvertex:f4;
g3.Objectivevalue:15.max15+7y1 5y2y1+y271Move:increasey1.4isreleased,2becomestight.Stopaty1=1.3y1 2y232y203Newvertexf2;
g3haslocalcoordinates(z1;z2):y104z1=3 3y1+2y2;z2=y2 y1+y235RewrittenLP:Currentvertex:f2;
g3.Objectivevalue:22.max22 7z 1z3132 1z+5z61Optimal:allci<0.3132z102Solve2;3(inoriginalLP)togetoptimalsolutionz203(x1;x2)=(1;4).1z 2z1313241z+1z431325f2;
g3Increasey1f3;
g4f1;
g2Increasex2f4;
g5f1;
g5217\n7.6.3LooseendsThereareseveralimportantissuesinthesimplexalgorithmthatwehaven'tyetmentioned.Thestartingvertex.Howdowendavertexatwhichtostartsimplex?Inour2Dand3Dexampleswealwaysstartedattheorigin,whichworkedbecausethelinearprogramshappenedtohaveinequalitieswithpositiveright-handsides.InageneralLPwewon'talwaysbesofortunate.However,itturnsoutthatndingastartingvertexcanbereducedtoanLPandsolvedbysimplex!Toseehowthisisdone,startwithanylinearprograminstandardform(recallSec-tion7.1.4),sinceweknowLPscanalwaysberewrittenthisway.mincTxsuchthatAx=bandx0:Werstmakesurethattheright-handsidesoftheequationsareallnonnegative:ifbi<0,justmultiplybothsidesoftheithequationby 1.ThenwecreateanewLPasfollows:Createmnewarticialvariablesz1;:::;zm0,wheremisthenumberofequations.Addzitotheleft-handsideoftheithequation.Lettheobjective,tobeminimized,bez1+z2++zm.ForthisnewLP,it'seasytocomeupwithastartingvertex,namely,theonewithzi=biforalliandallothervariableszero.Thereforewecansolveitbysimplex,toobtaintheoptimumsolution.Therearetwocases.Iftheoptimumvalueofz1++zmiszero,thenallzi'sobtainedbysimplexarezero,andhencefromtheoptimumvertexofthenewLPwegetastartingfeasiblevertexoftheoriginalLP,justbyignoringthezi's.Wecanatlaststartsimplex!Butwhatiftheoptimumobjectiveturnsouttobepositive?Letusthink.Wetriedtominimizethesumofthezi's,butsimplexdecidedthatitcannotbezero.Butthismeansthattheoriginallinearprogramisinfeasible:itneedssomenonzerozi'stobecomefeasible.ThisishowsimplexdiscoversandreportsthatanLPisinfeasible.Degeneracy.InthepolyhedronofFigure7.12vertexBisdegenerate.Geometrically,thismeansthatitistheintersectionofmorethann=3facesofthepolyhedron(inthiscase,2;3;4;5).Algebraically,itmeansthatifwechooseanyoneoffoursetsofthreeinequal-ities(f2;3;4g;f2;3;
g5;f2;4;
g5,andf3;4;
g5)andsolvethecorrespondingsystemofthreelinearequationsinthreeunknowns,we'llgetthesamesolutioninallfourcases:(0;300;100).Thisisaseriousproblem:simplexmayreturnasuboptimaldegeneratevertexsimplybecauseallitsneighborsareidenticaltoitandthushavenobetterobjective.Andifwemodifysimplexsothatitdetectsdegeneracyandcontinuestohopfromvertextovertexdespitelackofanyimprovementinthecost,itmayenduploopingforever.Onewaytoxthisisbyaperturbation:changeeachbibyatinyrandomamounttobii.Thisdoesn'tchangetheessenceoftheLPsincethei'saretiny,butithastheeffectofdiffer-entiatingbetweenthesolutionsofthelinearsystems.Toseewhygeometrically,imaginethatthefourplanes2;3;4;5werejoltedalittle.Wouldn'tvertexBsplitintotwovertices,very218\nclosetooneanother?Unboundedness.InsomecasesanLPisunbounded,inthatitsobjectivefunctioncanbemadearbitrarilylarge(orsmall,ifit'saminimizationproblem).Ifthisisthecase,simplexwilldiscoverit:inexploringtheneighborhoodofavertex,itwillnoticethattakingoutaninequalityandaddinganotherleadstoanunderdeterminedsystemofequationsthathasaninnityofsolutions.Andinfact(thisisaneasytest)thespaceofsolutionscontainsawholelineacrosswhichtheobjectivecanbecomelargerandlarger,allthewayto1.Inthiscasesimplexhaltsandcomplains.7.6.4TherunningtimeofsimplexWhatistherunningtimeofsimplex,foragenericlinearprogrammaxcTxsuchthatAx0andx0;wheretherearenvariablesandAcontainsminequalityconstraints?Sinceitisaniterativealgorithmthatproceedsfromvertextovertex,let'sstartbycomputingthetimetakenforasingleiteration.Supposethecurrentvertexisu.Bydenition,itistheuniquepointatwhichninequalityconstraintsaresatisedwithequality.Eachofitsneighborssharesn 1oftheseinequalities,soucanhaveatmostnmneighbors:choosewhichinequalitytodropandwhichnewonetoadd.Anaivewaytoperformaniterationwouldbetocheckeachpotentialneighbortoseewhetheritreallyisavertexofthepolyhedronandtodetermineitscost.Findingthecostisquick,justadotproduct,butcheckingwhetheritisatruevertexinvolvessolvingasystemofnequationsinnunknowns(thatis,satisfyingthenchoseninequalitiesexactly)andcheckingwhethertheresultisfeasible.ByGaussianelimination(seethefollowingbox)thistakesO(n3)time,givinganunappetizingrunningtimeofO(mn4)periteration.Fortunately,thereisamuchbetterway,andthismn4factorcanbeimprovedtomn,mak-ingsimplexapracticalalgorithm.Recallourearlierdiscussion(Section7.6.2)aboutthelocalviewfromvertexu.Itturnsoutthattheper-iterationoverheadofrewritingtheLPintermsofthecurrentlocalcoordinatesisjustO((m+n)n);thisexploitsthefactthatthelocalviewchangesonlyslightlybetweeniterations,injustoneofitsdeninginequalities.Next,toselectthebestneighbor,werecallthatthe(localviewof)theobjectivefunctionisoftheform“maxcu+c~y”wherecuisthevalueoftheobjectivefunctionatu.Thisimmediatelyidentiesapromisingdirectiontomove:wepickanyc~i>0(ifthereisnone,thenthecurrentvertexisoptimalandsimplexhalts).SincetherestoftheLPhasnowbeenrewrittenintermsofthey-coordinates,itiseasytodeterminehowmuchyicanbeincreasedbeforesomeotherinequalityisviolated.(Andifwecanincreaseyiindenitely,weknowtheLPisunbounded.)ItfollowsthattherunningtimeperiterationofsimplexisjustO(mn).Buthowmany m+niterationscouldtherebe?Naturally,therecan'tbemorethan,whichisanupperboundnonthenumberofvertices.Butthisupperboundisexponentialinn.Andinfact,thereareexamplesofLPsforwhichsimplexdoesindeedtakeanexponentialnumberofiterations.Inotherwords,simplexisanexponential-timealgorithm.However,suchexponentialexamplesdonotoccurinpractice,anditisthisfactthatmakessimplexsovaluableandsowidelyused.219\n220\nGaussianeliminationUnderouralgebraicdenition,merelywritingdownthecoordinatesofavertexinvolvessolvingasystemoflinearequations.Howisthisdone?Wearegivenasystemofnlinearequationsinnunknowns,sayn=4andx1 2x3=2x2+x3=3x1+x2 x4=4x2+3x3+x4=5Thehighschoolmethodforsolvingsuchsystemsistorepeatedlyapplythefollowingrule:ifweaddamultipleofoneequationtoanotherequation,theoverallsystemofequationsremainsequivalent.Forexample,adding 1timestherstequationtothethirdone,wegettheequivalentsystemx1 2x3=2x2+x3=3x2+2x3 x4=2x2+3x3+x4=5Thistransformationiscleverinthefollowingsense:iteliminatesthevariablex1fromthethirdequation,leavingjustoneequationwithx1.Inotherwords,ignoringtherstequation,wehaveasystemofthreeequationsinthreeunknowns:wedecreasednby1!Wecansolvethissmallersystemtogetx2;x3;x4,andthenplugtheseintotherstequationtogetx1.Thissuggestsanalgorithm—oncemoreduetoGauss.proceduregauss(E;X)Input:AsystemE=fe1;:::;engofequationsinnunknownsX=fx1;:::;xng:e1:a11x1+a12x2++a1nxn=b1;;en:an1x1+an2x2++annxn=bnOutput:Asolutionofthesystem,ifoneexistsifallcoefficientsai1arezero:haltwithmessage``eitherinfeasibleornotlinearlyindependent''ifn=1:returnb1=a11choosethecoefficientap1oflargestmagnitude,andswapequationse1;epfori=2ton:ei=ei (ai1=a11)e1(x2;:::;xn)P=gauss(E fe1g;X fx1g)x1=(b1 j>1a1jxj)=a11return(x1;:::;xn)(Whenchoosingtheequationtoswapintorstplace,wepicktheonewithlargestjap1jforreasonsofnumericalaccuracy;afterall,wewillbedividingbyap1.)GaussianeliminationusesO(n2)arithmeticoperationstoreducetheproblemsizefromnton 1,andthususesO(n3)operationsoverall.Toshowthatthisisalsoagoodestimateofthetotalrunningtime,weneedtoarguethatthenumbersinvolvedremainpolynomi-allybounded—forinstance,thatthesolution(x1;:::;xn)doesnotrequiretoomuchmoreprecisiontowritedownthantheoriginalcoefcientsaijandbi.Doyouseewhythisistrue?221\nLinearprogramminginpolynomialtimeSimplexisnotapolynomialtimealgorithm.Certainrarekindsoflinearprogramscauseittogofromonecornerofthefeasibleregiontoabettercornerandthentoastillbetterone,andsoonforanexponentialnumberofsteps.Foralongtime,linearprogrammingwasconsideredaparadox,aproblemthatcanbesolvedinpractice,butnotintheory!Then,in1979,ayoungSovietmathematiciancalledLeonidKhachiyancameupwiththeellipsoidalgorithm,onethatisverydifferentfromsimplex,extremelysimpleinitsconception(butsophisticatedinitsproof)andyetonethatsolvesanylinearprograminpolynomialtime.Insteadofchasingthesolutionfromonecornerofthepolyhedrontothenext,Khachiyan'salgorithmconnesittosmallerandsmallerellipsoids(skewedhigh-dimensionalballs).Whenthisalgorithmwasannounced,itbecameakindof“mathematicalSputnik,”asplashyachievementthathadtheU.S.establishmentworried,intheheightoftheColdWar,aboutthepossiblescienticsuperiorityoftheSovietUnion.Theellipsoidalgorithmturnedouttobeanimportanttheoreticaladvance,butdidnotcompetewellwithsimplexinpractice.Theparadoxoflinearprogrammingdeepened:Aproblemwithtwoalgorithms,onethatisefcientintheory,andonethatisefcientinpractice!AfewyearslaterNarendraKarmarkar,agraduatestudentatUCBerkeley,cameupwithacompletelydifferentidea,whichledtoanotherprovablypolynomialalgorithmforlinearprogramming.Karmarkar'salgorithmisknownastheinteriorpointmethod,becauseitdoesjustthat:itdashestotheoptimumcornernotbyhoppingfromcornertocorneronthesurfaceofthepolyhedronlikesimplexdoes,butbycuttingacleverpathintheinteriorofthepolyhedron.Anditdoesperformwellinpractice.ButperhapsthegreatestadvanceinlinearprogrammingalgorithmswasnotKhachiyan'stheoreticalbreakthroughorKarmarkar'snovelapproach,butanunexpectedconsequenceofthelatter:theercecompetitionbetweenthetwoapproaches,simplexandinteriorpoint,resultedinthedevelopmentofveryfastcodeforlinearprogramming.7.7Postscript:circuitevaluationTheimportanceoflinearprogrammingstemsfromtheastoundingvarietyofproblemsthatreducetoitandtherebybearwitnesstoitsexpressivepower.Inasense,thisnextoneistheultimateapplication.WearegivenaBooleancircuit,thatis,adagofgatesofthefollowingtypes.Inputgateshaveindegreezero,withvaluetrueorfalse.ANDgatesandORgateshaveindegree2.NOTgateshaveindegree1.Inaddition,oneofthegatesisdesignatedastheoutput.Here'sanexample.222\noutputANDNOTORANDORNOTtruefalsetrueTheCIRCUITVALUEproblemisthefollowing:whenthelawsofBooleanlogicareappliedtothegatesintopologicalorder,doestheoutputevaluatetotrue?Thereisasimple,automaticwayoftranslatingthisproblemintoalinearprogram.Createavariablexgforeachgateg,withconstraints0xg1.Addadditionalconstraintsforeachtypeofgate:ggggateggORANDNOTtruefalsexg=1xg=0hh0hh0hxgxhxgxhxg=1 xhxgxh0xgxh0xgxh+xh0xgxh+xh0 1Theseconstraintsforceallthegatestotakeonexactlytherightvalues—0forfalse,and1fortrue.Wedon'tneedtomaximizeorminimizeanything,andwecanreadtheanswerofffromthevariablexocorrespondingtotheoutputgate.Thisisastraightforwardreductiontolinearprogramming,fromaproblemthatmaynotseemveryinterestingatrst.However,theCIRCUITVALUEproblemisinasensethemostgeneralproblemsolvableinpolynomialtime!Afterall,anyalgorithmwilleventuallyrunonacomputer,andthecomputerisultimatelyaBooleancombinationalcircuitimplementedonachip.Ifthealgorithmrunsinpolynomialtime,itcanberenderedasaBooleancircuitcon-sistingofpolynomiallymanycopiesofthecomputer'scircuit,oneperunitoftime,withthevaluesofthegatesinonelayerusedtocomputethevaluesforthenext.Hence,thefactthatCIRCUITVALUEreducestolinearprogrammingmeansthatallproblemsthatcanbesolvedinpolynomialtimedo!Inournexttopic,NP-completeness,weshallseethatmanyhardproblemsreduce,muchthesameway,tointegerprogramming,linearprogramming'sdifculttwin.223\nAnotherpartingthought:bywhatothermeanscanthecircuitevaluationproblembesolved?Let'sthink—acircuitisadag.Andwhatalgorithmictechniqueismostappropriateforsolvingproblemsondags?That'sright:dynamicprogramming!Togetherwithlinearprogramming,theworld'stwomostgeneralalgorithmictechniques.224\nExercises7.1.Considerthefollowinglinearprogram.maximize5x+3y5x 2y0x+y7x5x0y0Plotthefeasibleregionandidentifytheoptimalsolution.7.2.DuckwheatisproducedinKansasandMexicoandconsumedinNewYorkandCalifornia.Kansasproduces15shnupellsofduckwheatandMexico8.Meanwhile,NewYorkconsumes10shnupellsandCalifornia13.Thetransportationcostspershnupellare$4fromMexicotoNewYork,$1fromMexicotoCalifornia,$2fromKansastoNewYork,and$3andfromKansastoCalifornia.Writealinearprogramthatdecidestheamountsofduckwheat(inshnupellsandfractionsofashnupell)tobetransportedfromeachproducertoeachconsumer,soastominimizetheoveralltransportationcost.7.3.Acargoplanecancarryamaximumweightof100tonsandamaximumvolumeof60cubicmeters.Therearethreematerialstobetransported,andthecargocompanymaychoosetocarryanyamountofeach,uptothemaximumavailablelimitsgivenbelow.Material1hasdensity2tons/cubicmeter,maximumavailableamount40cubicmeters,andrevenue$1,000percubicmeter.Material2hasdensity1ton/cubicmeter,maximumavailableamount30cubicmeters,andrevenue$1,200percubicmeter.Material3hasdensity3tons/cubicmeter,maximumavailableamount20cubicmeters,andrevenue$12,000percubicmeter.Writealinearprogramthatoptimizesrevenuewithintheconstraints.7.4.MoeisdecidinghowmuchRegularDuffbeerandhowmuchDuffStrongbeertoordereachweek.RegularDuffcostsMoe$1perpintandhesellsitat$2perpint;DuffStrongcostsMoe$1:50perpintandhesellsitat$3perpint.However,aspartofacomplicatedmarketingscam,theDuffcompanywillonlysellapintofDuffStrongforeachtwopintsormoreofRegularDuffthatMoebuys.Furthermore,duetopasteventsthatarebetterleftuntold,DuffwillnotsellMoemorethan3;000pintsperweek.Moeknowsthathecansellhowevermuchbeerhehas.FormulatealinearprogramfordecidinghowmuchRegularDuffandhowmuchDuffStrongtobuy,soastomaximizeMoe'sprot.Solvetheprogramgeometrically.7.5.TheCanineProductscompanyofferstwodogfoods,FriskyPupandHuskyHound,thataremadefromablendofcerealandmeat.ApackageofFriskyPuprequires1poundofcerealand1:5poundsofmeat,andsellsfor$7.ApackageofHuskyHounduses2poundsofcerealand1poundofmeat,andsellsfor$6.Rawcerealcosts$1perpoundandrawmeatcosts$2perpound.Italsocosts$1:40topackagetheFriskyPupand$0:60topackagetheHuskyHound.Atotalof240;000poundsofcerealand180;000poundsofmeatareavailableeachmonth.Theonlyproductionbottleneckisthatthefactorycanonlypackage110;000bagsofFriskyPuppermonth.Needlesstosay,managementwouldliketomaximizeprot.(a)Formulatetheproblemasalinearprogramintwovariables.225\n(b)Graphthefeasibleregion,givethecoordinatesofeveryvertex,andcirclethevertexmaxi-mizingprot.Whatisthemaximumprotpossible?7.6.Giveanexampleofalinearprogramintwovariableswhosefeasibleregionisinnite,butsuchthatthereisanoptimumsolutionofboundedcost.7.7.Findnecessaryandsufcientconditionsontherealsaandbunderwhichthelinearprogrammaxx+yax+by1x;y0(a)Isinfeasible.(b)Isunbounded.(c)Hasauniqueoptimalsolution.7.8.Youaregiventhefollowingpointsintheplane:(1;3);(2;5);(3;7);(5;11);(7;14);(8;15);(10;19):Youwanttondalineax+by=cthatapproximatelypassesthroughthesepoints(nolineisaperfectt).Writealinearprogram(youdon'tneedtosolveit)tondthelinethatminimizesthemaximumabsoluteerror,maxjaxi+byi cj:1i77.9.Aquadraticprogrammingproblemseekstomaximizeaquadratricobjectivefunction(withtermslike3x2or5xx)subjecttoasetoflinearconstraints.Giveanexampleofaquadraticprogram112intwovariablesx1;x2suchthatthefeasibleregionisnonemptyandbounded,andyetnoneoftheverticesofthisregionoptimizethe(quadratic)objective.7.10.Forthefollowingnetwork,withedgecapacitiesasshown,ndthemaximumowfromStoT,alongwithamatchingcut.4AD5G612210121SB20E6T2104CF57.11.Writethedualtothefollowinglinearprogram.maxx+y2x+y3x+3y5x;y0FindtheoptimalsolutionstobothprimalanddualLPs.226\n7.12.Forthelinearprogrammaxx1 2x3x1 x212x2 x31x1;x2;x30provethatthesolution(x1;x2;x3)=(3=2;1=2;0)isoptimal.7.13.Matchingpennies.Inthissimpletwo-playergame,theplayers(callthemRandC)eachchooseanoutcome,headsortails.Ifbothoutcomesareequal,CgivesadollartoR;iftheoutcomesaredifferent,RgivesadollartoC.(a)Representthepayoffsbya22matrix.(b)Whatisthevalueofthisgame,andwhataretheoptimalstrategiesforthetwoplayers?7.14.ThepizzabusinessinLittleTownissplitbetweentworivals,TonyandJoey.Theyareeachinvestigatingstrategiestostealbusinessawayfromtheother.Joeyisconsideringeitherloweringpricesorcuttingbiggerslices.Tonyislookingintostartingupalineofgourmetpizzas,orofferingoutdoorseating,orgivingfreesodasatlunchtime.Theeffectsofthesevariousstrategiesaresummarizedinthefollowingpayoffmatrix(entriesaredozensofpizzas,Joey'sgainandTony'sloss).TONYGourmetSeatingFreesodaJOEYLowerprice+20 3Biggerslices 1 2+1Forinstance,ifJoeyreducespricesandTonygoeswiththegourmetoption,thenTonywilllose2dozenpizzasworthofbusinesstoJoey.Whatisthevalueofthisgame,andwhataretheoptimalstrategiesforTonyandJoey?7.15.Findthevalueofthegamespeciedbythefollowingpayoffmatrix.00 1 101 2 1 1 111 10011 20 31 1 1 10 32 10 21 1(Hint:Considerthemixedstrategies(1=3;0;0;1=2;1=6;0;0;0)and(2=3;0;0;1=3).)7.16.Asaladisanycombinationofthefollowingingredients:(1)tomato,(2)lettuce,(3)spinach,(4)carrot,and(5)oil.Eachsaladmustcontain:(A)atleast15gramsofprotein,(B)atleast2andatmost6gramsoffat,(C)atleast4gramsofcarbohydrates,(D)atmost100milligramsofsodium.Furthermore,(E)youdonotwantyoursaladtobemorethan50%greensbymass.Thenutritionalcontentsoftheseingredients(per100grams)are227\ningredientenergyproteinfatcarbohydratesodium(kcal)(grams)(grams)(grams)(milligrams)tomato210.850.334.649.00lettuce161.620.202.378.00spinach37112.781.5874.697.00carrot3468.391.3980.70508.20oil8840.00100.000.000.00FindalinearprogrammingappletontheWebanduseittomakethesaladwiththefewestcaloriesunderthenutritionalconstraints.Describeyourlinearprogrammingformulationandtheoptimalsolution(thequantityofeachingredientandthevalue).CitetheWebresourcesthatyouused.7.17.Considerthefollowingnetwork(thenumbersareedgecapacities).4AC972ST265BD3(a)Findthemaximumowfandaminimumcut.(b)DrawtheresidualgraphGf(alongwithitsedgecapacities).Inthisresidualnetwork,marktheverticesreachablefromSandtheverticesfromwhichTisreachable.(c)Anedgeofanetworkiscalledabottleneckedgeifincreasingitscapacityresultsinanincreaseinthemaximumow.Listallbottleneckedgesintheabovenetwork.(d)Giveaverysimpleexample(containingatmostfournodes)ofanetworkwhichhasnobottleneckedges.(e)Giveanefcientalgorithmtoidentifyallbottleneckedgesinanetwork.(Hint:Startbyrunningtheusualnetworkowalgorithm,andthenexaminetheresidualgraph.)7.18.Therearemanycommonvariationsofthemaximumowproblem.Herearefourofthem.(a)Therearemanysourcesandmanysinks,andwewishtomaximizethetotalowfromallsourcestoallsinks.(b)Eachvertexalsohasacapacityonthemaximumowthatcanenterit.(c)Eachedgehasnotonlyacapacity,butalsoalowerboundontheowitmustcarry.(d)Theoutgoingowfromeachnodeuisnotthesameastheincomingow,butissmallerbyafactorof(1 u),whereuisalosscoefcientassociatedwithnodeu.Eachofthesecanbesolvedefciently.Showthisbyreducing(a)and(b)totheoriginalmax-owproblem,andreducing(c)and(d)tolinearprogramming.7.19.Supposesomeonepresentsyouwithasolutiontoamax-owproblemonsomenetwork.Givealineartimealgorithmtodeterminewhetherthesolutiondoesindeedgiveamaximumow.7.20.Considerthefollowinggeneralizationofthemaximumowproblem.YouaregivenadirectednetworkG=(V;E)withedgecapacitiesfceg.Insteadofasingle(s;t)pair,youaregivenmultiplepairs(s1;t1);(s2;t2);:::;(sk;tk),wherethesiaresourcesofGandthetaresinksofG.Youarealsogivenkdemandsd;:::;d.Thegoalistondkowsf(1);:::;f(k)i1kwiththefollowingproperties:228\nf(i)isavalidowfromstot.ii(1)(2)(k)Foreachedgee,thetotalowfe+fe++fedoesnotexceedthecapacityce.Thesizeofeachowf(i)isatleastthedemandd.iThesizeofthetotalow(thesumoftheows)isaslargeaspossible.Howwouldyousolvethisproblem?7.21.Anedgeofaownetworkiscalledcriticalifdecreasingthecapacityofthisedgeresultsinadecreaseinthemaximumow.Giveanefcientalgorithmthatndsacriticaledgeinanetwork.7.22.InaparticularnetworkG=(V;E)whoseedgeshaveintegercapacitiesce,wehavealreadyfoundthemaximumowffromnodestonodet.However,wenowndoutthatoneofthecapacityvaluesweusedwaswrong:foredge(u;v)weusedcuvwhereasitshouldhavebeencuv 1.Thisisunfortunatebecausetheowfusesthatparticularedgeatfullcapacity:fuv=cuv.Wecouldredotheowcomputationfromscratch,butthere'safasterway.ShowhowanewoptimalowcanbecomputedinO(jVj+jEj)time.7.23.AvertexcoverofanundirectedgraphG=(V;E)isasubsetoftheverticeswhichtoucheseveryedge—thatis,asubsetSVsuchthatforeachedgefu;vg2E,oneorbothofu;vareinS.Showthattheproblemofndingtheminimumvertexcoverinabipartitegraphreducestomax-imumow.(Hint:Canyourelatethisproblemtotheminimumcutinanappropriatenetwork?)7.24.Directbipartitematching.We'veseenhowtondamaximummatchinginabipartitegraphviareductiontothemaximumowproblem.Wenowdevelopadirectalgorithm.LetG=(V1[V2;E)beabipartitegraph(soeachedgehasoneendpointinV1andoneendpointinV2),andletM2Ebeamatchinginthegraph(thatis,asetofedgesthatdon'ttouch).AvertexissaidtobecoveredbyMifitistheendpointofoneoftheedgesinM.Analternatingpathisapathofoddlengththatstartsandendswithanon-coveredvertex,andwhoseedgesalternatebetweenMandE M.(a)Inthebipartitegraphbelow,amatchingMisshowninbold.Findanalternatingpath.EAFBGCHDI(b)ProvethatamatchingMismaximalifandonlyiftheredoesnotexistanalternatingpathwithrespecttoit.(c)DesignanalgorithmthatndsanalternatingpathinO(jVj+jEj)timeusingavariantofbreadth-rstsearch.(d)GiveadirectO(jVjjEj)algorithmforndingamaximalmatchinginabipartitegraph.7.25.Thedualofmaximumow.Considerthefollowingnetworkwithedgecapacities.229\nA12S1T31B(a)WritetheproblemofndingthemaximumowfromStoTasalinearprogram.(b)Writedownthedualofthislinearprogram.ThereshouldbeadualvariableforeachedgeofthenetworkandforeachvertexotherthanS;T.Nowwe'llsolvethesameprobleminfullgenerality.Recallthelinearprogramforageneralmaximumowproblem(Section7.2).(c)WritedownthedualofthisgeneralowLP,usingavariableyeforeachedgeandxuforeachvertexu6=s;t.(d)ShowthatanysolutiontothegeneraldualLPmustsatisfythefollowingproperty:foranydirectedpathfromstotinthenetwork,thesumoftheyevaluesalongthepathmustbeatleast1.(e)Whataretheintuitivemeaningsofthedualvariables?Showthatanys tcutinthenetworkcanbetranslatedintoadualfeasiblesolutionwhosecostisexactlythecapacityofthatcut.7.26.Inasatisablesystemoflinearinequalitiesa11x1++a1nxnb1...am1x1++amnxnbmwedescribethejthinequalityasforced-equalifitissatisedwithequalitybyeverysolutionPx=(x1;:::;xn)ofthesystem.Equivalently,Piajixibjisnotforced-equalifthereexistsanxthatsatisesthewholesystemandsuchthatiajixi0.P(a)Showthatthisisequivalenttondingans towfthatminimizeselefesubjecttosize(f)=1.Therearenocapacityconstraints.(b)Writetheshortestpathproblemasalinearprogram.(c)ShowthatthedualLPcanbewrittenasmaxxs xtxu xvluvforall(u;v)2E(d)Aninterpretationforthedualisgivenintheboxonpage210.Whyisn'tourdualLPidenticaltotheoneonthatpage?7.29.Hollywood.Almproducerisseekingactorsandinvestorsforhisnewmovie.Therearenavailableactors;actorichargessidollars.Forfunding,therearemavailableinvestors.Investorjwillprovidepjdollars,butonlyontheconditionthatcertainactorsLjf1;2;:::;ngareincludedinthecast(alloftheseactorsLjmustbechoseninordertoreceivefundingfrominvestorj).Theproducer'sprotisthesumofthepaymentsfrominvestorsminusthepaymentstoactors.Thegoalistomaximizethisprot.(a)Expressthisproblemasanintegerlinearprograminwhichthevariablestakeonvaluesf0;1g.(b)Nowrelaxthistoalinearprogram,andshowthattheremustinfactbeanintegraloptimalsolution(asisthecase,forexample,withmaximumowandbipartitematching).7.30.Hall'stheorem.ReturningtothematchmakingscenarioofSection7.3,supposewehaveabipar-titegraphwithboysontheleftandanequalnumberofgirlsontheright.Hall'stheoremsaysthatthereisaperfectmatchingifandonlyifthefollowingconditionholds:anysubsetSofboysisconnectedtoatleastjSjgirls.Provethistheorem.(Hint:Themax-owmin-cuttheoremshouldbehelpful.)7.31.Considerthefollowingsimplenetworkwithedgecapacitiesasshown.A10001000S1T10001000B(a)Showthat,iftheFord-Fulkersonalgorithmisrunonthisgraph,acarelesschoiceofupdatesmightcauseittotake1000iterations.Imagineifthecapacitieswereamillioninsteadof1000!Wewillnowndastrategyforchoosingpathsunderwhichthealgorithmisguaranteedtoter-minateinareasonablenumberofiterations.Consideranarbitrarydirectednetwork(G=(V;E);s;t;fceg)inwhichwewanttondthemax-imumow.Assumeforsimplicitythatalledgecapacitiesareatleast1,anddenethecapacityofans tpathtobethesmallestcapacityofitsconstituentedges.Thefattestpathfromstotisthepathwiththemostcapacity.231\n(b)Showthatthefattests tpathinagraphcanbecomputedbyavariantofDijkstra'salgorithm.(c)ShowthatthemaximumowinGisthesumofindividualowsalongatmostjEjpathsfromstot.(d)Nowshowthatifwealwaysincreaseowalongthefattestpathintheresidualgraph,thentheFord-FulkersonalgorithmwillterminateinatmostO(jEjlogF)iterations,whereFisthesizeofthemaximumow.(Hint:ItmighthelptorecalltheproofforthegreedysetcoveralgorithminSection5.4.)Infact,anevensimplerrule—ndingapathintheresidualgraphusingbreadth-rstsearch—guaranteesthatatmostO(jVjjEj)iterationswillbeneeded.232\nChapter8NP-completeproblems8.1SearchproblemsOverthepastsevenchapterswehavedevelopedalgorithmsforndingshortestpathsandminimumspanningtreesingraphs,matchingsinbipartitegraphs,maximumincreasingsub-sequences,maximumowsinnetworks,andsoon.Allthesealgorithmsareefcient,becauseineachcasetheirtimerequirementgrowsasapolynomialfunction(suchasn,n2,orn3)ofthesizeoftheinput.Tobetterappreciatesuchefcientalgorithms,considerthealternative:Inalltheseprob-lemswearesearchingforasolution(path,tree,matching,etc.)fromamonganexponentialpopulationofpossibilities.Indeed,nboyscanbematchedwithngirlsinn!differentways,agraphwithnverticeshasnn 2spanningtrees,andatypicalgraphhasanexponentialnum-berofpathsfromstot.Alltheseproblemscouldinprinciplebesolvedinexponentialtimebycheckingthroughallcandidatesolutions,onebyone.Butanalgorithmwhoserunningtimeis2n,orworse,isallbutuselessinpractice(seethenextbox).Thequestforefcientalgorithmsisaboutndingcleverwaystobypassthisprocessofexhaustivesearch,usingcluesfromtheinputinordertodramaticallynarrowdownthesearchspace.Sofarinthisbookwehaveseenthemostbrilliantsuccessesofthisquest,algorithmictech-niquesthatdefeatthespecterofexponentiality:greedyalgorithms,dynamicprogramming,linearprogramming(whiledivide-and-conquertypicallyyieldsfasteralgorithmsforproblemswecanalreadysolveinpolynomialtime).Nowthetimehascometomeetthequest'smostembarrassingandpersistentfailures.Weshallseesomeother“searchproblems,”inwhichagainweareseekingasolutionwithparticularpropertiesamonganexponentialchaosofal-ternatives.Butforthesenewproblemsnoshortcutseemspossible.Thefastestalgorithmsweknowforthemareallexponential—notsubstantiallybetterthananexhaustivesearch.Wenowintroducesomeimportantexamples.233\nThestoryofSissaandMooreAccordingtothelegend,thegameofchesswasinventedbytheBrahminSissatoamuseandteachhisking.Askedbythegratefulmonarchwhathewantedinreturn,thewisemanrequestedthatthekingplaceonegrainofriceintherstsquareofthechessboard,twointhesecond,fourinthethird,andsoon,doublingtheamountofriceuptothe64thsquare.Thekingagreedonthespot,andasaresulthewastherstpersontolearnthevaluable—-albeithumbling—lessonofexponentialgrowth.Sissa'srequestamountedto264 1=18;446;744;073;709;551;615grainsofrice,enoughricetopaveallofIndiaseveraltimesover!Allovernature,fromcoloniesofbacteriatocellsinafetus,weseesystemsthatgrowexponentially—forawhile.In1798,theBritishphilosopherT.RobertMalthuspublishedanessayinwhichhepredictedthattheexponentialgrowth(hecalledit“geometricgrowth”)ofthehumanpopulationwouldsoondepletelinearlygrowingresources,anargumentthatinuencedCharlesDarwindeeply.Malthusknewthefundamentalfactthatanexponentialsoonerorlatertakesoveranypolynomial.In1965,computerchippioneerGordonE.Moorenoticedthattransistordensityinchipshaddoubledeveryyearintheearly1960s,andhepredictedthatthistrendwouldcontinue.Thisprediction,moderatedtoadoublingevery18monthsandextendedtocomputerspeed,isknownasMoore'slaw.Ithasheldremarkablywellfor40years.Andthesearethetworootcausesoftheexplosionofinformationtechnologyinthepastdecades:Moore'slawandefcientalgorithms.ItwouldappearthatMoore'slawprovidesadisincentivefordevelopingpolynomialal-gorithms.Afterall,ifanalgorithmisexponential,whynotwaititoutuntilMoore'slawmakesitfeasible?Butinrealitytheexactoppositehappens:Moore'slawisahugeincen-tivefordevelopingefcientalgorithms,becausesuchalgorithmsareneededinordertotakeadvantageoftheexponentialincreaseincomputerspeed.Hereiswhy.If,forexample,anO(2n)algorithmforBooleansatisability(SAT)weregivenanhourtorun,itwouldhavesolvedinstanceswith25variablesbackin1975,31vari-ablesonthefastercomputersavailablein1985,38variablesin1995,andabout45variableswithtoday'smachines.Quiteabitofprogress—exceptthateachextravariablerequiresayearandahalf'swait,whiletheappetiteofapplications(manyofwhichare,ironically,re-latedtocomputerdesign)growsmuchfaster.Incontrast,thesizeoftheinstancessolvedbyanO(n)orO(nlogn)algorithmwouldbemultipliedbyafactorofabout100eachdecade.InthecaseofanO(n2)algorithm,theinstancesizesolvableinaxedtimewouldbemul-tipliedbyabout10eachdecade.EvenanO(n6)algorithm,polynomialyetunappetizing,wouldmorethandoublethesizeoftheinstancessolvedeachdecade.Whenitcomestothegrowthofthesizeofproblemswecanattackwithanalgorithm,wehaveareversal:expo-nentialalgorithmsmakepolynomiallyslowprogress,whilepolynomialalgorithmsadvanceexponentiallyfast!ForMoore'slawtobereectedintheworldweneedefcientalgorithms.AsSissaandMalthusknewverywell,exponentialexpansioncannotbesustainedin-denitelyinourniteworld.Bacterialcoloniesrunoutoffood;chipshittheatomicscale.Moore'slawwillstopdoublingthespeedofourcomputerswithinadecadeortwo.Andthenprogresswilldependonalgorithmicingenuity—orotherwiseperhapsonnovelideassuchasquantumcomputation,exploredinChapter10.234\nSatisabilitySATISFIABILITY,orSAT(recallExercise3.28andSection5.3),isaproblemofgreatpracticalimportance,withapplicationsrangingfromchiptestingandcomputerdesigntoimageanaly-sisandsoftwareengineering.Itisalsoacanonicalhardproblem.Here'swhataninstanceofSATlookslike:(x_y_z)(x_y)(y_z)(z_x)(x_y_z):ThisisaBooleanformulainconjunctivenormalform(CNF).Itisacollectionofclauses(theparentheses),eachconsistingofthedisjunction(logicalor,denoted_)ofseveralliterals,wherealiteraliseitheraBooleanvariable(suchasx)orthenegationofone(suchasx).Asatisfyingtruthassignmentisanassignmentoffalseortruetoeachvariablesothateveryclausecontainsaliteralwhosevalueistrue.TheSATproblemisthefollowing:givenaBooleanformulainconjunctivenormalform,eitherndasatisfyingtruthassignmentorelsereportthatnoneexists.Intheinstanceshownpreviously,settingallvariablestotrue,forexample,satiseseveryclauseexceptthelast.Isthereatruthassignmentthatsatisesallclauses?Withalittlethought,itisnothardtoarguethatinthisparticularcasenosuchtruthassignmentexists.(Hint:Thethreemiddleclausesconstrainallthreevariablestohavethesamevalue.)Buthowdowedecidethisingeneral?Ofcourse,wecanalwayssearchthroughalltruthassignments,onebyone,butforformulaswithnvariables,thenumberofpossibleassignmentsisexponential,2n.SATisatypicalsearchproblem.WearegivenaninstanceI(thatis,someinputdataspecifyingtheproblemathand,inthiscaseaBooleanformulainconjunctivenormalform),andweareaskedtondasolutionS(anobjectthatmeetsaparticularspecication,inthiscaseanassignmentthatsatiseseachclause).Ifnosuchsolutionexists,wemustsayso.Morespecically,asearchproblemmusthavethepropertythatanyproposedsolutionStoaninstanceIcanbequicklycheckedforcorrectness.Whatdoesthisentail?Foronething,Smustatleastbeconcise(quicktoread),withlengthpolynomiallyboundedbythatofI.ThisisclearlytrueinthecaseofSAT,forwhichSisanassignmenttothevariables.Toformalizethenotionofquickchecking,wewillsaythatthereisapolynomial-timealgorithmthattakesasinputIandSanddecideswhetherornotSisasolutionofI.ForSAT,thisiseasyasitjustinvolvescheckingwhethertheassignmentspeciedbySindeedsatiseseveryclauseinI.Laterinthischapteritwillbeusefultoshiftourvantagepointandtothinkofthisefcientalgorithmforcheckingproposedsolutionsasdeningthesearchproblem.Thus:AsearchproblemisspeciedbyanalgorithmCthattakestwoinputs,aninstanceIandaproposedsolutionS,andrunsintimepolynomialinjIj.WesaySisasolutiontoIifandonlyifC(I;S)=true.GiventheimportanceoftheSATsearchproblem,researchersoverthepast50yearshavetriedhardtondefcientwaystosolveit,butwithoutsuccess.Thefastestalgorithmswehavearestillexponentialontheirworst-caseinputs.Yet,interestingly,therearetwonaturalvariantsofSATforwhichwedohavegoodalgo-rithms.Ifallclausescontainatmostonepositiveliteral,thentheBooleanformulaiscalledaHornformula,andasatisfyingtruthassignment,ifoneexists,canbefoundbythegreedyalgorithmofSection5.3.Alternatively,ifallclauseshaveonlytwoliterals,thengraphthe-orycomesintoplay,andSATcanbesolvedinlineartimebyndingthestronglyconnected235\nFigure8.1Theoptimaltravelingsalesmantour,showninbold,haslength18.53233414326componentsofaparticulargraphconstructedfromtheinstance(recallExercise3.28).Infact,inChapter9,we'llseeadifferentpolynomialalgorithmforthissamespecialcase,whichiscalled2SAT.Ontheotherhand,ifwearejustalittlemorepermissiveandallowclausestocontainthreeliterals,thentheresultingproblem,knownas3SAT(anexampleofwhichwesawearlier),onceagainbecomeshardtosolve!ThetravelingsalesmanproblemInthetravelingsalesmanproblem(TSP)wearegivennvertices1;:::;nandalln(n 1)=2distancesbetweenthem,aswellasabudgetb.Weareaskedtondatour,acyclethatpassesthrougheveryvertexexactlyonce,oftotalcostborless—ortoreportthatnosuchtourexists.Thatis,weseekapermutation (1);:::; (n)oftheverticessuchthatwhentheyaretouredinthisorder,thetotaldistancecoveredisatmostb:d (1); (2)+d (2); (3)++d (n); (1)b:SeeFigure8.1foranexample(onlysomeofthedistancesareshown;assumetherestareverylarge).NoticehowwehavedenedtheTSPasasearchproblem:givenaninstance,ndatourwithinthebudget(orreportthatnoneexists).Butwhyareweexpressingthetravelingsalesmanprobleminthisway,wheninrealityitisanoptimizationproblem,inwhichtheshortestpossibletourissought?Whydressitupassomethingelse?Foragoodreason.Ourplaninthischapteristocompareandrelateproblems.Theframeworkofsearchproblemsishelpfulinthisregard,becauseitencompassesoptimizationproblemsliketheTSPinadditiontotruesearchproblemslikeSAT.Turninganoptimizationproblemintoasearchproblemdoesnotchangeitsdifcultyatall,becausethetwoversionsreducetooneanother.AnyalgorithmthatsolvestheoptimizationTSPalsoreadilysolvesthesearchproblem:ndtheoptimumtourandifitiswithinbudget,returnit;ifnot,thereisnosolution.Conversely,analgorithmforthesearchproblemcanalsobeusedtosolvetheoptimizationproblem.Toseewhy,rstsupposethatwesomehowknewthecostoftheoptimumtour;thenwecouldndthistourbycallingthealgorithmforthesearchproblem,usingtheoptimum236\ncostasthebudget.Fine,buthowdowendtheoptimumcost?Easy:Bybinarysearch!(SeeExercise8.1.)Incidentally,thereisasubtletyhere:Whydowehavetointroduceabudget?Isn'tanyoptimizationproblemalsoasearchprobleminthesensethatwearesearchingforasolutionthathasthepropertyofbeingoptimal?Thecatchisthatthesolutiontoasearchproblemshouldbeeasytorecognize,orasweputitearlier,polynomial-timecheckable.Givenapo-tentialsolutiontotheTSP,itiseasytochecktheproperties“isatour”(justcheckthateachvertexisvisitedexactlyonce)and“hastotallengthb.”Buthowcouldonechecktheproperty“isoptimal”?AswithSAT,therearenoknownpolynomial-timealgorithmsfortheTSP,despitemucheffortbyresearchersovernearlyacentury.Ofcourse,thereisanexponentialalgorithmforsolvingit,bytryingall(n 1)!tours,andinSection6.6wesawafaster,yetstillexponential,dynamicprogrammingalgorithm.Theminimumspanningtree(MST)problem,forwhichwedohaveefcientalgorithms,providesastarkcontrasthere.Tophraseitasasearchproblem,weareagaingivenadistancePmatrixandaboundb,andareaskedtondatreeTwithtotalweight(i;j)2Tdijb.TheTSPcanbethoughtofasatoughcousinoftheMSTproblem,inwhichthetreeisnotallowedtobranchandisthereforeapath.1Thisextrarestrictiononthestructureofthetreeresultsinamuchharderproblem.EulerandRudrataInthesummerof1735LeonhardEuler(pronounced“Oiler”),thefamousSwissmathemati-cian,waswalkingthebridgesoftheEastPrussiantownofKonigsberg¨.Afterawhile,henoticedinfrustrationthat,nomatterwherehestartedhiswalk,nomatterhowcleverlyhecontinued,itwasimpossibletocrosseachbridgeexactlyonce.Andfromthissillyambition,theeldofgraphtheorywasborn.Euleridentiedatoncetherootsofthepark'sdeciency.First,youturnthemapoftheparkintoagraphwhoseverticesarethefourlandmasses(twoislands,twobanks)andwhoseedgesarethesevenbridges:NorthernbankSmallBigislandislandSouthernbank1ActuallytheTSPdemandsacycle,butonecandeneanalternativeversionthatseeksapath,anditisnothardtoseethatthisisjustashardastheTSPitself.237\nThisgraphhasmultipleedgesbetweentwovertices—afeaturewehavenotbeenallowingsofarinthisbook,butonethatismeaningfulforthisparticularproblem,sinceeachbridgemustbeaccountedforseparately.Wearelookingforapaththatgoesthrougheachedgeexactlyonce(thepathisallowedtorepeatvertices).Inotherwords,weareaskingthisquestion:Whencanagraphbedrawnwithoutliftingthepencilfromthepaper?TheanswerdiscoveredbyEulerissimple,elegant,andintuitive:Ifandonlyif(a)thegraphisconnectedand(b)everyvertex,withthepossibleexceptionoftwovertices(thestartandnalverticesofthewalk),hasevendegree(Exercise3.26).ThisiswhyKonigsberg'¨sparkwasimpossibletotraverse:allfourverticeshaveodddegree.Toputitintermsofourpresentconcerns,letusdeneasearchproblemcalledEULERPATH:Givenagraph,ndapaththatcontainseachedgeexactlyonce.ItfollowsfromEuler'sobservation,andalittlemorethinking,thatthissearchproblemcanbesolvedinpolynomialtime.AlmostamillenniumbeforeEuler'sfatefulsummerinEastPrussia,aKashmiripoetnamedRudratahadaskedthisquestion:Canonevisitallthesquaresofthechessboard,withoutrepeatinganysquare,inonelongwalkthatendsatthestartingsquareandateachstepmakesalegalknightmove?Thisisagainagraphproblem:thegraphnowhas64ver-tices,andtwosquaresarejoinedbyanedgeifaknightcangofromonetotheotherinasinglemove(thatis,iftheircoordinatesdifferby2inonedimensionandby1intheother).SeeFigure8.2fortheportionofthegraphcorrespondingtotheupperleftcorneroftheboard.Canyoundaknight'stouronyourchessboard?Figure8.2Knight'smovesonacornerofachessboard.Thisisadifferentkindofsearchproblemingraphs:wewantacyclethatgoesthroughallvertices(asopposedtoalledgesinEuler'sproblem),withoutrepeatinganyvertex.Andthereisnoreasontosticktochessboards;thisquestioncanbeaskedofanygraph.LetusdenetheRUDRATACYCLEsearchproblemtobethefollowing:givenagraph,ndacyclethatvisitseachvertexexactlyonce—orreportthatnosuchcycleexists.2Thisproblemisominously2IntheliteraturethisproblemisknownastheHamiltoncycleproblem,afterthegreatIrishmathematicianwhorediscovereditinthe19thcentury.238\nFigure8.3Whatisthesmallestcutinthisgraph?reminiscentoftheTSP,andindeednopolynomialalgorithmisknownforit.TherearetwodifferencesbetweenthedenitionsoftheEulerandRudrataproblems.TherstisthatEuler'sproblemvisitsalledgeswhileRudrata'svisitsallvertices.Butthereisalsotheissuethatoneofthemdemandsapathwhiletheotherrequiresacycle.Whichofthesedifferencesaccountsforthehugedisparityincomputationalcomplexitybetweenthetwoproblems?Itmustbetherst,becausetheseconddifferencecanbeshowntobepurelycosmetic.Indeed,denetheRUDRATAPATHproblemtobejustlikeRUDRATACYCLE,exceptthatthegoalisnowtondapaththatgoesthrougheachvertexexactlyonce.Aswewillsoonsee,thereisapreciseequivalencebetweenthetwoversionsoftheRudrataproblem.CutsandbisectionsAcutisasetofedgeswhoseremovalleavesagraphdisconnected.Itisoftenofinteresttondsmallcuts,andtheMINIMUMCUTproblemis,givenagraphandabudgetb,tondacutwithatmostbedges.Forexample,thesmallestcutinFigure8.3isofsize3.Thisproblemcanbesolvedinpolynomialtimebyn 1max-owcomputations:giveeachedgeacapacityof1,andndthemaximumowbetweensomexednodeandeverysingleothernode.Thesmallestsuchowwillcorrespond(viathemax-owmin-cuttheorem)tothesmallestcut.Canyouseewhy?We'vealsoseenaverydifferent,randomizedalgorithmforthisproblem(page143).Inmanygraphs,suchastheoneinFigure8.3,thesmallestcutleavesjustasingletonvertexononeside—itconsistsofalledgesadjacenttothisvertex.Farmoreinterestingaresmallcutsthatpartitiontheverticesofthegraphintonearlyequal-sizedsets.Moreprecisely,theBALANCEDCUTproblemisthis:givenagraphwithnverticesandabudgetb,partitiontheverticesintotwosetsSandTsuchthatjSj;jTjn=3andsuchthatthereareatmostbedgesbetweenSandT.Anotherhardproblem.Balancedcutsariseinavarietyofimportantapplications,suchasclustering.Considerforexampletheproblemofsegmentinganimageintoitsconstituentcomponents(say,anelephantstandinginagrassyplainwithaclearblueskyabove).Agoodwayofdoingthisistocreateagraphwithanodeforeachpixeloftheimageandtoputanedgebetweennodeswhosecorrespondingpixelsarespatiallyclosetogetherandarealsosimilarincolor.Asingleobjectintheimage(liketheelephant,say)thencorrespondstoasetofhighlyconnectedverticesinthegraph.Abalancedcutisthereforelikelytodividethepixelsintotwoclusterswithoutbreakingapartanyoftheprimaryconstituentsoftheimage.Therstcutmight,forinstance,separatetheelephantontheonehandfromtheskyandfromgrassontheother.A239\nfurthercutwouldthenbeneededtoseparatetheskyfromthegrass.IntegerlinearprogrammingEventhoughthesimplexalgorithmisnotpolynomialtime,wementionedinChapter7thatthereisadifferent,polynomialalgorithmforlinearprogramming.Therefore,linearpro-grammingisefcientlysolvablebothinpracticeandintheory.Butthesituationchangescompletelyif,inadditiontospecifyingalinearobjectivefunctionandlinearinequalities,wealsoconstrainthesolution(thevaluesforthevariables)tobeinteger.ThislatterproblemiscalledINTEGERLINEARPROGRAMMING(ILP).Let'sseehowwemightformulateitasasearchproblem.WearegivenasetoflinearinequalitiesAxb,whereAisanmnmatrixandbisanm-vector;anobjectivefunctionspeciedbyann-vectorc;andnally,agoalg(thecounterpartofabudgetinmaximizationproblems).Wewanttondanonnegativeintegern-vectorxsuchthatAxbandcxg.Butthereisaredundancyhere:thelastconstraintcxgisitselfalinearinequalityandcanbeabsorbedintoAxb.So,wedeneILPtobefollowingsearchproblem:givenAandb,ndanonnegativeintegervectorxsatisfyingtheinequalitiesAxb,orreportthatnoneexists.Despitethemanycrucialapplicationsofthisproblem,andintenseinterestbyresearchers,noefcientalgorithmisknownforit.ThereisaparticularlycleanspecialcaseofILPthatisveryhardinandofitself:thegoalistondavectorxof0'sand1'ssatisfyingAx=1,whereAisanmnmatrixwith0 1entriesand1isthem-vectorofall1's.ItshouldbeapparentfromthereductionsinSection7.1.4thatthisisindeedaspecialcaseofILP.WecallitZERO-ONEEQUATIONS(ZOE).Wehavenowintroducedanumberofimportantsearchproblems,someofwhicharefa-miliarfromearlierchaptersandforwhichthereareefcientalgorithms,andotherswhicharedifferentinsmallbutcrucialwaysthatmakethemveryhardcomputationalproblems.Tocompleteourstorywewillintroduceafewmorehardproblems,whichwillplayarolelaterinthechapter,whenwerelatethecomputationaldifcultyofalltheseproblems.ThereaderisinvitedtoskipaheadtoSection8.2andthenreturntothedenitionsoftheseproblemsasrequired.Three-dimensionalmatchingRecalltheBIPARTITEMATCHINGproblem:givenabipartitegraphwithnnodesoneachside(theboysandthegirls),ndasetofndisjointedges,ordecidethatnosuchsetexists.InSection7.3,wesawhowtoefcientlysolvethisproblembyareductiontomaximumow.However,thereisaninterestinggeneralization,called3DMATCHING,forwhichnopolyno-mialalgorithmisknown.Inthisnewsetting,therearenboysandngirls,butalsonpets,andthecompatibilitiesamongthemarespeciedbyasetoftriples,eachcontainingaboy,agirl,andapet.Intuitively,atriple(b;g;p)meansthatboyb,girlg,andpetpgetalongwelltogether.Wewanttondndisjointtriplesandtherebycreatenharmonioushouseholds.CanyouspotasolutioninFigure8.4?240\nFigure8.4Amoreelaboratematchmakingscenario.Eachtripleisshownasatriangular-shapednodejoiningboy,girl,andpet.ChetAliceBobBeatriceAlCarolArmadilloBobcatCanaryIndependentset,vertexcover,andcliqueIntheINDEPENDENTSETproblem(recallSection6.7)wearegivenagraphandanintegerg,andtheaimistondgverticesthatareindependent,thatis,notwoofwhichhaveanedgebetweenthem.CanyoundanindependentsetofthreeverticesinFigure8.5?Howaboutfourvertices?WesawinSection6.7thatthisproblemcanbesolvedefcientlyontrees,butforgeneralgraphsnopolynomialalgorithmisknown.Figure8.5Whatisthesizeofthelargestindependentsetinthisgraph?Therearemanyothersearchproblemsaboutgraphs.InVERTEXCOVER,forexample,theinputisagraphandabudgetb,andtheideaistondbverticesthatcover(touch)everyedge.CanyoucoveralledgesofFigure8.5withsevenvertices?Withsix?(AnddoyouseetheintimateconnectiontotheINDEPENDENTSETproblem?)VERTEXCOVERisaspecialcaseofSETCOVER,whichweencounteredinChapter5.Inthatproblem,wearegivenasetEandseveralsubsetsofit,S1;:::;Sm,alongwithabudgetb.WeareaskedtoselectbofthesesubsetssothattheirunionisE.VERTEXCOVERisthe241\nspecialcaseinwhichEconsistsoftheedgesofagraph,andthereisasubsetSiforeachvertex,containingtheedgesadjacenttothatvertex.Canyouseewhy3DMATCHINGisalsoaspecialcaseofSETCOVER?AndnallythereistheCLIQUEproblem:givenagraphandagoalg,ndasetofgver-ticessuchthatallpossibleedgesbetweenthemarepresent.WhatisthelargestcliqueinFigure8.5?LongestpathWeknowtheshortest-pathproblemcanbesolvedveryefciently,buthowabouttheLONGESTPATHproblem?HerewearegivenagraphGwithnonnegativeedgeweightsandtwodistin-guishedverticessandt,alongwithagoalg.Weareaskedtondapathfromstotwithtotalweightatleastg.Naturally,toavoidtrivialsolutionswerequirethatthepathbesimple,containingnorepeatedvertices.Noefcientalgorithmisknownforthisproblem(whichsometimesalsogoesbythenameofTAXICABRIP-OFF).KnapsackandsubsetsumRecalltheKNAPSACKproblem(Section6.4):wearegivenintegerweightsw1;:::;wnandintegervaluesv1;:::;vnfornitems.WearealsogivenaweightcapacityWandagoalg(theformerispresentintheoriginaloptimizationproblem,thelatterisaddedtomakeitasearchproblem).WeseekasetofitemswhosetotalweightisatmostWandwhosetotalvalueisatleastg.Asalways,ifnosuchsetexists,weshouldsayso.InSection6.4,wedevelopedadynamicprogrammingschemeforKNAPSACKwithrunningtimeO(nW),whichwenotedisexponentialintheinputsize,sinceitinvolvesWratherthanlogW.Andwehavetheusualexhaustivealgorithmaswell,whichlooksatallsubsetsofitems—all2nofthem.IsthereapolynomialalgorithmforKNAPSACK?Nobodyknowsofone.Butsupposethatweareinterestedinthevariantoftheknapsackprobleminwhichtheintegersarecodedinunary—forinstance,bywritingIIIIIIIIIIIIfor12.Thisisadmittedlyanexponentiallywastefulwaytorepresentintegers,butitdoesdenealegitimateproblem,whichwecouldcallUNARYKNAPSACK.Itfollowsfromourdiscussionthatthissomewhatarticialproblemdoeshaveapolynomialalgorithm.Adifferentvariation:supposenowthateachitem'svalueisequaltoitsweight(allgiveninbinary),andtotopitoff,thegoalgisthesameasthecapacityW.(Toadaptthesillybreak-instorywherebywerstintroducedtheknapsackproblem,theitemsareallgoldnuggets,andtheburglarwantstollhisknapsacktothehilt.)ThisspecialcaseistantamounttondingasubsetofagivensetofintegersthataddsuptoexactlyW.SinceitisaspecialcaseofKNAPSACK,itcannotbeanyharder.Butcoulditbepolynomial?Asitturnsout,thisproblem,calledSUBSETSUM,isalsoveryhard.Atthispointonecouldask:IfSUBSETSUMisaspecialcasethathappenstobeashardasthegeneralKNAPSACKproblem,whyareweinterestedinit?Thereasonissimplicity.Inthecomplicatedcalculusofreductionsbetweensearchproblemsthatweshalldevelopinthischapter,conceptuallysimpleproblemslikeSUBSETSUMand3SATareinvaluable.242\n8.2NP-completeproblemsHardproblems,easyproblemsInshort,theworldisfullofsearchproblems,someofwhichcanbesolvedefciently,whileothersseemtobeveryhard.Thisisdepictedinthefollowingtable.Hardproblems(NP-complete)Easyproblems(inP)3SAT2SAT,HORNSATTRAVELINGSALESMANPROBLEMMINIMUMSPANNINGTREELONGESTPATHSHORTESTPATH3DMATCHINGBIPARTITEMATCHINGKNAPSACKUNARYKNAPSACKINDEPENDENTSETINDEPENDENTSETontreesINTEGERLINEARPROGRAMMINGLINEARPROGRAMMINGRUDRATAPATHEULERPATHBALANCEDCUTMINIMUMCUTThistableisworthcontemplating.Ontherightwehaveproblemsthatcanbesolvedefciently.Ontheleft,wehaveabunchofhardnutsthathaveescapedefcientsolutionovermanydecadesorcenturies.Thevariousproblemsontherightcanbesolvedbyalgorithmsthatarespecializedanddiverse:dynamicprogramming,networkow,graphsearch,greedy.Theseproblemsareeasyforavarietyofdifferentreasons.Instarkcontrast,theproblemsontheleftarealldifcultforthesamereason!Attheircore,theyareallthesameproblem,justindifferentdisguises!Theyareallequivalent:asweshallseeinSection8.3,eachofthemcanbereducedtoanyoftheothers—andback.PandNPIt'stimetointroducesomeimportantconcepts.Weknowwhatasearchproblemis:itsden-ingcharacteristicisthatanyproposedsolutioncanbequicklycheckedforcorrectness,inthesensethatthereisanefcientcheckingalgorithmCthattakesasinputthegiveninstanceI(thedataspecifyingtheproblemtobesolved),aswellastheproposedsolutionS,andoutputstrueifandonlyifSreallyisasolutiontoinstanceI.MoreovertherunningtimeofC(I;S)isboundedbyapolynomialinjIj,thelengthoftheinstance.WedenotetheclassofallsearchproblemsbyNP.We'veseenmanyexamplesofNPsearchproblemsthataresolvableinpolynomialtime.Insuchcases,thereisanalgorithmthattakesasinputaninstanceIandhasarunningtimepolynomialinjIj.IfIhasasolution,thealgorithmreturnssuchasolution;andifIhasnosolution,thealgorithmcorrectlyreportsso.TheclassofallsearchproblemsthatcanbesolvedinpolynomialtimeisdenotedP.Hence,allthesearchproblemsontheright-handsideofthetableareinP.243\nWhyPandNP?Okay,Pmuststandfor“polynomial.”ButwhyusetheinitialsNP(thecommonchatroomabbreviationfor“noproblem”)todescribetheclassofsearchproblems,someofwhichareterriblyhard?NPstandsfor“nondeterministicpolynomialtime,”atermgoingbacktotherootsofcomplexitytheory.Intuitively,itmeansthatasolutiontoanysearchproblemcanbefoundandveriedinpolynomialtimebyaspecial(andquiteunrealistic)sortofalgorithm,calledanondeterministicalgorithm.Suchanalgorithmhasthepowerofguessingcorrectlyateverystep.Incidentally,theoriginaldenitionofNP(anditsmostcommonusagetothisday)wasnotasaclassofsearchproblems,butasaclassofdecisionproblems:algorithmicquestionsthatcanbeansweredbyyesorno.Example:“IsthereatruthassignmentthatsatisesthisBooleanformula?”Butthistooreectsahistoricalreality:AtthetimethetheoryofNP-completenesswasbeingdeveloped,researchersinthetheoryofcomputationwereinterestedinformallanguages,adomaininwhichsuchdecisionproblemsareofcentralimportance.Aretheresearchproblemsthatcannotbesolvedinpolynomialtime?Inotherwords,isP6=NP?Mostalgorithmsresearchersthinkso.Itishardtobelievethatexponentialsearchcanalwaysbeavoided,thatasimpletrickwillcrackallthesehardproblems,famouslyunsolvedfordecadesandcenturies.AndthereisagoodreasonformathematicianstobelievethatP6=NP—thetaskofndingaproofforagivenmathematicalassertionisasearchproblemandisthereforeinNP(afterall,whenaformalproofofamathematicalstatementiswrittenoutinexcruciatingdetail,itcanbecheckedmechanically,linebyline,byanefcientalgorithm).SoifP=NP,therewouldbeanefcientmethodtoproveanytheorem,thuseliminatingtheneedformathematicians!Allinall,thereareavarietyofreasonswhyitiswidelybelievedthatP6=NP.However,provingthishasturnedouttobeextremelydifcult,oneofthedeepestandmostimportantunsolvedpuzzlesofmathematics.Reductions,againEvenifweacceptthatP6=NP,whataboutthespecicproblemsontheleftsideofthetable?Onthebasisofwhatevidencedowebelievethattheseparticularproblemshavenoefcientalgorithm(besides,ofcourse,thehistoricalfactthatmanyclevermathematiciansandcomputerscientistshavetriedhardandfailedtondany)?Suchevidenceisprovidedbyreductions,whichtranslateonesearchproblemintoanother.Whattheydemonstrateisthattheproblemsontheleftsideofthetableareall,insomesense,exactlythesameproblem,exceptthattheyarestatedindifferentlanguages.What'smore,wewillalsousereductionstoshowthattheseproblemsarethehardestsearchproblemsinNP—ifevenoneofthemhasapolynomialtimealgorithm,theneveryprobleminNPhasapolynomialtimealgorithm.ThusifwebelievethatP6=NP,thenallthesesearchproblemsarehard.WedenedreductionsinChapter7andsawmanyexamplesofthem.Let'snowspecializethisdenitiontosearchproblems.AreductionfromsearchproblemAtosearchproblemBisapolynomial-timealgorithmfthattransformsanyinstanceIofAintoaninstancef(I)ofB,togetherwithanotherpolynomial-timealgorithmhthatmapsanysolutionSoff(I)backintoasolutionh(S)ofI;seethefollowingdiagram.Iff(I)hasnosolution,thenneitherdoes244\nI.ThesetwotranslationproceduresfandhimplythatanyalgorithmforBcanbeconvertedintoanalgorithmforAbybracketingitbetweenfandh.AlgorithmforASolutionSoff(I)SolutionInstanceInstancef(I)Algorithmhh(S)ofIIfforBNosolutiontof(I)NosolutiontoIAndnowwecannallydenetheclassofthehardestsearchproblems.AsearchproblemisNP-completeifallothersearchproblemsreducetoit.Thisisaverystrongrequirementindeed.ForaproblemtobeNP-complete,itmustbeusefulinsolvingeverysearchproblemintheworld!Itisremarkablethatsuchproblemsexist.Buttheydo,andtherstcolumnofthetablewesawearlierislledwiththemostfamousexamples.InSection8.3weshallseehowalltheseproblemsreducetooneanother,andalsowhyallothersearchproblemsreducetothem.ThetwowaystousereductionsSofarinthisbookthepurposeofareductionfromaproblemAtoaproblemBhasbeenstraightforwardandhonorable:WeknowhowtosolveBefciently,andwewanttousethisknowledgetosolveA.Inthischapter,however,reductionsfromAtoBserveasomewhatperversegoal:weknowAishard,andweusethereductiontoprovethatBishardaswell!IfwedenoteareductionfromAtoBbyA !Bthenwecansaythatdifcultyowsinthedirectionofthearrow,whileefcientalgorithmsmoveintheoppositedirection.ItisthroughthispropagationofdifcultythatweknowNP-completeproblemsarehard:allothersearchproblemsreducetothem,andthuseachNP-completeproblemcontainsthecomplexityofallsearchproblems.IfevenoneNP-completeproblemisinP,thenP=NP.Reductionsalsohavetheconvenientpropertythattheycompose.IfA !BandB !C,thenA !C.Toseethis,observerstofallthatanyreductioniscompletelyspeciedbythepre-andpostprocessingfunctionsfandh(seethereductiondiagram).If(fAB;hAB)and(fBC;hBC)denethereductionsfromAtoBandfromBtoC,respectively,thenareductionfromAtoCisgivenbycompositionsofthesefunctions:fBCfABmapsaninstanceofAtoaninstanceofCandhABhBCsendsasolutionofCbacktoasolutionofA.ThismeansthatonceweknowaproblemAisNP-complete,wecanuseittoprovethatanewsearchproblemBisalsoNP-complete,simplybyreducingAtoB.SuchareductionestablishesthatallproblemsinNPreducetoB,viaA.245\nFigure8.6ThespaceNPofallsearchproblems,assumingP6=NP.NP−PcompleteIncreasingdifficultyFactoringOnelastpoint:westartedoffthisbookbyintroducinganotherfamouslyhardsearchproblem:FACTORING,thetaskofndingallprimefactorsofagiveninteger.ButthedifcultyofFACTORINGisofadifferentnaturethanthatoftheotherhardsearchproblemswehavejustseen.Forexample,nobodybelievesthatFACTORINGisNP-complete.Onemajordifferenceisthat,inthecaseofFACTORING,thedenitiondoesnotcontainthenowfamiliarclause“orreportthatnoneexists.”Anumbercanalwaysbefactoredintoprimes.Anotherdifference(possiblynotcompletelyunrelated)isthis:asweshallseeinChap-ter10,FACTORINGsuccumbstothepowerofquantumcomputation—whileSAT,TSPandtheotherNP-completeproblemsdonotseemto.246\nFigure8.7Reductionsbetweensearchproblems.AllofNPSAT3SATINDEPENDENTSET3DMATCHINGVERTEXCOVERCLIQUEZOESUBSETSUMILPRUDRATACYCLETSP8.3ThereductionsWeshallnowseethatthesearchproblemsofSection8.1canbereducedtooneanotherasdepictedinFigure8.7.Asaconsequence,theyareallNP-complete.Beforewetacklethespecicreductionsinthetree,let'swarmupbyrelatingtwoversionsoftheRudrataproblem.RUDRATA(s;t)-PATH !RUDRATACYCLERecalltheRUDRATACYCLEproblem:givenagraph,isthereacyclethatpassesthrougheachvertexexactlyonce?WecanalsoformulatethecloselyrelatedRUDRATA(s;t)-PATHproblem,inwhichtwoverticessandtarespecied,andwewantapathstartingatsandendingattthatgoesthrougheachvertexexactlyonce.IsitpossiblethatRUDRATACYCLEiseasierthanRUDRATA(s;t)-PATH?Wewillshowbyareductionthattheanswerisno.Thereductionmapsaninstance(G=(V;E);s;t)ofRUDRATA(s;t)-PATHintoaninstanceG0=(V0;E0)ofRUDRATACYCLEasfollows:G0issimplyGwithanadditionalvertexxandtwonewedgesfs;xgandfx;tg.Forinstance:GG0sstxtSoV0=V[fxg,andE0=E[ffs;xg;fx;tgg.HowdowerecoveraRudrata(s;t)-pathinGgivenanyRudratacycleinG0?Easy,wejustdeletetheedgesfs;xgandfx;tgfromthecycle.247\nRUDRATA(s;t)-PATHSolution:cycleDeleteedgesSolution:Instance:AddnodexG0=(V0;E0)RUDRATAfs;xg;fx;tgpathG=(V;E)andedgesnodess;tfs;xg;fx;tgCYCLENosolutionNosolutionToconrmthevalidityofthisreduction,wehavetoshowthatitworksinthecaseofeitheroutcomedepicted.1.WhentheinstanceofRUDRATACYCLEhasasolution.Sincethenewvertexxhasonlytwoneighbors,sandt,anyRudratacycleinG0mustconsec-utivelytraversetheedgesft;xgandfx;sg.Therestofthecyclethentraverseseveryothervertexenroutefromstot.Thusdeletingthetwoedgesft;xgandfx;sgfromtheRudratacyclegivesaRudratapathfromstotintheoriginalgraphG.2.WhentheinstanceofRUDRATACYCLEdoesnothaveasolution.InthiscasewemustshowthattheoriginalinstanceofRUDRATA(s;t)-PATHcannothaveasolutioneither.Itisusuallyeasiertoprovethecontrapositive,thatis,toshowthatifthereisaRudrata(s;t)-pathinG,thenthereisalsoaRudratacycleinG0.Butthisiseasy:justaddthetwoedgesft;xgandfx;sgtotheRudratapathtoclosethecycle.Onelastdetail,crucialbuttypicallyeasytocheck,isthatthepre-andpostprocessingfunctionstaketimepolynomialinthesizeoftheinstance(G;s;t).ItisalsopossibletogointheotherdirectionandreduceRUDRATACYCLEtoRUDRATA(s;t)-PATH.Together,thesereductionsdemonstratethatthetwoRudratavariantsareinessencethesameproblem—whichisnottoosurprising,giventhattheirdescriptionsareal-mostthesame.Butmostoftheotherreductionswewillseearebetweenpairsofproblemsthat,onthefaceofit,lookquitedifferent.Toshowthattheyareessentiallythesame,ourreductionswillhavetocleverlytranslatebetweenthem.3SAT !INDEPENDENTSETOnecanhardlythinkoftwomoredifferentproblems.In3SATtheinputisasetofclauses,eachwiththreeorfewerliterals,forexample(x_y_z)(x_y_z)(x_y_z)(x_y);andtheaimistondasatisfyingtruthassignment.InINDEPENDENTSETtheinputisagraphandanumberg,andtheproblemistondasetofgpairwisenon-adjacentvertices.WemustsomehowrelateBooleanlogicwithgraphs!Letusthink.Toformasatisfyingtruthassignmentwemustpickoneliteralfromeachclauseandgiveitthevaluetrue.Butourchoicesmustbeconsistent:ifwechoosexinoneclause,wecannotchoosexinanother.Anyconsistentchoiceofliterals,onefromeachclause,speciesatruthassignment(variablesforwhichneitherliteralhasbeenchosencantakeoneithervalue).So,letusrepresentaclause,say(x_y_z),byatriangle,withverticeslabeledx;y;z.Whytriangle?Becauseatrianglehasitsthreeverticesmaximallyconnected,andthusforcesus248\nFigure8.8Thegraphcorrespondingto(x_y_z)(x_y_z)(x_y_z)(x_y).yyyyxzxzxzxtopickonlyoneofthemfortheindependentset.Repeatthisconstructionforallclauses—aclausewithtwoliteralswillberepresentedsimplybyanedgejoiningtheliterals.(Aclausewithoneliteralissillyandcanberemovedinapreprocessingstep,sincethevalueofthevariableisdetermined.)Intheresultinggraph,anindependentsethastopickatmostoneliteralfromeachgroup(clause).Toforceexactlyonechoicefromeachclause,takethegoalgtobethenumberofclauses;inourexample,g=4.Allthatismissingnowisawaytopreventusfromchoosingoppositeliterals(thatis,bothxandx)indifferentclauses.Butthisiseasy:putanedgebetweenanytwoverticesthatcorrespondtooppositeliterals.TheresultinggraphforourexampleisshowninFigure8.8.Let'srecaptheconstruction.GivenaninstanceIof3SAT,wecreateaninstance(G;g)ofINDEPENDENTSETasfollows.GraphGhasatriangleforeachclause(orjustanedge,iftheclausehastwoliterals),withverticeslabeledbytheclause'sliterals,andhasadditionaledgesbetweenanytwoverticesthatrepresentoppositeliterals.Thegoalgissettothenumberofclauses.Clearly,thisconstructiontakespolynomialtime.However,recallthatforareductionwedonotjustneedanefcientwaytomapinstancesoftherstproblemtoinstancesofthesecond(thefunctionfinthediagramonpage245),butalsoawaytoreconstructasolutiontotherstinstancefromanysolutionofthesecond(thefunctionh).Asalways,therearetwothingstoshow.1.GivenanindependentsetSofgverticesinG,itispossibletoefcientlyrecoverasatis-fyingtruthassignmenttoI.Foranyvariablex,thesetScannotcontainverticeslabeledbothxandx,becauseanysuchpairofverticesisconnectedbyanedge.SoassignxavalueoftrueifScontainsavertexlabeledx,andavalueoffalseifScontainsavertexlabeledx(ifScontainsneither,thenassigneithervaluetox).SinceShasgvertices,itmusthaveonevertexperclause;thistruthassignmentsatisesthoseparticularliterals,andthussatisesallclauses.2.IfgraphGhasnoindependentsetofsizeg,thentheBooleanformulaIisunsatisable.249\nItisusuallycleanertoprovethecontrapositive,thatifIhasasatisfyingassignmentthenGhasanindependentsetofsizeg.Thisiseasy:foreachclause,pickanyliteralwhosevalueunderthesatisfyingassignmentistrue(theremustbeatleastonesuchliteral),andaddthecorrespondingvertextoS.DoyouseewhysetSmustbeindependent?SAT !3SATThisisaninterestingandcommonkindofreduction,fromaproblemtoaspecialcaseofitself.Wewanttoshowthattheproblemremainshardevenifitsinputsarerestrictedsomehow—inthepresentcase,evenifallclausesarerestrictedtohave3literals.Suchreductionsmodifythegiveninstancesoastogetridoftheforbiddenfeature(clauseswith4literals)whilekeepingtheinstanceessentiallythesame,inthatwecanreadoffasolutiontotheoriginalinstancefromanysolutionofthemodiedone.Here'sthetrickforreducingSATto3SAT:givenaninstanceIofSAT,useexactlythesameinstancefor3SAT,exceptthatanyclausewithmorethanthreeliterals,(a1_a2__ak)(wheretheai'sareliteralsandk>3),isreplacedbyasetofclauses,(a1_a2_y1)(y1_a3_y2)(y2_a4_y3)(yk 3_ak 1_ak);wheretheyi'sarenewvariables.Calltheresulting3SATinstanceI0.TheconversionfromItoI0isclearlypolynomialtime.Whydoesthisreductionwork?I0isequivalenttoIintermsofsatisability,becauseforanyassignmenttotheai's,893clauses.Thenreplaceitsrstappearancebyx1,itssecondappearancebyx2,andsoon,replacingeachofitskappearancesbyadifferentnewvariable.Finally,addtheclauses(x1_x2)(x2_x3)(xk_x1):Andrepeatforeveryvariablethatappearsmorethanthreetimes.250\nFigure8.9SisavertexcoverifandonlyifV Sisanindependentset.SItiseasytoseethatinthenewformulanovariableappearsmorethanthreetimes(andinfact,noliteralappearsmorethantwice).Furthermore,theextraclausesinvolv-ingx1;x2;:::;xkconstrainthesevariablestohavethesamevalue;doyouseewhy?Hencetheoriginalinstanceof3SATissatisableifandonlyiftheconstrainedinstanceissatisable.INDEPENDENTSET !VERTEXCOVERSomereductionsrelyoningenuitytorelatetwoverydifferentproblems.Otherssimplyrecordthefactthatoneproblemisathindisguiseofanother.ToreduceINDEPENDENTSETtoVERTEXCOVERwejustneedtonoticethatasetofnodesSisavertexcoverofgraphG=(V;E)(thatis,StoucheseveryedgeinE)ifandonlyiftheremainingnodes,V S,areanindependentsetofG(Figure8.9).Therefore,tosolveaninstance(G;g)ofINDEPENDENTSET,simplylookforavertexcoverofGwithjVj gnodes.Ifsuchavertexcoverexists,thentakeallnodesnotinit.Ifnosuchvertexcoverexists,thenGcannotpossiblyhaveanindependentsetofsizeg.INDEPENDENTSET !CLIQUEINDEPENDENTSETandCLIQUEarealsoeasytoreducetooneanother.DenethecomplementofagraphG=(V;E)tobeG=(V;E),whereEcontainspreciselythoseunorderedpairsofverticesthatarenotinE.ThenasetofnodesSisanindependentsetofGifandonlyifSisacliqueofG.Toparaphrase,thesenodeshavenoedgesbetweentheminGifandonlyiftheyhaveallpossibleedgesbetweentheminG.Therefore,wecanreduceINDEPENDENTSETtoCLIQUEbymappinganinstance(G;g)ofINDEPENDENTSETtothecorrespondinginstance(G;g)ofCLIQUE;thesolutiontobothisidentical.3SAT !3DMATCHINGAgain,twoverydifferentproblems.Wemustreduce3SATtotheproblemofnding,amongasetofboy-girl-pettriples,asubsetthatcontainseachboy,eachgirl,andeachpetexactlyonce.Inshort,wemustdesignsetsofboy-girl-pettriplesthatsomehowbehavelikeBooleanvariablesandgates!Considerthefollowingsetoffourtriples,eachrepresentedbyatriangularnodejoiningaboy,girl,andpet:251\np1b0g0p0p2g1b1p3Supposethatthetwoboysb0andb1andthetwogirlsg0andg1arenotinvolvedinanyothertriples.(Thefourpetsp0;:::;p3willofcoursebelongtoothertriplesaswell;forotherwisetheinstancewouldtriviallyhavenosolution.)Thenanymatchingmustcontaineitherthetwotriples(b0;g1;p0);(b1;g0;p2)orthetwotriples(b0;g0;p1);(b1;g1;p3),becausethesearetheonlywaysinwhichthesetwoboysandgirlscanndanymatch.Therefore,this“gadget”hastwopossiblestates:itbehaveslikeaBooleanvariable!Tothentransformaninstanceof3SATtooneof3DMATCHING,westartbycreatingacopyoftheprecedinggadgetforeachvariablex.Calltheresultingnodespx1;bx0;gx1,andsoon.Theintendedinterpretationisthatboybx0ismatchedwithgirlgx1ifx=true,andwithgirlgx0ifx=false.Nextwemustcreatetriplesthatsomehowmimicclauses.Foreachclause,sayc=(x_y_z),introduceanewboybcandanewgirlgc.Theywillbeinvolvedinthreetriples,oneforeachliteralintheclause.Andthepetsinthesetriplesmustreectthethreewayswherebytheclausecanbesatised:(1)x=true,(2)y=false,(3)z=true.For(1),wehavethetriple(bc;gc;px1),wherepx1isthepetp1inthegadgetforx.Hereiswhywechosep1:ifx=true,thenbx0ismatchedwithgx1andbx1withgx0,andsopetspx0andpx2aretaken.Inwhichcasebcandgccanbematchedwithpx1.Butifx=false,thenpx1andpx3aretaken,andsogcandbccannotbeaccommodatedthisway.Wedothesamethingfortheothertwoliteralsoftheclause,whichyieldtriplesinvolvingbcandgcwitheitherpy0orpy2(forthenegatedvariabley)andwitheitherpz1orpz3(forvariablez).Wehavetomakesurethatforeveryoccurrenceofaliteralinaclausecthereisadifferentpettomatchwithbcandgc.Butthisiseasy:byanearlierreductionwecanassumethatnoliteralappearsmorethantwice,andsoeachvariablegadgethasenoughpets,twofornegatedoccurrencesandtwoforunnegated.Thereductionnowseemscomplete:fromanymatchingwecanrecoverasatisfyingtruthassignmentbysimplylookingateachvariablegadgetandseeingwithwhichgirlbx0wasmatched.Andfromanysatisfyingtruthassignmentwecanmatchthegadgetcorrespondingtoeachvariablexsothattriples(bx0;gx1;px0)and(bx1;gx0;px2)arechosenifx=trueandtriples(bx0;gx0;px1)and(bx1;gx1;px3)arechosenifx=false;andforeachclausecmatchbcandgcwiththepetthatcorrespondstooneofitssatisfyingliterals.Butonelastproblemremains:inthematchingdenedattheendofthelastparagraph,somepetsmaybeleftunmatched.Infact,iftherearenvariablesandmclauses,thenexactly252\n2n mpetswillbeleftunmatched(youcancheckthatthisnumberissuretobepositive,becausewehaveatmostthreeoccurrencesofeveryvariable,andatleasttwoliteralsineveryclause).Butthisiseasytox:Add2n mnewboy-girlcouplesthatare“genericanimal-lovers,”andmatchthembytripleswithallthepets!3DMATCHING !ZOERecallthatinZOEwearegivenanmnmatrixAwith0 1entries,andwemustnda0 1vectorx=(x1;:::;xn)suchthatthemequationsAx=1aresatised,whereby1wedenotethecolumnvectorofall1's.Howcanweexpressthe3DMATCHINGprobleminthisframework?ZOEandILPareveryusefulproblemspreciselybecausetheyprovideaformatinwhichmanycombinatorialproblemscanbeexpressed.Insuchaformulationwethinkofthe0 1variablesasdescribingasolution,andwewriteequationsexpressingtheconstraintsoftheproblem.Forexample,hereishowweexpressaninstanceof3DMATCHING(mboys,mgirls,mpets,andnboy-girl-pettriples)inthelanguageofZOE.Wehave0 1variablesx1;:::;xn,onepertriple,wherexi=1meansthattheithtripleischosenforthematching,andxi=0meansthatitisnotchosen.Nowallwehavetodoiswriteequationsstatingthatthesolutiondescribedbythexi'sisalegitimatematching.Foreachboy(orgirl,orpet),supposethatthetriplescontaininghim(orher,orit)arethosenumberedj1;j2;:::;jk;theappropriateequationisthenxj1+xj2++xjk=1;whichstatesthatexactlyoneofthesetriplesmustbeincludedinthematching.Forexample,hereistheAmatrixforaninstanceof3DMATCHINGwesawearlier.01ChetAlice10000B00011CBobBeatriceBCB01100CBCAlCarolBB10001CCA=BB01000CCBB00110CCBB10001CC@00110A01000ArmadilloBobcatCanaryThevecolumnsofAcorrespondtothevetriples,whiletheninerowsareforAl,Bob,Chet,Alice,Beatrice,Carol,Armadillo,Bobcat,andCanary,respectively.Itisstraightforwardtoarguethatsolutionstothetwoinstancestranslatebackandforth.253\nZOE !SUBSETSUMThisisareductionbetweentwospecialcasesofILP:onewithmanyequationsbutonly0 1coefcients,andtheotherwithasingleequationbutarbitraryintegercoefcients.Thereductionisbasedonasimpleandtime-honoredidea:0 1vectorscanencodenumbers!Forexample,giventhisinstanceofZOE:011000B0001CBCA=BB0110CC;@1000A0100wearelookingforasetofcolumnsofAthat,addedtogether,makeuptheall-1'svector.Butifwethinkofthecolumnsasbinaryintegers(readfromtoptobottom),wearelookingforasubsetoftheintegers18;5;4;8thatadduptothebinaryinteger111112=31.AndthisisaninstanceofSUBSETSUM.Thereductioniscomplete!Exceptforonedetail,theonethatusuallyspoilsthecloseconnectionbetween0 1vec-torsandbinaryintegers:carry.Becauseofcarry,5-bitbinaryintegerscanaddupto31(forexample,5+6+20=31or,inbinary,001012+001102+101002=111112)evenwhenthesumofthecorrespondingvectorsisnot(1;1;1;1;1).Butthisiseasytox:Thinkofthecolumnvectorsnotasintegersinbase2,butasintegersinbasen+1—onemorethanthenumberofcolumns.Thisway,sinceatmostnintegersareadded,andalltheirdigitsare0and1,therecanbenocarry,andourreductionworks.ZOE !ILP3SATisaspecialcaseofSAT—or,SATisageneralizationof3SAT.Byspecialcasewemeanthattheinstancesof3SATareasubsetoftheinstancesofSAT(inparticular,theoneswithnolongclauses),andthedenitionofsolutionisthesameinbothproblems(anassignmentsatisfyingallclauses).Consequently,thereisareductionfrom3SATtoSAT,inwhichtheinputundergoesnotransformation,andthesolutiontothetargetinstanceisalsokeptunchanged.Inotherwords,functionsfandhfromthereductiondiagram(onpage245)areboththeidentity.Thissoundstrivialenough,butitisaveryusefulandcommonwayofestablishingthataproblemisNP-complete:SimplynoticethatitisageneralizationofaknownNP-completeproblem.Forexample,theSETCOVERproblemisNP-completebecauseitisageneraliza-tionofVERTEXCOVER(andalso,incidentally,of3DMATCHING).SeeExercise8.10formoreexamples.Oftenittakesalittleworktoestablishthatoneproblemisaspecialcaseofanother.ThereductionfromZOEtoILPisacaseinpoint.InILPwearelookingforanintegervectorxthatsatisesAxb,forgivenmatrixAandvectorb.TowriteaninstanceofZOEinthispreciseform,weneedtorewriteeachequationoftheZOEinstanceastwoinequalities(recallthetransformationsofSection7.1.4),andtoaddforeachvariablexitheinequalitiesxi1and xi0.254\nFigure8.10Rudratacyclewithpairededges:C=f(e1;e3);(e5;e6);(e4;e5);(e3;e7);(e3;e8)g.e1e2e3e8e7e5e4e6ZOE !RUDRATACYCLEIntheRUDRATACYCLEproblemweseekacycleinagraphthatvisitseveryvertexexactlyonce.WeshallproveitNP-completeintwostages:rstwewillreduceZOEtoageneralizationofRUDRATACYCLE,calledRUDRATACYCLEWITHPAIREDEDGES,andthenweshallseehowtogetridoftheextrafeaturesofthatproblemandreduceittotheplainRUDRATACYCLEproblem.InaninstanceofRUDRATACYCLEWITHPAIREDEDGESwearegivenagraphG=(V;E)andasetCEEofpairsofedges.Weseekacyclethat(1)visitsallverticesonce,likeaRudratacycleshould,and(2)foreverypairofedges(e;e0)inC,traverseseitheredgeeoredgee0—exactlyoneofthem.InthesimpleexampleofFigure8.10asolutionisshowninbold.Noticethatweallowtwoormoreparalleledgesbetweentwonodes—afeaturethatdoesn'tmakesenseinmostgraphproblems—sincenowthedifferentcopiesofanedgecanbepairedwithothercopiesofedgesinwaysthatdomakeadifference.NowforthereductionofZOEtoRUDRATACYCLEWITHPAIREDEDGES.GivenaninstanceofZOE,Ax=1(whereAisanmnmatrixwith0 1entries,andthusdescribesmequationsinnvariables),thegraphweconstructhastheverysimplestructureshowninFigure8.11:acyclethatconnectsm+ncollectionsofparalleledges.Foreachvariablexiwehavetwoparalleledges(correspondingtoxi=1andxi=0).Andforeachequationxj1++xjk=1involvingkvariableswehavekparalleledges,oneforeveryvariableappearingintheequation.Thisisthewholegraph.Evidently,anyRudratacycleinthisgraphmusttraversethem+ncollectionsofparalleledgesonebyone,choosingoneedgefromeachcollection.Thisway,thecycle“chooses”foreachvariableavalue—0or1—and,foreachequation,avariableappearinginit.Thewholereductioncan'tbethissimple,ofcourse.ThestructureofthematrixA(andnotjustitsdimensions)mustbereectedsomewhere,andthereisoneplaceleft:thesetCofpairsofedgessuchthatexactlyoneedgeineachpairistraversed.Foreveryequation(recalltherearemintotal),andforeveryvariablexiappearinginit,weaddtoCthepair(e;e0)whereeistheedgecorrespondingtotheappearanceofxiinthatparticularequation(ontheleft-handsideofFigure8.11),ande0istheedgecorrespondingtothevariableassignment255\nFigure8.11ReducingZOEtoRUDRATACYCLEWITHPAIREDEDGES.equationsvariablesxi=0(ontherightsideofthegure).Thiscompletestheconstruction.TakeanysolutionofthisinstanceofRUDRATACYCLEWITHPAIREDEDGES.Asdiscussedbefore,itpicksavalueforeachvariableandavariableforeveryequation.WeclaimthatthevaluesthuschosenareasolutiontotheoriginalinstanceofZOE.Ifavariablexihasvalue1,thentheedgexi=0isnottraversed,andthusalledgesassociatedwithxiontheequationsidemustbetraversed(sincetheyarepairedinCwiththexi=0edge).So,ineachequationexactlyoneofthevariablesappearinginithasvalue1—whichisthesameassayingthatallequationsaresatised.Theotherdirectionisstraightforwardaswell:fromasolutiontotheinstanceofZOEoneeasilyobtainsanappropriateRudratacycle.GettingRidoftheEdgePairs.SofarwehaveareductionfromZOEtoRUDRATACYCLEWITHPAIREDEDGES;butwearereallyinterestedinRUDRATACYCLE,whichisaspecialcaseoftheproblemwithpairededges:theoneinwhichthesetofpairsCisempty.Toaccomplishourgoal,weneed,asusual,tondawayofgettingridoftheunwantedfeature—inthiscasetheedgepairs.ConsiderthegraphshowninFigure8.12,andsupposethatitisapartofalargergraphGinsuchawaythatonlythefourendpointsa;b;c;dtouchtherestofthegraph.Weclaimthatthisgraphhasthefollowingimportantproperty:inanyRudratacycleofGthesubgraphshownmustbetraversedinoneofthetwowaysshowninboldinFigure8.12(b)and(c).Hereiswhy.Supposethatthecyclerstentersthesubgraphfromvertexacontinuingtof.Thenitmustcontinuetovertexg,becauseghasdegree2andsoitmustbevisitedimmediatelyafteroneofitsadjacentnodesisvisited—otherwisethereisnowaytoincludeitinthecycle.Hencewemustgoontonodeh,andhereweseemtohaveachoice.Wecouldcontinueontoj,orreturntoc.Butifwetakethesecondoption,howarewegoingtovisittherestofthesubgraph?(ARudratacyclemustleavenovertexunvisited.)Itiseasytoseethatthiswouldbeimpossible,andsofromhwehavenochoicebuttocontinuetojandfromtheretovisitthe256\nrestofthegraphasshowninFigure8.12(b).Bysymmetry,iftheRudratacycleentersthissubgraphatc,itmusttraverseitasinFigure8.12(c).Andthesearetheonlytwoways.Butthispropertytellsussomethingimportant:thisgadgetbehavesjustliketwoedgesfa;bgandfc;dgthatarepairedupintheRUDRATACYCLEWITHPAIREDEDGESproblem(seeFigure8.12(d)).Therestofthereductionisnowclear:toreduceRUDRATACYCLEWITHPAIREDEDGEStoRUDRATACYCLEwegothroughthepairsinConebyone.Togetridofeachpair(fa;bg;fc;dg)wereplacethetwoedgeswiththegadgetinFigure8.12(a).ForanyotherpairinCthatinvolvesfa;bg,wereplacetheedgefa;bgwiththenewedgefa;fg,wherefisfromthegadget:thetraversaloffa;fgisfromnowonanindicationthatedgefa;bgintheoldgraphwouldbetraversed.Similarly,fc;hgreplacesfc;dg.AfterjCjsuchreplacements(performedinpolynomialtime,sinceeachreplacementaddsonly12verticestothegraph)wearedone,andtheRudratacyclesintheresultinggraphwillbeinone-to-onecorrespondencewiththeRudratacyclesintheoriginalgraphthatconformtotheconstraintsinC.257\nFigure8.12Agadgetforenforcingpairedbehavior.(a)flmsabgknrcdhjpq(b)abcd(c)abcd(d)abC=f(fa;bg;fc;dg)gcd258\nRUDRATACYCLE !TSPGivenagraphG=(V;E),constructthefollowinginstanceoftheTSP:thesetofcitiesisthesameasV,andthedistancebetweencitiesuandvis1iffu;vgisanedgeofGand1+otherwise,forsome>1tobedetermined.ThebudgetoftheTSPinstanceisequaltothenumberofnodes,jVj.ItiseasytoseethatifGhasaRudratacycle,thenthesamecycleisalsoatourwithinthebudgetoftheTSPinstance;andthatconversely,ifGhasnoRudratacycle,thenthereisnosolution:thecheapestpossibleTSPtourhascostatleastn+(itmustuseatleastoneedgeoflength1+,andthetotallengthofalln 1othersisatleastn 1).ThusRUDRATACYCLEreducestoTSP.Inthisreduction,weintroducedtheparameterbecausebyvaryingit,wecanobtaintwointerestingresults.If=1,thenalldistancesareeither1or2,andsothisinstanceoftheTSPsatisesthetriangleinequality:ifi;j;karecities,thendij+djkdik(proof:a+bcholdsforanynumbers1a;b;c2).ThisisaspecialcaseoftheTSPwhichisofpracticalimportanceandwhich,asweshallseeinChapter9,isinacertainsenseeasier,becauseitcanbeefcientlyapproximated.Ifontheotherhandislarge,thentheresultinginstanceoftheTSPmaynotsatisfythetriangleinequality,buthasanotherimportantproperty:eitherithasasolutionofcostnorless,orallitssolutionshavecostatleastn+(whichnowcanbearbitrarilylargerthann).Therecanbenothinginbetween!AsweshallseeinChapter9,thisimportantgappropertyimpliesthat,unlessP=NP,noapproximationalgorithmispossible.ANYPROBLEMINNP !SATWehavereducedSATtothevarioussearchproblemsinFigure8.7.Nowwecomefullcircleandarguethatalltheseproblems—andinfactallproblemsinNP—reducetoSAT.Inparticular,weshallshowthatallproblemsinNPcanbereducedtoageneralizationofSATwhichwecallCIRCUITSAT.InCIRCUITSATwearegivena(Boolean)circuit(seeFigure8.13,andrecallSection7.7),adagwhoseverticesaregatesofvedifferenttypes:ANDgatesandORgateshaveindegree2.NOTgateshaveindegree1.Knowninputgateshavenoincomingedgesandarelabeledfalseortrue.Unknowninputgateshavenoincomingedgesandarelabeled“?”.Oneofthesinksofthedagisdesignatedastheoutputgate.Givenanassignmentofvaluestotheunknowninputs,wecanevaluatethegatesofthecircuitintopologicalorder,usingtherulesofBooleanlogic(suchasfalse_true=true),untilweobtainthevalueattheoutputgate.Thisisthevalueofthecircuitfortheparticularassignmenttotheinputs.Forinstance,thecircuitinFigure8.13evaluatestofalseundertheassignmenttrue;false;true(fromlefttoright).CIRCUITSATisthenthefollowingsearchproblem:Givenacircuit,ndatruthassignmentfortheunknowninputssuchthattheoutputgateevaluatestotrue,orreportthatnosuchassignmentexists.Forexample,ifpresentedwiththecircuitinFigure8.13wecouldhave259\nFigure8.13AninstanceofCIRCUITSAT.outputANDNOTORANDORANDtrue???returnedtheassignment(false;true;true)because,ifwesubstitutethesevaluestotheunknowninputs(fromlefttoright),theoutputbecomestrue.CIRCUITSATisageneralizationofSAT.Toseewhy,noticethatSATasksforasatisfyingtruthassignmentforacircuitthathasthissimplestructure:abunchofANDgatesatthetopjointheclauses,andtheresultofthisbigANDistheoutput.EachclauseistheORofitsliterals.AndeachliteraliseitheranunknowninputgateortheNOTofone.Therearenoknowninputgates.Goingintheotherdirection,CIRCUITSATcanalsobereducedtoSAT.Hereishowwecanrewriteanycircuitinconjunctivenormalform(theANDofclauses):foreachgateginthecircuitwecreateavariableg,andwemodeltheeffectofthegateusingafewclauses:gggGateggORANDNOTtruefalse(g)(g)h1h2h1h2h(g_h2)(g_h1)(g_h)(g_h1)(g_h2)(g_h)(g_h1_h2)(g_h1_h2)(Doyouseethattheseclausesdo,infact,forceexactlythedesiredeffect?)Andtonishup,ifgistheoutputgate,weforceittobetruebyaddingtheclause(g).TheresultinginstanceofSATisequivalenttothegiveninstanceofCIRCUITSAT:thesatisfyingtruthassignmentsofthisconjunctivenormalformareinone-to-onecorrespondencewiththoseofthecircuit.260\nNowthatweknowCIRCUITSATreducestoSAT,weturntoourmainjob,showingthatallsearchproblemsreducetoCIRCUITSAT.So,supposethatAisaprobleminNP.WemustdiscoverareductionfromAtoCIRCUITSAT.Thissoundsverydifcult,becauseweknowalmostnothingaboutA!AllweknowaboutAisthatitisasearchproblem,sowemustputthisknowledgetowork.Themainfeatureofasearchproblemisthatanysolutiontoitcanquicklybechecked:thereisanalgorithmCthatchecks,givenaninstanceIandaproposedsolutionS,whetherornotSisasolutionofI.Moreover,CmakesthisdecisionintimepolynomialinthelengthofI(wecanassumethatSisitselfencodedasabinarystring,andweknowthatthelengthofthisstringispolynomialinthelengthofI).RecallnowourargumentinSection7.7thatanypolynomialalgorithmcanberenderedasacircuit,whoseinputgatesencodetheinputtothealgorithm.Naturally,foranyinputlength(numberofinputbits)thecircuitwillbescaledtotheappropriatenumberofinputs,butthetotalnumberofgatesofthecircuitwillbepolynomialinthenumberofinputs.Ifthepolynomialalgorithminquestionsolvesaproblemthatrequiresayesornoanswer(asisthesituationwithC:“DoesSencodeasolutiontotheinstanceencodedbyI?”),thenthisanswerisgivenattheoutputgate.Weconcludethat,givenanyinstanceIofproblemA,wecanconstructinpolynomialtimeacircuitwhoseknowninputsarethebitsofI,andwhoseunknowninputsarethebitsofS,suchthattheoutputistrueifandonlyiftheunknowninputsspellasolutionSofI.Inotherwords,thesatisfyingtruthassignmentstotheunknowninputsofthecircuitareinone-to-onecorrespondencewiththesolutionsofinstanceIofA.Thereductioniscomplete.261\nUnsolvableproblemsAtleastanNP-completeproblemcanbesolvedbysomealgorithm—thetroubleisthatthisalgorithmwillbeexponential.Butitturnsoutthereareperfectlydecentcomputationalproblemsforwhichnoalgorithmsexistatall!OnefamousproblemofthissortisanarithmeticalversionofSAT.Givenapolynomialequationinmanyvariables,perhapsx3yz+2y4z2 7xy5z=6;arethereintegervaluesofx;y;zthatsatisfyit?Thereisnoalgorithmthatsolvesthisproblem.Noalgorithmatall,polynomial,exponential,doublyexponential,orworse!Suchproblemsarecalledunsolvable.Therstunsolvableproblemwasdiscoveredin1936byAlanM.Turing,thenastudentofmathematicsatCambridge,England.]WhenTuringcameupwithit,therewerenocomputersorprogramminglanguages(infact,itcanbearguedthatthesethingscameaboutlaterexactlybecausethisbrilliantthoughtoccurredtoTuring).Buttodaywecanstateitinfamiliarterms.Supposethatyouaregivenaprograminyourfavoriteprogramminglanguage,alongwithaparticularinput.Willtheprogrameverterminate,oncestartedonthisinput?Thisisaveryreasonablequestion.Manyofuswouldbeecstaticifwehadanalgorithm,callitterminates(p,x),thattookasinputalecontainingaprogramp,andaleofdatax,andaftergrindingaway,nallytolduswhetherornotpwouldeverstopifstartedonx.Buthowwouldyougoaboutwritingtheprogramterminates?(Ifyouhaven'tseenthisbefore,it'sworththinkingaboutitforawhile,toappreciatethedifcultyofwritingsuchan“universalinnite-loopdetector.”)Well,youcan't.Suchanalgorithmdoesnotexist!Andhereistheproof:Supposeweactuallyhadsuchaprogramterminates(p,x).Thenwecoulduseitasasubroutineofthefollowingevilprogram:functionparadox(z:file)1:ifterminates(z,z)goto1Noticewhatparadoxdoes:itterminatesifandonlyifprogramzdoesnotterminatewhengivenitsowncodeasinput.Youshouldsmelltrouble.Whatifweputthisprograminalenamedparadoxandweexecutedparadox(paradox)?Wouldthisexecutioneverstop?Ornot?Neitheranswerispossible.Sincewearrivedatthiscontradictionbyassumingthatthereisanalgorithmfortellingwhetherprogramsterminate,wemustconcludethatthisproblemcannotbesolvedbyanyalgorithm.Bytheway,allthistellsussomethingimportantaboutprogramming:Itwillneverbeautomated,itwillforeverdependondiscipline,ingenuity,andhackery.Wenowknowthatyoucan'ttellwhetheraprogramhasaninniteloop.Butcanyoutellifithasabufferoverrun?Doyouseehowtousetheunsolvabilityofthe“haltingproblem”toshowthatthis,too,isunsolvable?262\nExercises8.1.Optimizationversussearch.Recallthetravelingsalesmanproblem:TSPInput:Amatrixofdistances;abudgetbOutput:Atourwhichpassesthroughallthecitiesandhaslengthb,ifsuchatourexists.Theoptimizationversionofthisproblemasksdirectlyfortheshortesttour.TSP-OPTInput:AmatrixofdistancesOutput:Theshortesttourwhichpassesthroughallthecities.ShowthatifTSPcanbesolvedinpolynomialtime,thensocanTSP-OPT.8.2.Searchversusdecision.SupposeyouhaveaprocedurewhichrunsinpolynomialtimeandtellsyouwhetherornotagraphhasaRudratapath.Showthatyoucanuseittodevelopapolynomial-timealgorithmforRUDRATAPATH(whichreturnstheactualpath,ifitexists).8.3.STINGYSATisthefollowingproblem:givenasetofclauses(eachadisjunctionofliterals)andanintegerk,ndasatisfyingassignmentinwhichatmostkvariablesaretrue,ifsuchanassignmentexists.ProvethatSTINGYSATisNP-complete.8.4.ConsidertheCLIQUEproblemrestrictedtographsinwhicheveryvertexhasdegreeatmost3.CallthisproblemCLIQUE-3.(a)ProvethatCLIQUE-3isinNP.(b)WhatiswrongwiththefollowingproofofNP-completenessforCLIQUE-3?WeknowthattheCLIQUEproblemingeneralgraphsisNP-complete,soitisenoughtopresentareductionfromCLIQUE-3toCLIQUE.GivenagraphGwithverticesofdegree3,andaparameterg,thereductionleavesthegraphandtheparameterunchanged:clearlytheoutputofthereductionisapossibleinputfortheCLIQUEproblem.Furthermore,theanswertobothproblemsisidentical.Thisprovesthecorrectnessofthereductionand,therefore,theNP-completenessofCLIQUE-3.(c)ItistruethattheVERTEXCOVERproblemremainsNP-completeevenwhenrestrictedtographsinwhicheveryvertexhasdegreeatmost3.CallthisproblemVC-3.WhatiswrongwiththefollowingproofofNP-completenessforCLIQUE-3?WepresentareductionfromVC-3toCLIQUE-3.GivenagraphG=(V;E)withnodedegreesboundedby3,andaparameterb,wecreateaninstanceofCLIQUE-3byleavingthegraphunchangedandswitchingtheparametertojVj b.Now,asubsetCVisavertexcoverinGifandonlyifthecomplementarysetV CisacliqueinG.ThereforeGhasavertexcoverofsizebifandonlyifithasacliqueofsizejVj b.Thisprovesthecorrectnessofthereductionand,consequently,theNP-completenessofCLIQUE-3.(d)DescribeanO(jVj4)algorithmforCLIQUE-3.8.5.Giveasimplereductionfrom3DMATCHINGtoSAT,andanotherfromRUDRATACYCLEtoSAT.(Hint:Inthelattercaseyoumayusevariablesxijwhoseintuitivemeaningis“vertexiisthejthvertexoftheHamiltoncycle”;youthenneedtowriteclausesthatexpresstheconstraintsoftheproblem.)8.6.Onpage250wesawthat3SATremainsNP-completeevenwhenrestrictedtoformulasinwhicheachliteralappearsatmosttwice.263\n(a)Showthatifeachliteralappearsatmostonce,thentheproblemissolvableinpolynomialtime.(b)ShowthatINDEPENDENTSETremainsNP-completeeveninthespecialcasewhenallthenodesinthegraphhavedegreeatmost4.8.7.Consideraspecialcaseof3SATinwhichallclauseshaveexactlythreeliterals,andeachvariableappearsatmostthreetimes.Showthatthisproblemcanbesolvedinpolynomialtime.(Hint:createabipartitegraphwithclausesontheleft,variablesontheright,andedgeswheneveravariableappearsinaclause.UseExercise7.30toshowthatthisgraphhasamatching.)8.8.IntheEXACT4SATproblem,theinputisasetofclauses,eachofwhichisadisjunctionofexactlyfourliterals,andsuchthateachvariableoccursatmostonceineachclause.Thegoalistondasatisfyingassignment,ifoneexists.ProvethatEXACT4SATisNP-complete.8.9.IntheHITTINGSETproblem,wearegivenafamilyofsetsfS1;S2;:::;Sngandabudgetb,andwewishtondasetHofsizebwhichintersectseverySi,ifsuchanHexists.Inotherwords,wewantH\Si6=;foralli.ShowthatHITTINGSETisNP-complete.8.10.ProvingNP-completenessbygeneralization.Foreachoftheproblemsbelow,provethatitisNP-completebyshowingthatitisageneralizationofsomeNP-completeproblemwehaveseeninthischapter.(a)SUBGRAPHISOMORPHISM:GivenasinputtwoundirectedgraphsGandH,determinewhetherGisasubgraphofH(thatis,whetherbydeletingcertainverticesandedgesofHweobtainagraphthatis,uptorenamingofvertices,identicaltoG),andifso,returnthecorrespondingmappingofV(G)intoV(H).(b)LONGESTPATH:GivenagraphGandanintegerg,ndinGasimplepathoflengthg.(c)MAXSAT:GivenaCNFformulaandanintegerg,ndatruthassignmentthatsatisesatleastgclauses.(d)DENSESUBGRAPH:Givenagraphandtwointegersaandb,ndasetofaverticesofGsuchthatthereareatleastbedgesbetweenthem.(e)SPARSESUBGRAPH:Givenagraphandtwointegersaandb,ndasetofaverticesofGsuchthatthereareatmostbedgesbetweenthem.(f)SETCOVER.(ThisproblemgeneralizestwoknownNP-completeproblems.)(g)RELIABLENETWORK:Wearegiventwonnmatrices,adistancematrixdijandaconnec-tivityrequirementmatrixrij,aswellasabudgetb;wemustndagraphG=(f1;2;:::;ng;E)suchthat(1)thetotalcostofalledgesisborlessand(2)betweenanytwodistinctverticesiandjtherearerijvertex-disjointpaths.(Hint:Supposethatalldij'sare1or2,b=n,andallrij'sare2.WhichwellknownNP-completeproblemisthis?)8.11.TherearemanyvariantsofRudrata'sproblem,dependingonwhetherthegraphisundirectedordirected,andwhetheracycleorpathissought.ReducetheDIRECTEDRUDRATAPATHproblemtoeachofthefollowing.(a)The(undirected)RUDRATAPATHproblem.(b)TheundirectedRUDRATA(s;t)-PATHproblem,whichisjustlikeRUDRATAPATHexceptthattheendpointsofthepatharespeciedintheinput.8.12.Thek-SPANNINGTREEproblemisthefollowing.264\nInput:AnundirectedgraphG=(V;E)Output:AspanningtreeofGinwhicheachnodehasdegreek,ifsuchatreeexists.Showthatforanyk2:(a)k-SPANNINGTREEisasearchproblem.(b)k-SPANNINGTREEisNP-complete.(Hint:Startwithk=2andconsidertherelationbetweenthisproblemandRUDRATAPATH.)8.13.DeterminewhichofthefollowingproblemsareNP-completeandwhicharesolvableinpolyno-mialtime.IneachproblemyouaregivenanundirectedgraphG=(V;E),alongwith:(a)AsetofnodesLV,andyoumustndaspanningtreesuchthatitssetofleavesincludesthesetL.(b)AsetofnodesLV,andyoumustndaspanningtreesuchthatitssetofleavesispreciselythesetL.(c)AsetofnodesLV,andyoumustndaspanningtreesuchthatitssetofleavesisincludedinthesetL.(d)Anintegerk,andyoumustndaspanningtreewithkorfewerleaves.(e)Anintegerk,andyoumustndaspanningtreewithkormoreleaves.(f)Anintegerk,andyoumustndaspanningtreewithexactlykleaves.(Hint:AlltheNP-completenessproofsarebygeneralization,exceptforone.)8.14.ProvethatthefollowingproblemisNP-complete:givenanundirectedgraphG=(V;E)andanintegerk,returnacliqueofsizekaswellasanindependentsetofsizek,providedbothexist.8.15.ShowthatthefollowingproblemisNP-complete.MAXIMUMCOMMONSUBGRAPHInput:TwographsG1=(V1;E1)andG2=(V2;E2);abudgetb.Output:TwosetofnodesV0VandV0Vwhosedeletionleavesatleastbnodes1122ineachgraph,andmakesthetwographsidentical.8.16.Wearefeelingexperimentalandwanttocreateanewdish.Therearevariousingredientswecanchoosefromandwe'dliketouseasmanyofthemaspossible,butsomeingredientsdon'tgowellwithothers.Iftherearenpossibleingredients(numbered1ton),wewritedownannnmatrixgivingthediscordbetweenanypairofingredients.Thisdiscordisarealnumberbetween0:0and1:0,where0:0means“theygotogetherperfectly”and1:0means“theyreallydon'tgotogether.”Here'sanexamplematrixwhentherearevepossibleingredients.1234510.00.40.20.91.020.40.00.11.00.230.20.10.00.80.540.91.00.80.00.251.00.20.50.20.0Inthiscase,ingredients2and3gotogetherprettywellwhereas1and5clashbadly.Noticethatthismatrixisnecessarilysymmetric;andthatthediagonalentriesarealways0:0.Anysetofingredientsincursapenaltywhichisthesumofalldiscordvaluesbetweenpairsofingredients.Forinstance,thesetofingredientsf1;3;5gincursapenaltyof0:2+1:0+0:5=1:7.Wewantthispenaltytobesmall.265\nEXPERIMENTALCUISINEInput:n,thenumberofingredientstochoosefrom;D,thenn“discord”matrix;somenumberp0Output:Themaximumnumberofingredientswecanchoosewithpenaltyp.ShowthatifEXPERIMENTALCUISINEissolvableinpolynomialtime,thensois3SAT.8.17.ShowthatforanyprobleminNP,thereisanalgorithmwhichsolvesintimeO(2p(n)),wherenisthesizeoftheinputinstanceandp(n)isapolynomial(whichmaydependon).8.18.ShowthatifP=NPthentheRSAcryptosystem(Section1.4.2)canbebrokeninpolynomialtime.8.19.Akiteisagraphonanevennumberofvertices,say2n,inwhichnoftheverticesformacliqueandtheremainingnverticesareconnectedina“tail”thatconsistsofapathjoinedtooneoftheverticesoftheclique.Givenagraphandagoalg,theKITEproblemasksforasubgraphwhichisakiteandwhichcontains2gnodes.ProvethatKITEisNP-complete.8.20.InanundirectedgraphG=(V;E),wesayDVisadominatingsetifeveryv2ViseitherinDoradjacenttoatleastonememberofD.IntheDOMINATINGSETproblem,theinputisagraphandabudgetb,andtheaimistondadominatingsetinthegraphofsizeatmostb,ifoneexists.ProvethatthisproblemisNP-complete.8.21.Sequencingbyhybridization.OneexperimentalprocedureforidentifyinganewDNAsequencerepeatedlyprobesittodeterminewhichk-mers(substringsoflengthk)itcontains.Basedonthese,thefullsequencemustthenbereconstructed.Let'snowformulatethisasacombinatorialproblem.Foranystringx(theDNAsequence),let (x)denotethemultisetofallofitsk-mers.Inparticular, (x)containsexactlyjxj k+1elements.Thereconstructionproblemisnoweasytostate:givenamultisetofk-lengthstrings,ndastringxsuchthat (x)isexactlythismultiset.(a)ShowthatthereconstructionproblemreducestoRUDRATAPATH.(Hint:Constructadi-rectedgraphwithonenodeforeachk-mer,andwithanedgefromatobifthelastk 1charactersofamatchtherstk 1charactersofb.)(b)Butinfact,thereismuchbetternews.ShowthatthesameproblemalsoreducestoEULERPATH.(Hint:Thistime,useonedirectededgeforeachk-mer.)8.22.Intaskscheduling,itiscommontouseagraphrepresentationwithanodeforeachtaskandadirectededgefromtaskitotaskjifiisapreconditionforj.Thisdirectedgraphdepictstheprecedenceconstraintsintheschedulingproblem.Clearly,ascheduleispossibleifandonlyifthegraphisacyclic;ifitisn't,we'dliketoidentifythesmallestnumberofconstraintsthatmustbedroppedsoastomakeitacyclic.GivenadirectedgraphG=(V;E),asubsetE0EiscalledafeedbackarcsetiftheremovalofedgesE0rendersGacyclic.FEEDBACKARCSET(FAS):GivenadirectedgraphG=(V;E)andabudgetb,ndafeedbackarcsetofbedges,ifoneexists.(a)ShowthatFASisinNP.FAScanbeshowntobeNP-completebyareductionfromVERTEXCOVER.Givenaninstance(G;b)ofVERTEXCOVER,whereGisanundirectedgraphandwewantavertexcoverofsizeb,weconstructainstance(G0;b)ofFASasfollows.IfG=(V;E)hasnverticesv;:::;v,thenmake1nG0=(V0;E0)adirectedgraphwith2nverticesw;w0;:::;w;w0,andn+2jEj(directed)edges:11nn266\n(w;w0)foralli=1;2;:::;n.ii(w0;w)and(w0;w)forevery(v;v)2E.ijjiij(b)ShowthatifGcontainsavertexcoverofsizeb,thenG0containsafeedbackarcsetofsizeb.(c)ShowthatifG0containsafeedbackarcsetofsizeb,thenGcontainsavertexcoverofsize(atmost)b.(Hint:givenafeedbackarcsetofsizebinG0,youmayneedtorstmodifyitslightlytoobtainanotheronewhichisofamoreconvenientform,butisofthesamesizeorsmaller.Then,arguethatGmustcontainavertexcoverofthesamesizeasthemodiedfeedbackarcset.)8.23.IntheNODE-DISJOINTPATHSproblem,theinputisanundirectedgraphinwhichsomeverticeshavebeenspeciallymarked:acertainnumberof“sources”s1;s2;:::skandanequalnumberof“destinations”t1;t2;:::tk.Thegoalistondknode-disjointpaths(thatis,pathswhichhavenonodesincommon)wheretheithpathgoesfromsitoti.ShowthatthisproblemisNP-complete.Hereisasequenceofprogressivelystrongerhints.(a)Reducefrom3SAT.(b)Fora3SATformulawithmclausesandnvariables,usek=m+nsourcesanddestinations.Introduceonesource/destinationpair(sx;tx)foreachvariablex,andonesource/destinationpair(sc;tc)foreachclausec.(c)Foreach3SATclause,introduce6newintermediatevertices,oneforeachliteraloccurringinthatclauseandoneforitscomplement.(d)Noticethatifthepathfromsctotcgoesthroughsomeintermediatevertexrepresenting,say,anoccurrenceofvariablex,thennootherpathcangothroughthatvertex.Whatvertexwouldyouliketheotherpathtobeforcedtogothroughinstead?267\n268\nChapter9CopingwithNP-completenessYouarethejuniormemberofaseasonedprojectteam.Yourcurrenttaskistowritecodeforsolvingasimple-lookingprobleminvolvinggraphsandnumbers.Whatareyousupposedtodo?Ifyouareverylucky,yourproblemwillbeamongthehalf-dozenproblemsconcerninggraphswithweights(shortestpath,minimumspanningtree,maximumow,etc.),thatwehavesolvedinthisbook.Evenifthisisthecase,recognizingsuchaprobleminitsnaturalhabitat—grungyandobscuredbyrealityandcontext—requirespracticeandskill.Itismorelikelythatyouwillneedtoreduceyourproblemtooneoftheseluckyones—ortosolveitusingdynamicprogrammingorlinearprogramming.Butchancesarethatnothinglikethiswillhappen.Theworldofsearchproblemsisableaklandscape.Thereareafewspotsoflight—brilliantalgorithmicideas—eachilluminatingasmallareaaroundit(theproblemsthatreducetoit;twooftheseareas,linearanddynamicprogramming,areinfactdecentlylarge).Buttheremainingvastexpanseispitchdark:NP-complete.Whatareyoutodo?YoucanstartbyprovingthatyourproblemisactuallyNP-complete.Oftenaproofbygeneralization(recallthediscussiononpage254andExercise8.10)isallthatyouneed;andsometimesasimplereductionfrom3SATorZOEisnottoodifculttond.Thissoundslikeatheoreticalexercise,but,ifcarriedoutsuccessfully,itdoesbringsometangiblerewards:nowyourstatusintheteamhasbeenelevated,youarenolongerthekidwhocan'tdo,andyouhavebecomethenobleknightwiththeimpossiblequest.But,unfortunately,aproblemdoesnotgoawaywhenprovedNP-complete.Therealques-tionis,Whatdoyoudonext?Thisisthesubjectofthepresentchapterandalsotheinspirationforsomeofthemostimportantmodernresearchonalgorithmsandcomplexity.NP-completenessisnotadeathcerticate—itisonlythebeginningofafascinatingadventure.Yourproblem'sNP-completenessproofprobablyconstructsgraphsthatarecomplicatedandweird,verymuchunlikethosethatcomeupinyourapplication.Forexample,eventhoughSATisNP-complete,satisfyingassignmentsforHORNSAT(theinstancesofSATthatcomeupinlogicprogramming)canbefoundefciently(recallSection5.3).Or,supposethegraphsthatariseinyourapplicationaretrees.Inthiscase,manyNP-completeproblems,suchasINDEPENDENTSET,canbesolvedinlineartimebydynamicprogramming(recallSection6.7).269\nUnfortunately,thisapproachdoesnotalwayswork.Forexample,weknowthat3SATisNP-complete.AndtheINDEPENDENTSETproblem,alongwithmanyotherNP-completeproblems,remainssoevenforplanargraphs(graphsthatcanbedrawnintheplanewithoutcrossingedges).Moreover,oftenyoucannotneatlycharacterizetheinstancesthatcomeupinyourapplication.Instead,youwillhavetorelyonsomeformofintelligentexponentialsearch—proceduressuchasbacktrackingandbranchandboundwhichareexponentialtimeintheworst-case,but,withtherightdesign,couldbeveryefcientontypicalinstancesthatcomeupinyourapplication.WediscussthesemethodsinSection9.1.OryoucandevelopanalgorithmforyourNP-completeoptimizationproblemthatfallsshortoftheoptimumbutneverbytoomuch.Forexample,inSection5.4wesawthatthegreedyalgorithmalwaysproducesasetcoverthatisnomorethanlogntimestheoptimalsetcover.Analgorithmthatachievessuchaguaranteeiscalledanapproximationalgorithm.AswewillseeinSection9.2,suchalgorithmsareknownformanyNP-completeoptimizationproblems,andtheyaresomeofthemostcleverandsophisticatedalgorithmsaround.AndthetheoryofNP-completenesscanagainbeusedasaguideinthisendeavor,byshowingthat,forsomeproblems,thereareevenseverelimitstohowwelltheycanbeapproximated—unlessofcourseP=NP.Finally,thereareheuristics,algorithmswithnoguaranteesoneithertherunningtimeorthedegreeofapproximation.Heuristicsrelyoningenuity,intuition,agoodunderstandingoftheapplication,meticulousexperimentation,andofteninsightsfromphysicsorbiology,toattackaproblem.WeseesomecommonkindsinSection9.3.9.1Intelligentexhaustivesearch9.1.1BacktrackingBacktrackingisbasedontheobservationthatitisoftenpossibletorejectasolutionbylookingatjustasmallportionofit.Forexample,ifaninstanceofSATcontainstheclause(x1_x2),thenallassignmentswithx1=x2=0(i.e.,false)canbeinstantlyeliminated.Toputitdifferently,byquicklycheckinganddiscreditingthispartialassignment,weareabletopruneaquarteroftheentiresearchspace.Apromisingdirection,butcanitbesystematicallyexploited?Here'showitisdone.ConsidertheBooleanformula (w;x;y;z)speciedbythesetofclauses(w_x_y_z);(w_x);(x_y);(y_z);(z_w);(w_z):Wewillincrementallygrowatreeofpartialsolutions.Westartbybranchingonanyonevariable,sayw:Initialformula w=0w=1Pluggingw=0andw=1into ,wendthatnoclauseisimmediatelyviolatedandthusneitherofthesetwopartialassignmentscanbeeliminatedoutright.Soweneedtokeep270\nbranching.Wecanexpandeitherofthetwoavailablenodes,andonanyvariableofourchoice.Let'strythisone:Initialformula w=0w=1x=0x=1Thistime,weareinluck.Thepartialassignmentw=0;x=1violatestheclause(w_x)andcanbeterminated,therebypruningagoodchunkofthesearchspace.Webacktrackoutofthiscul-de-sacandcontinueourexplorationsatoneofthetworemainingactivenodes.Inthismanner,backtrackingexploresthespaceofassignments,growingthetreeonlyatnodeswherethereisuncertaintyabouttheoutcome,andstoppingifatanystageasatisfyingassignmentisencountered.InthecaseofBooleansatisability,eachnodeofthesearchtreecanbedescribedeitherbyapartialassignmentorbytheclausesthatremainwhenthosevaluesarepluggedintotheoriginalformula.Forinstance,ifw=0andx=0thenanyclausewithworxisinstantlysatisedandanyliteralworxisnotsatisedandcanberemoved.What'sleftis(y_z);(y);(y_z):Likewise,w=0andx=1leaves();(y_z);withthe“emptyclause”()rulingoutsatisability.Thusthenodesofthesearchtree,repre-sentingpartialassignments,arethemselvesSATsubproblems.Thisalternativerepresentationishelpfulformakingthetwodecisionsthatrepeatedlyarise:whichsubproblemtoexpandnext,andwhichbranchingvariabletouse.Sincetheben-etofbacktrackingliesinitsabilitytoeliminateportionsofthesearchspace,andsincethishappensonlywhenanemptyclauseisencountered,itmakessensetochoosethesubproblemthatcontainsthesmallestclauseandtothenbranchonavariableinthatclause.Ifthisclausehappenstobeasingleton,thenatleastoneoftheresultingbrancheswillbeterminated.(Ifthereisatieinchoosingsubproblems,onereasonablepolicyistopicktheonelowestinthetree,inthehopethatitisclosetoasatisfyingassignment.)SeeFigure9.1fortheconclusionofourearlierexample.Moreabstractly,abacktrackingalgorithmrequiresatestthatlooksatasubproblemandquicklydeclaresoneofthreeoutcomes:1.Failure:thesubproblemhasnosolution.2.Success:asolutiontothesubproblemisfound.3.Uncertainty.271\nFigure9.1Backtrackingrevealsthat isnotsatisable.(w_x_y_z);(w_x);(x_y);(y_z);(z_w);(w_z)w=0w=1(x_y_z);(x);(x_y);(y_z)(x_y);(y_z);(z);(z)x=0x=1z=0z=1(y_z);(y);(y_z)();(y_z)(x_y);()(x_y);(y);()y=0y=1(z);(z)()z=0z=1()()InthecaseofSAT,thistestdeclaresfailureifthereisanemptyclause,successiftherearenoclauses,anduncertaintyotherwise.Thebacktrackingprocedurethenhasthefollowingformat.StartwithsomeproblemP0LetS=fP0g,thesetofactivesubproblemsRepeatwhileSisnonempty:chooseasubproblemP2SandremoveitfromSexpanditintosmallersubproblemsP1;P2;:::;PkForeachPi:Iftest(Pi)succeeds:haltandannouncethissolutionIftest(Pi)fails:discardPiOtherwise:addPitoSAnnouncethatthereisnosolutionForSAT,thechooseprocedurepicksaclause,andexpandpicksavariablewithinthatclause.Wehavealreadydiscussedsomereasonablewaysofmakingsuchchoices.Withtherighttest,expand,andchooseroutines,backtrackingcanberemarkablyeffec-tiveinpractice.ThebacktrackingalgorithmweshowedforSATisthebasisofmanysuccessfulsatisabilityprograms.Anothersignofqualityisthis:ifpresentedwitha2SATinstance,itwillalwaysndasatisfyingassignment,ifoneexists,inpolynomialtime(Exercise9.1)!9.1.2Branch-and-boundThesameprinciplecanbegeneralizedfromsearchproblemssuchasSATtooptimizationproblems.Forconcreteness,let'ssaywehaveaminimizationproblem;maximizationwill272\nfollowthesamepattern.Asbefore,wewilldealwithpartialsolutions,eachofwhichrepresentsasubproblem,namely,whatisthe(costofthe)bestwaytocompletethissolution?Andasbefore,weneedabasisforeliminatingpartialsolutions,sincethereisnoothersourceofefciencyinourmethod.Torejectasubproblem,wemustbecertainthatitscostexceedsthatofsomeothersolutionwehavealreadyencountered.Butitsexactcostisunknowntousandisgenerallynotefcientlycomputable.Soinsteadweuseaquicklowerboundonthiscost.StartwithsomeproblemP0LetS=fP0g,thesetofactivesubproblemsbestsofar=1RepeatwhileSisnonempty:chooseasubproblem(partialsolution)P2SandremoveitfromSexpanditintosmallersubproblemsP1;P2;:::;PkForeachPi:IfPiisacompletesolution:updatebestsofarelseiflowerbound(Pi)0.ApartialsolutionisasimplepathabpassingthroughsomeverticesSV,whereSincludestheendpointsaandb.Wecandenotesuchapartialsolutionbythetuple[a;S;b]—infact,awillbexedthroughoutthealgorithm.Thecorrespondingsubproblemistondthebestcompletionofthetour,thatis,thecheapestcomplementarypathbawithintermediatenodesV S.Noticethattheinitialproblemisoftheform[a;fag;a]foranya2Vofourchoosing.Ateachstepofthebranch-and-boundalgorithm,weextendaparticularpartialsolution[a;S;b]byasingleedge(b;x),wherex2V S.TherecanbeuptojV Sjwaystodothis,andeachofthesebranchesleadstoasubproblemoftheform[a;S[fxg;x].Howcanwelower-boundthecostofcompletingapartialtour[a;S;b]?Manysophisticatedmethodshavebeendevelopedforthis,butlet'slookatarathersimpleone.TheremainderofthetourconsistsofapaththroughV S,plusedgesfromaandbtoV S.Therefore,itscostisatleastthesumofthefollowing:1.ThelightestedgefromatoV S.2.ThelightestedgefrombtoV S.3.TheminimumspanningtreeofV S.(Doyouseewhy?)Andthislowerboundcanbecomputedquicklybyaminimumspanningtreealgorithm.Figure9.2runsthroughanexample:eachnodeofthetreerepresentsapartialtour(specically,thepathfromtheroottothatnode)thatatsomestageisconsideredbythebranch-and-boundprocedure.Noticehowjust28partialsolutionsareconsidered,insteadofthe7!=5;040thatwouldariseinabrute-forcesearch.273\nFigure9.2(a)Agraphanditsoptimaltravelingsalesmantour.(b)Thebranch-and-boundsearchtree,exploredlefttoright.Boxednumbersindicatelowerboundsoncost.(a)11FEFE22GDGD11111111115HCHC1111ABAB2(b)A10BF8H810CE108EGCG1314810DH14DF108BD151211EGGGC8inf141411F8DHinf11G8G11H8HCost:11Cost:8274\n9.2ApproximationalgorithmsInanoptimizationproblemwearegivenaninstanceIandareaskedtondtheoptimumsolution—theonewiththemaximumgainifwehaveamaximizationproblemlikeINDEPEN-DENTSET,ortheminimumcostifwearedealingwithaminimizationproblemsuchastheTSP.ForeveryinstanceI,letusdenotebyOPT(I)thevalue(benetorcost)oftheoptimumsolution.Itmakesthemathalittlesimpler(andisnottoofarfromthetruth)toassumethatOPT(I)isalwaysapositiveinteger.Wehavealreadyseenanexampleofa(famous)approximationalgorithminSection5.4:thegreedyschemeforSETCOVER.ForanyinstanceIofsizen,weshowedthatthisgreedyalgorithmisguaranteedtoquicklyndasetcoverofcardinalityatmostOPT(I)logn.Thislognfactorisknownastheapproximationguaranteeofthealgorithm.Moregenerally,consideranyminimizationproblem.Supposenowthatwehaveanalgo-rithmAforourproblemwhich,givenaninstanceI,returnsasolutionwithvalueA(I).TheapproximationratioofalgorithmAisdenedtobeA(I)A=max:IOPT(I)Inotherwords,AmeasuresbythefactorbywhichtheoutputofalgorithmAexceedstheoptimalsolution,ontheworst-caseinput.Theapproximationratiocanalsobedenedformaximizationproblems,suchasINDEPENDENTSET,inthesameway—exceptthattogetanumberlargerthan1wetakethereciprocal.So,whenfacedwithanNP-completeoptimizationproblem,areasonablegoalistolookforanapproximationalgorithmAwhoseAisassmallaspossible.Butthiskindofguaranteemightseemalittlepuzzling:Howcanwecomeclosetotheoptimumifwecannotdeterminetheoptimum?Let'slookatasimpleexample.9.2.1VertexcoverWealreadyknowtheVERTEXCOVERproblemisNP-hard.VERTEXCOVERInput:AnundirectedgraphG=(V;E).Output:AsubsetoftheverticesSVthattoucheseveryedge.Goal:MinimizejSj.SeeFigure9.3foranexample.SinceVERTEXCOVERisaspecialcaseofSETCOVER,weknowfromChapter5thatitcanbeapproximatedwithinafactorofO(logn)bythegreedyalgorithm:repeatedlydeletethevertexofhighestdegreeandincludeitinthevertexcover.Andtherearegraphsonwhichthegreedyalgorithmreturnsavertexcoverthatisindeedlogntimestheoptimum.AbetterapproximationalgorithmforVERTEXCOVERisbasedonthenotionofamatching,asubsetofedgesthathavenoverticesincommon(Figure9.4).Amatchingismaximalifnomoreedgescanbeaddedtoit.Maximalmatchingswillhelpusndgoodvertexcovers,andmoreover,theyareeasytogenerate:repeatedlypickedgesthataredisjointfromtheoneschosenalready,untilthisisnolongerpossible.275\nFigure9.3Agraphwhoseoptimalvertexcover,shownshaded,isofsize8.Figure9.4(a)Amatching,(b)itscompletiontoamaximalmatching,and(c)theresultingvertexcover.(a)(b)(c)Whatistherelationshipbetweenmatchingsandvertexcovers?Hereisthecrucialfact:anyvertexcoverofagraphGmustbeatleastaslargeasthenumberofedgesinanymatchinginG;thatis,anymatchingprovidesalowerboundonOPT.Thisissimplybecauseeachedgeofthematchingmustbecoveredbyoneofitsendpointsinanyvertexcover!Findingsuchalowerboundisakeystepindesigninganapproximationalgorithm,becausewemustcomparethequalityofthesolutionfoundbyouralgorithmtoOPT,whichisNP-completetocompute.Onemoreobservationcompletesthedesignofourapproximationalgorithm:letSbeasetthatcontainsbothendpointsofeachedgeinamaximalmatchingM.ThenSmustbeavertexcover—ifitisn't,thatis,ifitdoesn'ttouchsomeedgee2E,thenMcouldnotpossiblybemaximalsincewecouldstilladdetoit.ButourcoverShas2jMjvertices.AndfromthepreviousparagraphweknowthatanyvertexcovermusthavesizeatleastjMj.Sowe'redone.Here'sthealgorithmforVERTEXCOVER.FindamaximalmatchingMEReturnS=fallendpointsofedgesinMgThissimpleprocedurealwaysreturnsavertexcoverwhosesizeisatmosttwiceoptimal!Insummary,eventhoughwehavenowayofndingthebestvertexcover,wecaneasilyndanotherstructure,amaximalmatching,withtwokeyproperties:1.Itssizegivesusalowerboundontheoptimalvertexcover.276\n2.Itcanbeusedtobuildavertexcover,whosesizecanberelatedtothatoftheoptimalcoverusingproperty1.Thus,thissimplealgorithmhasanapproximationratioofA2.Infact,itisnothardtondexamplesonwhichitdoesmakea100%error;henceA=2.9.2.2ClusteringWeturnnexttoaclusteringproblem,inwhichwehavesomedata(textdocuments,say,orimages,orspeechsamples)thatwewanttodivideintogroups.Itisoftenusefultodene“dis-tances”betweenthesedatapoints,numbersthatcapturehowcloseorfartheyarefromoneanother.Oftenthedataaretruepointsinsomehigh-dimensionalspaceandthedistancesaretheusualEuclideandistance;inothercases,thedistancesaretheresultofsome“similaritytests”towhichwehavesubjectedthedatapoints.Assumethatwehavesuchdistancesandthattheysatisfytheusualmetricproperties:1.d(x;y)0forallx;y.2.d(x;y)=0ifandonlyifx=y.3.d(x;y)=d(y;x).4.(Triangleinequality)d(x;y)d(x;z)+d(z;y).Wewouldliketopartitionthedatapointsintogroupsthatarecompactinthesenseofhavingsmalldiameter.k-CLUSTERInput:PointsX=fx1;:::;xngwithunderlyingdistancemetricd(;);integerk.Output:ApartitionofthepointsintokclustersC1;:::;Ck.Goal:Minimizethediameteroftheclusters,maxmaxd(xa;xb):jxa;xb2CjOnewaytovisualizethistaskistoimaginenpointsinspace,whicharetobecoveredbykspheresofequalsize.Whatisthesmallestpossiblediameterofthespheres?Figure9.5showsanexample.ThisproblemisNP-hard,buthasaverysimpleapproximationalgorithm.Theideaistopickkofthedatapointsasclustercentersandtothenassigneachoftheremainingpointstothecenterclosesttoit,thuscreatingkclusters.Thecentersarepickedoneatatime,usinganintuitiverule:alwayspickthenextcentertobeasfaraspossiblefromthecenterschosensofar(seeFigure9.6).Pickanypoint12Xasthefirstclustercenterfori=2tok:LetibethepointinXthatisfarthestfrom1;:::;i 1(i.e.,thatmaximizesminj0producesaninstanceI(G;C)oftheTSPsuchthat:(i)IfGhasaRudratapath,thenOPT(I(G;C))=n,thenumberofverticesinG.(ii)IfGhasnoRudratapath,thenOPT(I(G;C))n+C.ThismeansthatevenanapproximatesolutiontoTSPwouldenableustosolveRUDRATAPATH!Let'sworkoutthedetails.ConsideranapproximationalgorithmAforTSPandletAdenoteitsapproximationratio.FromanyinstanceGofRUDRATAPATH,wewillcreateaninstanceI(G;C)ofTSPusingthespecicconstantC=n A.WhathappenswhenalgorithmAisrunonthisTSPinstance?Incase(i),itmustoutputatouroflengthatmostAOPT(I(G;C))=n A,whereasincase(ii)itmustoutputatouroflengthatleastOPT(I(G;C))>n A.ThuswecangureoutwhetherGhasaRudratapath!Hereistheresultingprocedure:GivenanygraphG:computeI(G;C)(withC=nA)andrunalgorithmAonitiftheresultingtourhaslengthn A:concludethatGhasaRudratapathelse:concludethatGhasnoRudratapathThistellsuswhetherornotGhasaRudratapath;bycallingtheprocedureapolynomialnumberoftimes,wecanndtheactualpath(Exercise8.2).We'veshownthatifTSPhasapolynomial-timeapproximationalgorithm,thenthereisapolynomialalgorithmfortheNP-completeRUDRATAPATHproblem.So,unlessP=NP,therecannotexistanefcientapproximationalgorithmfortheTSP.9.2.4KnapsackOurlastapproximationalgorithmisforamaximizationproblemandhasaveryimpressiveguarantee:givenany>0,itwillreturnasolutionofvalueatleast(1 )timestheoptimalvalue,intimethatscalesonlypolynomiallyintheinputsizeandin1=.TheproblemisKNAPSACK,whichwerstencounteredinChapter6.Therearenitems,withweightsw1;:::;wnandvaluesv1;:::;vn(allpositiveintegers),andthegoalistopickthemostvaluablecombinationofitemssubjecttotheconstraintthattheirtotalweightisatmostW.EarlierwesawadynamicprogrammingsolutiontothisproblemwithrunningtimeO(nW).Usingasimilartechnique,arunningtimeofO(nV)canalsobeachieved,whereVisthesumofthevalues.Neitheroftheserunningtimesispolynomial,becauseWandVcanbeverylarge,exponentialinthesizeoftheinput.280\nLet'sconsidertheO(nV)algorithm.InthebadcasewhenVislarge,whatifwesimplyscaledownallthevaluesinsomeway?Forinstance,ifv1=117;586;003;v2=738;493;291;v3=238;827;453;wecouldsimplyknockoffsomeprecisionandinsteaduse117,738,and238.Thisdoesn'tchangetheproblemallthatmuchandwillmakethealgorithmmuch,muchfaster!Nowforthedetails.Alongwiththeinput,theuserisassumedtohavespeciedsomeapproximationfactor>0.Discardanyitemwithweight>WLetvmax=maxiviRescalevaluesvb=bvnciivmaxRunthedynamicprogrammingalgorithmwithvaluesfvbigOutputtheresultingchoiceofitemsLet'sseewhythisworks.Firstofall,sincetherescaledvaluesvbiareallatmostn=,thedynamicprogramisefcient,runningintimeO(n3=).NowsupposetheoptimalsolutiontotheoriginalproblemistopicksomesubsetofitemsS,withtotalvalueK.TherescaledvalueofthissameassignmentisXXnXnnvb=vv 1K n:iiivmaxvmaxvmaxi2Si2Si2STherefore,theoptimalassignmentfortheshrunkenproblem,callitSb,hasarescaledvalueofatleastthismuch.Intermsoftheoriginalvalues,assignmentSbhasavalueofatleastXXvmaxnvmaxvivbiK n=K vmaxK(1 ):nvmaxni2Sbi2Sb9.2.5TheapproximabilityhierarchyGivenanyNP-completeoptimizationproblem,weseekthebestapproximationalgorithmpossible.Failingthis,wetrytoprovelowerboundsontheapproximationratiosthatareachievableinpolynomialtime(wejustcarriedoutsuchaproofforthegeneralTSP).Alltold,NP-completeoptimizationproblemsareclassiedasfollows:Thoseforwhich,liketheTSP,noniteapproximationratioispossible.Thoseforwhichanapproximationratioispossible,buttherearelimitstohowsmallthiscanbe.VERTEXCOVER,k-CLUSTER,andtheTSPwithtriangleinequalitybelonghere.(Fortheseproblemswehavenotestablishedlimitstotheirapproximability,buttheselimitsdoexist,andtheirproofsconstitutesomeofthemostsophisticatedresultsinthiseld.)DownbelowwehaveamorefortunateclassofNP-completeproblemsforwhichap-proximabilityhasnolimits,andpolynomialapproximationalgorithmswitherrorratiosarbitrarilyclosetozeroexist.KNAPSACKresideshere.281\nFinally,thereisanotherclassofproblems,betweenthersttwogivenhere,forwhichtheapproximationratioisaboutlogn.SETCOVERisanexample.(Ahumblingreminder:AllthisiscontingentupontheassumptionP6=NP.Failingthis,thishierarchycollapsesdowntoP,andallNP-completeoptimizationproblemscanbesolvedexactlyinpolynomialtime.)Analpointonapproximationalgorithms:oftenthesealgorithms,ortheirvariants,per-formmuchbetterontypicalinstancesthantheirworst-caseapproximationratiowouldhaveyoubelieve.9.3LocalsearchheuristicsOurnextstrategyforcopingwithNP-completenessisinspiredbyevolution(whichis,afterall,theworld'sbest-testedoptimizationprocedure)—byitsincrementalprocessofintroducingsmallmutations,tryingthemout,andkeepingthemiftheyworkwell.Thisparadigmiscalledlocalsearchandcanbeappliedtoanyoptimizationtask.Here'showitlooksforaminimizationproblem.letsbeanyinitialsolutionwhilethereissomesolutions0intheneighborhoodofsforwhichcost(s0)w(S)do:setS=S0(a)ShowthatthisisanapproximationalgorithmforMAXCUTwithratio2.(b)Butisitapolynomial-timealgorithm?9.10.Letuscallalocalsearchalgorithmexactwhenitalwaysproducestheoptimumsolution.Forexample,thelocalsearchalgorithmfortheminimumspanningtreeproblemintroducedinProb-lem9.5isexact.Foranotherexample,simplexcanbeconsideredanexactlocalsearchalgorithmforlinearprogramming.(a)Showthatthe2-changelocalsearchalgorithmfortheTSPisnotexact.(b)Repeatforthedne-changelocalsearchalgorithm,wherenisthenumberofcities.2(c)Showthatthe(n 1)-changelocalsearchalgorithmisexact.(d)IfAisanoptimizationproblem,deneA-IMPROVEMENTtobethefollowingsearchproblem:GivenaninstancexofAandasolutionsofA,ndanothersolutionofxwithbettercost(orreportthatnoneexists,andthussisoptimum).Forexample,inTSPIMPROVEMENTwearegivenadistancematrixandatour,andweareaskedtondabettertour.ItturnsoutthatTSPIMPROVEMENTisNP-complete,andsoisSETCOVERIMPROVEMENT.(Canyouprovethis?)(e)Wesaythatalocalsearchalgorithmhaspolynomialiterationifeachexecutionofthelooprequirespolynomialtime.Forexample,theobviousimplementationsofthe(n 1)-changelocalsearchalgorithmfortheTSPdenedabovedonothavepolynomialiteration.Showthat,unlessP=NP,thereisnoexactlocalsearchalgorithmwithpolynomialiterationfortheTSPandSETCOVERproblems.293\n294\nChapter10QuantumalgorithmsThisbookstartedwiththeworld'soldestandmostwidelyusedalgorithms(theonesforaddingandmultiplyingnumbers)andanancienthardproblem(FACTORING).Inthislastchapterthetablesareturned:wepresentoneofthelatestalgorithms—anditisanefcientalgorithmforFACTORING!Thereisacatch,ofcourse:thisalgorithmneedsaquantumcomputertoexecute.QuantumphysicsisabeautifulandmysterioustheorythatdescribesNatureinthesmall,atthelevelofelementaryparticles.Oneofthemajordiscoveriesoftheninetieswasthatquantumcomputers—computersbasedonquantumphysicsprinciples—areradicallydiffer-entfromthosethatoperateaccordingtothemorefamiliarprinciplesofclassicalphysics.Surprisingly,theycanbeexponentiallymorepowerful:asweshallsee,quantumcomputerscansolveFACTORINGinpolynomialtime!Asaresult,inaworldwithquantumcomputers,thesystemsthatcurrentlysafeguardbusinesstransactionsontheInternet(andarebasedontheRSAcryptosystem)willnolongerbesecure.10.1Qubits,superposition,andmeasurementInthissectionweintroducethebasicfeaturesofquantumphysicsthatarenecessaryforunderstandinghowquantumcomputerswork.1Inordinarycomputerchips,bitsarephysicallyrepresentedbylowandhighvoltagesonwires.Buttherearemanyotherwaysabitcouldbestored—forinstance,inthestateofahydrogenatom.Thesingleelectroninthisatomcaneitherbeinthegroundstate(thelowestenergyconguration)oritcanbeinanexcitedstate(ahighenergyconguration).Wecanusethesetwostatestoencodeforbitvalues0and1,respectively.Letusnowintroducesomequantumphysicsnotation.Wedenotethegroundstateofourelectronby0,sinceitencodesforbitvalue0,andlikewisetheexcitedstateby1.Thesearethetwopossiblestatesoftheelectroninclassicalphysics.Manyofthemostcounterintuitiveaspectsofquantumphysicsarisefromthesuperpositionprinciplewhichstatesthatifa1ThiseldissostrangethatthefamousphysicistRichardFeynmanisquotedashavingsaid,“IthinkIcansafelysaythatnooneunderstandsquantumphysics.”Sothereislittlechanceyouwillunderstandthetheoryindepthafterreadingthissection!Butifyouareinterestedinlearningmore,seetherecommendedreadingatthebook'send.295\nFigure10.1Anelectroncanbeinagroundstateorinanexcitedstate.IntheDiracnotationusedinquantumphysics,thesearedenoted0and1.Butthesuperpositionprinciplesaysthat,infact,theelectronisinastatethatisalinearcombinationofthesetwo:00+11.Thiswouldmakeimmediatesenseifthe'swereprobabilities,nonnegativerealnumbersaddingto1.Butthesuperpositionprincipleinsiststhattheycanbearbitrarycomplexnum-bers,aslongasthesquaresoftheirnormsaddupto1!groundstate0excitedstate1superposition00+11quantumsystemcanbeinoneoftwostates,thenitcanalsobeinanylinearsuperpositionofthosetwostates.Forinstance,thestateoftheelectroncouldwellbep10+p11or22p10 p11;oraninnitenumberofothercombinationsoftheform0+1.The0122coefcient0iscalledtheamplitudeofstate0,andsimilarlywith1.And—ifthingsaren'talreadystrangeenough—the'scanbecomplexnumbers,aslongastheyarenormalizedsopthatjj2+jj2=1.Forexample,p10+p2i1(whereiistheimaginaryunit, 1)isa0155perfectlyvalidquantumstate!Suchasuperposition,00+11,isthebasicunitofencodedinformationinquantumcomputers(Figure10.1).Itiscalledaqubit(pronounced“cubit”).Thewholeconceptofasuperpositionsuggeststhattheelectrondoesnotmakeupitsmindaboutwhetheritisinthegroundorexcitedstate,andtheamplitude0isameasureofitsinclinationtowardthegroundstate.Continuingalongthislineofthought,itistemptingtothinkof0astheprobabilitythattheelectronisinthegroundstate.Butthenhowarewetomakesenseofthefactthat0canbenegative,orevenworse,imaginary?Thisisoneofthemostmysteriousaspectsofquantumphysics,onethatseemstoextendbeyondourintuitionsaboutthephysicalworld.Thislinearsuperposition,however,istheprivateworldoftheelectron.Forustogetaglimpseoftheelectron'sstatewemustmakeameasurement,andwhenwedoso,wegetasinglebitofinformation—0or1.Ifthestateoftheelectronis00+11,thentheoutcomeofthemeasurementis0withprobabilityj0j2and1withprobabilityj1j2(luckilywenormalizedsoj0j2+j1j2=1).Moreover,theactofmeasurementcausesthesystemtochangeitsstate:iftheoutcomeofthemeasurementis0,thenthenewstateofthesystemis0(thegroundstate),andiftheoutcomeis1,thenewstateis1(theexcitedstate).Thisfeatureofquantumphysics,thatameasurementdisturbsthesystemandforcesittochoose(inthiscasegroundorexcitedstate),isanotherstrangephenomenonwithnoclassicalanalog.Thesuperpositionprincipleholdsnotjustfor2-levelsystemsliketheonewejustdescribed,butingeneralfork-levelsystems.Forexample,inrealitytheelectroninthehydrogenatomcanbeinoneofmanyenergylevels,startingwiththegroundstate,therstexcitedstate,the296\nFigure10.2Measurementofasuperpositionhastheeffectofforcingthesystemtodecideonaparticularstate,withprobabilitiesdeterminedbytheamplitudes.state0withprobjj2000+111statewithprobjj21secondexcitedstate,andsoon.Sowecouldconsiderak-levelsystemconsistingofthegroundstateandtherstk 1excitedstates,andwecoulddenotetheseby0;1;2;:::;k 1.ThesuperpositionprinciplewouldthensaythatthegeneralquantumstateofthesystemisP0+1++k 1,wherek 1jj2=1.Measuringthestateofthesystem01k 1j=0jwouldnowrevealanumberbetween0andk 1,andoutcomejwouldoccurwithprobabilityjjj2.Asbefore,themeasurementwoulddisturbthesystem,andthenewstatewouldactuallybecomejorthejthexcitedstate.Howdoweencodenbitsofinformation?Wecouldchoosek=2nlevelsofthehydrogenatom.Butamorepromisingoptionistousenqubits.Letusstartbyconsideringthecaseoftwoqubits,thatis,thestateoftheelectronsoftwohydrogenatoms.Sinceeachelectroncanbeineitherthegroundorexcitedstate,inclassi-calphysicsthetwoelectronshaveatotaloffourpossiblestates—00,01,10,or11—andarethereforesuitableforstoring2bitsofinformation.Butinquantumphysics,thesuperpositionprincipletellsusthatthequantumstateofthetwoelectronsisalinearcombinationofthefourclassicalstates,=0000+0101+1010+1111;Pnormalizedsothat2jxj2=1.2Measuringthestateofthesystemnowreveals2bitsx2f0;1gofinformation,andtheprobabilityofoutcomex2f0;1g2isjxj2.Moreover,asbefore,iftheoutcomeofmeasurementisjk,thenthenewstateofthesystemisjk:ifjk=10,forexample,thentherstelectronisintheexcitedstateandthesecondelectronisinthegroundstate.Aninterestingquestioncomesuphere:whatifwemakeapartialmeasurement?Forinstance,ifwemeasurejusttherstqubit,whatistheprobabilitythattheoutcomeis0?Thisissimple.Itisexactlythesameasitwouldhavebeenhadwemeasuredbothqubits,namely,22nRecallthatf0;1gdenotesthesetconsistingofthefour2-bitbinarystringsandingeneralf0;1gdenotesthesetofalln-bitbinarystrings.297\nEntanglementSupposewehavetwoqubits,therstinthestate00+11andthesecondinthestate00+11.Whatisthejointstateofthetwoqubits?Theansweris,the(tensor)productofthetwo:0000+0101+1010+1111.Givenanarbitrarystateoftwoqubits,canwespecifythestateofeachindividualqubitinthisway?No,ingeneralthetwoqubitsareentangledandcannotbedecomposedintothestatesoftheindividualqubits.Forexample,considerthestate=p100+p111,which22isoneofthefamousBellstates.Itcannotbedecomposedintostatesofthetwoindividualqubits(seeExercise10.1).Entanglementisoneofthemostmysteriousaspectsofquantummechanicsandisultimatelythesourceofthepowerofquantumcomputation.22Prf1stbit=0g=Prf00g+Prf01g=j00j+j01j.Fine,buthowmuchdoesthispartialmeasurementdisturbthestateofthesystem?Theansweriselegant.Iftheoutcomeofmeasuringtherstqubitis0,thenthenewsuperpositionisobtainedbycrossingoutalltermsofthatareinconsistentwiththisoutcome(thatis,whoserstbitis1).Ofcoursethesumofthesquaresoftheamplitudesisnolonger1,sowemustrenormalize.Inourexample,thisnewstatewouldbe00000101new=q+q:2222j00j+j01jj00j+j01jFinally,letusconsiderthegeneralcaseofnhydrogenatoms.Thinkofnasafairlysmallnumberofatoms,sayn=500.Classicallythestatesofthe500electronscouldbeusedtostore500bitsofinformationintheobviousway.Butthequantumstateofthe500qubitsisalinearsuperpositionofall2500possibleclassicalstates:Xxx:x2f0;1gnItisasifNaturehas2500scrapsofpaperontheside,eachwithacomplexnumberwrittenonit,justtokeeptrackofthestateofthissystemof500hydrogenatoms!Moreover,ateachmoment,asthestateofthesystemevolvesintime,itisasthoughNaturecrossesoutthecomplexnumberoneachscrapofpaperandreplacesitwithitsnewvalue.Letusconsidertheeffortinvolvedindoingallthis.Thenumber2500ismuchlargerthanestimatesofthenumberofelementaryparticlesintheuniverse.Where,then,doesNaturestorethisinformation?Howcouldmicroscopicquantumsystemsofafewhundredatomscontainmoreinformationthanwecanpossiblystoreintheentireclassicaluniverse?SurelythisisamostextravaganttheoryabouttheamountofeffortputinbyNaturejusttokeepatinysystemevolvingintime.Inthisphenomenonliesthebasicmotivationforquantumcomputation.Afterall,ifNa-tureissoextravagantatthequantumlevel,whyshouldwebaseourcomputersonclassicalphysics?Whynottapintothismassiveamountofeffortbeingexpendedatthequantumlevel?Butthereisafundamentalproblem:thisexponentiallylargelinearsuperpositionistheprivateworldoftheelectrons.Measuringthesystemonlyrevealsnbitsofinformation.Asbefore,theprobabilitythattheoutcomeisaparticular500-bitstringxisjxj2.Andthenewstateaftermeasurementisjustx.298\nFigure10.3Aquantumalgorithmtakesn“classical”bitsasitsinput,manipulatesthemsoastocreateasuperpositionoftheir2npossiblestates,manipulatesthisexponentiallylargesuperpositiontoobtainthenalquantumresult,andthenmeasurestheresulttoget(withtheappropriateprobabilitydistribution)thenoutputbits.Forthemiddlephase,thereareelementaryoperationswhichcountasonestepandyetmanipulatealltheexponentiallymanyamplitudesofthesuperposition.InputxOutputyn-bitstringn-bitstringExponentialsuperposition10.2TheplanAquantumalgorithmisunlikeanyyouhaveseensofar.Itsstructurereectsthetensionbetweentheexponential“privateworkspace”ofann-qubitsystemandthemerenbitsthatcanbeobtainedthroughmeasurement.Theinputtoaquantumalgorithmconsistsofnclassicalbits,andtheoutputalsoconsistsofnclassicalbits.ItiswhilethequantumsystemisnotbeingwatchedthatthequantumeffectstakeoverandwehavethebenetofNatureworkingexponentiallyhardonourbehalf.Iftheinputisann-bitstringx,thenthequantumcomputertakesasinputnqubitsinstatex.Thenaseriesofquantumoperationsareperformed,bytheendofwhichthestatePofthenqubitshasbeentransformedtosomesuperpositionyyy.Finally,ameasurementismade,andtheoutputisthen-bitstringywithprobabilityjyj2.Observethatthisoutputisrandom.Butthisisnotaproblem,aswehaveseenbeforewithrandomizedalgorithmssuchastheoneforprimalitytesting.Aslongasycorrespondstotherightanswerwithhighenoughprobability,wecanrepeatthewholeprocessafewtimestomakethechanceoffailureminiscule.Nowletuslookmorecloselyatthequantumpartofthealgorithm.Someofthekeyquantumoperations(whichwewillsoondiscuss)canbethoughtofaslookingforcertainkindsofpatternsinasuperpositionofstates.Becauseofthis,itishelpfultothinkofthealgorithmashavingtwostages.Intherststage,thenclassicalbitsoftheinputare“unpacked”intoanexponentiallylargesuperposition,whichisexpresslysetupsoastohaveanunderlyingpatternorregularitythat,ifdetected,wouldsolvethetaskathand.Thesecondstagethenconsistsofasuitablesetofquantumoperations,followedbyameasurement,whichreveals299\nthehiddenpattern.Allthisprobablysoundsquitemysteriousatthemoment,butmoredetailsareontheway.InSection10.3wewillgiveahigh-leveldescriptionofthemostimportantoperationthatcanbeefcientlyperformedbyaquantumcomputer:aquantumversionofthefastFouriertransform(FFT).WewillthendescribecertainpatternsthatthisquantumFFTisideallysuitedtodetect,andwillshowhowtorecasttheproblemoffactoringanintegerNintermsofdetectingpreciselysuchapattern.Finallywewillseehowtosetuptheinitialstageofthequantumalgorithm,whichconvertstheinputNintoanexponentiallylargesuperpositionwiththerightkindofpattern.ThealgorithmtofactoralargeintegerNcanbeviewedasasequenceofreductions(andeverythingshownhereinitalicswillbedenedingoodtime):FACTORINGisreducedtondinganontrivialsquarerootof1moduloN.FindingsucharootisreducedtocomputingtheorderofarandomintegermoduloN.Theorderofanintegerispreciselytheperiodofaparticularperiodicsuperposition.Finally,periodsofsuperpositionscanbefoundbythequantumFFT.Webeginwiththelaststep.10.3ThequantumFouriertransformRecallthefastFouriertransform(FFT)fromChapter2.IttakesasinputanM-dimensional,complex-valuedvector(whereMisapowerof2,sayM=2m),andoutputsanM-dimensionalcomplex-valuedvector:2311112361!!2!M 1723067067661!2!4!2(M 1)7767617676176716..767627=p6.7627;64..75M61!j!2j!(M 1)j764..75.67.6.7M 14..5M 11!(M 1)!2(M 1)!(M 1)(M 1)pwhere!isacomplexMthrootofunity(theextrafactorofMisnewandhastheeffectofensuringthatifthejij2addupto1,thensodothejij2).Althoughtheprecedingequa-tionsuggestsanO(M2)algorithm,theclassicalFFTisabletoperformthiscalculationinjustO(MlogM)steps,anditisthisspeedupthathashadtheprofoundeffectofmakingdigitalsig-nalprocessingpracticallyfeasible.WewillnowseethatquantumcomputerscanimplementtheFFTexponentiallyfaster,inO(log2M)time!Butwait,howcananyalgorithmtaketimelessthanM,thelengthoftheinput?Thepointisthatwecanencodetheinputinasuperpositionofjustm=logMqubits:afterall,thissuperpositionconsistsof2mamplitudevalues.Inthenotationweintroducedearlier,wePwouldwritethesuperpositionas=M 1jwhereistheamplitudeofthem-bitj=0jibinarystringcorrespondingtothenumberiinthenaturalway.Thisbringsupanimportant300\nFigure10.4TheclassicalFFTcircuitfromChapter2.InputvectorsofMbitsareprocessedinasequenceofm=logMlevels. !00 "#4414$%222&'64634()141*+545524,-36632./74677point:thejnotationisreallyjustanotherwayofwritingavector,wheretheindexofeachentryofthevectoriswrittenoutexplicitlyinthespecialbracketsymbol.Startingfromthisinputsuperposition,thequantumFouriertransform(QFT)manip-ulatesitappropriatelyinm=logMstages.AteachstagethesuperpositionevolvessothatitencodestheintermediateresultsatthesamestageoftheclassicalFFT(whosecircuit,withm=logMstages,isreproducedfromChapter2inFigure10.4).AswewillseeinSection10.5,thiscanbeachievedwithmquantumoperationsperstage.Ultimately,aftermsuchstagesandm2=log2Melementaryoperations,weobtainthesuperpositionthatcorrespondstothedesiredoutputoftheQFT.SofarwehaveonlyconsideredthegoodnewsabouttheQFT:itsamazingspeed.Nowitistimetoreadtheneprint.TheclassicalFFTalgorithmactuallyoutputstheMcomplexPnumbers;:::;.Incontrast,theQFTonlypreparesasuperposition=M 1j.0M 1j=0And,aswesawearlier,theseamplitudesarepartofthe“privateworld”ofthisquantumsystem.Thustheonlywaytogetourhandsonthisresultisbymeasuringit!Andmeasuringthestateofthesystemonlyyieldsm=logMclassicalbits:specically,theoutputisindexjwithprobabilityjjj2.So,insteadofQFT,itwouldbemoreaccuratetocallthisalgorithmquantumFouriersampling.Moreover,eventhoughwehaveconnedourattentiontothecaseM=2minthissection,thealgorithmcanbeimplementedforarbitraryvaluesofM,andcanbesummarizedasfollows:301\nPInput:Asuperpositionofm=logMqubits,=M 1j.j=0jMethod:UsingO(m2)=O(log2M)quantumoperationsperformthequantumFFTPtoobtainthesuperposition=M 1j.j=0jOutput:Arandomm-bitnumberj(thatis,0jM 1),fromtheprobabilitydistributionPr[j]=jjj2.QuantumFouriersamplingisbasicallyaquickwayofgettingaveryroughideaabouttheoutputoftheclassicalFFT,justdetectingoneofthelargercomponentsoftheanswervector.Infact,wedon'tevenseethevalueofthatcomponent—weonlyseeitsindex.Howcanweusesuchmeagerinformation?InwhichapplicationsoftheFFTisjusttheindexofthelargecomponentsenough?Thisiswhatweexplorenext.10.4PeriodicitySupposethattheinputtotheQFT,=(0;1;:::;M 1),issuchthati=jwheneverijmodk,wherekisaparticularintegerthatdividesM.Thatis,thearrayconsistsofM=krepetitionsofsomesequence(0;1;:::;k 1)oflengthk.Moreover,supposethatexactlyoneoftheknumbers0;:::;k 1isnonzero,sayj.Thenwesaythatisperiodicwithperiodkandoffsetj.Figure10.5Examplesofperiodicsuperpositions.period30369M 6M 3period4159M 7M 3Itturnsoutthatiftheinputvectorisperiodic,wecanusequantumFouriersamplingtocomputeitsperiod!Thisisbasedonthefollowingfact,provedinthenextbox:SupposetheinputtoquantumFouriersamplingisperiodicwithperiodk,forsomekthatdividesM.ThentheoutputwillbeamultipleofM=k,anditisequallylikelytobeanyofthekmultiplesofM=k.Nowalittlethoughttellsusthatbyrepeatingthesamplingafewtimes(repeatedlypreparingtheperiodicsuperpositionanddoingFouriersampling),andthentakingthegreatestcommondivisorofalltheindicesreturned,wewillwithveryhighprobabilitygetthenumberM=k—andfromittheperiodkoftheinput!302\nTheFouriertransformofaperiodicvectorSupposethevector=(0;1;:::;M 1)isperiodicwithperiodkandwithnooffset(thatis,thenonzerotermsare0;k;2k;:::).Thus,MX=k 1q=kjk:Mj=0WewillshowthatitsFouriertransform=(0;1;:::;M 1)isalsoperiodic,withperiodM=kandnooffset.PClaim=p1k 1jM.kj=0kpProof.Intheinputvector,thecoefcient`isk=Mifkdivides`,andiszerootherwise.Wecanplugthisintotheformulaforthejthcoefcientof:MX 1pMX=k 11j`kjikj=p!`=!:MM`=0i=0Thesummationisageometricseries,1+!jk+!2jk+!3jk+,containingM=ktermsandwithratio!jk(recallthat!isacomplexMthrootofunity).Therearetwocases.Iftheratioisexactly1,whichhappensifjk0modM,thenthesumoftheseriesissimplythenumberofterms.Iftheratioisn't1,wecanapplytheusualformulaforgeometricseriestondthatthesumis1 !jk(M=k)=1 !Mj=0.p1 !jk1 !jkThereforejis1=kifMdividesjk,andiszerootherwise.Moregenerally,wecanconsidertheoriginalsuperpositiontobeperiodicwithperiodk,butwithsomeoffsetl
查看更多
