Download pdf Computational complexity of counting and sampling discrete mathematics and its applicat

Computational Complexity

of Counting and Sampling Discrete Mathematics and Its Applications 1st Edition Istvan Miklos

Visit to download the full and correct content document: https://ebookmeta.com/product/computational-complexity-of-counting-and-sampling-d iscrete-mathematics-and-its-applications-1st-edition-istvan-miklos/

More products digital (pdf, epub, mobi) instant download maybe you interests ...

Combinatorics of Permutations Discrete Mathematics and Its Applications 3rd Edition Miklos Bona

https://ebookmeta.com/product/combinatorics-of-permutationsdiscrete-mathematics-and-its-applications-3rd-edition-miklosbona/

Discrete Mathematics and Its Applications 7e Kenneth Rosen

https://ebookmeta.com/product/discrete-mathematics-and-itsapplications-7e-kenneth-rosen/

The Lambert W Function Its Generalizations and Applications Discrete Mathematics and Its Applications 1st Edition István Mez■

https://ebookmeta.com/product/the-lambert-w-function-itsgeneralizations-and-applications-discrete-mathematics-and-itsapplications-1st-edition-istvan-mezo/

Instructor's Resource Guide To Accompany Discrete Mathematics and Its Applications, 8e Kenneth H. Rosen

https://ebookmeta.com/product/instructors-resource-guide-toaccompany-discrete-mathematics-and-its-applications-8e-kenneth-hrosen/

Discrete Mathematics: Essentials and Applications 1st Edition Ali Grami

https://ebookmeta.com/product/discrete-mathematics-essentialsand-applications-1st-edition-ali-grami/

Computational Science and its Applications 1st Edition Abul Hassan Siddiqi

https://ebookmeta.com/product/computational-science-and-itsapplications-1st-edition-abul-hassan-siddiqi/

Geometry and Discrete Mathematics A Selection of Highlights Benjamin Fine

https://ebookmeta.com/product/geometry-and-discrete-mathematicsa-selection-of-highlights-benjamin-fine/

Geometry and Discrete Mathematics 2nd Edition Benjamin Fine

https://ebookmeta.com/product/geometry-and-discretemathematics-2nd-edition-benjamin-fine/

Progress in Pure and Applied Discrete Mathematics Vol 1

Probabilistic Methods in Discrete Mathematics 2nd Edition V.F. Kolchin

https://ebookmeta.com/product/progress-in-pure-and-applieddiscrete-mathematics-vol-1-probabilistic-methods-in-discretemathematics-2nd-edition-v-f-kolchin/

Computational Complexity of Counting and Sampling

Computational Complexity of Counting and Sampling

István Miklós

Rényi Institute, Budapest, Hungary

CRC Press

Taylor & Francis Group

6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742

© 2019 by Taylor & Francis Group, LLC

CRC Press is an imprint of Taylor & Francis Group, an Informa business

No claim to original U.S. Government works

Printed on acid-free paper

Version Date: 20190201

International Standard Book Number-13: 978-1-138-03557-7 (Paperback)

International Standard Book Number-13: 978-1-138-07083-7 (Hardback)

This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, please access www. copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.

Library of Congress Cataloging-in-Publication Data

Names: Miklos, Istvan (Mathematician), author.

Title: Computational complexity of counting and sampling / Istvan Miklos.

Description: Boca Raton : Taylor & Francis, 2018. | Includes bibliographical references.

Identifiers: LCCN 2018033716 | ISBN 9781138035577 (pbk.)

Subjects: LCSH: Computational complexity. | Sampling (Statistics)

Classification: LCC QA267.7 .M55 2018 | DDC 511.3/52--dc23

LC record available at https://lccn.loc.gov/2018033716

Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com

and the CRC Press Web site at http://www.crcpress.com

Tothememoryofmybelovedwife, ´ AgnesNy´ul (1972–2018)

1.5Randomdecisionalgorithms:RP,BPP.Papadimitriou’stheo-

2.1.3.1Countingthecoinsequencessumminguptoa

2.1.3.2Calculatingthepartitionpolynomial.....

2.1.3.3Findingtherecursionforoptimizationwithalgebraicdynamicprogramming........

2.1.3.4Countingthetotalsumofweights......

2.1.3.5Countingthecoinsequenceswhentheorder doesnotcount.................

2.2Counting,optimizing,deciding.................

2.3Thezooofcountingandoptimizationproblemssolvablewith algebraicdynamicprogramming................

2.3.1Regulargrammars,HiddenMarkovModels......

2.3.2Sequencealignmentproblems,pairHiddenMarkov Models..........................

2.3.3Context-freegrammars..................

2.3.4Walksondirectedgraphs................

2.4Limitationsofthealgebraicdynamicprogrammingapproach

3Linearalgebraicalgorithms.Thepowerofsubtracting

3.1Division-freealgorithmsforcalculatingthedeterminantand Pfaffian..............................

3.2Kirchhoff’smatrix-treetheorem................

3.3TheBEST(deBruijn-Ehrenfest-Smith-Tutte)algorithm..

3.4TheFKT(Fisher-Kasteleyn-Temperley)algorithm......

3.5Thepowerofsubtraction....................

4#P-completecountingproblems

4.1Approximation-preserving#P-completeproofs........

4.1.1 #3SAT

4.1.2Calculatingthepermanentofanarbitrarymatrix... 170

4.1.3Countingthemostparsimonioussubstitutionhistories onanevolutionarytree.................

4.1.4 #IS and #Mon-2SAT .................

4.2#P-completeproofsnotpreservingtherelativeerror.....

4.2.1 #DNF, #3DNF ..................... 186

4.2.2Countingthesequencesofagivenlengththataregular grammarcangenerate.................. 187

4.2.3Computingthepermanentofanon-negativematrixand countingperfectmatchingsinbipartitegraphs.... 188

4.2.4Countingthe(notnecessarilyperfect)matchingsofa bipartitegraph...................... 190

4.2.5Countingthelinearextensionsofaposet....... 191

4.2.6Countingthemostparsimonioussubstitutionhistories onastartree.......................

4.2.7Countingthe(notnecessarilyperfect)matchingsina planargraph.......................

4.2.8Countingthesubtreesofagraph............

4.2.9NumberofEulerianorientationsinaEuleriangraph.

4.3Furtherreadingandopenproblems..............

4.3.1Furtherresults......................

4.3.2#BIS-completeproblems................

5Holographicalgorithms

5.2.1#X-matchings......................

5.2.2#Pl-3-(1,1)-Cyclechain..................

5.2.3#Pl-3-NAE-ICE.....................

6.1Generatingrandomnumbers..................

7.1Relaxationtimeandsecond-largesteigenvalue........

7.2TechniquestoproverapidmixingofMarkovchains.....

7.2.1Cheeger’sinequalitiesandtheisoperimetricinequality

7.2.2MixingofMarkovchainsonfactorizedstatespaces..

7.2.3Canonicalpathsandmulticommodityflow.......

7.2.4CouplingofMarkovchains...............

7.2.5MixingofMarkovchainsondirectproductspaces..

7.3Self-reduciblecountingproblems................

7.3.1TheJerrum-Valiant-Vaziranitheorem.........

7.3.2Dichotomytheoryontheapproximabilityofselfreduciblecountingproblems...............

7.4Furtherreadingandopenquestions..............

7.4.1Furtherreading......................

7.4.2Openproblems......................

7.5Exercises.............................

7.6Solutions.............................

8Approximablecountingandsamplingproblems

8.1Samplingwiththerejectionmethod..............

8.1.1#Knapsack........................

8.1.2Edge-disjointtreerealizationswithoutcommoninternal vertices..........................

8.2SamplingwithMarkovchains.................

8.2.1Linearextensionsofposets...............

8.2.2Countingthe(notnecessarilyperfect)matchingsofa graph...........................

8.2.3Samplingrealizationsofbipartitedegreesequences..

8.2.4BalancedrealizationsofaJDM.............

8.2.5CountingthemostparsimoniousDCJscenarios....

8.2.6Samplingandcountingthe k-coloringsofagraph...

8.3Furtherresultsandopenproblems...............

8.3.1Furtherresults......................

8.3.2Openproblems......................

8.4Exercises.............................

Preface

Theideatowriteabookonthecomputationalcomplexityofcountingand samplingcametoourmindin2016February,whenMikl´osB´onaco-organized aDagstuhlseminarwithMichaelAlbert,EinarSteingr´ımsson,andme.We realizedthatmanyoftheenumerativecombinatoristsknowlittleaboutcomputerscience,andclearly,thereisademandforabookthatintroduces thecomputationalaspectsofenumerativecombinatorics.Similarly,thereare physicists,bioinformaticians,engineers,statisticians,andotherappliedmathematicians,whodevelopanduseMarkovchainMonteCarlomethods,butare notawareofthetheoreticalcomputerscientificbackgroundofsampling.

Theaimofthisbookistogiveabroadoverviewofthecomputationalcomplexityofcountingandsampling,fromverysimplethingslikelinearrecurrences,tohighleveltopicslikeholographicreductionsandmixingofMarkov chains.Sincethebookstartswiththebasics,eagerMSc,PhDstudents,and youngpostdoctoralresearchersdevotedtocomputerscience,combinatorics, and/orstatisticsmightstartstudyingthisbook.Thebookisalsouniquein thewaythatitfocusesequallyoncomputationallyeasyandhardproblems, andhighlightsthoseeasyproblemsthathavehardvariants.Forexample,it iseasytocountthegenerationsofaregulargrammarthatproducesequences oflength n.Ontheotherhand,itishardtocountthenumberofsequences oflength n thataregulargrammarcangenerate.

Thereisaspecialemphasisonbioinformatics-relatedproblemsinthehope ofbringingtheoryandapplicationscloser.Abunchofopenproblemsare drawntotheattentionoftheorists,whomightfindtheminterestingand challengingenoughtoworkonthem.Wealsobelievethattherewillbeapplied mathematicianswhowanttodeepentheirunderstandingofthetheoryof sampling,andwillbehappytoseethatthetheoryisexplainedviaexamples theyalreadyknow.

Manyofthetopicsareintroducedviaworked-outexamples,andalong listofexercisescanbefoundattheendofeachchapter.Exercisesmarked with * haveadetailedsolution,whilehintscanbefoundonexercisesmarked with ◦.Unsolvedexercisesvaryfromverysimpletochallenging.Therefore, instructorswillfindappropriateexercisesforstudentsatalllevels.

Althoughthebookstartswiththebasics,itstillneedsprerequisites.Backgroundinbasiccombinatorics,graphtheory,linearandabstractalgebra,and probabilitytheoryisexpected.Adiscussiononcomputationalcomplexityis

verybrieflypresentedatthebeginningofthebook.However,TuringMachines and/orothermodelsofcomputationsarenotexplainedinthisbook.

Wewantedtogiveathoroughoverviewofthefield.Still,severaltopicsare omittedornotdiscussedindetailinthisbook.Asthebookfocusesonclassifyingeasyandhardcomputationalproblems,verylittleispresentedonimproved runningtimesandasymptoticoptimalityofalgorithms.Forexample,divide andconqueralgorithms,likethecelebrated“fourRussiansspeed-up”,cannotbefoundinthisbook.Similarly,thelogarithmicSobolevinequalitiesare notdiscussedindetailinthechapteronthemixingofMarkovchains.Many beautifultopics,likestochasticcomputingofthevolumeofconvexbodies, monotonecircuitcomplexity,#BIS-completecountingproblems,Fibonacci gates,pathcoupling,andcouplingfromthepastarementionedonlyvery brieflyduetolimitedspace.

Writingthisbookwasgreatfun.Thisworkcouldnothavebeenaccomplishedwithoutthehelpofmycolleagues.IwouldliketothankMikl´osB´ona forsuggestingtowritethisbook.Also,thewholeteamatCRCPressis thankedfortheirsupport.SpecialthanksshouldgotoJin-YiCai,CatherineGreenhill,andZolt´anKir´alyfordrawingmyattentiontoseveralpapers Ihadnotbeenawareof.IwouldliketothankK´alm´anCziszter,M´aty´as Domokos,P´eterErd˝os,JotunHein,P´eterP´alP´alfy,LajosR´onyai,andMikl´os Simonovitsforfruitfuldiscussions.Andr´asR´aczwasvolunteeredtoreadthe firsttwochaptersofthebookandtocomment,forwhichIwouldliketo warmlythankhim.Lastbutnotleast,Iwillalwaysremembermybeloved wife, ´ AgnesNy´ul,whosupportedthewritingofthisbooktilltheendofher lastdays,andwho,unfortunately,passedawaybeforethepublicationofthis book.

ListofFigures

1.1Thegadgetgraphreplacingadirectededgeintheproofof Theorem16.Seetextfordetails................ 21

2.1Astair-stepshapeofheight5tiledwith5rectangles.The horizontallinesinsidetheshapearehighlightedasdotted. Thefourcirclesindicatethefourcornersoftherectangleat thetopleftcornerthatcutsthestair-stepshapeintotwo smallerones.Seetextfordetails................ 40

2.2 a) Nested, b) separated, c) crossingbasepairs.Eachbase pairisrepresentedbyanarcconnectingtheindexpositions ofthebasepair.......................... 80

4.1ThedirectedacyclicgraphrepresentationoftheCNF (x1 ∨ x2 ∨ x3 ∨ x4) ∧ (x2 ∨ x3 ∨ x4).Logicalvaluesarepropagatedontheedges,anedgecrossedwithatilde(∼)means negation.Eachinternalnodehastwoincomingedgesandone outgoingedge.Theoperationperformedbyanodemight bealogicalOR(∨)oralogicalAND(∧).Theoutcome oftheoperationistheincomeattheotherendofthe outgoingedge........................... 169

4.2Thetrack T5 forthevariable x5 when x5 isaliteralin C2 and C5 and x5 isaliteralin C3 172

4.3Theinterchange R3 fortheclause C3 =(x1 ∨ x5 ∨ x8).Note thatinterchangesdonotdistinguishliterals xi and xi.Each edgeinandabovethelineofthejunctionsgoesfromleft toright,andeachedgebelowthelineofthejunctionsgoes fromrighttoleft.Junctionswithoutlabelsaretheinternal junctions............................. 173

4.4Constructingasubtree Tcj foraclause cj .Thesubtreeis builtinthreephases.First,elementarysubtreesareconnected withacombtogetaunitsubtree.Inthesecondphasethe sameunitsubtreeisrepeatedseveraltimes,“blowingup”the tree.Inthethirdphase,theblown-uptreeisamendedwitha constantsize,depth3fullybalancedtree.Thesmallersubtrees constructedinthepreviousphasearedenotedwithatriangle inthenextphase.Seealsotextfordetails........... 177

4.5 a) Acherrymotif,i.e.,twoleavesconnectedwithaninternal node. b) Acomb,i.e.,afullyunbalancedtree. c) Atreewith 3cherrymotifsconnectedwithacomb.Theassignmentsfor 4adjacencies, α1, α2, α3 and αx areshownatthebottom foreachleaf. αi, i =1, 2, 3aretheadjacenciesrelatedtothe logicalvariables bi,and αx isanextraadjacency.Notethat Fitch’salgorithmgivesambiguityforalladjacencies αi atthe rootofthissubtree........................ 180

4.6Theunweightedsubgraphreplacingtheedge(v,w)witha weightof3.Seetextfordetails................. 190

4.7TheHassediagramofaclauseposet.Seetextfordetails... 192

4.8Theposet PΦ,p.Ovalsrepresentanantichainofsize p 1.For sakeofclarity,onlytheliteralandsomeoftheclausevertices fortheclause cj =(xi1 ∨ xi2 ∨ xi3 )arepresentedhere.See alsothetextfordetails..................... 193

4.9Thegadgetcomponent∆1 forreplacingacrossinginanonplanargraph.Seetextfordetails................ 201

4.10Thegadgetcomponent∆2 forreplacingacrossinginanonplanargraph.Seetextfordetails................ 201

4.11ThegadgetΓreplacingacrossinginanon-planargraph.See textfordetails.......................... 202

4.12Thegadget∆replacingavertexinaplanar,3-regulargraph. Seetextfordetails........................ 204

5.1Anedge-weightedbipartiteplanargraphasanillustrativeexampleofthe#X-matchingsproblem.Seetextfordetails.. 224

5.2Thematchgridsolvingthe#X-matchingproblemforthe graphinFigure5.1.Theedgeslabeledby ei belongtothe setofedges C,andareconsideredtohaveweights1.Seethe textfordetails.......................... 226

5.3Anexampleprobleminstancefortheproblem#Pl-3-NAEICE................................ 233

5.4Thematchgridsolvingthe#Pl-3-NAE-ICEproblemforthe probleminstanceinFigure5.3.Theedgesbelongingtothe edgeset C aredotted.Therecognizermatchgatesareput intodashedcircles........................ 233

7.1Thestructureof Y = Y l Y u.Anon-filledellipse(witha simplelineboundary)representsthespace Yx foragiven x Thesolidblackellipsesrepresenttheset S withsomeofthem (the Sl)belongingtothelowerpart Y l,andtherest(the Su) belongingtotheupperpart(Y u)................ 285

7.2When Sl isnotanegligiblepartof S,thereisaconsiderable flowgoingoutfrom Sl towithin Y l,implyingthattheconditionalflowgoingoutfrom S cannotbesmall.Seetextfor detailsandrigorouscalculations................ 287

7.3When Sl isanegligiblepartof S,thereisaconsiderableflow goingoutfrom Su into Y l \Sl.Seetextfordetailsandrigorous calculations............................

8.1Constructionoftheauxiliarybipartitegraph Gi anda RSO {(v1,w), (v2,r)} →{(v1,r), (v2,w)} taking(x1,y1)into (x2,y2).............................. 339

8.2Anexampleoftwogenomeswith7markers.......... 341

ListofTables

4.1Thenumberofscenariosondifferentelementarysubtreesof theunitsubtreeofthesubtree Tcj forclause cj = x1 ∨ x2 ∨ x3. Columnsrepresentthe14differentelementarysubtrees,the topologyoftheelementarysubtreeisindicatedonthetop. Theblackdotsmeanextrasubstitutionsontheindicatededge duetothecharactersintheauxiliarypositions;thenumbers representthepresence/absenceofadjacenciesontheleftleaf ofaparticularcherrymotif,seetextfordetails.Therowstartingwith#indicatesthenumberofrepeatsoftheelementary subtrees.Furtherrowsrepresentthelogicaltrue/falsevalues oftheliterals,forexample,001means x1 =FALSE, x2 = FALSE, x3 =TRUE.Thevaluesinthetableindicatethe numberofscenarios,raisedtotheappropriatepowerdueto multiplicityoftheelementarysubtrees.Itiseasytocheckthat theproductofthenumbersinthefirstlineis2136 × 376 and inanyotherlinesis2156 × 364

4.2Constructingthe50sequencesforaclause.Seetextforexplanation...............................

Chapter1

Backgroundoncomputational complexity

Incomputationalcomplexitytheory,wedistinguishdecision,optimization, countingandsamplingproblems.Althoughthisbookisaboutthecomputationalcomplexityofcountingandsampling,countingandsamplingproblems arerelatedtodecisionandoptimizationproblems.Countingproblemsarealwaysatleastashardastheirdecisioncounterparts.Indeed,ifwecantell, say,thenumberofperfectmatchingsinagraph G,thennaturallywecantell ifthereexistsaperfectmatchingin G: G containsaperfectmatchingifand onlyifthenumberofperfectmatchingsin G isatleast1.

Optimizationproblemsarealsorelatedtocountingandsampling.Aswe aregoingtoshowinthischapter,itishardtocountthecyclesinadirected graphaswellassamplingthemsinceitishardtofindthelongestcycleina graph.Thismightbesurprisinginthelightthatfindingacycleinagraphis aneasyproblem.Therearenumerousothercaseswhenthecountingversion ofaneasydecisionproblemishardsincefindinganoptimalsolutionishard inspiteofthefactthatfindingone(arbitrary)solutioniseasy.Although webrieflyreviewthemaincomplexityclassesofdecisionandoptimization problemsin Sections1.2 and 1.5,weassumereadershavepriorknowledgeon them.Possiblereferencesoncomputationalcomplexityare[8,140,160].

Whenwearetalkingabouteasyandhardproblems,weusetheconvention ofcomputationalcomplexitythataproblemisdefinedasaneasycomputationalproblemifthereisapolynomialrunningtimealgorithmtosolveit.Very rarelywecanunconditionallyprovethatapolynomialrunningtimealgorithm

Computationalcomplexityofcountingandsampling doesnotexistforacomputationalproblem.However,wecanprovethatno polynomialrunningtimealgorithmexistsforcertaincountingproblemsifno polynomialrunningtimealgorithmexistsforcertainharddecisionproblems. Thisfactalsounderlineswhydiscussingdecisionproblemsisinevitableina bookaboutcomputationalcomplexityofcountingandsampling.

Whenexactcountingishard,approximatecountingmightbeeasyorhard. Surprisingly,hardcountingproblemsmightbeeasytoapproximatestochastically,however,therearecountingproblemsthatwecannotapproximatewell. Weconjecturethattheyarehardtoapproximate,andthisisapointwhere stochasticapproximationsarealsorelatedtorandomapproachestodecision problems.Particularly,ifnorandomalgorithmexistsforcertainharddecision problemsthatruninpolynomialtimeandisanybetterthanrandomguessing, thenthereisnoefficientgoodapproximationforcertaincountingproblems.

Inthischapter,wegiveabriefintroductiontocomputationalcomplexity andshowhowcomputationalcomplexityofcountingandsamplingisrelated tocomputationalcomplexityofdecisionandoptimizationproblems.

1.1Generaloverviewofcomputationalproblems

A computationalproblem isamathematicalobjectrepresentingacollection ofquestionsthatcomputersmightbeabletosolve.Thequestionsbelonging toacomputationalproblemarealsocalled probleminstances.Anexampleof a decisionproblem isthetriangleproblemwhichasksifthereistriangleina finitegraph.Theprobleminstancesarethefinitegraphsandtheanswerfor anyprobleminstanceis“yes”or“no”dependingonwhetherornotthereisa triangleinthegraph.Inthiscomputationalproblem,atriangleinagraphis calleda witness or solution.Ingeneral,thewitnessesofaprobleminstanceare themathematicalobjectsthatcertifythattheanswerforthedecisionproblem is“yes”.Anexampleforan optimizationproblem isthecliqueproblemwhich askswhatthelargestclique(completesubgraph)isinafinitegraph.The probleminstancesareagainthefinitegraphsandthesolutionsarethelargest cliquesinthegraphs.

Anydecisionoroptimizationproblemhasitsnaturalcountingcounterpart problemaskingthenumberofwitnessesorsolutions.Forexample,wecanask howmanytrianglesagraphhas,aswellashowmanylargestcliquesagraph has.

Computationalproblemsmightbesolvedwithalgorithms.Wecanclassify algorithmsbasedontheirproperties.Algorithmsmightbeexactorapproximate,mightbedeterministicorrandom,andprobablytheirmostimportant featureisiftheyarefeasibleorinfeasible.Todefinefeasibility,wehaveto definehowtomeasureit.Largerprobleminstancesmightneedmorecomputationalsteps,alsocalledrunningtime.Therefore,itisnaturaltomeasurethe

complexityofanalgorithmwiththenecessarycomputationalstepsasafunctionoftheinput(probleminstance)size.Thesizeoftheprobleminstance isdefinedasthenumberofbitsnecessarytodescribeit.Acomputational problemisdefinedas tractable ifitsrunningtimegrowswithapolynomial functionofthesizeoftheinput,and intractable ifitsrunningtimegrows exponentiallyorevenmorewiththesizeoftheinput.Thisdefinitionignores constantfactors,theorderofthepolynomialandtypicalinputsizes.This meansthattheoreticallytractableproblemsmightbeinfeasibleinpractice, and viceversa,theoreticallyintractableproblemsmightbefeasibleinpracticeifthetypicalinputsizesaresmall.Interestedreaderscanfindaseriesof exercisesexploringthisphenomenaattheendofthechapter(Exercises1–3). Inpractice,mostofthetractablealgorithmsruninatmostcubictime,and theirconstantfactorislessthan10.Thesealgorithmsarenotonlytheoreticallytractablebutalsofeasibleinpractice.Thegivendefinitionsoftractable andintractableproblemsdonotcoverallalgorithmsastherearefunctions thatgrowfasterthananypolynomialfunctionbutslowerthananyexponential function.Suchfunctionsarecalled superpolynomial and subexponential.Althoughthereareremarkablecomputationalproblems,mostnotablythegraph isomorphismproblem[9,10],whichisconjecturedtohavesuperpolynomial andsubexponentialrunningtimealgorithmsinthebestcase,suchproblems arerelativelyrare,andnotdiscussedindetailinthisbook.

Observethatboththesizeoftheprobleminstanceandthenumberof computationalstepsarenotpreciselydefined.Indeed,agraph,forexample, mightbeencodedbyitsadjacencymatrixorbythelistofedgesinit.These encodingsmighthavedifferentnumbersofbits.Similarly,onmanycomputers, differentoperationsmighthavedifferentrunningtimes:thetimenecessaryto multiplytwonumbersmightbemuchmorethanthetimeneededtoadd twonumbers.Togetrigorousmathematicaldefinitions,theoreticalcomputer scienceintroducedmathematicalmodelsofcomputations;thebestknown aretheTuringmachines.Inthisbook,weavoidtheseformaldescriptions ofcomputations.Thereasonforthisisthatweareinterestedinonlythe order oftherunningtime,andconstantfactorsarehiddeninthe O (bigO, ordo)notation.Evenifsizesaredefinedindifferentways,differentdefinitions almostneverhaveexponential(ormoreprecisely,superpolynomial)gaps.For example,ifagraphhas n vertices,thenitmighthave O(n2)edges.However,it doesnotmakeatheoreticaldifferenceifanalgorithmongraphsrunsin O(n3) timeor O(m1 5)time,where n isthenumberofverticesand m isthenumber ofedges:bothfunctionsarepolynomials.Theonlydifferencewhenthereisan exponentialgapbetweentwoconceptsofinputsizesiswhenwedistinguishthe valueofthenumberandthenumberofbitsnecessarytodescribeanumber. Whenwewouldliketoemphasizethattheinputsizeisthevalueofthe number,wewillsaythattheinputnumbersaregiven inunary.Atypical exampleisthe subsetsum problem,whereweaskifwecanselectasubsetof integerswhosesumisaprescribedvalue W .Thereisadynamicprogramming algorithmtosolvethisproblemwhoserunningtimeispolynomialwiththe

Computationalcomplexityofcountingandsampling

valueof W .However,itisaharddecisionproblemif W isnotgiveninunary [108].

1.2Deterministicdecisionproblems:P,NP,NPcomplete

Definition1. Incomputationalcomplexitytheory, P istheclassthatcontains thedecisionproblemssolvableinpolynomialtime.

ExamplesfordecisionproblemsinParethefollowing:

• Theperfectmatchingproblemasksifagraphhasaperfectmatching. Aperfectmatchingisasetofindependentedgesthatcoversallvertices [60].

• Thesubstringproblemasksifasequence A isasubstringofsequence B.Asubstringisaseriesofconsecutivecharactersofasequence,for example, A = aba isasubstringof B = bbabaaab sincethethird,fourth andfifthcharactersof B isindeedsequence A

• Theprimalitytestingproblemasksifapositiveintegernumberisa primenumber.Surprisingly,thisproblemcanbesolvedinpolynomial timeeveniftheinputsizeisthenumberofdigitsnecessarytowrite downthenumber[3].

Oneofthemostimportantandunsolvedquestionsintheoreticalcomputer scienceiswhetherornotPisequaltoNP.Formally,thecomplexityclass NPcontainstheproblemsthatcanbesolvedinpolynomialtimewithnondeterministicTuringmachines.ThenameNPstandsfor“non-deterministic polynomial”.SincewedonotintroduceTuringmachinesinthisbook,an alternative,equivalentdefinitionisgivenhere.

Definition2. Thecomplexityclass NP containsthedecisionproblemsfor whichsolutionscanbeverifiedinpolynomialtime.

ThisdefinitionismoreintuitivethantheformaldefinitionusingTuring machines.ExamplesforproblemsinNParethefollowing.

• Thek-cliqueproblemasksifthereisacliqueofsize k inagraph,that isasubgraphisomorphictothecompletegraph Kk

• Thetwopartitioningproblemasksifthereisapartitioningofafiniteset ofintegernumbersintotwosubsetssuchthatthesumofthenumbers inthetwosubsetsisthesame.

• Thefeasibilityofanintegerprogrammingquestionasksifthereisa listofintegernumbers x1,x2,...xn satisfyingasetoflinearinequalities havingtheform

ItiseasytoseethattheseproblemsareindeedinNP.Ifsomebodyselects vertices v1,v2,...,vk,itiseasytoverifythatforall i,j ∈{1, 2,...,k},there isanedgebetween vi and vj .Ifsomebodyprovidesapartitioningofasetof numbers,itiseasytocalculatethesumsofthesubsetsandcheckifthetwo sumsarethesame.Also,itiseasytoverifythatassignmentstothevariables x1,x2,...,xn satisfyanyinequalityunder(1.1).

Inmanycases,findingasolutionseemstobeharderthanverifyinga solution.ThereareproblemsinNPforwhichnopolynomialrunningtime algorithmisknown.Wecannotprovethatsuchanalgorithmdoesnotexist, however,wecanprovethatthesehardcomputationalproblemsareashard asanyproblemsinNP.Topreciselystatethis,wefirstneedthefollowing definitions.

Definition3. Let A and B betwocomputationalproblems.Wesaythat A hasa polynomialreduction to B,ifapolynomialrunningtimealgorithm existsthatsolvesanyprobleminstance x ∈ A bygeneratingprobleminstances y1,y2,...,yk allin B andsolves x usingthesolutionsfor y1,y2,...yk.The computationaltimegeneratingprobleminstances y1,y2,...yk countsinthe runningtimeofthealgorithm,butthecomputationaltimespentinsolving theseprobleminstancesisnotconsideredintheoverallrunningtime.Wealso saythat A is polynomiallyreducible to B.

Example1. Anindependentsetinagraphisasubsetoftheverticessuch thatnotwoverticesinitareadjacent.The k-independentsetproblemasksif thereisanindependentsetofsize k inagraph.

The k-independentsetproblemispolynomiallyreducibletothe k-clique problem.Indeed,agraphcontainsanindependentsetofsize k ifandonlyif itscomplementcontainsacliqueofsize k.Takingthecomplementofagraph canbedoneinpolynomialtime.

Similarly,the k-cliqueproblemisalsopolynomiallyreducibletothe kindependentsetproblem.

Polynomialreductionisanimportantconceptincomputationalcomplexity.Ifacomputationalproblem A ispolynomiallyreducibleto B and B can besolvedinpolynomialtime,then A alsocanbesolvedinpolynomialtime. Similarly,if B ispolynomiallyreducibleto A,and A canbesolvedinpolynomialtime,then B canbesolvedinpolynomialtime,aswell.Therefore,if A and B aremutuallypolynomiallyreducibletoeachother,theneitherbothof themornoneofthemcanbesolvedinpolynomialtime.Thesethoughtslead tothefollowingdefinitions.

Computationalcomplexityofcountingandsampling

Definition4. Acomputationalproblemisinthecomplexityclass NP-hard if everyprobleminNPispolynomiallyreducibletoit.The NP-complete problemsaretheintersectionofNPandNP-hard.

WhatfollowsfromthedefinitionisthatPisequaltoNPifandonly ifthereisapolynomialrunningtimealgorithmthatsolvesanNP-complete problem.ItiswidelybelievedthatPisnotequaltoNP,andthus,thereare nopolynomialrunningtimealgorithmsforNP-completeproblems.

ItisabsolutelynottrivialthatNP-completeproblemsexist.Belowwe defineadecisionproblemandstatethatitisNP-complete.

Definition5. InBooleanlogic,a literal isalogicalvariableoritsnegation. A disjunctiveclause isalogicalexpressionofliteralsandORoperators(∨).

A conjunctivenormalform or CNF isaconjunctionofdisjunctiveclauses, thatis,disjunctiveclausesconnectedwiththelogical AND (∧)operator.A conjunctivenormalform Φ is satisfiable ifthereisanassignmentoflogical variablesin Φ suchthatthevalueof Φ isTRUE.Suchanassignmentis calleda satisfyingassignment.Thedecisionproblemifthereisasatisfying assignmentofaconjunctivenormalformiscalledthe satisfiabilityproblem anddenotedby SAT

Theorem1. ForanydecisionproblemAinNPandanyprobleminstance xinA,thereexistsaconjunctivenormalform Φ suchthat Φ issatisfiableif andonlyiftheanswerfortheprobleminstancexis“yes”.Furthermore,for anyx,suchaconjunctivenormalformcanbeconstructedinpolynomialtime ofthesizeofx.Sinceverifyingthatanassignmentofthelogicalvariablesisa satisfyingassignmentcanbeclearlydoneinpolynomialtime,andthus,SAT isinNP,thisalsomeansthatSATisanNP-completeproblem.

Wedonotprovethistheoremhere;theproofcanbefoundinanystandard textbookoncomputationalcomplexity,seeforexample[72].Thesatisfiability istheonlyproblemforwhichwecandirectlyproveNP-completeness.For allotherdecisionproblems,NP-completenessisprovedbypolynomialreductionoftheSATproblemorotherNP-completeproblemstothosedecision problems.Indeed,thefollowingtheoremholds.

Theorem2. Let A beanNP-completeproblemandlet B beadecisionprobleminNP.If A ispolynomiallyreducibleto B,then B isalsoNP-complete.

Proof. Theproofisbasedonthefactthatthesumaswellasthecomposition oftwopolynomialsarealsopolynomials.

StephenCookprovedin1971thatSATisNP-complete[44],andRichard Karpdemonstratedin1972thatmanynaturalcomputationalproblemsare NP-completebyreducingSATtothem[108].ThesefamousKarp’s21NPcompleteproblemsdroveattentiontoNP-completenessandinitiatedthestudy ofthePversusNPproblem.ThequestionwhetherornotPequalsNPhas

becomethemostfamousunsolvedproblemincomputationalcomplexitytheory.In2000,theClayInstituteoffered$1millionforaproofordisproofthat PequalsNP[1].

BelowwegivealistofNP-completeproblemsthatwearegoingtousein proofsoftheoremsaboutcomputationalcomplexityofcountingandsampling.

Definition6. Let G =(V,E) beadirectedgraph.A Hamiltonianpath is adirectedpaththatvisitseachvertexexactlyonce.A Hamiltoniancycle isa directedcyclethatcontainseachvertexexactlyonce.

Basedonthisdefinition,wecandefinethefollowingtwoproblems.

Problem1.

Name: H-Path.

Input: adirectedgraph, G =(V,E).

Output: “yes”if G hasaHamiltonianpath,“no”otherwise.

Problem2.

Name: H-Cycle.

Input: adirectedgraph, G =(V,E).

Output: “yes”if G hasaHamiltoniancycle,“no”otherwise.

Theorem3. [108]Both H-Path and H-Cycle areinNP-complete.

Itisalsohardtodecideifagraphcontainsalargeindependentset.

Problem3.

Name: LargeIS.

Input: apositiveinteger m andagraph G inwhicheveryindependentsethas sizeatmost m.

Output: “yes”if G hasanindependentsetofsize m,and“no”otherwise.

Theorem4. [73]Thedecisionproblem LargeIS isinNP-complete.

Thesubsetsum(seebelow)isaninfamousNP-completeproblem.Itis polynomiallysolvableiftheweightsaregiveninunary,however,itbecomes hardforlargeweights.

Problem4.

Name: SubsetSum.

Input: asetofnumbers, S = {x1,x1,...,xn} andanumber m.

Output: “yes”ifthereisasubset A ⊆ S suchthat x∈A x = m,otherwise “no”.

Theorem5. [108]Thedecisionproblem SubsetSum isinNP-complete.

1.3Deterministiccounting:FP,#P,#P-complete

Definition7. Thecomplexityclass #P containsthecountingproblemsthat askforthenumberofwitnessesofthedecisionproblemsinNP.If A denotes aprobleminNP,then #A denotesitscountingcounterpart.

Sincethedecisionversionsof#PproblemsareinNP,thereisawitness thatcanbeverifiedinpolynomialtime.Thisdoesnotautomaticallyimply thatallwitnessescanbeverifiedinpolynomialtime,althoughitnaturally holdsinmanycases.Whenitisquestionablethatallsolutionscanbeverified inpolynomialtime,apolynomialupperboundmustbegiven,andonlythose witnessescountthatcanbeverifiedinthattime.

Forexample,#SATdenotesthecountingproblemthatasksforthenumber ofsatisfyingassignmentsofconjunctivenormalforms.Somecountingproblemsaretractable.Formally,theybelongtotheclassoftractablefunction problems.

Definition8. A functionproblem isacomputationalproblemwheretheoutputismorecomplexthanasimple“yes”or“no”answer.Thecomplexityclass FP (FunctionPolynomial-Time)istheclassoffunctionproblemsthatcanbe solvedinpolynomialtimewithanalgorithm.

Wecandefinethe#P-hardand#P-completeclassesanalogouslytothe NP-hardandNP-completeclasses.

Definition9. Acomputationalproblemisin #P-hard ifanyproblemin#P ispolynomiallyreducibletoit.The #P-complete classistheintersectionof #Pand#P-hard.

Asonecannaturallyguess,#SATisa#P-completeproblem.Indeed,the followingtheoremholds.

Theorem6. Foreveryproblem#Ain#P,andeveryprobleminstance x in#A,thereexistsaconjunctivenormalform Φ suchthatthenumberof satisfyingassignmentsof Φ istheanswerfor x.Furthermore,sucha Φ can beconstructedinpolynomialtimeofthesizeoftheprobleminstance x.Since #SATisin#P,thismeansthat#SATisa#P-completeproblem.

Itisclearthat#P ⊆ FPifandonlyifthereexistsapolynomialrunningtimealgorithmfora#P-completeproblem.Itisalsotrivialtoseethat #P ⊆ FPimpliesP=NP.However,wedonotknowifthereverseistrue, namely,whetherornotP=NPimpliesthatcountingthewitnessesofany #P-completeproblemiseasy.Still,wehavethefollowingnon-trivialresult.

Theorem7. [161]IfP=NP,thenforanyproblem#Ain#Pandany polynomial p,thereisapolynomialtimealgorithmfor#Asuchthatitapproximatesanyinstanceof#Awithinamultiplicativeapproximationfactor 1+ 1 p

ByassumingthatPisnotequaltoNP,wecannotexpectapolynomialrunningtimealgorithmcountingthewitnessesofanNP-completeproblem.Naturally,thecountingversionsofmanyNP-completeproblemsare#P-complete. However,wedonotknowifitistruethatforanyNP-completeproblem A, itscountingversion#A is#P-complete.Ontheotherhand,thereareeasy decisionproblemswhosecountingversionis#P-complete.Belowweshowtwo ofthem.

Definition10. The permanent ofan n × n matrix M = {mi,j } isdefinedas

where Sn isthesetofpermutationsofnumbers 1, 2,...,n

Theorem8. Computingthepermanentis#P-hard.Computingthepermanentofa0-1matrixisstill#P-hard.

Wearegoingtoprovethistheoremin Chapter4.Calculatingthepermanentisrelatedtocountingtheperfectmatchingsinagraph,asstatedand provedbelow.

Theorem9. Computingthenumberofperfectmatchingsinabipartitegraph isa#P-completecountingproblem.

Proof. Wereducethepermanentofa0-1matrixtocomputingthenumberof perfectmatchingsinabipartitegraph.

Let A = {ai,j } beanarbitrary n×n matrixcontaining0sand1s.Construct abipartitegraph G =(U,V,E)suchthatthereisanedgebetween ui and vj ifandonlyif ai,j =1.

Matrix A containsonly0sand1s,thereforeforanypermutation σ,

is1ifeach ai,σ(i) is1and0otherwise.Let S denotethesubsetofpermutations forwhichtheproductis1.Let M denotethesetofperfectmatchingsin G Clearly,thereisbijectionbetween S and M:if σ ∈ S ,thenassignthe perfectmatchingto σ thatcontainstheedges(ui,vσ(i)).Thisisindeeda perfectmatching,sinceeach(ui,vσ(i)) ∈ E duetothedefinitionof S and A, andeachvertexiscoveredbyexactlyoneedgesince σ isapermutation.Itis alsoclearthatthismappingisaninjection,if σ1 = σ2,thentheirimagesare alsodifferent.

Similarly,if M ∈M isaperfectmatching,thenforeach i,itcontainsan edge(ui,vj ).Thenassignto M thepermutationthatmaps i to j.Itisindeed apermutationduetothedefinitionofperfectmatching,andthesoobtained σ isindeedin S duetothedefinitionof S and A

Computationalcomplexityofcountingandsampling

Therefore,thenumberofperfectmatchingsin G isthepermanentof A Sinceconstructing G canbeclearlydoneinpolynomialtime,thisisapolynomialreduction,andthus,computingthenumberofperfectmatchingsina bipartitegraphis#P-hard.Sincethiscountingproblemisalsoin#P,itisin #P-complete.

LeslieValiantdefinedtheclasses#P-hardand#P-complete,andproved thatcomputingthepermanentofamatrixis#P-hard,itisstill#P-hard tocomputethepermanentofamatrixiftheentriesarerestrictedtothe {0, 1} set,andthus,countingtheperfectmatchingsinabipartitegraphis #P-complete[175].Thisisquitesurprising,sincedecidingifthereisaperfect matchinginabipartitegraphisinP[94].

Weintroduceanother#P-completecountingproblem,whichisevenmore surprisinginthesensethatitsdecisionversionisabsolutelytrivial.Theproblemisalsorelatedtofindingthevolumeofaconvexbody.Itisalsothefirst exampleforthefactthathardcountingproblemsmightbeeasytoapproximate,aswearegoingtodiscusslateroninthischapter.

Definition11. A partiallyorderedset orshort:a poset isapair (A, ≤) where A isasetand ≤ isareflexive,antisymmetricandtransitiverelation on A,thatis,forany a,b,c ∈ A

(a) a ≤ a,

(b) if a = b and a ≤ b,thentherelation b ≤ a doesnothold,

(c) a ≤ b ∧ b ≤ c =⇒ a ≤ c.

Themeaningofthenameisthattheremightbeelements a,b ∈ A such thatneither a ≤ b nor b ≤ a hold.Anexampleforpartialorderedsetsiswhen A =2X forsomeset X,andtherelationis ⊆.Furtherexamplesare: A isthe naturalnumbersand a ≤ b if a|b, A isthesetofsubgroupsofagroupand a ≤ b is a isasubgroupof b,etc.

Itiseasytoseethatanyposetcanbeextendedtoatotalordering,such thatforany a,b ∈ A, a ≤ b impliesthat a ≤t b,where ≤t isthedefining relationofthetotalordering.Suchatotalorderingiscalleda linearextension oftheposet.Wecanaskhowmanylinearextensionsaposethas.

Problem5.

Name: #LE.

Input: apartiallyorderedset(P, ≤).

Output: thenumberoflinearextensionsof(P, ≤).

Observethatthedecisionversionof#LEistrivial:whatevertheposetis, theanswerisalways“yes”tothequestioniftheposethasalinearextension. Therefore,itisverysurprisingthatthefollowingtheoremholds.

Theorem10. #LEisa#P-completeproblem.

Wearegoingtoprovethistheoremin Chapter4.#LEisrelatedtocomputingthevolumeofaconvexbody.Wecandefineapolytopeforeachfinite posetinthefollowingway.

Definition12. Let (A, ≤) beafiniteposetof n elements.Theposetpolytope isaconvexbodyintheEuclidianspace Rn inwhicheachpoint (x1,x2,...,xn) satisfiestheinequalities

1.forall i, 0 ≤ xi ≤ 1,

2.forall ai ≤ aj , xi ≤ xj .

Theorem11. Thevolumeoftheposetpolytopeofaposet P =(A, ≤) is 1 n! timesthenumberoflinearextensionsof P where n = |A|.

Proof. Anytotalorderingisalsoapartialordering,sowecandefineitspolytope.Theintersectionofthepolytopesoftwototalorderingshas0measure (thepossiblecommonfacetsofthepolytopes).Therefore,itissufficientto provethattheposetpolytopeofanytotalorderingofasetofsize n hasvolume 1 n! .Thereisanaturalbijectionbetweenthepermutationsoflength n andthetotalorderingsofasetofsize n: ai ≤ aj inatotalordering(A, ≤)if andonlyif σ(i) <σ(j)inpermutation σ.Theunionofthe n!posetpolytopes isthe n-dimensionalunitcube,whichhasvolume1.Eachposetpolytopehas thesamevolumesincetheycanbetransformedintoeachotherwithlinear transformationspreservingthevolume.Indeed,thematricesoftheselinear transformationsinthestandardbasisarepermutationmatrices.Therefore, thedeterminantoftheminabsolutevalueis1.Theintersectionsofthese polytopeshave0measure.Thenindeed,thevolumeoftheposetpolytopeof anytotalorderingofasetofsize n is 1 n! .Theposetpolytopeofanypartial orderingistheunionoftheposetpolytopesofitslinearextensions,therefore itsvolumeisindeed 1 n! timesthenumberoflinearextensionsoftheposet.

Polytopesarerelatedtoothercountingproblems,too,seeforexample, Exercise5.

1.4Computingthevolumeofaconvexbody,deterministicversusstochasticcase

ThecorollaryofTheorem10isthatitis#P-hardtofindthevolumeofa convexbodydefinedastheintersectionofhalfspacesgivenbylinearinequalities.Computingthevolumeofaconvexbodyisintrinsicallyhard.However, thecomputationbecomeseasyifstochasticcomputationsareallowed.Inthis section,wepresenttworesultshighlightingthattheremightbeexponential gapsbetweenrandomanddeterministiccomputations.