ManagementofDataCenterNetworks
Editedby NadjibAitsaadi
UniversitésParis-Saclay,UVSQ,DAVID,F-78035,Versailles,France
Copyright©2021byTheInstituteofElectricalandElectronicsEngineers,Inc.Allrights reserved.
PublishedbyJohnWiley&Sons,Inc.,Hoboken,NewJersey.
PublishedsimultaneouslyinCanada.
Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmitted inanyformorbyanymeans,electronic,mechanical,photocopying,recording,scanning, orotherwise,exceptaspermittedunderSection107or108ofthe1976UnitedStates CopyrightAct,withouteitherthepriorwrittenpermissionofthePublisher,or authorizationthroughpaymentoftheappropriateper-copyfeetotheCopyright ClearanceCenter,Inc.,222RosewoodDrive,Danvers,MA01923,(978)750-8400,fax (978)750-4470,oronthewebatwww.copyright.com.RequeststothePublisherfor permissionshouldbeaddressedtothePermissionsDepartment,JohnWiley&Sons,Inc., 111RiverStreet,Hoboken,NJ07030,(201)748-6011,fax(201)748-6008,oronlineat http://www.wiley.com/go/permission.
LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveused theirbesteffortsinpreparingthisbook,theymakenorepresentationsorwarrantieswith respecttotheaccuracyorcompletenessofthecontentsofthisbookandspecifically disclaimanyimpliedwarrantiesofmerchantabilityorfitnessforaparticularpurpose.No warrantymaybecreatedorextendedbysalesrepresentativesorwrittensalesmaterials. Theadviceandstrategiescontainedhereinmaynotbesuitableforyoursituation.You shouldconsultwithaprofessionalwhereappropriate.Neitherthepublishernorauthor shallbeliableforanylossofprofitoranyothercommercialdamages,includingbutnot limitedtospecial,incidental,consequential,orotherdamages.
Forgeneralinformationonourotherproductsandservicesorfortechnicalsupport, pleasecontactourCustomerCareDepartmentwithintheUnitedStatesat(800) 762-2974,outsidetheUnitedStatesat(317)572-3993orfax(317)572-4002.
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthat appearsinprintmaynotbeavailableinelectronicformats.Formoreinformationabout Wileyproducts,visitourwebsiteatwww.wiley.com.
LibraryofCongressCataloging-in-PublicationDataappliedfor :
ISBN:9781119647423
CoverDesign:Wiley
CoverImage:©BillDonnelley/WTDesign
Setin9.5/12.5ptSTIXTwoTextbySPiGlobal,Chennai,India
10987654321
TomydaughtersL.H.S.
Contents
AbouttheEditor xi
Contributors xiii
Acronyms xv
Introduction xvii
1ArchitecturesofDataCenterNetworks:Overview 1 BoutheinaDab,IlhemFajjari,DallalBelabed,andNadjibAitsaadi
1.1TaxonomyofDCNArchitectures 1
1.1.1ClassificationofDCNArchitectures 2
1.1.2Switch-CentricDCNArchitecturesOverview 3
1.1.2.1Tree-BasedDCN 3
1.1.2.2HierarchicalDCNArchitecture 4
1.1.2.3FlatDCNArchitecture 6
1.1.3Server-CentricDCNArchitecturesOverview 7
1.1.4EnhancedDCNArchitecturesOverview 10
1.1.4.1OpticalDCNArchitecture 10
1.1.4.2WirelessDCNArchitecture 12
1.2ComparisonBetweenDCNArchitectures 15
1.3ProposedHDCNArchitecture 15
1.3.1HDCNArchitectureBasedonMSDCModel 19
1.3.1.1ECMPProtocol 19
1.3.260GHzTechnologyinHDCN 20
1.3.3BeamformingTechniqueinHDCN 21
1.4Conclusion 23 References 23
2DataCenterOptimizationTechniques 29 DallalBelabed
2.1EthernetSwitchingandRouting 29
2.2DataCenterOptimizationTechniques 38
2.2.1VirtualNetworkEmbedding 38
2.2.2ServerConsolidation 40
2.2.3TrafficEngineering 43
2.2.3.1Link-StateTrafficEngineering 44
2.2.3.2MPLSTrafficEngineering 44
2.2.3.3TCPProportionalFairnessModel 46
2.3Conclusion 49 Bibliography 51
3ResourceManagementinHybrid(Wired/Wireless) DataCenterNetworks 57 BoutheinaDab,IlhemFajjari,andNadjibAitsaadi
3.1RoutingandWirelessChannelAllocationProblematicin HDCN 58
3.1.1RoutingandWirelessChannelAssignmentChallengesin HDCN 59
3.1.2RoutingandWirelessChannelAssignmentCriteriain HDCN 61
3.2WirelessChannelAllocationStrategiesforOne-Hop CommunicationsinHDCN 62
3.2.1ChannelAllocationProbleminWirelessNetworks 63
3.2.2Omni-DirectionalAntennasBasedStrategies 63
3.2.3Beamforming-BasedStrategies 67
3.3OnlineJointRoutingandWirelessChannelAllocationStrategies inHDCN 69
3.3.1JointRoutingandChannelAssignmentinMesh Networks 70
3.3.2OnlineJointRoutingandChannelAssignmentStrategiesin HDCN 71
3.4JointBatchRoutingandChannelAllocationStrategiesin HDCN 75
3.5JointBatchRoutingandChannelAllocationStrategiesin HDCN 75
3.6Summary 77
3.7Conclusion 80 References 80
4Inter-DataCenterNetworks:RoutingandReliabilityin VirtualNetworkBackbone 85 OussamaSoualah,IlhemFajjari,andNadjibAitsaadi
4.1OverviewofBasicVirtualNetworkEmbeddingWithout ReliabilityConstraint 85
4.1.1OnlineApproaches 86
4.1.2BatchApproaches 87
4.2OverviewofVirtualNetworkEmbeddingwithReliability Constraint 89
4.2.1DistributedApproaches 89
4.2.2CentralizedApproaches 91
4.2.2.1SubstrateRouterFailures 91
4.2.2.2SubstrateLinkFailures 92
4.2.2.3SubstrateRouterandLinkFailures 94
4.2.2.4RegionalFailures 95
4.2.3Summary 101
4.3Conclusion 101 References 101
5AnEvaluationMethodofOptimalCostSavingina DataCenterwithProactiveManagement 105 RubenMilocco,PascaleMinet,ÉricRenault,andSelmaBoumerdassi
5.1Introduction 106
5.2RelatedWork 108
5.3FrameworkforDCModeling 111
5.3.1NotationsandAssumptions 111
5.3.2EnergyComputation 111
5.3.2.1Single-ResourceCase 111
5.3.2.2ExtensiontotheMulti-resourceCase 114
5.4CostFormulation 114
5.4.1Example 115
5.4.2Methodology 116
5.4.3RelativeEnergyCostSaving 116
5.4.4UpperBoundComputation 118
5.5ApplicationtoaRealDC 118
5.5.1Generalities 119
5.5.1.1SelectionoftheSamplingInterval 119
5.5.1.2SelectionofPossibleValuesfortheCosts 119
5.5.1.3DynamicCapacityProvisioningBasedonEnergy Prediction 119
x Contents
5.5.2ApplicationtoaGoogleDataset 120
5.5.2.1EnergyComputation 120
5.5.2.2EvaluationoftheUpperBound 122
5.5.2.3ComputationoftheRelativeEnergyCostSaving 123
5.5.2.4DiscussionofResults 124
5.6Conclusion 124 References 125
Index 129
AbouttheEditor
NadjibAitsaadi,PhD,isaFullProfessorinNetworksandTelecommunicationsatUVSQParis-SaclayUniversityinFrance.Heisamemberof DAVIDLaboratoryandleadstheNextGenerationNetworksTeam.Prof. AitsaadiearnedaPhDin2010fromSorbonneUniversityinNetworks& Telecommunicationsandgraduatedin2016witha“Habilitation”diploma fromUniversityParisEst(UPE).Hismainresearchfieldsarethesecurity andQoSoptimizationofcellularnetworks(5G,6G,HAP),IoT,DCN,V2X, MEC,NFV/SDN,andmore.Theresultshavebeenpublishedinmany majorjournalssuchasIEEEJSAC,IEEETVT,ElsevierComNet,Elsevier ComCom,etc.andmajorconferencessuchIEEESECON,IEEELCN,IEEE MASS,ACMMSWiM,IEEE/IFIPNOMS/IM,IEEEICC,IEEEGLOBECOM,etc.Prof.AitsaadichairsmanytracksinIEEE/IFIPconferencessuch asIEEEGLOBECOM,IEEE/IFIPIM,IEEE/IFIPCIoT,etc.andheisvery activeinIEEETechnicalCommitteesTCIIN.
Contributors
NadjibAitsaadi
UniversitésParis-Saclay,UVSQ
DAVID
Versailles
France
DallalBelabed
AirbusDefenseandSpace,Airbus Saint-QuentinenYvelines Elancourt
France
SelmaBoumerdassi CEDRIC,CNAM
Paris
France
BoutheinaDab VMware,Hauts-de-Seine LaDefense
France and LiSSiLab,UPEC,ValdeMarne
VitrysurSeine
France
IlhemFajjari OrangeLabs,Orange Hauts-de-Seine
Chatillon
France
RubenMilocco GCAyS,UNComahue Neuquen
Argentina
PascaleMinet Inria
Paris
France
EricRenault LIGM,Univ.GustaveEiffel,CNRS ESIEEParis Marne-la-Vallée
France
OussamaSoualah OS-Consulting Athis-Mons,Essonne
France and LiSSiLab,UPEC,VitrysurSeine ValdeMarne
France
Acronyms
DCNDataCenterNetwork
ECMPEqualCostMulti-Path
GMPLSGeneralizedMulti-ProtocolLabelSwitching
HDCNHybridDataCenterNetwork
IGPInteriorGatewayProtocol
LISPLocator/IdentifierSeparationProtocol
MSDCMassivelyDataCenter
MPTCPMultipathTCP
NFVNetworkFunctionVirtualization
OSPFOpenShortestPathFirst
QoSQualityofService
SCTPStreamControlTransmissionProtocol
SDNSoftwareDefinedNetwork
SNSubstrateNetwork
STPSpanningTreeProtocol
STTStatelessTransportTunneling
TCITagControlInformation
TETrafficEngineering
ToRTopofRack
TPIDTagProtocolIDentifier
TRILLTransparentInterconnectionofLotsofLinks
VDCVirtualDataCenter
VLCVisibleLightCommunication
VNEVirtualNetworkEmbedding
VXLANVirtualExtensibleLAN
WTUWirelessTransmissionUnit
Introduction
Thankstotheadventofthelong-awaitedfifth-generation(5G)mobile networks,mobiledataandonlineservicesarebecomingwidelyaccessible. Discussionsofthisnewstandardhavetakenplaceinbothindustryand academiatodesignthisemergingarchitecture.Themainnearfuture objectiveistoensurethecapabilitytorespondtothedifferentapplications needssuchasvideos,games,websearching,etc.,whileensuringahigher datarateandanenhancedQualityofService(QoS).Whilenoofficial standardizationisyetdeliveredfor5G,expertsassurethat,theimpressive proliferationofsmartdeviceswillleadtotheexplosionoftrafficdemand. Billionsofconnectedusersareexpectedtodeployamyriadofapplications.
Inthisrespect,recentstatisticselaboratedbyCISCOVisualNetworking Index(VNI)highlightthattheannualglobalIPtrafficwillroughlytriple overthenext5yearsandwillreach2.3zettabytesby2020.Morespecifically,itisexpectedthatsmartphonestrafficwillimpressivelyincreasefrom 8% in2015to30% ofthetotalofIPtrafficin2020.Mobiledatatrafficper monthwillgrowfrom7Exabytesin2016to49Exabytesby2021.Inparticular,tremendousvideotrafficwillbecrossingIPnetworkstoreach82% ofthe totalityofIPtraffic.Itisalsoexpectedthatthenumberofconnectedmobile deviceswillbemorethanthreetimesthesizeoftheglobalpopulationby 2020.Inthisregard,futurenetworksareanticipatedtosupportandconnect plentyofdevices,whileofferinghigherdatarateandlowerlatency.
Tocopewiththisunprecedentedtrafficexplosion,theserviceproviders areurgedtorethinktheirnetworkarchitectures.Infact,efficientscalable physicalinfrastructures,e.g.,DataCenters(DCs),arerequiredtosupport thedrasticallyincreasingnumberofbothonlineservicesandusers.
TomanagetheirDCsinfrastructure,manyofgiantservicetenantsare resortingtovirtualizationtechnologiesmakinguseofSoftwareDefinedNetworking(SDN)andNetworkFunctionsVirtualization(NFV).
Ontheonehand,SDNcontrollersoffertheopportunitytoimplement morepowerfulalgorithmsthankstoareal-timecentralizedcontrolleveraginganaccurateviewofthenetwork.Indeed,thankstotheseparation oftheforwardingandthecontrolplanes,themanagements’complexity ofthenetworkinfrastructureisconsiderablyreducedwhileproviding tremendouscomputationalpowercomparedtolegacydevices.Ontheother hand,thankstoNFVparadigm,networkfunctionsandcommunication servicesarefirstsoftwarizedandthencloudified,sothattheycanbeon demandorchestratedandmanagedascloud-nativeITapplications.Itis straightforwardtoseethattheseapproachesarecomplimentary.They offeranewwaytodesignandmanagedatacenterswhileguaranteeinga highlevelofflexibilityandscalability.ThenewemergingSDNandNFV technologiesrequiresscalableinfrastructures.Tothatend,agreatdeal ofeffortshavebeendevotedtothedesignofefficientDCarchitectures. Indeed,Internetgiantsrampeduptheirinvestmentindatacenters/IT infrastructuresandpouredinbillionsofdollarstowidentheirglobal presenceandimprovetheircompetitivenessintheCloudmarket.
Inthiscontext,thelatestCapitalExpenditure(CAPEX)ofthefive largest-scaleInternetoperators,Apple,Google,Microsoft,Amazon,and Facebook,increasedby9.7% in2016inordertoinvestindesigningtheir DCs.Overthepastyears,thesecompanieshavespent,intotal,acapital of115$billions,tobuildouttheirDCs.Forinstance,Googlehasinvested millionsofdollarsinexpandingitsdatacentersspreadallovertheworld: Taiwan,LatinAmerica,Singapore,etc.Facebookhasstarted,since2010, buildingoutitsownDCsinAltoona,Iowa,andNorthCarolina.Inthis regard,efficientlydesigningdatacentersisacrucialtasktoensurescalabilityrequiredtomeettoday’smassiveworkloadofCloudapplications. Moreover,itismandatorytodeploythepropermechanismsforroutingand resourceallocationtocommunicationflowsinDCs.
Todealwiththesechallenges,weinvestigate,inthisbook,aradically newmethodologychangingthedesignoftraditionalDataCenterNetwork (DCN)whileensuringscalabilityandenhancingperformance.First, inChapter1,wewilloverviewtheDataCenternetworkarchitecture. Then,inChapter2,wewillsummarizethemainrelatedDCnetworks routingstrategiesatlayer-2,layer-3,anduplayers.Besides,anoverview ofTrafficEngineering(TE)techniquesfromlink-statewhileconsidering TCPfairnessmodels.Next,inChapter3,wewilloverviewtherelated workaddressingintra-datacenterresourceallocationandroutinginboth wiredand/orwireless(i.e.,hybrid)datacenternetworks.Afterwards,in Chapter4,wewillsummarizethemainreliablevirtualnetworkembedding strategiesconnectinggeographicallydistributeddatacenters.Thanksto
networkfunctionvirtualization,CAPEXandOPEXaredeeplyreduced whileensuringtherequestedqualityofserviceismorecomplex.Finally, inChapter5,wewillprovideamethodologytoevaluatetheenergycost reductioninDCbroughtbyproactivemanagement,whilekeepingahigh levelofusersatisfaction.
ArchitecturesofDataCenterNetworks:Overview
BoutheinaDab 1,2 ,IlhemFajjari 3 ,DallalBelabed 4 ,and NadjibAitsaadi 5
1 VMware,Hauts-de-Seine,LaDefense,France
2 LiSSiLab,UPEC,ValdeMarne,VitrysurSeine,France
3 OrangeLabs,Orange,Hauts-de-Seine,Chatillon,France
4 AirbusDefenseandSpace,Airbus,Saint-QuentinenYvelines,Elancourt,France
5 UniversitésParis-Saclay,UVSQ,DAVID,F-78035,Versailles,France
Abstract
Todealwiththewidespreaduseofcloudservicesandtheunprecedented trafficgrowth,thescaleoftheDataCenterhasimportantlyincreased. Therefore,itiscrucialtodesignnovelefficientnetworkarchitecturesableto satisfytherequirementsonbandwidth.Asakeyphysicalinfrastructure,Data CenterNetwork(DCN)designinghaswidelybeenahotresearchfocus. ThischapterreviewsthemainDCNarchitecturespropoundedintheliterature.Todoso,ataxonomyofDCNdesignswillbeproposed,whileanalyzing indeptheachstructureofthegivenclassification.Then,wewillprovidea qualitativecomparisonbetweenthesedifferentDCNgroups.Finally,wewill presenthybridDCNarchitecturebasedonwiredandwirelessarchitecture.
1.1TaxonomyofDCNArchitectures
Inthissection,wepresentataxonomyoftheexistentDataCenterNetwork (DCN)architectureswithadetailedreviewofeachdrawnclass.Ingeneral, severalcriteriahavetobeconsideredtodesignrobustDCNs,namely, highnetworkperformance,efficientresourceutilization,fullavailable bandwidth,highscalability,easycabling,etc.Todealwiththeaforementionedchallenges,apanoplyofsolutionshavebeendesigned.Mainly,
ManagementofDataCenterNetworks, FirstEdition.EditedbyNadjibAitsaadi. ©2021TheInstituteofElectricalandElectronicsEngineers,Inc. Published2021byJohnWiley&Sons,Inc.
wecandistinguishtworesearchdirections.Inthefirstone,wiredDCN architectureshavebeenupgradedtobuildadvancedcost-effectivetopologiesabletoscaleupdatacenters.Thesecondapproachhasresortedto deployingnewnetworktechniqueswithintheexistingDCNsoastohandle thechallengesencounteredinthepriorarchitectures.Hereafter,wewill giveadetailedtaxonomyofthesetechniques.
1.1.1ClassificationofDCNArchitectures
Withregardtotheaforementionedresearchdirections,wecanidentify threemaingroupsofDCNarchitectures,namely, switch-centric DCN, server-centric DCN,and enhanced DCN.Eachgroupincludesavariety ofcategoriesthatwewilldetailhereafter.
● Switch-centricDCNarchitecture:switchesare,mostly,responsiblefor network-relatedfunctions,whereastheservershandleprocessingtasks. Thefocusofsuchadesignistoimprovethetopologysoastoincrease networkscale,reduceoversubscription,andspeedupflowtransmission. Switch-centricarchitecturescanbeclassifiedintothreemaincategories accordingtotheirstructuralproperties:
1. Traditionaltree-basedDCNarchitecture:representsaspecifickind ofswitch-centricarchitecture,whereswitchesarelinkedina multirootedform.
2. HierarchicDCNarchitecture:isaswitch-centricDCN,wherenetwork componentsarearrangedinmultiplelayers.Eachlayercharacterizes trafficdifferently.
3. FlatDCNarchitecture:compressesthethreeswitchlayersintoonlyone ortwoswitchlayers,inordertosimplifythemanagementandmaintenanceoftheDCN.
● Server-centricDCNarchitecture:serversareenhancedtohandlenetworkingfunctions,whereasswitchesareusedonlytoforwardpackets. Basically,serversaresimultaneouslyend-hostsandrelayingnodesfor multihopcommunications.Usually,server-centricDCNarerecursively definedmultileveltopologies.
● EnhancedDCNarchitecture:isaspecificDCNwhichistailoredfor futureCloudcomputingservices.Indeed,thefutureresearchdirection attemptstodeploynetworkingtechniquessoastodealwithwiredDCN designslimitations.Recently,avarietyoftechnologieshavebeenused inthiscontext,namely,opticalswitchingandwirelesscommunications.Accordingly,wedistinguishtwomainclassesofenhancedDCN architectures:
1. OpticalDCN :makesuseofopticaldevicestospeedupcommunications.Itcanbeeither:(i)all-opticalDCN(i.e.withcompletelyoptical devices)or(ii)hybridopticalDCN(i.e.bothopticalandEthernet switches).
2. WirelessDCN :deployswirelessinfrastructureinordertoenhance networkperformance,andmaybe:(i)fullywirelessDCN(i.e.only wirelessdevices)or(ii)HybridDCN(i.e.bothwirelessandwired devices).
Figure1.1illustratesthetaxonomyofcurrentDCNarchitectures.Inthe following,wewilldetaileachcategoryanddiscusstheirimpactonCloud computingperformance.
1.1.2Switch-CentricDCNArchitecturesOverview
1.1.2.1Tree-BasedDCN
ThetraditionalDCNistypicallybasedonamultiroottreearchitecture.The latterisathree-tiertopologycomposedbythreelayersofswitches.Thetop level(i.e.root)representsthecorelayer,themiddlelevelistheaggregation layer,whilethebottomlevelisknownastheaccesslayer.Thecoredevices arecharacterizedbyhighcapacitiescomparedwithaggregationandaccess switches.Typically,thecoreswitches’uplinksconnectthedatacenterto theInternet.Ontheotherhand,theaccesslayerswitchescommonlyuse 1Gbpsdownlinkinterfacesand10Gbpsuplinkinterfaces,whileaggregationswitchesprovide10Gbpslinks.Accessswitches(i.e.topofrack,ToRs) interconnectserversinthesamerack.Aggregationlayerallowstheconnectionbetweenaccessswitchesandthedataforwarding.Itisworthnoting thattheabovevaluesofnetworkinterfacecardsthroughputarecontinuouslyincreasing.Forinstance,nowadaysitiseasyandnotreallyexpensive
Figure1.1 TaxonomyofDCNarchitectures.
Figure1.2 Traditionaltree-basedDCNarchitecture.
todeployinterfaceswith25and100Gbps.Anillustrationoftree-basedDCN architectureisdepictedinFigure1.2.
Unfortunately,traditionalDCNsstruggletoresisttotheincreasingtraffic demand.First,coreswitchesarepronetobottlenecksissuesassoonasthe workloadsreachthepeak.Moreover,insuchaDCN,severaldownlinksof aToRswitchsharethesameuplinkwhichlimitstheavailablebandwidth. Second,DCNscalabilitystronglydependsonthenumberofswitchports. Therefore,theuniquewaytoscalethistopologyistoincreasethenumber ofnetworkdevices.However,thesesolutionsresultsinhighconstruction costsandenergyconsumption.Third,tree-basedDCNsuffersfromserious resiliencyproblems.Forinstance,ifafailurehappensonsomeoftheaggregationswitches,thenserversarelikelytoloseconnectionwithothers.In addition,resourceutilizationisnotefficientlybalanced.Foralltheaforementionedreasons,researchersputforwardalternativeDCNtopologies.
1.1.2.2HierarchicalDCNArchitecture
HierarchicaltopologyarrangestheDCNcomponentsinmultiplelayers.The keyinsightbehindthismodelistoreducethecongestionbyminimizing theoversubscriptioninlower-layerswitchesusingtheupper-layerdevices. Intheliterature,wefindseveralhierarchicDCNexamples,namely,CLOS, FatTree,andVL2.Hereafter,wewilldescribeeachoneofthem.
