FederatedLearningforFutureIntelligent WirelessNetworks
Editedby YaoSun
UniversityofGlasgow,UK
ChaoqunYou
SingaporeUniversityofTechnologyandDesign,Singapore
GangFeng
UniversityofElectronicScienceandTechnologyofChina,China
LeiZhang
UniversityofGlasgow,UK
Copyright©2024byTheInstituteofElectricalandElectronicsEngineers,Inc. Allrightsreserved.
PublishedbyJohnWiley&Sons,Inc.,Hoboken,NewJersey. PublishedsimultaneouslyinCanada.
Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmittedinany formorbyanymeans,electronic,mechanical,photocopying,recording,scanning,orotherwise, exceptaspermittedunderSection107or108ofthe1976UnitedStatesCopyrightAct,without eitherthepriorwrittenpermissionofthePublisher,orauthorizationthroughpaymentofthe appropriateper-copyfeetotheCopyrightClearanceCenter,Inc.,222RosewoodDrive,Danvers, MA01923,(978)750-8400,fax(978)750-4470,oronthewebatwww.copyright.com.Requeststo thePublisherforpermissionshouldbeaddressedtothePermissionsDepartment,JohnWiley& Sons,Inc.,111RiverStreet,Hoboken,NJ07030,(201)748-6011,fax(201)748-6008,oronlineat http://www.wiley.com/go/permission.
Trademarks:WileyandtheWileylogoaretrademarksorregisteredtrademarksofJohnWiley& Sons,Inc.and/oritsaffiliatesintheUnitedStatesandothercountriesandmaynotbeused withoutwrittenpermission.Allothertrademarksarethepropertyoftheirrespectiveowners. JohnWiley&Sons,Inc.isnotassociatedwithanyproductorvendormentionedinthisbook.
LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbest effortsinpreparingthisbook,theymakenorepresentationsorwarrantieswithrespecttothe accuracyorcompletenessofthecontentsofthisbookandspecificallydisclaimanyimplied warrantiesofmerchantabilityorfitnessforaparticularpurpose.Nowarrantymaybecreatedor extendedbysalesrepresentativesorwrittensalesmaterials.Theadviceandstrategiescontained hereinmaynotbesuitableforyoursituation.Youshouldconsultwithaprofessionalwhere appropriate.Further,readersshouldbeawarethatwebsiteslistedinthisworkmayhave changedordisappearedbetweenwhenthisworkwaswrittenandwhenitisread.Neitherthe publishernorauthorsshallbeliableforanylossofprofitoranyothercommercialdamages, includingbutnotlimitedtospecial,incidental,consequential,orotherdamages.
Forgeneralinformationonourotherproductsandservicesorfortechnicalsupport,please contactourCustomerCareDepartmentwithintheUnitedStatesat(800)762-2974,outsidethe UnitedStatesat(317)572-3993orfax(317)572-4002.
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsin printmaynotbeavailableinelectronicformats.FormoreinformationaboutWileyproducts, visitourwebsiteatwww.wiley.com.
LibraryofCongressCataloging-in-PublicationDataAppliedfor: [HardbackISBN:9781119913894]
CoverDesign:Wiley
CoverImage:©BluePlanetStudio/Shutterstock
Setin9.5/12.5ptSTIXTwoTextbyStraive,Chennai,India
Contents
AbouttheEditors xv
Preface xvii
1FederatedLearningwithUnreliableTransmissioninMobile
EdgeComputingSystems 1
ChenyuanFeng,DaquanFeng,ZhongyuanZhao,HowardH.Yang,and TonyQ.S.Quek
1.1SystemModel 1
1.1.1LocalModelTraining 1
1.1.2UpdateResultFeedbackviatheWirelessChannels 2
1.1.3GlobalModelAveraging 3
1.2ProblemFormulation 4
1.2.1ModelAccuracyLoss 4
1.2.2CommunicationLoss �� C t,m 6
1.2.3SampleSelectionLoss �� S t,m 6
1.2.4ModelTrainingLoss �� M t,m 7
1.2.5ProblemFormulation 8
1.2.5.1ObjectiveFunction 8
1.2.5.2EnergyConsumptionConstraint 9
1.2.5.3UserSelectionConstraint 9
1.2.5.4DataVolumeConstraintofLocalTrainingDatasets 9
1.3AJointOptimizationAlgorithm 10
1.3.1CompressionOptimization 10
1.3.1.1OptimizationofAt,m 10
1.3.1.2OptimizationofDt,m 12
1.3.2JointOptimizationofAt,m andDt,m 13
1.3.3OptimizationofSampleSelection 13
1.3.4OptimizationofUserSelection 15
1.3.5AJointOptimizationAlgorithm 15
1.4SimulationandExperimentResults 16 Bibliography 21
2FederatedLearningwithnon-IIDdatainMobileEdge ComputingSystems 23
ChenyuanFeng,DaquanFeng,ZhongyuanZhao,GeyongMin,and HancongDuan
2.1SystemModel 23
2.1.1LocalModelTraining 23
2.1.2FederatedAveraging 24
2.2PerformanceAnalysisandAveragingDesign 24
2.2.1TheAnalysisofExpectedWeightDivergence 26
2.2.1.1TheAnalysisofExpectedDataDistributionDivergence ��{m } 27
2.2.1.2AnUpperBoundof �� (tK ) 28
2.2.2RethinkingtheSettingsofFederatedAveragingWeights 28
2.3DataSharingScheme 30
2.3.1DataSharing 30
2.3.2ProblemFormation 31
2.3.2.1ObjectiveFunction 32
2.3.3OptimizationConstraints 33
2.3.4AJointOptimizationAlgorithm 34
2.3.4.1CPUCycleFrequencyOptimizationSubproblem 34
2.3.4.2TransmitPowerAllocationSubproblem 35
2.3.4.3SharingDatasetOptimizationSubproblem 37
2.3.4.4UserSelectionOptimizationSubproblem 39
2.3.4.5AJointOptimizationAlgorithm 41
2.4SimulationResults 42 Bibliography 47
3HowManyResourcesAreNeededtoSupportWirelessEdge Networks 49
Yi-JingLiu,GangFeng,YaoSun,andShuangQin
3.1Introduction 49
3.2SystemModel 50
3.2.1FLModel 51
3.2.1.1LossFunction 51
3.2.1.2UpdatingModel 52
3.2.2ComputingResourceConsumptionModel 52
3.2.3CommunicationResourceConsumptionModel 53
3.2.3.1Uplink 53
3.2.3.2DownLink 53
3.3WirelessBandwidthandComputingResourcesConsumedfor SupportingFL-EnabledWirelessEdgeNetworks 54
3.3.1SINRAnalysis(UplinkDirection) 54
3.3.1.1ProbabilityDensityFunction(PDF)ofSINR 54
3.3.1.2TransmissionSuccessProbabilityofLocalModels 55
3.3.2SNRAnalysis(DownlinkDirection) 57
3.3.3WirelessBandwidthNeededforTransmittingLocal/Global Models 57
3.3.4ComputingResourcesNeededforTrainingLocalModels 58
3.4TheRelationshipbetweenFLPerformanceandConsumed Resources 59
3.4.1LocalModelAccuracy 59
3.4.2GlobalModelAccuracy 60
3.5DiscussionsofThreeCases 62
3.5.1Case1:SufficientCommunicationResourcesandComputing Resources 62
3.5.2Case2:SufficientComputingResourcesandInsufficient CommunicationResources 63
3.5.3Case3:SufficientCommunicationResourcesandInsufficient ComputingResources 64
3.6NumericalResultsandDiscussion 67
3.6.1SimulationSetting 67
3.6.2SimulationResults 68
3.6.2.1VerifyingAnalyticalResults 68
3.6.2.2MeasuringthePerformanceofFLSettings 71
3.6.2.3ExaminingtheTrade-OffbetweentheComputingand CommunicationResourcesunderFLFramework 73
3.7Conclusion 75
3.8ProofofCorollary3.2 76
3.9ProofofCorollary3.3 77 References 81
4DeviceAssociationBasedonFederatedDeepReinforcement LearningforRadioAccessNetworkSlicing 85 Yi-JingLiu,GangFeng,YaoSun,andShuangQin
4.1Introduction 85
4.2SystemModel 87
4.2.1NetworkModel 87
4.2.2RANSlicing 88
4.2.3ServiceRequirements 89
4.2.4HandoffCost 89
Contents
4.3ProblemFormulation 90
4.3.1ProblemStatement 90
4.3.2MarkovDecisionProcessModelingforDeviceAssociation 92
4.3.2.1State 92
4.3.2.2Action 93
4.3.2.3TransitionProbability 93
4.3.2.4Reward 93
4.4HybridFederatedDeepReinforcementLearningforDevice Association 94
4.4.1FrameworkofHDRL 94
4.4.1.1DRLonSmartDevices 94
4.4.1.2HorizontalModelAggregation(hDRL)Level 95
4.4.1.3VerticalModelAggregation(vDRL)Level 95
4.4.2AlgorithmofHorizontalModelAggregation 97
4.4.2.1DDQNforTrainingLocalModel 97
4.4.2.2UpdateModels 98
4.4.3AlgorithmofVerticalModelAggregation 98
4.4.4HDRLAlgorithmforDeviceAssociation 100
4.4.5ConvergenceAnalysis 102
4.5NumericalResults 103
4.5.1SimulationSettings 104
4.5.2NumericalResultsandDiscussions 105
4.6Conclusion 109 Acknowledgment 110 References 110
5DeepFederatedLearningBasedonKnowledgeDistillation andDifferentialPrivacy 113 HuiLin,FengYu,andXiaodingWang
5.1Introduction 113
5.2RelatedWork 115
5.3SystemModel 118
5.3.1SecurityModel 118
5.4TheImplementationDetailsoftheProposedStrategy 119
5.4.1SecurityAnalysis 119
5.5PerformanceEvaluation 120
5.5.1ExperimentalEnvironment 120
5.5.2ExperimentalResults 120
5.6Conclusions 122 Bibliography 123
6FederatedLearning-BasedBeamManagementinDense MillimeterWaveCommunicationSystems 127 QingXueandLiuYang
6.1Introduction 127
6.1.1PriorWork 128
6.1.2Contributions 129
6.2SystemModel 130
6.3ProblemFormulationandAnalysis 133
6.4FL-BasedBeamManagementinUDmmN 135
6.4.1MarkovDecisionProcessModel 135
6.4.2FL-BasedBeamManagement 136
6.4.2.1DataCleaning 136
6.4.2.2ModelTraining 138
6.4.3UserAssociationinUDmmN 141
6.4.3.1DownlinkMeasurements 141
6.4.3.2UserPerception 141
6.4.3.3MultipleAssociation 142
6.5PerformanceEvaluation 143
6.5.1ComparisonwithBFSandEDB 146
6.5.2ComparisonwithBMDLandBMCL 148
6.6Conclusions 150 Bibliography 150
7Blockchain-EmpoweredFederatedLearningApproachforAn IntelligentandReliableD2DCachingScheme 155 RunzeCheng,YaoSun,YijingLiu,LeXia,DaquanFeng,and MuhammadImran
7.1Introduction 155
7.2RelatedWork 157
7.2.1Learning-BasedD2DCachingSchemes 157
7.2.2Blockchain-EnabledD2DCachingSchemes 158
7.3SystemModel 159
7.3.1D2DNetwork 159
7.3.2ContentCachingSchemes 159
7.3.3TransmissionLatency 160
7.4ProblemFormulationandDRL-BasedModelTraining 160
7.4.1ProblemFormulation 161
7.4.1.1Action 161
7.4.1.2State 161
7.4.1.3RewardandReturn 161
7.4.2DRL-BasedLocalModelTraining 163
x Contents
7.5Privacy-PreservedandSecureBDRFLCachingSchemeDesign 165
7.5.1TaskandRequirementsPublication 166
7.5.2AppropriateUESelection 166
7.5.3LocalModelTraining 166
7.5.4AreaModelUpdateandRecording 169
7.5.5GlobalModelUpdateandRecording 169
7.6ConsensusMechanismandFederatedLearningModelUpdate 170
7.6.1Double-LayerBlockchainConsensusMechanism 170
7.6.2FLAreaModelUpdateinSubchainLayer 171
7.6.3FLGlobalModelUpdateinMainchainLayer 172
7.7SimulationResultsandDiscussions 173
7.7.1SimulationSetting 174
7.7.2NumericalResults 175
7.8Conclusion 177 References 178
8Heterogeneity-AwareDynamicSchedulingforFederated EdgeLearning 181 KunGuo,ZihanChen,HowardH.Yang,andTonyQ.S.Quek
8.1Introduction 181
8.2RelatedWorks 184
8.3SystemModelforFEEL 185
8.3.1FlowofFEELwithScheduling 186
8.3.2DelayandEnergyModelinFEEL 187
8.3.2.1DelayModel 187
8.3.2.2EnergyModel 188
8.4Heterogeneity-AwareDynamicSchedulingProblemFormulation 189
8.4.1ConvergenceofFEELwithScheduling 189
8.4.2SchedulingPolicywithSequentialTransmission 190
8.4.3ProblemFormulation 191
8.5DynamicSchedulingAlgorithmDesignandAnalysis 192
8.5.1Benchmark: R-RoundLookaheadAlgorithm 192
8.5.2DISCO:DynamicSchedulingAlgorithm 193
8.5.3AlgorithmAnalysis,ComplexityReduction,andImplementation Discussion 196
8.5.3.1AlgorithmAnalysis 196
8.5.3.2ComplexityReduction 197
8.5.3.3ImplementationDiscussion 197
8.6EvaluationResults 197
8.6.1ParameterSettings 198
8.6.2NumericalResults 200
8.6.3ExperimentalResults 203
8.7Conclusions 208
8.A.1ProofofTheorem8.2 208
8.A.2ProofofTheorem8.3 209
8.A.2.1FeasibilityProof 210
8.A.2.2OptimalityProof 211 References 212
9RobustFederatedLearningwithReal-WorldNoisy Data 215 JingyiXu,ZihanChen,TonyQ.S.Quek,andKaiFongErnestChong
9.1Introduction 215
9.1.1WorkPriorto FedCorr 215
9.2RelatedWork 217
9.2.1FederatedMethods 217
9.2.2LocalIntrinsicDimension(LID) 218
9.2.2.1EstimationofLID 219
9.3FedCorr 219
9.3.1Preliminaries 221
9.3.1.1DataPartition 221
9.3.1.2NoiseModel 222
9.3.1.3LIDScoresforLocalModels 222
9.3.2FederatedPreprocessingStage 222
9.3.2.1ClientIterationandFractionScheduling 223
9.3.2.2MixupandLocalProximalRegularization 223
9.3.2.3IdentificationofNoisyClientsandNoisySamples 224
9.3.3FederatedFine-TuningStage 225
9.3.4FederatedUsualTrainingStage 226
9.4Experiments 226
9.4.1ExperimentalSetup 227
9.4.1.1Baselines 227
9.4.1.2ImplementationDetails 227
9.4.2ComparisonwithState-of-the-ArtMethods 228
9.4.2.1IIDSettings 228
9.4.2.2Non-IIDSettings 228
9.4.2.3CombinationwithOtherFLMethods 231
9.4.2.4ComparisonofCommunicationEfficiency 231
9.4.3AblationStudy 232
9.5FurtherRemarks 232 Bibliography 234
10AnalogOver-the-AirFederatedLearning:Designand Analysis 239
HowardH.Yang,ZihanChen,andTonyQ.S.Quek
10.1Introduction 239
10.2SystemModel 241
10.3AnalogOver-the-AirModelTraining 242
10.3.1SalientFeatures 243
10.3.2Heavy-TailedInterference 244
10.4ConvergenceAnalysis 245
10.4.1Preliminaries 245
10.4.2ConvergenceRateofAirFL 246
10.4.3KeyObservations 250
10.5NumericalResults 250
10.6Conclusion 253 Bibliography 253
11FederatedEdgeLearningforMassiveMIMOCSI Feedback 257
ShiJin,YimingCui,andJiajiaGuo
11.1Introduction 257
11.2SystemModel 259
11.2.1ChannelModelandSignalModel 259
11.2.2DL-BasedCSIFeedback 259
11.3FEELforDL-BasedCSIFeedback 260
11.3.1BasicAutoencoderArchitecture 260
11.3.2FEEL-BasedTrainingFramework 261
11.3.2.1Motivation 261
11.3.2.2TrainingFramework 261
11.3.3ParameterQuantizationintheFEEL-BasedTraining Framework 263
11.3.3.1KeyIdea 263
11.3.3.2Details 264
11.4SimulationResults 264
11.4.1SimulationSettings 264
11.4.1.1ChannelGeneration 264
11.4.1.2TrainingSettings 264
11.4.1.3PerformanceofFEEL-BasedTraining Framework 265
11.5Conclusion 268 Bibliography 269
12User-CentricDecentralizedFederatedLearningfor Autoencoder-BasedCSIFeedback 273 ShiJin,JiajiaGuo,YanLv,andYimingCui
12.1Autoencoder-BasedCSIFeedback 273
12.1.1CSIFeedbackinMassiveMIMO 273
12.1.2SystemModel 274
12.1.3AE-BasedFeedbackFramework 275
12.2User-CentricOnlineTrainingforAE-BasedCSIFeedback 275
12.2.1Motivation 275
12.2.2KeyIdeaofUser-CentricOnlineTraining 276
12.2.3ThreeOnlineTrainingFrameworks 277
12.2.3.1EditBeforeEncoder 277
12.2.3.2EditAfterEncoder 278
12.2.3.3FinetuningEncoder 279
12.3MultiuserOnlineTrainingUsingDecentralizedFederated Learning 279
12.3.1Motivation 279
12.3.2DecentralizedFederatedLearningFramework 280
12.3.2.1KeyIdeaofDecentralizedFederatedLearning 280
12.3.2.2MergeFunction 281
12.3.2.3NetworkConnectivityArchitecture 282
12.4NumericalResults 283
12.4.1SimulationSetting 283
12.4.1.1ChannelGeneration 283
12.4.1.2TrainingDetails 283
12.4.2PerformanceofUser-CentricOnlineTraining 283
12.4.3PerformanceofMultiuserOnlineTrainingUsingDFL 284
12.4.3.1PerformanceEvaluationofDFLframework 284
12.4.3.2NMSEUnderDifferentGossipNumbers 286
12.4.3.3NMSEUnderDifferentNumbersofUEParticipation 286
12.4.3.4NMSEUnderMismatchedChannels 286
12.5Conclusion 287
Bibliography 287
Index 291
AbouttheEditors
YaoSun iscurrentlyalecturerwithJamesWattSchoolofEngineering,theUniversityofGlasgow,Glasgow,UnitedKingdom.Hehasextensiveresearchexperienceandhaspublishedwidelyinwirelessnetworkingresearch.Hehaswonthe IEEECommunicationSocietyofTAOSBestPaperAwardin2019ICC,IEEEIoT JournalBestPaperAwardin2022,andBestPaperAwardin22ndICCT.Hehas beentheguesteditorforspecialissuesofseveralinternationaljournals.Hehas servedasTPCChairforUCET2021andTPCmemberforanumberofinternationalflagshipconferences,includingICC2022,VTCSpring2022,GLOBECOM 2020,andWCNC2019.Hisresearchinterestsincludeintelligentwirelessnetworking,semanticcommunications,blockchainsystems,andresourcemanagementin next-generationmobilenetworks.HeisaseniormemberofIEEE.
ChaoqunYou isapostdoctoralresearchfellowattheSingaporeUniversityof TechnologyandDesign(SUTD).ShereceivedtheBSdegreeincommunication engineeringandthePhDdegreeincommunicationandinformationsystemsfrom theUniversityofElectronicScienceandTechnologyofChina(UESTC)in2013 and2020,respectively.ShewasavisitingstudentattheUniversityofTorontofrom 2015to2017.Hercurrentresearchinterestsincludemobileedgecomputing,networkvirtualization,O-RAN,federatedlearning,and6G.
GangFeng receivedhisBEngandMEngdegreesinelectronicengineering fromtheUniversityofElectronicScienceandTechnologyofChina(UESTC)in 1986and1989,respectively,andthePhDdegreeinInformationEngineering fromtheChineseUniversityofHongKongin1998.Atpresent,heisaprofessor withtheNationalKeyLaboratoryofWirelessCommunications,UESTCof China.Hisresearchinterestsincluderesourcemanagementinwirelessnetworks, next-generationcellularnetworks,etc.Dr.FengisaseniormemberofIEEE.
LeiZhang isaProfessorofTrustworthySystemsattheUniversityofGlasgow.Hehascombinedacademiaandindustryresearchexperienceonwireless communicationsandnetworksanddistributedsystemsforIoT,blockchain,
xvi AbouttheEditors
andautonomoussystems.His20patentshavebeengranted/filedin30+ countries/regions.Hepublished3booksand150+ papersinpeer-reviewedjournals, conferences,andeditedbooks.HereceivedtheIEEEInternetofThingsJournal BestPaperAward2022,IEEEComSocTAOSTechnicalCommitteeBestPaper Award2019,andIEEEICEICT’21BestPaperAward.
Preface
Ithasbeenconsideredoneofthekeymissingcomponentsintheexisting5G networkandiswidelyrecognizedasoneofthemostsought-afterfunctionsfor next-generation6Gcommunicationsystems.Nowadays,therearemorethan 10billionInternet-of-Things(IoT)equipmentand5billionsmartphonesthatare equippedwithartificialintelligence(AI)-empoweredcomputingmodulessuchas AIchipsandGPU.Ontheonehand,theuserequipment(UE)canbepotentially deployedascomputingnodestoprocesscertainemergingservicetaskssuchas crowdsensingtasksandcollaborativetasks,whichpavesthewayforapplying AIinedgenetworks.Ontheotherhand,intheparadigmofmachinelearning (ML),thepowerfulcomputingcapabilityontheseUEscandecoupleMLfrom acquiring,storing,andtrainingdataindatacentersasconventionalmethods.
Federatedlearning(FL)hasbeenwidelyacknowledgedasoneofthemostessentialenablerstobringnetworkedgeintelligenceintoreality,asitcanenablecollaborativetrainingofMLmodelswhileenhancingindividualuserprivacyanddata security.EmpoweredbythegrowingcomputingcapabilitiesofUEs,FLtrainsML modelslocallyoneachdevicewheretherawdataneverleavesthedevice.Specifically,FLusesaniterativeapproachthatrequiresanumberofglobaliterationsto achieveaglobalmodelaccuracy.Ineachglobaliteration,UEstakeanumberof localiterationsuptoalocalmodelaccuracy.Asaresult,theimplementationofFL atedgenetworkscanalsodecreasethecostsoftransmittingrawdata,relievethe burdenonbackbonenetworks,reducethelatencyforreal-timedecisions.
ThisbookwouldexplorerecentadvancesinthetheoryandpracticeofFL,especiallywhenitisappliedtowirelesscommunicationsystems.Indetail,thebook coversthefollowingaspects:
1)principlesandfundamentalsofFL; 2)performanceanalysisofFLinwirelesscommunicationsystems;
3)howfuturewirelessnetworks(say6Gnetworks)enableFLaswellashowFL frameworks/algorithmscanbeoptimizedwhenapplyingtowirelessnetworks (6G);
4)FLapplicationstoverticalindustriesandsometypicalcommunication scenarios.
Chapter1investigatestheoptimizationdesignofFLintheedgenetwork.First, anoptimizationproblemisformulatedtomanagethetrade-offbetweenmodel accuracyandtrainingcost.Second,ajointoptimizationalgorithmisdesignedto optimizethemodelcompression,sampleselection,anduserselectionstrategies, whichcanapproachastationaryoptimalsolutioninacomputationallyefficient way.Finally,theperformanceoftheproposedoptimizationschemeisevaluated bynumericalsimulationandexperimentresults,whichshowthatboththeaccuracylossandthecostofFLintheedgenetworkcanbereducedsignificantlyby employingtheproposedalgorithm.
Chapter2studiesnon-IIDdatamodelforFL,derivesatheoreticalupperbound, andredesignsthefederatedaveragingschemetoreducetheweightdifference.To furthermitigatetheimpactofnon-IIDdata,adata-sharingschemeisdesigned tojointlyminimizetheaccuracyloss,theenergyconsumption,andlatencywith constrainedresourceofedgesystems.Thenacomputation-efficientalgorithmis proposedtoapproachtheoptimalsolutionandprovidetheexperimentresultsto evaluateourproposedschemes.
Chapter3theoreticallyanalyzestheperformanceandcostofrunningFL, whichisimperativetodeeplyunderstandtherelationshipbetweenFLperformanceandmultiple-dimensionalresources.Inthischapter,weconstructan analyticalmodeltoinvestigatetherelationshipbetweentheFLmodelaccuracy andconsumedresourcesinFL-enabledwirelessedgenetworks.Basedonthe analyticalmodel,weexplicitlyquantifythemodelaccuracy,computingresources, andcommunicationresources.Numericalresultsvalidatetheeffectivenessof ourtheoreticalmodelingandanalysisanddemonstratethetrade-offbetweenthe communicationandcomputingresourcesforachievingacertainmodelaccuracy.
Chapter4proposesanefficientdeviceassociationschemeforradioaccessnetwork(RAN)slicingbyexploitingafederatedreinforcementlearningframework, withtheaimtoimprovenetworkthroughput,whileguaranteeinguserprivacyand datasecurity.Specially,weusedeepreinforcementlearningtotrainlocalmodelsonUEsunderahybridFLframework,wherehorizontallyFLisemployedfor parameteraggregationonBS,whileverticallyFLisemployedforaccessselection aggregationontheencryptedparty.Numericalresultsshowthatourproposed schemecanachievesignificantperformancegainsintermsofnetworkthroughput andcommunicationefficiencyincomparisonwithsomeknownstate-of-the-art solutions.
Chapter5proposesadeepFLalgorithmthatutilizesknowledgedistillation anddifferentialprivacytosafeguardprivacyduringthedatafusionprocess. OurapproachinvolvesaddingGaussiannoiseatdifferentstagesofknowledgedistillation-basedFLtoensureprivacyprotection.Ourexperimental resultsdemonstratethatthisstrategyprovidesbetterprivacypreservationwhile achievinghigh-precisionIoTdatafusion.
Chapter6presentsanovelsystematicbeamcontrolschemetotacklethe formulatedbeammanagementproblem,whichisdifficultduetothenonconvex objectivefunction.ThedoubledeepQ-network(DDQN)underaFLframework isemployedtosolvetheaboveoptimizationproblem,therebyfulfillingadaptive andintelligentbeammanagementinmmwavenetworks.Intheproposedbeam managementschemebasedonfederatedlearning(BMFL),thenon-raw-data aggregationcantheoreticallyprotectuserprivacywhilereducinghandoffcosts. Moreover,adatacleaningtechniqueisusedbeforethelocalmodeltraining,with theaimtofurtherstrengthentheprivacyprotectionwhileimprovingthelearning convergencespeed.Simulationresultsdemonstratetheperformancegainofthe proposedBMFLscheme.
Chapter7proposesadouble-layerblockchain-baseddeepreinforcementfederatedlearning(BDRFL)schemetoensureprivacy-preservedandcaching-efficient D2Dnetworks.InBDRFL,adouble-layerblockchainisutilizedtofurtherenhance datasecurity.SimulationresultsfirstverifytheconvergenceofBDRFL-based algorithmandthendemonstratethatthedownloadlatencyoftheBDRFL-based cachingschemecanbesignificantlyreducedunderdifferenttypesofattacks whencomparedwithsomeexistingcachingpolicies.
Chapter8aimstodesignadynamicschedulingpolicytoexplorethespectrumflexibilityforheterogeneousfederatededgelearning(FEEL)soasto facilitatethedistributedintelligenceinedgenetworks.Thischapterproposesa heterogeneity-awaredynamicschedulingproblemtominimizethegloballoss function,withconsiderationofstragglerandlimiteddeviceenergyissues.By solvingtheformulatedproblem,weproposeadynamicschedulingalgorithm (DISCO),tomakeanintelligentdecisiononthesetandorderofscheduleddevices ineachcommunicationround.Theoreticalanalysisrevealsthatundercertain conditions,learningperformanceandenergyconstraintscanbeguaranteedinthe DISCO.Finally,wedemonstratethesuperiorityoftheDISCOthroughnumerical andexperimentalresults,respectively.
Chapter9discussesFedCorr,ageneralmultistageframeworktotackleheterogeneouslabelnoiseinFL,whichdoesnotmakeanyassumptionsonthenoise modelsoflocalclientswhilestillmaintainingclientdataprivacy.Boththeoretical analysisandexperimentresultsdemonstratetheperformancegainofthisnovel FLframework.
Preface
Chapter10providesageneraloverviewoftheanalogover-the-airfederated learning(AirFL)system.Specially,weillustratethegeneralsystemarchitecture andhighlightthesalientfeatureofAirFLthatadoptsanalogtransmissions forfast(butnoisy)aggregationofintermediateparameters.Then,weestablish anewconvergenceanalysisframeworkthattakesintoaccounttheeffectsof fadingandinterferencenoise.Ouranalysisunveilstheimpactsfromtheintrinsic propertiesofwirelesstransmissionsontheconvergenceperformanceofAirFL. Thetheoreticalfindingsarecorroboratedbyextensivesimulations.
Chapter11investigatesaFEEL-basedtrainingframeworktoDL-basedchannel stateinformation(CSI)feedback.InFEEL,eachUEtrainsanautoencodernetworklocallyandexchangesmodelparametersviathebasestation.Therefore,data privacyisbetterprotectedcomparedwithcentralizedlearningbecausethelocal CSIdatasetsarenotrequiredtobeuploaded.NeuralnetworkparameterquantizationisthenintroducedtotheFEEL-basedtrainingframeworktoreducecommunicationoverhead.ThesimulationresultsindicatethattheproposedFEEL-based trainingframeworkcanachievecomparableperformancewithcentralized learning.
Chapter12proposesauser-centriconlinetrainingstrategyinwhichtheUEcan collectCSIsamplesinthestableareaandadjustthepretrainedencoderonline tofurtherimproveCSIreconstructionaccuracy.Moreover,theproposedonline trainingframeworkisextendedtothemultiuserscenariotoimproveperformance sequentially.ThekeyideaistoadoptdecentralizedFLwithoutBSparticipation tocombinethesharingofchannelknowledgeamongUEs,whichiscalledcrowd intelligence.SimulationresultsshowthatthedecentralizedFL-aidedframework hashigherfeedbackaccuracythantheAEwithoutonlinetraining.
November2023
FederatedLearningwithUnreliableTransmissionin
MobileEdgeComputingSystems
ChenyuanFeng 1 ,DaquanFeng 1 ,ZhongyuanZhao 2 ,HowardH.Yang 3 ,and TonyQ.S.Quek 4
1 ShenzhenKeyLaboratoryofDigitalCreativeTechnology,TheGuangdongProvinceEngineering LaboratoryforDigitalCreativeTechnology,TheGuangdong-HongKongJointLaboratoryforBigData ImagingandCommunication,CollegeofElectronicsandInformationEngineering,ShenzhenUniversity, Shenzhen,Guangdong,China
2 StateKeyLaboratoryofNetworkingandSwitchingTechnology,SchoolofInformationand CommunicationEngineering,BeijingUniversityofPostsandTelecommunications,Beijing,China
3 ZhejiangUniversity/UniversityofIllinoisatUrbana-ChampaignInstitute,ZhejiangUniversity, TheCollegeofInformationScienceandElectronicEngineering,Haining,Zhejiang,China
4 InformationSystemsTechnologyandDesignPillar,SingaporeUniversityofTechnologyandDesign, Singapore
1.1SystemModel
ConsiderthedeploymentofFLinanMECscenario,whichconsistsofanedge accesspoint E andmultipleusers U1 , … , UM .Anedgecomputingserver SE isequippedwith E,whilealocalcomputingunit Sm isequippedwith Um , m = 1, … , M .AsshowninFigure1.1,theedgecomputingserver SE andlocal computingunits S1 , … , SM canactasthecomputingserverandtheclients, respectively,whichcaninteractwitheachotherviathewirelesschannels between E and U1 , , UM .
Asintroducedpreviously,FLcanbeimplementedbyupdatingthelocalmodels andtheglobalmodeliteratively.Inparticular,wefocusonthe tthiteration,which canbeintroducedasfollows.
1.1.1LocalModelTraining
Inthisphase,eachuserupdatesthelocalmodelindependentlybasedonitslocal collecteddata.Withoutlossofgenerality,wefocusonaspecificuser Um ,thelocal modelof Um canbeupdatedasfollows:
FederatedLearningforFutureIntelligentWirelessNetworks,FirstEdition. EditedbyYaoSun,ChaoqunYou,GangFeng,andLeiZhang. ©2024TheInstituteofElectricalandElectronicsEngineers,Inc.Published2024byJohnWiley&Sons,Inc.
1FederatedLearningwithUnreliableTransmissioninMobileEdgeComputingSystems
access point
Figure1.1 ThesystemmodelofwirelessFL.
where wt,m and wt 1,m denotetheupdateresultsof Um ’slocalmodelduringthe tthand (t 1)-thiterations,respectively, t,m denotesthetrainingdatasetfor updating wt,m ,whichisrandomlyselectedfrom m , t,m ⊆ m , m denotes thelocaldatasetlocatedat Um , ��t isthelearningrateofthe tthiteration,and ∇F (wt 1,m , t,m ) isthegradientoflossfunction F (wt 1,m , t,m ) withrespectto wt 1,m .Inthischapter,thelossfunctionisdefinedastheempiricalriskwith respectto wt,m ,whichcanbedefinedasfollows:
where l(wt,m ; x) denotesthelossfunctionofthedataelement x,and Nt,m denotes thesizeof t,m .
1.1.2UpdateResultFeedbackviatheWirelessChannels
Whenthelocalmodeltrainingprocedureisaccomplished, Um shouldtransmitits updateresult wt,m to E viathewirelesschannel.Intheexistingworks,theserver randomlyselectstheuserssinceitisassumedthatthecommunicationsbetween thecomputingserverandtheclientsareideal.However,itcannotbeensured intheMECsystemsduetotheunreliablewirelesstransmissioncircumstances, whichwillcauseaccuracylossofFLmodels.Therefore,inthischapter,onlythe userswithhighcommunicationreliabilityandlowmodelaccuracylossarescheduledtoparticipateineachiterationofglobalmodelaveraging.Inparticular,the schedulingstatusof Um forthe tthiterationofglobalaveragingischaracterizedby aBooleanvariable zt,m ,i.e., zt,m = { 1, if Um isscheduled 0, if Um isnotscheduled. (1.3)
If Um isscheduled,itsupdateresult wt,m canbemodeledasa d × 1vector,which isusuallywithhighdimension,especiallyforthedeepneuralnetworkmodels. Therefore,toimprovetheefficiencyofupdateresultfeedback, modelsparsification and parameterquantization techniquesshouldbeemployedtocompress wt,m .Asintroducedpreviously, wt,m canbetransformedintoasparseformvia modelsparsification,whichcanbeexpressedasfollows:
where At,m denotesa d × d sparsificationmatrixfor wt,m .
Next,eachelementof st,m isquantizedindependentlybyemployinguniform quantization.ThequantizationerrorcanbeapproximatedasanadditiveGaussian noise,whichisindependentwith wt,m .Thenthequantizedparametervectorcan beexpressedas
where qt,m denotesa d × 1quantizationnoisevector,i.e., qt,m ∼ (��, ��t,m ), and ��t,m denotesthecovariancematrix.Duetotheimplementationofindependentquantization,eachelementof qt,m isindependentwitheachother,i.e., ��{q t,m i (q t,m i )H }= �� t,m i , ��{q t,m i (q t,m j )H }= 0, i ≠ j,where q t,m i and q t,m j denotethe ith and jthelementsof qt,m ,respectively.Therefore, ��t,m isadiagonalmatrix,which canbedenotedas ��t,m = diag{q t,m 1 , … , q t,m d }.
Aftermodelsparsificationandparameterquantization, xt,m issuitableforbasebandprocessingandwirelesstransmissions.Inthischapter,theflatfadingmodel isemployedtocharacterizethewirelesschannelsbetween Um and E.Therefore, thechannelfadingcanbeassumedtobeunchangedduringthetransmissionof xt,m .Thentheobservationof xt,m at E canbeexpressedas
,
(1.6) where ht,m capturestheflatchannelfadingofthewirelesslinkbetween Um and E, nt,m denotestheadditivewhiteGaussiannoiseat E,i.e., nt,m ∼ (��,�� 2 t ��L ), ��L denotesa d × d identitymatrix,and �� 2 t isthepowerofnoise.
1.1.3GlobalModelAveraging
Torecovertheupdateresultsoflocalmodels, yt,m shouldbefirstdecompressedby E.Inthischapter,theminimummean-squareerror(MMSE)criterionisemployed, andthedecompressionresultcanbewrittenas
(1.7)
where Dt,m isa d × d decompressionmatrixof yt,m , denotesasetthatconsistsof allthepossiblequantizedparametervectors,i.e., wq ∈ .Sinceeachelementof
4
1FederatedLearningwithUnreliableTransmissioninMobileEdgeComputingSystems
thequantizedmodelparametervectorcanbedetectedindividually,recallingthe computationalcomplexityofMMSE,thecomplexityofthisdetectionisalinear functionofthevectordimension.1
Thentheglobalmodelcanbeupdatedbyaveragingthedecompressedresultsof localmodels.AsintroducedinKonecnýetal.[2016],theupdateresultofglobal modelcanbeexpressedas
where wt,F and wt 1,F denotetheglobalmodelforthe tthand (t 1)-thiterations, respectively, zt,m isdefinedby(1.3), N = ∑M m=1 Nm .
Afterglobalmodelaveraging, wt,F issentbacktotheusers.Since wt,F aretransmittedviadownlinktransmissions,whichcanacquiremoreradioresourceand highertransmitpowerthanthelocalmodelupdatephase.Therefore,itcanbe assumedthat wt,F isreceivedsuccessfullybyalltheusers.Thenthelocalmodel ofeachusercanbeupdatedas wt,m = wt,F , m = 1
1.2ProblemFormulation
Theperformanceofexistinglearningtechniquesismainlydeterminedbythe accuracyofgeneratedlearningmodels.Itisdifficulttobemodeledinatractable form,andthuscannotbeoptimizedbyemployingtheexistingresourcemanagementschemesintheMECsystems.Inthissection,wefirstderivedaclosed-form upperboundofmodelaccuracyloss,whichisanefficientmetrictoevaluatethe qualityofFLmodels.Thenanoptimizationproblemisformulatedtoimprovethe modelaccuracyandtrainingefficiencyofFLwithlimitedbudgetofcomputation andcommunicationresources.
1.2.1ModelAccuracyLoss
AsintroducedinKonecnýetal.[2016],theobjectiveofmodeltrainingistominimizetheexpectedrisk,whichcanbeestimatedbyemployingthetheempirical riskgivenby(1.2).Therefore,themodelaccuracyloss,whichcanbedefinedasthe
1Toimplement(1.7)atedgeserver,itrequirestofeedbacksparsificationmatrix At,m ,andthis communicationoverheadcanbereducedsignificantlywhen wt,m ∈ ℝd isdividedintomultiple segments.Moreover,thedetectioncanbedonewithchannelestimation,eachdevicecan feedback Dt,m insteadofchannelstateinformation.
expectedmeansquareerror(MSE)withrespecttothetheempiricalriskandthe expectedrisk,isanefficientmetrictoevaluatetheaccuracyperformanceofgeneratedmodels.Inthischapter,themodelaccuracylossafter t iterationsofglobal averagingcanbeexpressedas
where F (wt,F ) denotestheempiricalriskwithrespectto wt,F forthe tthiteration, G(w∗ ) denotestheminimumexpectedriskofourstudiedFLmodel,i.e., G(w∗ )= min w ��{l(w, x)},and w∗ denotesthetheoreticallyoptimalparameter vector.
AsintroducedinShaietal.[2010],toguaranteetheconvergenceofFL,the followingfeatureswithrespecttotheempiricalriskshouldbesatisfied:
1. F (w, ) is L-Lipchitz,i.e.,forany w1 and w2 ,wehave
,
2. F (w, ) is λ-stronglyconvex.i.e.,forany w1 and w2 ,and0 ⩽ �� ⩽ 1,thefollowinginequalitycanbeestablished: F (�� w1 +(1 �� )w2 ,
Bysubstituting(1.8)into(1.9),anupperboundof t canbeexpressedasfollows duetotheconvexityof F (w, ): t = �� ⎧ ⎪ ⎨ ⎪ ⎩ ‖ ‖ ‖ ‖ ‖ ‖ F ( M ∑ m=1 Nm N [(1 zt,m )wt 1,F + zt,m wt,m ]) G (w∗ ) ‖ ‖ ‖
N [(1 zt,m )F (wt 1,F )+ zt,m F (wt,m )− G(w∗ )] ‖ ‖ ‖ ‖ ‖ ‖ 2 ⎫ ⎪ ⎬ ⎪ ⎭
⎢
⎣ (1 zt,m )�� {∥ F (wt 1,F )− G(w∗ )∥2 } ⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟
1FederatedLearningwithUnreliableTransmissioninMobileEdgeComputingSystems
IntheparadigmsofFL,eachiterationcanbeoptimizedindependently.Without lossofgenerality,wefocusontheoptimizationofthe tthiterationinthischapter, andthus t 1 in(1.10)canbetreatedasaconstant.Therefore, t ismainlydecided by ��t,m ,whichcanbederivedasfollows:
+
{∥ G(wt,m )− G(w∗ )∥2 }
where F (wt,m ) isgivenby(1.2),and G(wt,m ) denotestheexpectedriskwithrespect to wt,m .Asshownin(1.11), ��t,m isdeterminedbythecommunicationloss �� C t,m , sampleselectionloss �� S t,m ,andmodeltrainingloss �� T t,m ,whichcanbederivedas follows.
1.2.2CommunicationLoss �� C
Thecommunicationlossisjointlydecidedbythemodelcompressionlossandthe communicationerrorcausedbywirelesstransmissions.Thenanupperboundof �� C t,m canbeprovidedbythefollowinglemma.
Lemma1.1 WhenF (w, ) isL-Lipchitz,anupperboundofcommunicationloss �� C t,m in (1.11) canbeexpressedas
whereat
isthepowerofquantizationnoise,and �� 2 t isthepower ofGaussiannoise
1.2.3SampleSelectionLoss �� S t ,m
Althoughthetrainingdataset t,m isgeneratedbasedonthedistributionof x,the sampleselectionbiasstillexistsduetothelimitedsizeof t,m .Since wt,m istrained basedon t,m ,thesampleselectionbiascauseserrorpropagationduringthemodel trainingprocedure,whichischaracterizedassampleselectionlossinthischapter.
Asshownin(1.11), �� S t,m isdefinedasthedifferencebetweentheempiricalrisk withrespectto t,m andtheexpectedriskwithrespecttothedistributionofdata elements.AsintroducedinDevroyeetal.[1996],anupperboundof �� S t,m canbe providedasfollows.
Remark1.1 ((Devroyeetal.[1996]),Theorem12.6)When F (w, ) isconvex,a tractableupperboundof �� S t,m canbeexpressedasfollows:
1.2.4ModelTrainingLoss
AsintroducedinSection1.1, wt,m isgeneratedbyemployingstochasticgradient descent(SGD)method,whichisoptimizedinaniterativeway.Theoretically,it canapproachtheoptimaltrainingresult w∗ (Rakhlinetal.[2017]).However,there alwaysexistsagapbetween wt,m and w∗ sinceonlyafewiterationscanbeexecuted toguaranteethecomputingefficiency.Inthischapter,itiscapturedasmodeltrainingloss �� M t,m in(1.11).Toevaluateitsimpactonthemodelaccuracy,thefollowing lemmaisprovided.
Lemma1.2 WhenF (w, ) isL-Lipchitzand λ-stronglyconvex,and ∇F (w, ) is withfinitescale,i.e., ��{∥∇F (w, )∥2 } ⩽ G2 , anupperboundofmodeltrainingloss �� M t,m canbewrittenas
Bysubstituting(1.11),(1.12),(1.13),and(1.14)into(1.10),anupperboundof t canbederived,whichisgivenbythefollowingcorollary.
Corollary1.1 Anupperboundofmodelaccuracyloss t givenby(1.9),which isdenotedas t ,canbeexpressedas
Itisworthmentioningthatitisunnecessarytoobtain F (wt,F ) and G(w∗ ) with highcomputationcostsbasedonthefollowingschemes.First,wecanobtain F (wt 1,F ) withoutadditionalcomputationoverheadsincethegloballosscanbe computedasalinearcombinationoflocalloss F (wt,F , um ) withrespecttoeach participant,andthelocallosscanbeobtainedby Um vialocalmodeltraining. Second,thetruevalueof G(w∗ ) isknownforsomespecificlearningtasks,such asthesupervisedmulticlassificationtasks.AsintroducedinMcMahanetal. [2017],sinceallimagedatacanbelabeledcorrectlywhentheglobaloptimal parametersetting w∗ canbeobtained,thecorrespondingminimumexpectedrisk canbesetas G(w∗ )= 0forthesupervisedmulticlassificationtasks.Moreover, whenthetruevalueof G(w∗ ) isunknown,zerocanbetreatedasalowerbound oftheexpectedriskfunction,andwealsohave F (wt 1,F ) ⩾ G(w∗ ) since w∗ is globaloptimalsolution.Byfar,anupperboundof t 1 canbederivedas t 1 = �� {∥ F (wt 1,F )− G(w
1 stillcancapture therelationshipbetweenthemodelaccuracylossandtheselectionstrategiesof participantsanddatasamples,whichcanbesubstitutedinto(1.15)insteadof t 1
1.2.5ProblemFormulation
1.2.5.1ObjectiveFunction
SincetheresourceofMECsystemsisrestricted,asophisticatedtrade-offbetween themodelaccuracyandcostshouldbekepttoguaranteethatahigh-qualitymodel canbegeneratedinacost-efficientway.Inthischapter,twocategoriesofcostsare considered,whicharethecomputationandcommunicationcosts,respectively.
Eachdataelementisprocessedbythesamelearningmodel,whichmeans thecomputationcostisproportionaltothesizeofemployedtrainingdataset.In accordancewith(Zhaoetal.[2020],Chenetal.[2021]),wemodelthecomputationcostasalinearfunctionof Nt,m .Thecompressionandcommunicationcost ismodeledasthemutualinformationbetween xt,m and wt,m sincethemutual informationcancharacterizethecapacityofwirelesslinkforacompressedsignal tobesuccessfullyrecovered(Zhangetal.[2007]).Thisstandardrate-distortion theoreticmetric(ElGamalandKim[2011])intheanalysisofcompressiondesign canprovideusefulinsighttodeployFLwithsuccessfulcompressionschemes. ThenthemodelcostofFLcanbeexpressedas
where ��1 and ��2 denotetheweightsofcomputationandcommunicationcosts, respectively.
Inthischapter,ourtargetistojointlyminimizethemodelaccuracylossandthe costsofFL,andtheobjectivefunctioncanbemodeledas
where w1 and w2 denotetheweightsof t and t ,respectively.
1.2.5.2EnergyConsumptionConstraint
Duetothelimitationofbatteryvolume,theenergyconsumptionofeachuser shouldberestrictedindependently.Duringthe tthiteration,theuserenergyconsumptionismainlycausedbylocalmodeltrainingandupdateresultfeedback. Inparticular,theenergyconsumptionoflocalmodeltraininggrowslinearlywith respecttothesizeoftrainingdataset t,m ,andthetransmitenergyconsumption canbederivedbasedon(1.5).Thentheenergyconsumptionconstraintcanbe expressedasfollows:
where �� denotesthecoefficientofenergyconsumptioncausedbyprocessinga singledataelementinthelocalmodeltrainingphase,and Pm isthemaximum energyconsumptionforthe tthiterationofFL.
1.2.5.3UserSelectionConstraint
AsintroducedinSection1.1,onlyapartofusersareselectedtofeedbacktheir updateresults.Toguaranteetheconvergenceperformanceandcomputingefficiency,themaximumnumberofselectedusersinthe tthiterationissetas Kt , Kt ⩽ M .Recalling(1.3),thefollowingconstraintofuserselectionisestablished:
1.2.5.4DataVolumeConstraintofLocalTrainingDatasets
Byemployingsophisticatedsampleselectionstrategies,theimpactofsample selectionlosscanbemitigatedsignificantly.Inparticular,thelocaltraining data-set t,m isselectedrandomlyfrom m ,andthusitsdatavolumeshould followtheconstraint
where
denotesthedatavolumeof m ,and ℕ denotesthesetofnonnegative integers.Inthischapter,wefocusontheminimizationofmodelaccuracylossand costofFLintheMECsystems,whichcanbecapturedbytheupperboundgiven by(1.15).Therefore,theoptimizationproblemcanbeformulatedasfollows:
1.3AJointOptimizationAlgorithm
Theestablishedoptimizationproblemgivenby(1.21)isanonlinearandnonconvexproblem.Inthissection,toobtainatractablesolution,itisfirstdecoupledasthreeindependentsubproblems,andaniterativeoptimizationalgorithm isdesigned.Thenweprovethattheproposedalgorithmcanapproachastationary optimalsolution.
1.3.1CompressionOptimization
Asshownin(1.15)and(1.16), At,m and Dt,m onlydeterminetheaccuracylossand costofcommunications,andthusanequivalentformoftheobjectivefunction canbeexpressedasfollowsbyremovingthetermsthatarenotrelatedwith At,m and Dt,m :
Equation(1.22)indicatesthatthecompressionofeachusercanbeoptimizedindependentlybyminimizing Q1,m ,whichisrestrictedbyanindividualenergyconsumptionconstraint.Therefore,theoptimizationofcompressionanddecompressionmatricescanbetransformedinto M -dependentsubproblems,whichcanbe expressedas min
Wefirstconsidersolving At,m and Dt,m independently,andthenpropose Algorithm1.1tooptimizethemjointly.
1.3.1.1OptimizationofAt ,m
Theoptimizationproblemof At,m isidenticalwith(1.25)when Dt,m isfixed. First,weverifytheconvexityof G1,m , G2,m ,and G3,m ,whichcanbeprovidedby thefollowinglemma.Since Q1,m G1,m , G2,m ,and G3,m canbetreatedasfunctions withrespectto At,m ,theyaredenotedas Q1,m (At,m ), G1,m (At,m ), G2,m (At,m ),and G3,m (At,m ) inthispart,respectively.
