Algorithms and architectures for parallel processing 14th international conference ica3pp 2014 dalia

and Architectures for Parallel Processing 14th International Conference ICA3PP 2014 Dalian China

August 24 27 2014 Proceedings Part II 1st Edition Xian-He Sun

Visit to download the full and correct content document: https://textbookfull.com/product/algorithms-and-architectures-for-parallel-processing-1 4th-international-conference-ica3pp-2014-dalian-china-august-24-27-2014-proceedin gs-part-ii-1st-edition-xian-he-sun/

More products digital (pdf, epub, mobi) instant download maybe you interests ...

Algorithms and Architectures for Parallel Processing

14th International Conference ICA3PP 2014 Dalian China

August 24 27 2014 Proceedings Part I 1st Edition XianHe Sun

https://textbookfull.com/product/algorithms-and-architecturesfor-parallel-processing-14th-international-conferenceica3pp-2014-dalian-china-august-24-27-2014-proceedingspart-i-1st-edition-xian-he-sun/

Algorithms and Architectures for Parallel Processing

18th International Conference ICA3PP 2018 Guangzhou

China November 15 17 2018 Proceedings Part I Jaideep

Vaidya

https://textbookfull.com/product/algorithms-and-architecturesfor-parallel-processing-18th-international-conferenceica3pp-2018-guangzhou-china-november-15-17-2018-proceedings-parti-jaideep-vaidya/

Algorithms and Architectures for Parallel Processing

18th International Conference ICA3PP 2018 Guangzhou

China November 15 17 2018 Proceedings Part III Jaideep

Vaidya

https://textbookfull.com/product/algorithms-and-architecturesfor-parallel-processing-18th-international-conferenceica3pp-2018-guangzhou-china-november-15-17-2018-proceedings-partiii-jaideep-vaidya/

Algorithms and Architectures for Parallel Processing

18th International Conference ICA3PP 2018 Guangzhou

China November 15 17 2018 Proceedings Part IV Jaideep

Vaidya

https://textbookfull.com/product/algorithms-and-architecturesfor-parallel-processing-18th-international-conferenceica3pp-2018-guangzhou-china-november-15-17-2018-proceedings-partiv-jaideep-vaidya/

Algorithms and Architectures for Parallel Processing

13th International Conference ICA3PP 2013 Vietri sul Mare Italy

December 18 20 2013 Proceedings Part II 1st

Edition Peter Benner

https://textbookfull.com/product/algorithms-and-architecturesfor-parallel-processing-13th-international-conferenceica3pp-2013-vietri-sul-mare-italydecember-18-20-2013-proceedings-part-ii-1st-edition-peter-benner/

Algorithms and Architectures for Parallel Processing

ICA3PP 2018 International Workshops Guangzhou China

November 15 17 2018 Proceedings Ting Hu

https://textbookfull.com/product/algorithms-and-architecturesfor-parallel-processing-ica3pp-2018-international-workshopsguangzhou-china-november-15-17-2018-proceedings-ting-hu/

Intelligent Virtual Agents 14th International Conference IVA 2014 Boston MA USA August 27 29 2014 Proceedings 1st Edition Timothy Bickmore

https://textbookfull.com/product/intelligent-virtual-agents-14thinternational-conference-iva-2014-boston-ma-usaaugust-27-29-2014-proceedings-1st-edition-timothy-bickmore/

Algorithms and Architectures for Parallel Processing

16th International Conference ICA3PP 2016 Granada Spain

December 14 16 2016 Proceedings 1st Edition Jesus Carretero

https://textbookfull.com/product/algorithms-and-architecturesfor-parallel-processing-16th-international-conferenceica3pp-2016-granada-spain-december-14-16-2016-proceedings-1stedition-jesus-carretero/

Algorithms and Architectures for Parallel Processing

13th International Conference ICA3PP 2013 Vietri sul Mare Italy December 18 20 2013 Proceedings Part I 1st Edition Antonio Balzanella

https://textbookfull.com/product/algorithms-and-architecturesfor-parallel-processing-13th-international-conferenceica3pp-2013-vietri-sul-mare-italydecember-18-20-2013-proceedings-part-i-1st-edition-antonio-

Xian-he Sun Wenyu Qu Ivan Stojmenovic

Wanlei Zhou Zhiyang Li Hua Guo

Geyong Min Tingting Yang Yulei Wu

Lei Liu (Eds.)

Algorithms and Architectures for Parallel Processing

14th International Conference, ICA3PP 2014 Dalian, China, August 24–27, 2014

Proceedings, Part II

LectureNotesinComputerScience8631

CommencedPublicationin1973

FoundingandFormerSeriesEditors: GerhardGoos,JurisHartmanis,andJanvanLeeuwen

EditorialBoard

DavidHutchison LancasterUniversity,UK

TakeoKanade CarnegieMellonUniversity,Pittsburgh,PA,USA

JosefKittler UniversityofSurrey,Guildford,UK

JonM.Kleinberg CornellUniversity,Ithaca,NY,USA

AlfredKobsa UniversityofCalifornia,Irvine,CA,USA

FriedemannMattern ETHZurich,Switzerland

JohnC.Mitchell StanfordUniversity,CA,USA

MoniNaor

WeizmannInstituteofScience,Rehovot,Israel

OscarNierstrasz UniversityofBern,Switzerland

C.PanduRangan IndianInstituteofTechnology,Madras,India

BernhardSteffen TUDortmundUniversity,Germany

DemetriTerzopoulos UniversityofCalifornia,LosAngeles,CA,USA

DougTygar UniversityofCalifornia,Berkeley,CA,USA

GerhardWeikum MaxPlanckInstituteforInformatics,Saarbruecken,Germany

Xian-heSunWenyuQuIvanStojmenovic WanleiZhouZhiyangLiHuaGuo

GeyongMinTingtingYangYuleiWu LeiLiu(Eds.)

AlgorithmsandArchitectures forParallelProcessing

14thInternationalConference,ICA3PP2014

Dalian,China,August24-27,2014

Proceedings,PartII

VolumeEditors

Xian-heSun

IllinoisInstituteofTechnology,Chicago,IL,USA,e-mail:sun@iit.edu

WenyuQu

DalianMaritimeUniversity,China,e-mail:wenyu@dlmu.edu.cn

IvanStojmenovic

UniversityofOttawa,ON,Canada,e-mail:ivan@site.ottawa.ca

WanleiZhou

DeakinUniversity,Burwood,VIC,Australia,e-mail:wanlei.zhou@deakin.edu.au

ZhiyangLi

DalianMaritimeUniversity,China,e-mail:lizy0205@gmail.com

HuaGuo

BeiHangUniversity,Beijing,China,e-mail:hguo@buaa.edu.cn

GeyongMin

UniversityofBradford,UK,e-mail:g.min@brad.ac.uk

TingtingYang

DalianMaritimeUniversity,China,e-mail:yangtingting820523@163.com

YuleiWu

ChineseAcademyofSciences,Beijing,China,e-mail:yulei.frank.wu@gmail.com

LeiLiu

ShandongUniversity,JinanCity,China,e-mail:l.liu@sdu.edu.cn

ISSN0302-9743e-ISSN1611-3349 ISBN978-3-319-11193-3e-ISBN978-3-319-11194-0 DOI10.1007/978-3-319-11194-0

SpringerChamHeidelbergNewYorkDordrechtLondon LibraryofCongressControlNumber:2014947719

LNCSSublibrary:SL1–TheoreticalComputerScienceandGeneralIssues

©SpringerInternationalPublishingSwitzerland2014

Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation, broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection withreviewsorscholarlyanalysisormaterialsuppliedspecificallyforthepurposeofbeingenteredand executedonacomputersystem,forexclusiveusebythepurchaserofthework.Duplicationofthispublication orpartsthereofispermittedonlyundertheprovisionsoftheCopyrightLawofthePublisher’slocation, inistcurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Permissionsforuse maybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violationsareliabletoprosecution undertherespectiveCopyrightLaw.

Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateofpublication, neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityforanyerrorsor omissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,withrespecttothe materialcontainedherein.

Typesetting: Camera-readybyauthor,dataconversionbyScientificPublishingServices,Chennai,India Printedonacid-freepaper

SpringerispartofSpringerScience+BusinessMedia(www.springer.com)

Preface

Welcometotheproceedingsofthe14thInternationalConferenceonAlgorithms andArchitecturesforParallelProcessing(ICA3PP2014)heldinDalian,China.

ICA3PP2014isthe14thinthisseriesofconferencesstartedin1995thatare devotedtoalgorithmsandarchitecturesforparallelprocessing.Asapplications ofcomputingsystemshavepermeatedineveryaspectofdailylife,thepower ofcomputingsystemhasbecomeincreasinglycritical.Thisconferenceprovides aforumforacademicsandpractitionersfromcountriesaroundtheworldto exchangeideasforimprovingtheefficiency,performance,reliability,security, andinteroperabilityofcomputingsystemsandapplications.

Itisourgreathonortointroducetheprogramfortheconference.Thanksto theProgramCommittee’shardwork,wewereabletofinalizethetechnicalprogram.Intheselectionprocess,eachpaperwasassignedtoatleast4PCmembers asreviewers.TheauthorsandthosePCmembersfromthesameinstitutionwere separatedinthereviewingprocesstoavoidconflictsofinterests.Wereceived285 submissionsfromallovertheworld.Thelargenumberofsubmissionsindicated continuedexcitementinthefieldworldwide.Themanuscriptshavebeenranked accordingtotheiroriginalcontribution,quality,presentation,andrelevanceto thethemesoftheconference.Intheend,70(24.56%)paperswereacceptedas themainconferencepapersandinclusionintheconference.

ICA3PP2014obtainedthesupportofmanypeopleandorganizationsaswell asthegeneralchairswhosemainresponsibilitywasvarioustaskscarriedoutby otherwillingandtalentedvolunteers.Wewanttoexpressourappreciationto ProfessorXian-HeSunforacceptingourinvitationtobethekeynote/invited speaker.

WewouldliketogiveourspecialthankstotheprogramchairsoftheconferencefortheirhardandexcellentworkonorganizingtheProgramCommittee, outstandingreviewprocesstoselecthigh-qualitypapers,andmakinganexcellent conferenceprogram.Wearegratefultoallworkshoporganizersfortheirprofessionalexpertiseandexcellenceinorganizingtheattractiveworkshops/symposia, andothercommitteechairs,advisorymembersandPCmembersfortheirgreat support.Weappreciateallauthorswhosubmittedtheirhigh-qualitypapersto themainconferenceandworkshops/symposia.

Wethankallofyouforparticipatinginthisyear’sICA3PP2014conference, andhopeyoufindthisconferencestimulatingandinteresting.

July2014IvanStojmenovic WanleiZhou

Organization

GeneralChairs

IvanStojmenovicOttawaUniversity,Canada

WanleiZhouDeakinUniversity,Australia

ProgramChairs

XianheSunIllinoisInstituteofTechnology,USA WenyuQuDalianMaritimeUniversity,China

PublicityChairs

JaimeLloretMauriPolytechnicUniversityofValencia,Spain

Al-SakibKhanPathanInternationalIslamicUniversityMalaysia, Malaysia

PublicationChair

YangXiangDeakinUniversity,Australia

SteeringCommitteeChairs

AndrzejGoscinskiDeakinUniversity,Australia

YiPanGeorgiaStateUniversity,USA

YangXiangDeakinUniversity,Australia

WorkshopChairs

MianxiongDongNationalInstituteofInformationand CommunicationsTechnology,Japan

LeiLiuShandongUniversity,China

LocalOrganizingChair

ZhiyangLiDalianMaritimeUniversity,China

RegistrationChair

WeijiangLiuDalianMaritimeUniversity,China

FinanceChair

ZhaobinLiuDalianMaritimeUniversity,China

WebChairs

YangShangDalianMaritimeUniversity,China TingtingWangDalianMaritimeUniversity,China

ProgramCommitteeMembers

ZafeiriosPapazachosQueen’sUniversityofBelfast,UK PaoloTrunfioUniversityofCalabria,Italy

Chao-TungYangTunghaiUniversity,Taiwan YongZhaoUniversityofElectronicScienceandTechnology ofChina,China Xingquan(Hill)ZhuFloridaAtlanticUniversity,USA GiandomenicoSpezzanoICAR-CNR,Italy YasuhikoTakenagaTheUniversityofElectro-Communications, Japan

SushilPrasadUniversityofGeorgia,USA TanselOzyerTOBBUniversityofEconomicsand Technology,Turkey DengPanFloridaInternationalUniversity,USA ApostolosPapadopoulosAristotleUniversity ofThessaloniki,Greece EricPardedeLaTrobeUniversity,Australia KarampelasPanagiotisHellenicAmericanUniversity,Greece PaulLuUniversityofAlberta,Canada KameshMadduriPennStateUniversity,USA

Ching-HsienHsuChungHuaUniversity,Taiwan MuhammadKhurramKhanKingSaudUniversity,SaudiArabia MorihiroKugaKumamotoUniversity,Japan WeiweiFangBeijingJiaotongUniversity,China FrancoFrattolilloUniversit`adelSannio,Italy LongxiangGaoDeakinUniversity,Australia JavierGarc´ıaUniversityCarlosIII,Spain MichaelGlassUniversityofErlangen-Nuremberg,Germany DavidE.SinghUniversidadCarlosIIIdeMadrid,Spain MarionOswaldTUWien,Austria RajkumarBuyyaTheUniversityofMelbourne,Australia

Yue-ShanChangNationalTaipeiUniversity,Taiwan ChristianEngelmanOakRidgeNationalLab,USA

AlessioBechiniUniversityofPisa,Italy

HideharuAmanoKeioUniversity,Japan

WeiWeiXi’anUniversityofTechnology,China

ToshihiroYamauchiOkayamaUniversity,Japan

BoYangUniversityofElectronicScienceandTechnology ofChina,China

LaurenceT.YangSt.FrancisXavierUniversity,Canada SheraliZeadallyUniversityoftheDistrictofColumbia,USA

SotiriosG.ZiavrasNJIT,USA

GennaroDellaVecchiaGennaroDellaVecchia-ICAR-CNR,Italy

OlivierTerzoIstitutoSuperioreMarioBoella,Italy

HiroyukiTomiyamaRitsumeikanUniversity,Japan

TomoakiTsumuraNagoyaInstituteofTechnology,Japan

LuisJavierGarc´ıaVillalbaUniversidadComplutensedeMadrid(UCM), Spain

GaocaiWangGuangxiUniversity,China

ChenWangCSIROICTCentre,Australia

MartineWedlakeIBM,USA

WeiXueTsinghuaUniversity,China

EdwinShaUniversityofTexasatDallas,USA

SachinShettyTennesseeStateUniversity,USA

Ching-LungSuNationalYunlinUniversityofScienceand Technology,Taiwan

AnthonySulistioHighPerformanceComputingCenterStuttgart (HLRS),Germany

MagdalenaSzmajduchCracowUniversityofTechnology(CDN PartnerCracow),Poland JieTaoUniversityofKarlsruhe(KarlsruheInstituteof Technology),Germany

DanaPetcuWestUniversityofTimisoara,Romania FlorinPopUniversityPolitehnicaofBucharest,Romania RajeevRajeIndianaUniversity-PurdueUniversity Indianapolis,USA

FrancoiseSailhanCNAM,France

SubhashSainiNASA,USA

ErichSchikutaUniversityofVienna,Austria

AlbaAmatoSecondUniversityofNaples,Italy

CosimoAnglanoUniversit`adelPiemonteOrientale,Italy

LadjelBellatrecheENSMA,France

AteetBhallaOrientalInstituteofScienceandTechnology, India

SurendraBynaLawrenceBerkeleyNationalLab,USA AleksanderByrskiAGHUniversityofScienceandTechnology, Poland

JuanM.MarinUniversityofMurcia,Spain FrancescoMoscatoSecondUniversityofNaples,Italy HirotakaOnoKyushuUniversity,Japan

FabrizioPetriniIBMResearch,USA

StefanoMarroneSecondUniversityofNaples,Italy

AlejandroMasrurTechnologyUniversityofMunich,Germany SusumuMatsumaeSagaUniversity,Japan WeiLuKeeneUniversity,USA

AmitMajumdarSanDiegoSupercomputerCenter,USA

TomasMargalefUniversitatAutonomadeBarcelona,Spain Che-RungLeeNationalTsingHuaUniversity,Taiwan

KeqinLiStateUniversityofNewYorkatNewPaltz, USA

MauroIaconoSecondUniversityofNaples,Italy ShadiIbrahimInria,France

HelenKaratzaAristotleUniversityofThessaloniki,Greece Soo-KyunKimPaiChaiUniversity,Korea EdmundLaiMasseyUniversity,NewZealand KarlFuerlingerLudwig-Maximilians-UniversityMunich, Germany

JoseDanielGarciaUniversityCarlosIIIofMadrid,Spain HaraldGjermundrodUniversityofNicosia,Cyprus HoucineHassanUniversidadPolitecnicadeValencia,Spain Rapha¨elCouturierUniversityofFranche-Comt´e,France

EugenDeduUniversityofFranche-Comt´e,France

CiprianDobreUniversityPolitehnicaofBucharest,Romania MassimoCafaroUniversityofSalento,Italy

Ruay-ShiungChangNationalDongHwaUniversity,Taiwan DanChenUniversityofGeosciences,China

Zizhong(Jeffrey)ChenUniversityofCaliforniaatRiverside,USA JingChenNationalChengKungUniversity,Taiwan CarmelaComitoUniversityofCalabria,Italy YujieXuDalianMaritimeUniversity,China

NatalijaVlajicYorkUniversity,Canada KenjiSaitoKeioUniversity,Japan

ThomasRauberUniversityofBayreuth,Germany

PilarHereroUniversidadPolitecnicadeMadrid,Spain TaniaCerquitelliPolitecnicodiTorino,Italy Tzung-ShiChenNationalUniversityofTainan,Taiwan

DavidExp´ositoUniversityCarlosIII,Spain

PeterStrazdinsTheAustralianNationalUniversity,Australia

UweTangenRuhr-UniversitaetBochum,Germany

LucaTasquierSecondUniversityofNaples,Italy RafaelSantosNationalInstituteforSpaceResearch,Brazil GeorgeBosilcaUniversityofTennessee,USA EsmondNgLawrenceBerkeleyNationalLab,USA LaurentLefevreLaurentLefevre,Inria,UniversityofLyon, France

GiuseppinaCretellaSecondUniversityofNaples,Italy GregoireDanoyUniversityofLuxembourg,Luxembourg BernabeDorronsoroUniversityofLille1,France MassimoFiccoSecondUniversityofNaples,Italy

JorgeBernalBernabeUniversityofMurcia,Spain

TableofContents–PartII

ParallelDataProcessinginDynamicHybridComputingEnvironment UsingMapReduce ................................................ 1 BingTang,HaiwuHe,andGillesFedak

FastScalablek-means++AlgorithmwithMapReduce ................ 15 YujieXu,WenyuQu,ZhiyangLi,ChangqingJi,YuanyuanLi,and YinanWu

AccelerationofSolvingNon-EquilibriumIonizationviaTracerParticles andMapReduceonEulerianMesh 29 JianXiao,XingyuXu,JizhouSun,XinZhou,andLiJi

AContinuousVirtualVector-BasedAlgorithmforMeasuring CardinalityDistribution 43 XuefeiZhou,WeijiangLiu,ZhiyangLi,andWenwenGao

Hmfs:EfficientSupportofSmallFilesProcessingoverHDFS .......... 54 CairongYan,TieLi,YongfengHuang,andYanglanGan

UtilizingMultipleXeonPhiCoprocessorsonOneComputeNode 68 XinnanDong,JunChai,JingYang,MeiWen,NanWu,XingCai, ChunyuanZhang,andZhaoyunChen

HPSO:PrefetchingBasedSchedulingtoImproveDataLocalityfor MapReduceClusters 82 MingmingSun,HangZhuang,XuehaiZhou,KunLu,and ChanglongLi

ServiceSchedulingAlgorithminVehicleEmbeddedMiddleware ........ 96 JuanLuo,XinJin,andFengWu

SimilarSamplesCleaninginSpeculativeMultithreading .............. 108 YuxiangLi,YinliangZhao,andBinLiu

Equi-joinforMultipleDatasetsBasedonTimeCostEvaluation Model 122 HongZhu,LiboXia,MieyiXie,andKeYan

IdentifyingFileSimilarityinLargeDataSetsbyModuloFileLength ... 136 YongtaoZhou,YuhuiDeng,XiaoguangChen,andJunjieXie

Conpy:ConcolicExecutionEngineforPythonApplications ........... 150 TingChen,Xiao-songZhang,Rui-dongChen,BoYang,and YangBai

APlatformforStockMarketSimulationwithDistributedAgent-Based Modeling 164

ChunyuWang,CeYu,HutongWu,XiangChen,YueleiLi,and XiaotaoZhang

C2CU:ACUDACProgramGeneratorforBulkExecutionofa SequentialAlgorithm 178

DaisukeTakafuji,KojiNakano,andYasuakiIto

DynamicallySpawningSpeculativeThreadstoImproveSpeculative PathExecution 192

MeirongLi,YinliangZhao,andYouTao

AParallelAlgorithmofKirchhoffPre-stackDepthMigrationBasedon GPU 207

YidaWang,ChaoLi,YangTian,HaihuaYan,ChanghaiZhao,and JianleiZhang

AnAlgorithmtoEmbedaFamilyofNode-Disjoint3DMeshesinto LocallyTwistedCubes 219 LantaoYouandYuejuanHan

GPUAccelerationofFindingMaximumEigenvalueofPositive Matrices 231

NingTian,LongjiangGuo,ChunyuAi,MeiruiRen,andJinbaoLi

ImprovingSpeculationAccuracywithInter-threadFetchingValue Prediction 245

FanXu,LiShen,ZhiyingWang,HuiGuo,BoSu,andWeiChen

TowardsEfficientDistributedSPARQLQueriesonLinkedData 259 XuejinLi,ZhendongNiu,andChunxiaZhang

MRFS:ADistributedFilesSystemwithGeo-replicatedMetadata 273 JiongyuYu,WeigangWu,DiYang,andNingHuang

AnAdvancedDataRedistributionApproachtoAcceleratethe Scale-DownProcessofRAID-6 286 CongjinDu,ChentaoWu,andJieLi

ThreadMappingandParallelOptimizationforMICHeterogeneous ParallelSystems 300 TaoJu,ZhengdongZhu,YinfengWang,LiangLi,andXiaosheDong

EfficientStorageSupportforReal-TimeNear-DuplicateVideo Retrieval

ZhenhuaNie,YuHua,DanFeng,QiuyuLi,andYuanyuanSun

RepairingMultipleDataLossesbyParallelMax-minTreesBasedon RegeneratingCodesinDistributedStorageSystems

312

325 PengfeiYou,YuxingPeng,ZhenHuang,andChangjianWang

ExploitingContentLocalitytoImprovethePerformanceandReliability ofPhaseChangeMemory ......................................... 339

SuzhenWu,ZaifaXi,BoMao,andHongJiang

Computing,CommunicationandControl TechnologiesinIntelligentTransportationSystem (3CinITS2014)

ApplicationofSupportVectorMachineintheDecision-Makingof Maneuvering ....................................................

352 ZhuangQi,ZhengChang,HanbangSong,andXinyuZhang

MobilePhoneDataRevealtheSpatiotemporalRegularityofHuman Mobility ........................................................ 359

ZihanSun,HanxiaoZhou,JianfengZheng,andYuhaoQin

ResearchonLarge-ScaleVesselRidingTidalCurrenttoPromote EfficiencyofFairway 366 KangZhou,RanDai,andXingwangYue

AVertex-ClusteringAlgorithmBasedontheCluster-Clique ........... 376 DeqiangWang,BinZhang,andKelunWang

DesignedSlideModeControllerforShipAutopilotwithSteeringGear Saturation

386 Gao-Xiaori,Hong-Biguang,Xing-Shengwei,andLi-Tieshan

396 WangDelongandRenHongxiang

BusArrivalTimePredictionandRelease:System,Databaseand AndroidApplicationDesign 404 JunhaoFu,LeiWang,MingyangPan,ZhongyiZuo,andQianYang

OnKeyTechniquesofaRadarRemoteTelemetryandMonitoring System ......................................................... 417 JianglingHao,MingyangPan,DeqiangWang,LiningZhao,and DepengZhao

PSCShip-SelectingModelBasedonImprovedParticleSwarm OptimizationandBPNeuralNetworkAlgorithm

425 TingtingYang,ZhonghuaSun,ShounaWang, ChengmingYang,andBinLin

LRPONBasedInfrastructureLayoutPlanningofBackboneNetworks forMobileCloudServicesinTransportation 436

SongYingge,DongJie,LinBin,andDingNing

InfrastructureDeploymentandDimensioningofRelayed-Based HeterogeneousWirelessAccessNetworksforGreenIntelligent

LinBin,GuoJiamei,HeRongxi,andYangTingting

VesselMotionPatternRecognitionBasedonOne-WayDistanceand SpectralClusteringAlgorithm .....................................

WenyaoMa,ZhaolinWu,JiaxuanYang,andWeifengLi

NavigationSafetyAssessmentofShipinRoughSeasBasedonBayesian Network 470

FengdeQu,FengwuWang,ZongmoYang,andJianSun

OptimizationofShipSchedulingBasedonOne-WayFairway

JunLin,Xin-yuZhang,YongYin,Jin-taoWang,andShunYao ResearchonVirtualCrewPathPlanningSimulatorBasedonA* Algorithm

HuilongHao,HongxiangRen,andDajunChen

DajunChen,HongxiangRen,andHuilongHao

TheAssessmentofRiskofCollisionbetweenTwoShipsAvoiding CollisionbyAlteringCourse ....................................... 507

WeifengLi,WenyaoMa,JiaxuanYang,GuoyouShi,and RobertDesrosiers

TheMergingAlgorithmofRadarSimulationDatainNavigational Simulator .......................................................

ShunYao,Xin-yuZhang,YongYin,XinXiong,andJunLin

LimingXue,ZhihuaiLi,andWeixinLuan

SimulationofMaritimeJointSea-AirSearchTrendUsing3DGIS

XingShengwei,WangRenda,YangXuefeng,andLiuJiandao

QuantitativeAnalysisfortheDevelopmentofMaritimeTransport Efficiency 543

WenboZhang,ZhaolinWu,YongLiu,andZebingLi

SecurityandPrivacyinComputerandNetwork Systems(SPCNS2014)

ImageCompressionBasedonTime-DomainLappedTransformand QuadtreePartition 553

XiuhuaMa,JiwenDong,andLeiWang

TheApplicabilityandSecurityAnalysisofIPv6TunnelTransition Mechanisms ..................................................... 560 WeiMi

QOSPerformanceAnalysisforFlexibleWorkflowSupportingException Handling ........................................................ 571

XiaoyanZhu,JingleZhang,andBoWang

AnalysisofPropagationCharacteristicsofVariantWorms 581 TaoLiu,CanZhang,MingjingCao,andRupingWu

ADesignofNetworkBehavior-BasedMalwareDetectionSystemfor Android 590

YinchengQi,MingjingCao,CanZhang,andRupingWu

DetectionandDefenseTechnologyofBlackholeAttacksinWireless SensorNetwork .................................................. 601 HuishengGao,RupingWu,MingjingCao,andCanZhang

AnImprovedRemoteDataPossessionCheckingProtocolinCloud Storage ......................................................... 611 EnguangZhouandZhoujunLi

FaultLocalizationofConcurrencyBugsandItsApplicationinWeb Security 618 ZhenyuanJiang

FeatureSelectionTowardOptimizingInternetTrafficBehavior Identification 631 ZhenxiangChen,LizhiPeng,ShupengZhao,LeiZhang,and ShanJing

ID-BasedAnonymousMulti-receiverKeyEncapsulationMechanism withSenderAuthentication ....................................... 645 BoZhang,TaoSun,andDairongYu

EnergyEfficientRoutingwith aTree-BasedParticleSwarm OptimizationApproach 659

GuodongWang,HuaWang,andLeiLiu

AContext-AwareFrameworkforSaaSServiceDynamicDiscoveryin Clouds ......................................................... 671

ShaochongLiandHao-pengChen

PortingthePrincetonOceanModeltoGPUs

ShizhenXu,XiaomengHuang,YanZhang,YongHu, HaohuanFu,andGuangwenYang

WebServiceRecommendationviaExploitingTemporalQoS Information .....................................................

ChaoZhou,WancaiZhang,andBoLi

OptimizingandScalingHPCGonTianhe-2:EarlyExperience

XianyiZhang,ChaoYang,FangfangLiu,YiqunLiu,andYutongLu

YichaoCheng,HongAn,ZhitaoChen,FengLi,ZhaohuiWang, XiaJiang,andYiPeng

AGPUImplementationofClipping-FreeHalftoningUsingtheDirect

HiroakiKoge,YasuakiIto,andKojiNakano AReliableandSecureGPU-AssistedFileSystem ....................

Shang-ChiehLin,Yu-ChengLiao,andYarsunHsu

EfficientDetectionofClonedAttacksforLarge-ScaleRFIDSystems 85 XiulongLiu,HengQi,KeqiuLi,JieWu,WeilianXue, GeyongMin,andBinXiao

ProbabilityBasedAlgorithmsforGuaranteeingtheStabilityof RechargeableWirelessSensorNetworks 100 YiyiGao,CeYu,JianXiao,JizhouSun,GuiyuanJiang,and HuiWang

PTASforMinimum k -PathConnectedVertexCover inGrowth-BoundedGraphs ....................................... 114 YanChu,JianxiFan,WenjunLiu,andCheng-KuanLin

ASimpleandEffectiveLongDurationContact-BasedUtilityMetric forMobileOpportunisticNetworking 127 ChyouhwaChen,Wei-ChungTeng,andYu-RenWu

AdaptiveQoSandSecurityforVideoTransmissionoverWireless Networks:ACognitive-BasedApproach ............................. 138 WalidAbdallah,SukkyuLee,HwagnamKim,and NoureddineBoudriga

VirtualNetworkMappingAlgorithminWirelessDataCenter Networks ....................................................... 152 JuanLuo,WenfengHe,KeqinLi,andYalingGuo

AWeightedCentroidBasedTrackingSysteminWirelessSensor Networks 166 HongyangLiu,QianqianRen,LongjiangGuo,JinbaoLi,HuiXu, HuJin,NanWang,andChengjieSong

ASmartphoneLocationIndependentActivityRecognitionMethod BasedontheAngleFeature 179 ChanghaiWang,JianzhongZhang,MengLi,YuanYuan,and YuweiXu

ReliableandEnergyEfficientRoutingAlgorithmforWirelessHART .... 192 QunZhang,FengLi,LeiJu,ZhipingJia,andZhaopengZhang

ADSCP-BasedMethodofQoSClassMappingbetweenWLANand EPSNetwork .................................................... 204 YaoLiu,GangLu,WeiZhang,FenglingCai,andQianKong

HostoSink:ACollaborativeSchedulinginHeterogeneous Environment 214 XiaofeiLiao,XiaobaoXiang,HaiJin,WeiZhang,andFengLu

LoadBalancinginMapReduceBasedonDataLocality 229 YiChen,ZhaobinLiu,TingtingWang,andLuWang

RD-PCA:ATrafficConditionDataImputationMethodBasedon RobustDistance ................................................. 242 XueJinWan,YongDu,andJiongWang

Network-AwareRe-Scheduling:TowardsImprovingNetwork PerformanceofVirtualMachinesinaDataCenter ................... 255 GangyiLuo,ZhuzhongQian,MianxiongDong,KaoruOta,and SangluLu

ANovelPetri-NetBasedResourceConstrainedMulti-project SchedulingMethod ............................................... 270 WenbinHuandHuanWang

InterconnectionNetworkReconstructionforFault-Toleranceof Torus-ConnectedVLSIArray 285 LongtingZhu,JigangWu,GuiyuanJiang,andJizhouSun

AnAntColonyOptimizationAlgorithmforVirtualNetwork Embedding 299 WenjieCao,HuaWang,andLeiLiu

Temperature-AwareSchedulingBasedonDynamicTime-Slice Scaling ......................................................... 310 GangyongJia,YouweiYuan,JianWan,CongfengJiang, XiLi,andDongDai

AnImprovedEnergy-EfficientSchedulingforPrecedenceConstrained TasksinMultiprocessorClusters 323 XinLi,YanhengZhao,YibinLi,LeiJu,andZhipingJia

HierarchicalEventualLeaderElectionforDynamicSystems ........... 338 HuaguanLi,WeigangWu,andYuZhou

EfficientResourceProvisioningforMobileMediaTrafficManagement inaCloudComputingEnvironment 352 MohammadMehediHassan,MuhammadAl-Qurishi, BiaoSong,andAtifAlamri

ACommunityCloudforaReal-TimeFinancialApplicationRequirements,ArchitectureandMechanisms ........................ 364 MarceloDutra OsandGra¸caBressan

StrategiesforEvacuatingfromanAffectedAreawithOneorTwo Groups ......................................................... 378

QiWei,YuanShi,BoJiang,andLijuanWang

ANovelAdaptiveWebServiceSelectionAlgorithmBasedonAnt ColonyOptimizationforDynamicWebServiceComposition ........... 391 DenghuiWang,HaoHuang,andChangshengXie

AnOptimizationVMDeploymentforMaximizingEnergyUtilityin CloudEnvironment ..............................................

JinhaiWang,ChuanheHuang,QinLiu,KaiHe,JingWang, PengLi,andXiaohuaJia

PerformanceEvaluationofLight-WeightedVirtualizationforPaaSin Clouds 415 XuehaiTang,ZhangZhang,MinWang,YifangWang, QingqingFeng,andJizhongHan

AnAccessControlSchemewithDirectCloud-AidedAttribute RevocationUsingVersionKey ..................................... 429 JiaoliShi,ChuanheHuang,JingWang,KaiHe,andJinhaiWang

DaluZhang,XiangJin,DejiangZhou,JianpengWang,andJiaqiZhu

ANear-ExactDefragmentationSchemetoImproveRestorePerformance forCloudBackupSystems 457 RongyuLai,YuHua,DanFeng,WenXia,MinFu,andYifanYang

AMusicRecommendationMethodforLarge-ScaleMusicLibraryona HeterogeneousPlatform ..........................................

YaoZheng,LiminXiao,WenqiTang,andLiRuan

GPU-AcceleratedVerificationoftheCollatz Conjecture 483 TakumiHonda,YasuakiIto,andKojiNakano

ReducingtheInterconnectionLengthfor3DFault-TolerantProcessor Arrays ..........................................................

GuiyuanJiang,JigangWu,JizhouSun,andLongtingZhu

497

FeatureEvaluationforEarlyStageInternetTrafficIdentification 511 LizhiPeng,HongliZhang,BoYang,andYuehuiChen

Hyper-StarGraphs:SomeTopologicalPropertiesandanOptimal NeighbourhoodBroadcastingAlgorithm ............................ 526 F.Zhang,K.Qiu,andJ.S.Kim

CustomizedNetwork-on-ChipforMessageReduction 535 HongweiWang,SiyuLu,YouhuiZhang,GuangwenYang,and WeiminZheng

Athena:AFault-Tolerant,EfficientandApplicableRoutingMechanism forDataCenters ................................................. 549 LijunLyu,JunjieXie,YuhuiDeng,andYongtaoZhou

Performance-AwareDataPlacementinHybridParallelFileSystems .... 563 ShuibingHe,Xian-HeSun,BoFeng,andKunFeng

SecurityAnalysisandProtectionBasedonSmaliInjectionforAndroid Applications ..................................................... 577 JunfengXu,ShoupengLi,andTaoZhang

The1stInternationalWorkshoponEmergingTopics inWirelessandMobileComputing(ETWMC2014)

ANovelKeyManagementSchemeinVANETs 587 GuihuaDuan,YunXiao,RuiJu,andHongSong

DesignandImplementationofNetworkHardDisk ................... 596 HongSong,JialongXu,andXiaoqiangCai

CombiningSupervisedandUnsupervisedLearningforAutomatic AttackSignatureGenerationSystem ............................... 607 LiliYang,JieWang,andPingZhong

TheStudyontheIncreasingStrategyofDetectingMovingTargetin WirelessSensorNetworks 619 JialongXu,ZhigangChen,AnfengLiu,andHongSong

ACRC-BasedLightweightAuthenticationProtocolforEPCglobal Class-1Gen-2Tags ............................................... 632 ZhicaiShi,YongxiangXia,YuZhang,YihanWang,andJianDai

TestCasePrioritizationBasedonGeneticAlgorithmandTest-Points Coverage 644

WeixiangZhang,BoWei,andHuisenDu

SAEP:SimulatedAnnealingBasedEnsembleProjectingMethodfor SolvingConditionalNonlinearOptimalPerturbation 655 ShichengWen,ShijinYuan,BinMu,HongyuLi,andLeiChen

ConvertingPtolemyIIModelsto SpaceExforAppliedVerification ..... 669 ShiweiRan,JinzhiLin,YingWu,JianzhongZhang,andYuweiXu

ResearchonInterestSearchingMechanisminSNSLearning Community ..................................................... 684 RenfengWang,JunpeiLiu,HainingSun,andZhihuaiLi

The5thInternationalWorkshoponIntelligent CommunicationNetworks(IntelNet2014)

ImprovingtheFrequencyAdaptiveCapabilityofHybridImmune DetectorMaturationAlgorithm .................................... 691 JunganChen,ShaoZhongZhang,andDanjiangChen

Cluster-BasedTimeSynchronizationProtocolforWirelessSensor Networks ....................................................... 700 JianZhang,ShipingLin,andDandanLiu

AFastCABACAlgorithmforTransformCoefficientsinHEVC 712 NanaShan,WeiZhou,andZheminDuan

AImprovedPageRankAlgorithmBasedonPageLinkWeight 720 XinshengWang,JianchuMa,KaiyuanBi,andZhihuaiLi

ComputationOffloadingManagementforVehicularAdHocCloud 728 BoLi,YijianPei,HaoWu,ZhiLiu,andHaixiaLiu

AnApproachtoModelComplexBigDataDrivenCyberPhysical Systems ........................................................ 740 LichenZhang

The5thInternationalWorkshoponWireless NetworksandMultimedia(WNM2014)

ReliableTransmissionwithMultipathandRedundancyforWireless MeshNetworks .................................................. 755

WenzeShi,TakeshiIkenaga,DaikiNobayashi,XinchunYin,and YebinXu

CommunityRoamer:ASocial-BasedRoutingAlgorithmin OpportunisticMobileNetworks 768 TieyingZhu,ChengWang,andDandanLiu

ASelf-adaptiveReliablePacketTransmissionSchemeforWireless MeshNetworks .................................................. 781 WenzeShi,TakeshiIkenaga,DaikiNobayashi,XinchunYin,and HuiXu

DistributedEfficientNodeLocalizationinWirelessSensorNetworks UsingtheBacktrackingSearchAlgorithm

AlanOliveiradeS´a,NadiaNedjah,andLuizadeMacedoMourelle UserSpecificQoSandItsApplicationinResourcesSchedulingfor WirelessSystem 809 ChaoHeandRichardD.Gitlin ADistributedStorageModelforSensorNetworks

LeeLuanLing

RelationbetweenIrregularSamplingandEstimatedCovariancefor Closed-LoopTrackingMethod .....................................

Bei-beiMiaoandXue-boJin

ParallelDataProcessinginDynamicHybrid ComputingEnvironmentUsingMapReduce

BingTang1 ,HaiwuHe2 ,andGillesFedak2

1 SchoolofComputerScienceandEngineering, HunanUniversityofScienceandTechnology, Xiangtan411201,China btang@hnust.edu.cn

2 UniversityofLyon,LIPLaboratory, UMRCNRS-ENSLyon-INRIA-UCBLyon5668, 46all´eed’Italie,69364LyonCedex07,France {haiwu.he,gilles.fedak}@inria.fr

Abstract. AnovelMapReducecomputationmodelinhybridcomputingenvironmentcalledHybridMRisproposedinthepaper.Usingthis model,highperformanceclusternodesandheterogeneousdesktopPCs inInternetorIntranetcanbeintegratedtoformahybridcomputingenvironment.Inthisway,thecomputationandstoragecapabilityoflargescaledesktopPCscanbefullyutilizedtoprocesslarge-scaledatasets. HybridMRreliesonahybriddistributedfilesystemcalledHybridDFS, andatime-outmethodhasbeenusedinHybridDFStopreventvolatility ofdesktopPCs,andfilereplicationmechanismisusedtorealizereliable storage.Anewnodepriority-basedfairscheduling(NPBFS)algorithm hasbeendevelopedinHybridMRtoachievebothdatastoragebalance andjobassignmentbalancebyassigningeachnodeaprioritythrough quantifyingCPUspeed,memorysizeandI/Obandwidth.Performance evaluationresultsshowthattheproposedhybridcomputationmodelnot onlyachievesreliableMapReducecomputation,reducestaskresponse timeandimprovestheperformanceofMapReduce,butalsoreducesthe computationcostandachievesagreenercomputingmode.

Keywords: HybridComputingEnvironment,DistributedFileSystem, MapReduce,VolunteerComputing,Fault-tolerance.

1Introduction

Inthepastdecade,DesktopGridandVolunteerComputingSystems(DGVCS’s) havebeenprovedaneffectivesolutiontoprovidescientistswithtensofTeraFLOPSfromhundredsofthousandsofresources.DGVCS’sutilizefreecomputing,networkandstorageresourcesofidledesktopPCsdistributedoverIntranet orInternetenvironmentsforsupportinglarge-scalecomputationandstorage. DGVCS’shavebeenoneofthelargestandmostpowerfuldistributedcomputingsystemsintheworld,offeringahighreturnoninvestmentforapplications

X.-h.Sunetal.(Eds.):ICA3PP2014,PartII,LNCS8631,pp.1–14,2014. c SpringerInternationalPublishingSwitzerland2014

fromawiderangeofscientificdomains,includingcomputationalbiology,climate prediction,andhigh-energyphysics[1][2][9].

MapReduceisanemergingprogrammingmodelfordataintensiveapplication whichwasfirstintroducedbyGooglein2004[4],andhasattractedalotof attentionsrecently.Hadoopisanopen-sourceimplementationofMapReduce, whichiswidelyusedinYahoo,FacebookandAmazon.

Recently,therearesomeotherMapReduceimplementationsthataredesigned forlarge-scaleparalleldataprocessing specializedondesktopgridorvolunteer resourcesinIntranetorInternet,suchasBitDew-MapReduce[12],MOON[8], P2P-MapReduce[10],VMR[3],etc.However,becausethereexiststhecorrelationofvolunteerordesktopfailures,inordertoachievelong-termandsustained highthroughput,MapReduceimplementationsadaptedtovolatiledesktopenvironmentscannotlackthesupportofhighreliableclusternodes.

Tothisend,thispaperpresentsahybridcomputingenvironment,inwhich theclusternodesandthevolunteercomputingnodesareintegrated.Forthishybridcomputingenvironment,weproposeandimplementaMapReduceparallel computationmodelthattakesadvantagesofthecomputingcapabilityofthese twokindsofresourcetoexecutereliableMapReducetasks.

Themainchallengesincludethreeaspects:thefirstishowtodealwithtask failurescausedbyunreliablevolunteercomputingnodefailures,andthesecondishowtostoretheinputdata,theintermediatedataandthefinalresults forMapReduceapplications,andthethirdishowtoachieveMapReducetask scheduling.

Tosolvetheaboveproblems,weproposedHybridMR,anewMapReduce implementationforhybridcomputingenvironment.Similartothedesignof Hadoop,HybridMRisalsodecomposedintotwolayers,namely,datastorage layerandMapReducetaskschedulingandexecutionlayer.First,ahybridstoragesystemcalledHybridDFScomposedofclusternodesandvolunteernodes isimplemented,thenMapReducetaskschedulingisimplemented.Inorderto solvethevolatilityofvolunteernodes,wedesignedandimplementedanode fault-tolerancemechanismbasedonthe“heartbeat”andtime-outmethod.Furthermore,anoptimizedschedulertaking intoaccountperformancedifferences betweenclusternodesandvolunteerdesktopnodesisalsoimplemented.

2BackgroundandRelatedWork

2.1MapReduce

MapReducemodelborrowssomeideasfromfunctionalprogramming.MapReduceapplicationsarebasedonamaster-slavemodel.AMapReducesystem includestwobasiccomputingunits,MapandReduce.TheMapReduceprogrammingmodelallowstheusertodefineaMapfunctionandaReducefunctionto realizelarge-scaledataprocessingandanalyzing.Inthefirststep,inputdataare dividedintochunksanddistributedinadistributedfilesystem,suchasHDFS, GFS.Inthesecondstep,MappernodesapplytheMapfunctiononeachfile chunk.Then,thePartitionphaseachievessplittingthekeysspaceonMapper

node,sothateachReducernodegetsapartofthekeyspace.Thisistypically donebyapplyingahashfunctiontothekeysalthoughprogrammerscandefine theirownpartitionfunction.Thenewdataproducedarecalledtheintermediate results.Inshort,theMapfunctionprocessesa(key,value)pairandreturnsa listofintermediate(key,value)pairs:

DuringtheShufflephase,intermediateresultsaresenttotheircorresponding Reducer.IntheReducephase,ReducernodesapplytheReducefunctionto mergeallintermediatevalueshavingthesameintermediatekey:

Attheend,alltheresultscanbeassembledandsentbacktothemasternode, andthisistheCombinephase.

2.2MapReduceonNon-dedicatedComputingResources

BesidestheoriginalMapReduceimplementationbyGoogle[4],severalother MapReduceimplementationshavebeenrealizedwithinothersystems.SomefocusedonprovidingmoreefficientimplementationsofMapReducecomponents, suchasthescheduler[13]andtheI/Osystem,whileothersfocusedonadapting theMapReducemodeltospecificcomputingenvironments,likeshared-memory systems,graphicsprocessors,multi-coresystems,volunteercomputingenvironmentsandDesktopGrids[12].

BitDew-MapReduceproposedbyTangetal.[12]isspecificallydesignedto supportMapReduceapplicationsinDesktopGrids,andexploitstheBitDew middleware[5],whichisaprogrammableenvironmentforautomaticandtransparentdatamanagementonDesktopGrid,GridandCloud.BitDewreliesona specificsetofmetadatatodrivekeydatamanagementoperations,namelylife cycle,distribution,placement,replicationandfault-tolerancewithahighlevel ofabstraction.

Marozzoetal.[10]proposedP2P-MapReducewhichexploitsapeer-to-peer modeltomanagenodechurn,masterfailures,andjobrecoveryinadecentralized buteffectiveway,soastoprovideamorereliableMapReducemiddlewarethat canbeeffectivelyexploitedindynamicCloudinfrastructures.

AnothersimilarworkisVMR[3],avolunteercomputingsystemabletorun MapReduceapplicationsontopofvolunteerresources,spreadthroughoutthe Internet.VMRleveragesusersbandwidththroughtheuseofinter-clientcommunication,andusesalightweighttaskvalidationmechanism.

AnothersystemthatsharessomeofthekeyideaswithHybridMRisMOON [8].ItisasystemdesignedtosupportMapReducejobsonopportunisticenvironments.ItextendsHadoopwithadaptivetaskanddataschedulingalgorithms toofferreliableMapReduceserviceson ahybridresourcearchitecture.

Therearealsosomeworkaboutusingnodeavailabilitypredictionmethod toenableHadooprunningonunreliableDesktopGridorusingnon-dedicated computingresources[6][7].

3SystemArchitecture

InthissectionwedescribethearchitectureofHybridMR.First,wepresentan overviewofthesystem,thenwefocusonthealgorithmsandimplementation ofthemaincomponentsofHybridMRandwehighlightthemainscheduling algorithm.

3.1GeneralOverview

HybridMRiscomposedofreliableclusternodesandvolatiledesktopPCs,which issimplebuteffective.MapReduceapplicationscanberuninthishybridenvironmenttoanalyzeandprocesslargeamountsofdatasets.Thearchitectureof proposedhybridMapReducecomputingsystemisshowninFig.1.

Fig.1. ArchitectureofhybridMapReducecomputingsystem

AsisshowninFig.1,thesystemisdesignedwithahierarchicalarchitecture. Thetoplayeristheuserlayer,andthemiddlelayeristheservicelayer,and thebottomlayeristheresourcelayer.Fourdifferentservicecomponentsareimplementedinservicelayer,namely,datastorageservice,metadataservice,data schedulerservice,Map/Reducetaskschedulerservice.Resourcelayercontains twotypesofresource:thefirstisreliableclusternodes(ClusterWorkers),and

Upload data

Schedule data/task

Submit task

Task status report

Download final results

Worker

Schedule task/data

Task status report

thesecondislargenumberofunreliablevolunteernodes(DesktopWorkers), whichjointhesysteminavoluntaryway.Thesetwotypesofresourceareboth computingandstorageresources,

SimilartoexistingMapReducesystems,datastoragelayerandMapReduce taskschedulinglayerarealsoseparatedinourproposedmodel.Theproposed modelreliesonahybriddistributedfilesystem,calledHybridDFS,whichcan alsoberunindependentlyasasub-component.HybridDFShassimilarcharacteristicswithHDFSandGFSthatdataarestoredinblock.Thedifferenceis thatHybridDFSdefinestwodifferenttypesofdatastoragenodes,thereliable clusternodesandunreliablevolunteernodes.Tosumup,inourproposedmodel weimplemented:

– ClientNode,providesinterfacetoaccessdataandsubmitjobs;

NameNode,providesmetadataservices;

– DataNode,providesdatastorageservices;

– WorkerNode,providesMap/Reducetaskcomputingservices;

– TrackerNode,providesMap/Reducetaskmonitoringservices.

Amongthem,DataNodeandWorkerNodecanbedeployedinclusternodes orvolunteernodes,whileNameNodeandTrackerNodecanonlybeconfigured inserver.Themainworkingprincipleofthesystemisshownasfollows:

–Step1: ClientNodeuploadsinputdatathatwillbeanalyzedandprocessed toHybridDFS;

–Step2: ClientNodesubmitstask,specifyingthedatastoredinHybridDFS whichwillbeprocessed;

–Step3: ScheduledbydataschedulerandMapReducescheduler,theMap tasksandReducetasksareallocatedtoclusternodesandvolunteernodes. Inthemeanwhile,MapReduceschedulercontrolsthetransmissionofintermediatedata;

–Step4: Clusternodesandvolunteernodesregularlysend“heartbeat”signalstoMapReduceschedulertoreporttaskstatus;

–Step5: Onceallofthetaskshavecompleted,ClientNodecandownload finalresultsfromHybridDFS.

3.2DesignOverviewofHybridDFS

InHybridDFS,eachnodecontributesacertainspacetostorefiles.Aswecansee inFig.2,largefileisfirstseparatedintochunks,thenallchunksstoredindifferent locations.Asvolunteernodesarevolatile,thechunksstoredinvolunteernodes maybecomeunavailable.Therefore,replicationapproachisutilizedtoachieve fault-tolerance.

HybridDFSisdesignedtosupportlargefiles.Afileissplitintooneormore blocksandtheseblocksarestoredinasetofDataNodes.Allblocksinafile exceptthelastblockarethesamesize. Theblocksofafilearereplicatedfor fault-tolerance.Theblocksizeandreplicationfactorareconfigurableperfile,and atypicalblocksizeis64MB.Usersorapplicationscandefinethereplication

Fig.2. TheprincipleoffileseparationandstorageforlargefilesinHybridDFS

factor Rs :Rv atfilecreationtimeinHybridDFS,where Rs meansthenumberof replicasofthefilestoredinclusternodes,and Rv meansthenumberofreplicas ofthefilestoredinvolunteernodes.Forexample,1:2meansstoringonecopyin clusternodes,andtwocopiesinvolunteernodesatthesametime.

Unlikeprevioussystems,HybridDFSdoesn’tdifferentiatetransientfailure frompermanentfailureparticularly[11].Wedefinethreenodestatuses: online, offline,and unconnected,andthestatusmigrationchartisshowninFig.3. Differentwithotherswhichusuallyconsiderthestatus dead,thereisaspecial unconnected status.Thefailuredetectionisachievedbythemethodofperiodicallysynchronization(“heartbeat”).Itusesasimpletimeoutthresholdapproach todetectbothshort-termfailureandlong-termfailure.Wedefinetwothresholdsinthismodel:SynchronizationIntervalTime(SIT)andFailureTimeout Time(FTT).Ifthefailuretimeoutperiodhasexpired,anodefailureisdetected (thatbecomes offline ).Inordertotoleratenodefailures,especiallythevolunteernodefailures,HybridDFSusesa Timeout methodtodetectnodefailures.

Fig.3. Nodestatusmigrationchart

1: Node on 0: Node off

Fig.4. Nodesynchronizationandtimeout-basednodefailuredetectionmethod.The detailedmigrationofthreesituations:1)nodemigrationfrom online to offline ;2)node migrationfrom offline to online ;3)re-join(recover)inashort-timefrom unconnected to online.

Astheresponseof“heartbeat”report,thereplicasoffileblocksaredistributed todifferentvolunteerPCsorclusternodes.

Volatilenodesdeclaretheiravailabilitytothesystemthroughperiodicalsynchronization(anintervalofSIT)withtheserver.Duringeachsynchronization, thevalueofvariable alivetime isupdatedtothecurrenttime.Ifthedifference betweenthevalueofvariable alivetime andthecurrenttimeexceedsFTT,this nodebecomes offline.ThedetailedmigrationofthreesituationsaredemonstratedinFig.4.Thegreendotsstandforperiodicallynodesynchronizationor nodejoiningtothesystem,andthereisalsoaupdatingofthevariable alivetime associatedwitheachgreendo t.Thebluedotsstandfor unconnected,whilethe reddotsstandfor offline whichmeansannodefailureisdetected.Boththeblue dotsandreddotsindicatethatanodesynchronizationisexpected,becausethe nodehasalreadylostthecommunicationwiththeserver.

MapReduceapplicationsdemandadvancedrequirementsforHybridDFS.HybridDFSactsasthedatastoragelayer,whilestoragenodesshouldalsorun MapReducetasks.HybridDFSencapsulatesmethodsandinterfaces,whichallowMapReduceapplicationsgettoknowhowthedataisseparated,andthe physicallocationofblockscanbequeried,andthetasksarescheduledtostoragenodesbyMapReducescheduler.

3.3MapReduceAlgorithmandImplementation

Theclientsubmitsthejobthatspecifiesthedatatobeprocessedwhichhas alreadybeenstoredinHybridDFS.BycallingHybridDFSAPIinterface,data blockingmethodandthephysicalstoragelocationofeachdatablockareobtained.Accordingtothefilereplicationattributes Rs :Rv,onecopyofeachdata blockinchosentorunMaptask.Thelargedatatobeanalyzedandprocessed isdenotedby Data,whichisdividedinto n blocks,andeachblockisdenoted by di .TheintermediateresultsforselectedblockswhenMaptaskcompleted