Organization
Acivs2013wasorganizedbyPoznanUniversityofTechnology,locatedinPoland.
SteeringCommittee
JacquesBlanc-TalonDGA,France
WilfriedPhilipsGhentUniversity/iMinds,Belgium
DanPopescuCSIRO,Australia
PaulScheundersUniversityofAntwerp,Belgium
OrganizingCommittee
ZuzannaDomagaaPoznanUniversityofTechnology,Poland MichalFularzPoznanUniversityofTechnology,Poland MarekKraftPoznanUniversityofTechnology,Poland AdamSchmidtPoznanUniversityofTechnology,Poland KrzysztofWalasPoznanUniversityofTechnology,Poland
ProgramCommittee
AlinAchimUniversityofBristol,UK
HamidAghajanStanfordUniversity,USA
MarcAntoniniUniversitdeNice-SophiaAntipolis,France
MarieBabelInria-IRISA,France
PhilippeBolonUniversityofSavoie,France
DonBoneCannonInformationSystemsResearch, Australia
SalahBourennaneEcoleCentraledeMarseille,France
DanDumitruBurdescuUniversityofCraiova,Romania
JocelynChanussotGrenobleInstituteofTechnology,France
JenniferDavidsonIowaStateUniversity,USA
ArturodelaEscaleraHuesoUniversidadCarlosIIIdeMadrid,Spain
EricDebreuveUniversityofNice-SophiaAntipolis,France
ZuzannaDomagaaPoznanUniversityofTechnology,Poland FrdricDufauxENST,France
MichalFularzPoznanUniversityofTechnology,Poland JrmeGillesUCLA,USA
GeorgyGimel’farbTheUniversityofAuckland,NewZealand MarkkuHauta-KasariUniversityofEasternFinland,Finland
DimitrisIakovidisTechnologicalEducationalInstituteofLamia, Greece
ArtoKaarnaLappeenrantaUniversityofTechnology, Finland
ZoltanKatoUniversityofSzeged,Hungary RonKimmelTechnion,Israel
MarekKraftPoznanUniversityofTechnology,Poland
HamidKrimNorthCarolinaStateUniversity,USA
KennethLamTheHongKongPolytechnicUniversity, SARChina
PatrickLeCalletPolytechNantes/UniversitdeNantes,France
AlessandroLeddaArtesisUniversityCollegeAntwerp,Belgium
GonzaloPajaresMartinsanzUniversidadComplutense,Spain JavierMateosUniversityofGranada,Spain
FabriceMriaudeauUniversitdeBourgogne,France
JeanMeunierUniversitdeMontral,Canada
AdrianMunteanuVrijeUniversiteitBrussel,Belgium FernandoPereiraInstitutoSuperiorTcnico,Portugal
StuartPerryCanonInformationSystemsResearch Australia,Australia
WojciechPieczynskiTELECOMSudParis,France MarcPierrot-DeseillignyIGN,France
AleksandraPizuricaGhentUniversity/iMinds,Belgium
WilliamPuechLIRMM,France
GianniRamponiTriesteUniversity,Italy PaoloRemagninoKingstonUniversity,UK
PatriceRondaoAlfaceAlcatel-LucentBellLabs,Belgium
AdamSchmidtPoznanUniversityofTechnology,Poland MubarakShahUniversityofCentralFlorida,USA
AndrzejSluzekKhalifaUniversity,UnitedArabEmirates
HuguesTalbotESIEE,France
MarcVanDroogenbroeckUniversityofLige,Belgium
PeterVeelaertGhentUniversity/iMinds,Belgium
NicoleVincentUniversitParisDescartes,France
KrzysztofWalasPoznanUniversityofTechnology,Poland GeraldZaunerFachhochschuleOber¨osterreich,Austria PavelZemcikBrnoUniversityofTechnology,CzechRepublic DjemelZiouSherbrookeUniversity,Canada
Reviewers
AlinAchimUniversityofBristol,UK
HamidAghajanStanfordUniversity,USA
MarieBabelInria-IRISA,France
JacquesBlanc-TalonDGA,France
NyanBoBoGentUniversity/iMinds,Belgium
PhilippeBolonUniversityofSavoie,France
DonBoneCannonInformationSystemsResearch, Australia
SalahBourennaneEcoleCentraledeMarseille,France
DanDumitruBurdescuUniversityofCraiova,Romania
JocelynChanussotGrenobleInstituteofTechnology,France
ThierryChateauInstitutPascal,France
GabrielaCsurkaXeroxResearchCentreEurope,France
BoguslawCyganekAGHUniversityofScienceandTechnology, Poland
EmmanuelD’AngeloAdvancedSiliconS.A.,Switzerland
ArturodelaEscaleraHuesoUniversidadCarlosIIIdeMadrid,Spain
EricDebreuveUniversityofNice-SophiaAntipolis,France
IvanaDespotovicGhentUniversity/iMinds,Belgium
SeverineDubuissonLaboratoired’InformatiquedeParis6,France
FrdricDufauxENST,France
JrmeGillesUCLA,USA
GeorgyGimel’farbTheUniversityofAuckland,NewZealand BartGoossensGhentUniversity/iMinds,Belgium
SebastianGruenwedelGhentUniversity,Belgium
MarkkuHauta-KasariUniversityofEasternFinland,Finland
DimitrisIakovidisTechnologicalEducationalInstituteofLamia, Greece
ArtoKaarnaLappeenrantaUniversityofTechnology, Finland
RichardKleihorstXetalandGhentUniversity,Belgium
MarekKraftPoznanUniversityofTechnology,Poland
KennethLamTheHongKongPolytechnicUniversity, SARChina
PatrickLeCalletPolytechNantes/UniversitdeNantes,France
AlessandroLeddaArtesisUniversityCollegeAntwerp,Belgium
DominiqueLuzeauxDGA,France
HenriMaitreTelecomParisTech,France
AntoineManzaneraENSTAParisTech,France
GonzaloPajaresMartinsanzUniversidadComplutense,Spain
JavierMateosUniversityofGranada,Spain
JeanMeunierUniversitdeMontral,Canada
AdrianMunteanuVrijeUniversiteitBrussel,Belgium
SergioOrjuelaVargasGhentUniversity,Belgium
FernandoPereiraInstitutoSuperiorTcnico,Portugal
StuartPerryCanonInformationSystemsResearch Australia,Australia
WilfriedPhilipsGhentUniversity/iMinds,Belgium
AleksandraPizuricaGhentUniversity/iMinds,Belgium
DanPopescuCSIRO,Australia
GianniRamponiTriesteUniversity,Italy PatriceRondaoAlfaceAlcatel-LucentBellLabs,Belgium PaulScheundersUniversityofAntwerp,Belgium AdamSchmidtPoznanUniversityofTechnology,Poland MubarakShahUniversityofCentralFlorida,USA AndrzejSluzekKhalifaUniversity,UnitedArabEmirates HuguesTalbotESIEE,France
GuyThoonenUniversityofAntwerp,Belgium MarcVanDroogenbroeckUniversityofLige,Belgium DavidVanHammeGhentUniversity/iMinds,Belgium PeterVeelaertGhentUniversity/iMinds,Belgium NicoleVincentUniversitParisDescartes,France KrzysztofWalasPoznanUniversityofTechnology,Poland GeraldZaunerFachhochschuleOber¨osterreich,Austria PavelZemcikBrnoUniversityofTechnology,CzechRepublic DjemelZiouSherbrookeUniversity,Canada WitoldZorskiCyberneticsFaculty,MilitaryUniversity ofTechnology,Poland
Acquisition,Pre-processingandCoding
EfficientLowComplexitySVCVideoTransraterwithSpatial Scalability 1 ChristopheDeknudt,Fran¸cois-XavierCoudoux,PatrickCorlay, MarcGazalet,andMohamedGharbi
JanuszCichowski,AndrzejCzy˙zewski,andBo˙zenaKostek
Mart´ınMontalvo,Jos´eM.Guerrero,JuanRomeo,Mar´ıaGuijarro, Jes´usM.delaCruz,andGonzaloPajares
KalyanKumarHalder,MuratTahtali,andSreenathaG.Anavatti
YasirSalih,AamirSaeedMalik,NicolasWalter,D´esir´eSidib´e, NaufalSaad,andFabriceMeriaudeau
Florina-CristinaCalnegru
Biometry
Real-TimeFacePoseEstimationinChallengingEnvironments ......... 114 MlikiHazar,HammamiMohamed,andBen-AbdallahHanˆene
HumanMotionCaptureUsingDataFusionofMultipleSkeleton Data 126
Jean-ThomasMasse,Fr´ed´ericLerasle,MichelDevy,Andr´eMonin, OlivierLefebvre,andSt´ephaneMas
RecognizingConversationalInteractionBasedon3DHumanPose 138 JingjingDeng,XianghuaXie,BenDaubney,HuiFang,and PhilW.Grant
Upper-BodyPoseEstimationUsingGeodesicDistances andSkin-Color 150
SebastianHandrichandAyoubAl-Hamadi
ANewApproachforHandAugmentationBasedonPatchModelling 162 OmerRashidAhmadandAyoubAl-Hamadi
HiddenMarkovModelsforModelingOccurrenceOrderofFacial TemporalDynamics 172 KhadoudjaGhanem
AdaptiveTwoPhaseSparseRepresentationClassifierforFace Recognition ..................................................... 182
FadiDornaika,YoussofElTraboulsi,andAmmarAssoum
AutomaticUser-SpecificAvatarParametrisationandEmotion Mapping ........................................................ 192
StephanieBehrens,AyoubAl-Hamadi,RobertNiese,and EickeRedweik
ClassificationandRecognition
OptimizingContextual-BasedOptimum-ForestClassification throughSwarmIntelligence ....................................... 203
DanielOsaku,RodrigoNakamura,Jo˜aoPapa,AlexandreLevada, F´abioCappabianco,andAlexandreFalc˜ao
AMobileImagingSystemforMedicalDiagnostics ................... 215 SamiVarjoandJariHannuksela
FastRoadNetworkExtractionfromRemotelySensedImages .......... 227 VladimirA.KrylovandJamesD.B.Nelson
PartialNear-DuplicateDetectioninRandomImagesbyaCombination ofDetectors .....................................................
Andrzej ´ Sluzek
ObjectRecognitionandModelingUsingSIFTFeatures 250 AlessandroBruno,LucaGreco,andMarcoLaCascia
PaintingSceneRecognitionUsingHomogenousShapes 262 RazvanGeorgeCondorovici,CorneliuFlorea,andConstantinVertan
ANovelGraphBasedClusteringTechniqueforHybridSegmentation ofMulti-spectralRemotelySensedImages ...........................
BiplabBanerjee,PradeepKumarMishra,SurenderVarma,and BuddhirajuKrishnaMohan
Depth,3DandTracking
PlanarSegmentationbyTime-of-FlightCameras .....................
RudiPenne,LucMertens,andBartRibbens
AnEfficientNormal-ErrorIterativeAlgorithmforLineTriangulation ... 298 QiangZhang,YanWu,MingLiu,andLichengJiao
MovingObjectDetectionSysteminAerialVideoSurveillance 310 AhlemWalha,AliWali,andAdelM.Alimi
AnIndoorRGB-DDatasetfortheEvaluationofRobotNavigation Algorithms 321 AdamSchmidt,MichalFularz,MarekKraft,AndrzejKasi´nski,and MichalNowicki
Real-TimeDepthMapBasedPeopleCounting ....................... 330 FrantiˇsekGalˇc´ıkandRadoslavGargal´ık
TrackingofaHandheldUltrasonicSensorforCorrosionControl onPipeSegmentSurfaces .........................................
ChristianBendicks,ErikLilienblum,ChristianFreye,and AyoubAl-Hamadi
NizarK.SallemandMichelDevy
EfficientImplementationsandFrameworks
AResourceAllocationFrameworkforAdaptiveSelectionofPoint MatchingStrategies 366 QuentinDeNeyerandChristopheDeVleeschouwer
VTApi:AnEfficientFrameworkforComputerVisionDataManagement andAnalytics
PetrChmelar,MartinPesek,TomasVolf,JaroslavZendulka,and VojtechFroml
ComputationalMethodsforSelectiveAcquisitionofDepth Measurements:AnExperimentalEvaluation
389 PierrePayeur,PhillipCurtis,andAna-MariaCretu
ANewColorImageDatabaseTID2013:InnovationsandResults
402 NikolayPonomarenko,OlegIeremeiev,VladimirLukin,LinaJin, KarenEgiazarian,JaakkoAstola,BenoitVozel,KacemChehdi, MarcoCarli,FedericaBattisti,andC.-C.JayKuo
PerformanceEvaluationofVideoAnalyticsforSurveillanceOn-Board Trains 414
ValentinaCasola,MarianaEsposito,FrancescoFlammini, NicolaMazzocca,andConcettaPragliola
GPU-AcceleratedHumanMotionTrackingUsingParticleFilter
BoguslawRymut,BogdanKwolek,andTomaszKrzeszowski
LowLevelImageAnalysisandSegmentation
ModellingLineandEdgeFeaturesUsingHigher-OrderRiesz
RossMarchantandPaulJackway
SemanticApproachinImageChangeDetection
AdrienGressin,NicoleVincent,Cl´ementMallet,and NicolasPaparoditis
SmallTargetDetectionImprovementinHyperspectralImage 460 TaoLin,JulienMarot,andSalahBourennane
TheObjectiveEvaluationofImageObjectSegmentationQuality 470 RanShi,KingNgiNgan,andSongnanLi
AModificationofDiffusionDistanceforClusteringandImage Segmentation 480 EduardSojkaandJanGaura
FlexibleMulti-modalGraph-BasedSegmentation ....................
WillemP.Sanberg,LuatDo,andPeterH.N.deWith
TheDivideandSegmentMethodforParallelImageSegmentation
504 ThalesSehnK¨orting,EmilianoFerreiraCastejon,and LeilaMariaGarciaFonseca
UnsupervisedSegmentationforTransmissionImagingofCarbon Black ........................................................... 516 LydieLuengo,H´el`eneLaurent,SylvieTreuillet,IsabelleJolivet,and EmmanuelGomez
TreeSymbolsDetectionforGreenSpaceEstimation 526
AdrianSrokaandMarcinLuckner
HierarchicalLayeredMeanShiftMethods 538
Milan ˇ Surkala,KarelMozdˇreˇn,RadovanFusek,andEduardSojka
GloballySegmentationUsingActiveContoursandBeliefFunction ..... 546 FouedDerraz,MiloudBoussahla,andLaurentPeyrodie
VideoAnalytics
AutomaticMonitoringofPigActivityUsingImageAnalysis ........... 555 MohammadAminKashiha,ClaudiaBahr,SanneOtt, ChristelP.H.Moons,TheoA.Niewold,FrankTuyttens,and DanielBerckmans
IMM-BasedTrackingandLatencyControlwithOff-the-ShelfIPPTZ Camera ......................................................... 564
PierrickPaillet,RomaricAudigier,FredericLerasle,and Quoc-CuongPham
EvaluationofTrafficSignRecognitionMethodsTrained onSyntheticallyGeneratedData 576 BorisMoiseev,ArtemKonev,AlexanderChigorin,and AntonKonushin
RobustMulti-cameraPeopleTrackingUsingMaximumLikelihood Estimation ......................................................
584 NyanBoBo,PeterVanHese,SebastianGruenwedel, JunzhiGuan,JorgeNi˜no-Casta˜neda,DirkVanHaerenborgh, DimitriVanCauwelaert,PeterVeelaert,andWilfriedPhilips
APerception-BasedInterpretationoftheKernel-BasedObject Tracking ........................................................ 596 VittoriaBruniandDomenicoVitulano
EfficientDetectionandTrackingofRoadSignsBasedonVehicle MotionandStereoVision 608 Chang-WonChoi,Sung-InChoi,andSoon-YongPark
IncrementalPrincipalComponentAnalysis-BasedSparse RepresentationforFacePoseClassification 620 YuyaoZhang,Y.Benhamza,KhalidIdrissi,andChristopheGarcia
PersonDetectionwithaComputationTimeWeightedAdaBoost .......
632 AlhayatAliMekonnen,Fr´ed´ericLerasle,andArianeHerbulot
PerspectiveMultiscaleDetectionofVehiclesforReal-TimeForward CollisionAvoidanceSystems 645
JuanDiegoOrtega,MarcosNieto,AndoniCortes,andJulianFlorez
LearningandPropagationofDominantColorsforFastVideo Segmentation .................................................... 657
C´edricVerleysenandChristopheDeVleeschouwer
AKey-PoseSimilarityAlgorithmforMotionDataRetrieval 669
JanSedmidubsky,JakubValcik,andPavelZezula
TrainingwithCorruptedLabelstoReinforceaProbablyCorrect TeamsportPlayerDetector 682
PascalineParisot,BerkSevilmi¸s,andChristopheDeVleeschouwer
SphericalCenter-SurroundforVideoSaliencyDetectionUsingSparse Sampling ....................................................... 695
HamedRezazadeganTavakoli,EsaRahtu,andJanneHeikkil¨ a
SemanticConceptDetectionUsingDenseCodewordMotion 705 ClaudiuT˘anaseandBernardM´erialdo
ChristopheDeknudt,Fran¸cois-XavierCoudoux,PatrickCorlay,MarcGazalet, andMohamedGharbi
I.E.M.N.,OAEDepartment,UMR8520,UniversityofValenciennes LeMontHouy59313ValenciennesCedex9,France christophe.deknudt@gmail.com {francois-xavier.coudoux,patrick.corlay, marc.gazalet,mohamed.gharbi}@univ-valenciennes.fr http://www.univ-valenciennes.fr/DOAE/index-doae
Abstract. InthispaperweproposeanewH.264SVCtransratingarchitectureforspatiallyscalableSVCcompressedvideostreams.Thealgorithmislowcomplexitybased,itappliestospatiallyscalablepre-encoded videostreamsandallowsfinebitrategranularitywhilekeepinghighest spatialresolution.Simulationresultsdemonstratethattranscodedbit streamsproducesatisfyingpicturequalityevenatbitratereductionup to66%.ThecomparisonwithMGScompressedvideostreamsshows thattheproposedtransratingaslgorithmofferssatisfyingperformances comparedtoMGSwhenbitratereductionremainslimited.Moreover qualityscalabilityisobtainedthankstoouralgorithmeveniftheSVC compressedvideobitstreamhasnotbeenprocessedusingMGSscalabilityrightfromthestart1 .
Keywords: Videocompression,H.264,SVC,SpatialScalability, Transcoding,MediumGrainScalability(MGS).
1Introduction
Severalmultimediaservicessuchasvideoondemand(VoD)andvideostreamingarebasedonthewideuseofpre-encodedvideostreams[1].ScalableVideo Coding(SVC)constitutesanattractivesolutioninordertopre-encodevideo contentswiththepossibilitytofurtheraccesstoavarietyofformats,temporal resolutionand/orqualitylevels[2]-[3].Consequently,thenumberofavailable scalablevideocontentsisexpectedtowidelyincreaseinthenextfewyearsgiven thatstate of the artH.264/AVCcompressionstandardintegratescurrentlya
1 ThisworkwassupportedinpartbytheFrenchNationalResearchAgencythrough theTOSCANEproject.
C.DeknudtiswithSoftthinks,VilleneuvedAscq,France.
F.-X.Coudoux,P.CorlayM.GazaletandM.GharbiarewithInstitutdElectronique, deMicrolectroniqueetdeNanotechnologie.
J.Blanc-Talonetal.(Eds.):ACIVS2013,LNCS8192,pp.1–12,2013. c SpringerInternationalPublishingSwitzerland2013
scalableextensionwhileascalableversionofthenewemergingHEVCstandardisunderdevelopment.Theso-calledspatial,temporal,andso-calledSNR scalabilitiescantheoreticallybecombinedtogetherinordertoaccountforthe widevarietyofnewdigitalimageformats,frameratesanddeliverynetworks. Inthiscase,however,theSVCencodingprocessbecomesverycomplexandcan leadtoasignificantlossincodingefficiencycomparedtosingle-layercoding. Whenpre-encodedstreams aretransmitted,itcanb ecomenecessarytoreduce theinitialbitrateofthecompressedbitstreamaccordingtotheavailablebandwidthoruserdemand.Inthiscase,bitrateconversionalsocalledtransrating mayberequiredasanalternativesolutiontotheinherentscalabilityofferedby SVC[4].TherehasbeenalotofrecentscientificworkonH.264/AVCandSVC transratingalsocalleddynamicshaping[5]-[6]-[7].Inparticular,VandeWalle etal.havebeenveryactiveinthisresearchfieldandhaveproposedseveral originalcontributions.In[8],theyproposedalowcomplexitySNRtranscodingforH.264/AVCcompressedstreamswhichisbasedonrequantization.In [9],theypresentanefficientarchitectureforH.264/AVCtoSNRscalableSVC fasttranscoding.Itshouldbenoticedthatthepresentauthorshavealsoproposedinapreviousworkalowcomplexity transratingarchitecturebasedonfrequencyselectivityforH.264/AVCvideobitstreamswithoutdriftforintra-coded frames[10].
Inthispapertheprevioustransratingarchitectureisextendedtothecaseof SVCpre-encodedvideostreamswithspatialscalabilityonly,i.e.SNRscalabilityhasnotbeenconsideredatfirstduringtheencodingstage.Inthiscase,the so-calledbaselayercorrespondstothelowestspatialresolution,andtheneach supplementaryenhancementlayerallowsincreasingthespatialresolution.Basically,bitratereductionofspatiallyscalableencodedvideostreamsismadepossiblebyremovingonespatialenhancementlayeratatimefromthecompressed bitstream.Unfortunately,suchrateadaptationleadstoverycoarsegranularity hencepoorflexibilityandefficiency.Moreover,spatialresolutiondecreaseseach timethebitrateisreduced.
WeproposeanewapproachforefficientSVCtransratingofspatiallyscalable compressedvideostreams.Oursolutionpermittoobtainafinerbitrategranularity.Toourknowledge,thispaperpresentsthefirstsolutionfortransratingofSVC compressedvideostreamssupportingspatialscalabilityonly.Inwhatfollows,we onlydescribetheproposedalgorithmbasedonsimpledyadicspatialscalability withtwolayers.However,thesolutioncanbeextendedtoarbitraryresolution factorsandmultipleenhancementlayers.Theproposedalgorithmconsistsinremovingselectivelyresidualtransform coefficientsfromtheenhancementlayerof highestspatialresolution.Inordertoguaranteerealtimeprocessing,itshould beoflowcomputationalcomplexitycomparedtoatraditionalfulldecode-full recodeapproach.Hence,thetransratingsolutionisbasedonopen-looparchitecture.Twosolutionsareproposedbasedonathoroughanalysisofmacroblock encodingmodes:first,theselectionoftransformcoefficientsisappliedtoall macroblocksofthehighestenhancementlayergiventhattheencodingprocess ismodifiedsuchthatintra-layerintra-predictionisforbidden.Then,thesecond
EfficientLowComplexitySVCVideoTransraterwithSpatialScalability3
algorithmislimitedtomacroblocksencodedbymeansofinter-frameprediction withnorestrictionofthecodingprocess.Simulationresultsshowthatbothalgorithmsofferfinerategranularity,withverysatisfyingvisualqualityofthe transratedvideosequencefreeofseveredrifterrordistortionwhilepreserving highestspatialresolution.
Thepaperisorganizedasfollows.InSection2,wegiveabriefoverviewof SVCspatialscalabilityandthecorrespondingencodingtools.Then,thetransratingsolutionsbasedonfrequencyselectivityarepresentedindetail.Simulation resultsaregiveninSection4.AcomparisonwithacombinedhybridMedium GranularityScalability(MGS)/spatialscalabilityapproachisproposed.Weshow thattheproposedsolutionprovidesgoodresultsformoderatebitratereductions.Moreover,thehybridMGS/spatialscalabilitysolutionintroducesabitrate increaseand,aboveall,thissolutionmustbeinitiallyplannedattheencoding stagewhiletheproposedtransratingschemeallowsbitrate/qualityadaptation ofaSVCbitstreamevenifthisstreamonlysupportsspatialscalability.Finally, Section5givestheconclusions.
2OverviewofSVCSpatialScalability
Fig.1illustratestheconceptofspatialscalabilitywithtwospatiallayers.Inthis case,theSVCcodingstructureisorganizedintwodependencylayers:abase layernotedhereLayer0,andanenhancementlayercorrespondingtoresidual dataofhigherspatialresolutionnotedhereLayer1.
Fig.1. IllustrationofSVCspatialscalabilitywithtwolayers:thedifferentencoding modesarenoted(a)to(d)
Foreachdependencylayer,thebasicconceptsofmotion-compensatedpredictionandintra-prediction(solidarrows)areusedasinsingle-layercoding. Classically,theintra,predicted,andbidirectionalpicturesarenotedI,P,andB, respectively.Additionally,thesocalledinter-layerprediction(dottedanddashed arrows)isintroducedinSVCtoexploitredundancybetweendependencylayers andhenceincreasecodingefficiency[11].Ifweconsideronemacroblockbelonging totheenhancementlayer,differentencodingmodescanbeencounteredwhich arenoted(a)to(d)inFig.1:
(a) Intra-layerintra-prediction :intra-predictionisappliedonneighbouringblocks fromtheenhancementlayer;
4C.Deknudtetal.
(b) Motion-compensatedintra-layerprediction :inthiscase,previousand/orfutureframesusedformotionestimationbelongtotheenhancementlayer;
(c) Inter-layerintra-prediction :inthiscase,knownasINTRA BLprediction mode,thecorrespondingblockinthebaselayerisup-sampledpriortobe usedaspredictionsignal;
(d) Motion-compensatedinter-layerprediction :previousand/orfutureframes usedformotionestimationbelongtothebaselayer.
In[12],itwasbeenshownthatthetransratingprocessisstronglydependent ofthemacroblocktype.InordertoprovideastatisticalanalysisofthemacroblockmodesusedinSVCcompressedstreams,thewell-known City, Crew, Mobcal, Harbour and Soccer sequenceshavebeenencodedusingtheJSVM9.15 referencesoftwarealgorithmwithspatialscalability.HerethebaselayerconsistsintheCIF-resolutionsequence, whiletheenhancementlayerconsistsin the4CIF-resolutionversion.Differentquantizationparametersareused,namely QP=18,24,30and36,andtheIBBBPBBBPBBBIhierarchicalGoPstructure isconsidered.ThesameQPvalueisusedforbothbaseandenhancementlayer. Table1givesstatisticsofthemacroblo cktypesintheenhancementlayerslices asafunctionoftheframetype,namelyI,PorB,forallsequences.
Table1. Distribution(in%)ofmacroblocktypesinenhancementlayerslices
AllSlices Types IntraSlicesPredicted Slices Bidirect. Slices Skipped21-1625 Baselayermode26-2521
Intra16×16629220 Predicted6-37Bidirectional41--54 IntraBaselayerMode-71--
Weverifylogicallythatbidirectionalmacroblocksarethemostencountered sinceBslicesarepredominantlypresentintheconsideredhierarchicalGoPstructure.Thebaselayermodemacroblocksareencodedusinginter-layerprediction whatevertheintra-frameorinter-framecodingtype.Themacroblocksencoded usingintra-frameareeitheroftypeINTRA 16×16(6%,intra-layerprediction) orINTRA BLtype(includedinthe26%baselayermode,inter-layerprediction).WenotethatwiththisJSVMencoderrevision,themacroblocksusing intra-layerintra-frameprediction(intrapredictionusedinH.264/AVC)areonly ofINTRA 16×16type.WeverifythatINTRA 16×16macroblocksusingintraframeintra-layerpredictioncorrespondto29%inIslices,22%inPslices,and 0%inBslices,respectively.
InthenextSection,wewillfirstproposetorestricttheintra-frameprediction onlytotheINTRA BLmacroblocktypeinordertoavoiddrifterror.Wediscuss theimpactofthislimitationonbothbitrateandvideoquality.
3DescriptionoftheTwoProposedAlgorithms
Fig.2givesthegeneralblockdiagramoftheproposedtransratingarchitecture inthespecificcaseofthreelayers:onebaselayer,andtwospatialenhancement layers.
Fig.2. BlockdiagramoftheproposedSVCtransratingarchitecturewithspatial scalability
Obviously,theproposedarchitecturecanbeappliedtoaSVCcompressedbit streamwithspatialscalabilityandanynumberofenhancementlayers.
Theproposedsolutionisbasedonfrequencyselectivity:inthepresentcase,it consistsinselectivelyremovingresidua ltransformcoefficientsfrommacroblocks ofthehighestenhancementlayer,theotherlayersbeingleftuntouched.Hence, lowfrequencycoefficientsthathavesignificantvisualinfluenceareleftunchanged andkeptinazigzagorderuntilagivenfrequencyposition(FP)whichvaries from1(onlytheDCcoefficientremains)to16(allcoefficientsarepreserved). Practically,theFPparameterusedtotransrateintra-codedblocks(notedhereafterFPintra )canbechosenindependentoftheoneusedtotransrateinter-coded blocks(notedhereafterFPinter )inordertoincreasethetransraterflexibility.
Unfortunately,duetothewideuseofpredictivecodingbySVC,suchcoefficientremovalleadstoso-calleddrifterrorinthereconstructedvideosequence. ConsideringspatialscalableSVC,drifterrormayappearintoasingleframe,a groupofpictures,oralayer.Ithasbeenshownin[13]thatdrifterrorremains perceptuallynegligibleinthecaseofinter-prediction.Itistrueforawidevariety ofvideocontentssequenceswithamoderatemotion.
Also,weverifyonthebasisofsubjectiveevaluationofreconstructedsequences thatdrifterrorisacceptablewhendealingwithINTRA BLmacroblocks,asthe baselayerwhichservesasreferenceisleftuntouched.However,aproblemarises whenconsideringintra-layerintra-prediction.Inthiscase,theremovaloftransformcoefficientsleadstodrifterrorwhichseverelydegradesthereconstructed videosequence.
Hence,itisnecessarytospecificallyadaptthetransratingprocesstothis macroblocktype.Twosolutionshavebeenproposed:
1.ThefirstsolutionneedstoslightlymodifythespatialscalableSVCencoder, buttheresultingspatialscalableSVCbitstreamremainsfullycompatible
withanyspatialscalableSVCdecoder.Suchmodificationispossiblewhen consideringthatthevideostreamsarepre-encoded,likeinVODorstreaming:inthiscase,videoserviceprovidershasaperfectcommandoftheencoder andcanthereforerestricttheproprietaryspatialscalableSVCencodertoa limitednumberofauthorizedmacroblo cktypesifnecessary.Consequently, weproposetomaketheintra-layerintra-predictionmacroblocktypeunauthorizedduringthespatialscalableSVCencodingprocess.Weverifythat thisrestrictionleadstoanegligibleaveragerateoverheadofabout1.5%. Suchoverheadisclearlyacceptableasinreturn,thetransratingprocessis simplifiedbyindistinctlyapplyingfrequencyselectivitytoanymacroblock typeinthehighestenhancementlayer.
2.Becausedrifterrorismainlyvisuallyannoyinginthecaseofintra-prediction, weproposeinthesecondsolutiontoapplyfrequencyselectiononlyto macroblocksencodedbymeansofinter-prediction.Thislimitationslightly reducestheperformanceofthetransratingalgorithmbutoffersthegreatadvantagethatthetransratingalgorithmisoflowercomputationalcomplexity andiscompatiblewithanyspatialscalableSVC-compressedvideostream.
InthefollowingSection,wedemonstratetheperformancesofthetwosolutions intermsofbitratereductionaswellasreconstructedvideoquality.
4SimulationResults
Extensivesimulationswereperformedinordertoevaluatethetwotransrating architectures.Thewell-known City, Crew, Harbour, Soccer and Mobcal sequences havebeenencodedusingtheJSVM9.15referencesoftwarealgorithmwithspatial scalability.Thesesequenceshavebeenchosenbecausetheyofferawiderange ofspatialandtemporalcomplexitycharacteristics.Asmentionedpreviously,the baselayerconsistsintheCIF-resolutionsequence,whiletheenhancementlayer consistsinthe4CIF-resolutionversion.Differentquantizationparameterswere used,namelyQP=18,24,30and36,andaIBBBPBBBPBBBIhierarchicalGoP structurewasconsidered.Targetedbit ratevaluesareachievedbymodifying QPsettings.
Inafirstapproach,theFPintra andFPinter transratingparametersarechosen equal,i.eFPintra =FPinter =FP.However,itshouldbenotedthatFPintra and FPinter canbechosendifferentleadingtheoreticallyto256possibleintermediate bitrates,evenifsomeofthe(FPintra ,FPinter )combinationsmightresultinthe samebitrate.Weusethewell-knownpeaksignal-to-noiseratio(PSNR)inorder toevaluatethevideoqualitybecauseitr emainsnowadaysthemostwidelyused objectivemetricinthevideocommunity.Resultsaregivenherefortheluminance component;similarresultswereobtainedforthechrominancecomponents.
First,weconsiderthetransratingsolutionforwhichintra-predictionislimitedtotheINTRA BLmodeonlyinthehighestenhancementlayer.Remember thatthissolutionrequiresthemodificationofthespatialscalableSVCencoding process.Fig.3givestherate-distortionperformancesofthefirstspatialscalable
SVCtransratingarchitecturewithspatialscalabilityforthe City sequence.Similarresultswereobtainedwithothersequences.Onlytheresultsobtainedfor FPvaryingfrom1to10aregivenforclarityofthefigure.ForeachQPvalue, twoanchorsarealsogivenforcomparison:
– OnehighanchorlocatedontherightcorrespondingtothePSNRandcorrespondingbitratevalueswhenbothbaseandenhancementlayersareavailable(thiscouldbeconsideredasthespecialcase:FP=16,fullquality); –
Onelowanchorlocatedonbottom-leftwhichcorrespondstothecasewhen onlytheCIFbaselayerisavailableandup-scaledto4CIFspatialresolution usingtheup-samplingfilterdescribedin[14].
Fig.3. SVCtransratingoftheupperspatialenhancementlayer:PSNRasafunction ofbitrate- City sequence
Initially,onlythesetwoversions,i.e.onlythesetwobitrates/qualitylevels, areavailablethankstospatialscalability.Wecannotethatmaximalbitrate reductionvalueislogicallyreducedwhenthequantizationparameterincreases. Indeed,thestrongerthequantization,thelessthenumberofremainingresidual coefficients.However,themaximalbitratereductionisverysignificantandvaries upto66%(City sequence,QP=18andFP=1).ItshouldbenotedthateachFP parametervalueleadstoanintermediatebitratethusoffersgreatflexibility whilekeepinghighestspatialresolution.
Intermsofvideoquality,wecomputethePSNRvaluesforallintermediate bitratesresultingfromspatialscalableSVCtransrating.Wenotethat:
ThePSNRdifferencebetweenthetwoanchorsFP=16(notransrating)and FP=1(DCcoefficientonly)ismaximalforQP=18andisequaltoabout15 dB.ThisPSNRdifferenceisabout4dBforQP=36.
– ThePSNRcorrespondingtotheup-sampledbaselayerislogicallyalways lowerthanthePSNRobtainedwhentheenhancementlayerisavailable,even withFP=1.Inthislatercase,thePSNRdifferencetendstoincreasewhen thecompressionratiobecomeshigher.
AssuggestedinSection2,wenowrestrictfrequencyselectivitytothemacroblocksencodedbymeansofmotion-compensatedprediction.Fig.4showsfor the City sequenceanddifferentquantizationparametersvalues,acomparison betweenthetwofollowingcases:
– Macro-blocksusingintra-aswellasinter-imagepredictionarebothtransrated(FPintra =FPinter varyingfrom1to16,dashedcurve);
– Onlymacro-blocksusinginter-imagepredictionaretransrated(FPintra =16 andFPinter varyingfrom1to16,plaincurve).
Fig.4. SVCtransratingoftheupperspatialenhancementlayer(Intermacroblocks): PSNRasafunctionofbitrate- City sequence
Onceagain,simulationresultsaresimilarfortheothersequences.Theresults fortheothersequencesareavailablein[15].Weverifyforallsequencesandall quantizationparametervaluesthatthePSNRisbetterwhenthetransrating operationisrestrictedtointer-imageprediction.ThePSNRdifferencewiththe up-sampledbaselayercaseismoreimportant.However,themaximumbitrate reductionisreduced.Thisdifferencetends toincreasewhenhigherquantization parametersareused.Visuallyspeaking,itshouldalsobenoticedthatnospurious discontinuitiesappearattheboundariesbetweeninter-codedandintra-coded blocksaftertransrating.
Fig.5givesanillustrativevisualexampleoftheperformancesobtainedwith theproposedspatialscalableSVCtransrater.
Fig.5a)andFig.5d)correspondrespectivelyto:
– TheCIFversionreconstructedfromthebaselayerandupsampledtothe highestresolution.Thecorrespondingvisualqualityispoorasthereconstructedpicturesuffersfromaseverelossofdetails.ThePSNRisequalto 27dB;
–
Thefull4-CIFversionreconstructedfrombothbaseandenhancementlayers. Itcorrespondstothebestvisualquality,withaPSNRequalsto43.13dB.
Fig.5. IllustrativeexampleoftheperformancesoftheproposedSVCtransratingsolution,Harboursequence(image#000,QP=18,intra-coded),fromtoptobottom:(a) upsampledbaselayer(PSNR-Y=27dB);(b)FP=3(PSNR-Y=31,84dB);(c)FP=6 (PSNR-Y=38,09dB);(d)full-qualitynottranscodedversionwithFP=16(PSNRY=43,13dB).
Thankstotheproposedspatialscalability-basedSVCtransratingalgorithm,it ispossibletoadaptthebitrateandtohaveaccesstointermediatevisualquality levels.ThisisillustratedinFig.5b)andFig.5c),whichcorrespondtotransrated versionswithhighestspatialresolutionandFP=3andFP=6,respectively.The correspondingpicturesarecharacterizedbyhighestspatialresolutionandclearly exhibitabettervisualqualitywithenhancedtexturesanddetails(forexample, seeriggingaswellasnumbersonthesailandcharactersonthehull),with increasedPSNRvaluesof31.84dBand38.09dB,respectively.
Theseresultsareencouraginganditispossibletoproposeanarchitecturefor transcodingH.264spatialscalableSVCastheintermodetolimitthecomplexity ofthetranscoder.Thus,thisarchitecturedoesnotrequireforcingmacroblocks
10C.Deknudtetal.
fromtheintra-layerintra-imagemodetotheinter-layerintra-imageone(INTRA BL).Theadvantageofthissolutionisnotnegligiblesinceitispossibleto transcodeanyspatialscalableSVCcompressedvideostream.Recallthatinthis case,however,itisnecessarytoimplementthetwotypesofentropycoding:the CAVLC(choseninthiswork)andCABAC.
Toconclude,theapproachproposedinourtransratingschemeleadstoprogressivebitratereductionasprovidedbyqualityscalability(alsocalledSNR scalability)usingMediumGranularityScalability(MGS).ThankstoMGS,video throughputcanbedynamicallyadaptedbydiscardinganyenhancementlayer NALunitfromaSNRscalablebitstream.Inaddition,SVCprovidesthepossibilitytodistributetheenhancementlayertransformcoefficientsamongseveral slicesqualityrefinementlayerswitheachofthemcontainingrefinementcoefficientsforparticulartransformbasisfunctionsonly.Hence,itisinterestingto comparetheperformancesoftheproposedtransratingarchitecturetotheones obtainedthankstothisspecificMGSfeature.Experimentshavebeenperformed ondifferentvideotestsequencesandleadtosimilarresults.Inpractice,there aremanydifferentwaystoconfigureSVC layerstomeetspecificrequirements. Inwhatfollows,acomparisonismade fortheMobcalsequencebetweenthe proposedsolution(case1)andacaseofSVCusingMGS(case2):
– Case1:weapplytheproposedtransratingalgorithmonaspatialscalable SVCcompressedvideobitstreamwiththefollowingcharacteristics:
• baselayer:720x576,25fps,QP=28
• enhancementlayer:1280x720,50fps,QP=28
• sevenfrequencypositionsareselected:FP1(DConly),FP3,FP4,FP5, FP6,FP7andFP16. –
Case2:weconsideraSVCcompressedvideobitstreamwithcombined spatialandSNR(MGS)scalabilities:
• baselayer(BL):720x576,25fps,QP=28
• enhancementlayer1(EL1):1280x720,50fps,QP=32
• enhancementlayer2(EL2):1280x720,50fps,QP=28(usingMGS).Seven pointsareselectedwithrespecttoFPvalues;theycorrespondtothefollowingEL2transformcoefficientrepartitionintheMGSoriginalslice: DC,DC:AC2,DC:AC3,DC:AC4,DC:AC5,DC:AC6,andDC:AC15.
Resultsforthe Mobcal sequencearegiveninFig.6below.
WenoteherethattheproposedtransratingschemeoffersbetterresultscomparedtoMGS,forbitrateshigherthan8Mb/s.Forlowerbitrates,theresults areinfavouroftheMGSscheme.Inparticular,thePSNRcorrespondingto theenhancementlayer1withMGS(QP=32)issignificantlyhigher.Itshould benotedhoweverthattheperformancesobtainedwithourtransratingsolution shouldbeimprovedbyconsideringQPvalueshigherthan32.Anyway,ourtransratingsolutionremainsveryattractiveinthecasewhenonlyspatialscalability hasbeenoriginallyusedattheSVCencodingstage.
Practically,SVCsuffersfromhighcomputationalcomplexity;moreover,itis noteasytooptimizecodingparameterswhendifferenttypesofscalabilityare appliedtogether.Consequently,videocontentsareoftenencodedbymeansof
Fig.6. ComparisonbetweenSVCtransratingusingspatialscalabilityandMGS- Mobcal sequence
SVCusingonescalabilityatatime.InthecaseofspatialscalableSVCbit streams,onlytwoqualityscalabilitylevelsareavailable:BLcombinedwithEL (fullhigh-definitionversion)orBLonly(standarddefinitionversioneventually upsampled).Conversely,ourtransratingarchitectureauthorizestorefineprogressivelyvideoqualityofspatiallyscalablebitstreamswhilekeepinghighest spatialresolutionlevel.Clearlythisconstitutesagreatadvantageforthisoriginal transratingmethod.
5Conclusion
Inthispaper,wepresentedanoriginalsolutionfortransratingofSVCcompressedvideobitstreamssupportingspatialscalabilityonly.Theproposedarchitectureisopen-loopandlowcomplexity,toensurerealtimeprocessingofthe enhancementlayer.Transratingbyfrequencyselectionisappliedtomacroblocks oftheenhancementlayerandleadstointermediatebitrateswithgracefulvideo degradationbetweenthequalityoftheupsampledbaselayerandthequalitycorrespondingtotheadditionofthefullenhancementlayer.Dependingon thequantizationparameterusedforspatialscalableSVCencoding,wedemonstratedthattheoutputbitrateaftertransratingcanbereducedupto66%,while preservinghighestspatialresolution.Amongthetwodevelopedtransratingsolutions,thesecondonerestrictsthetransratingprocessonlytomacroblocksusing inter-pictureprediction(intermode).Thislimitationallowsreducingthecomputationalcomplexityofthesolutionwhileavoidingdrifterror.Moreover,this secondtranscodingarchitecturehasthegreatadvantagetobecompatiblewith anyspatialscalableSVCcompressedvideostream.Furtherworkwillconcern
12C.Deknudtetal.
firstamechanismtodynamicallycontroltheFPparameterbasedonthedesired bitratereduction,aswellastheeffectsofvaryingFPvaluesaccordingtothe picture/slice/macroblocktypes.Theauthorswillalsoconsidertheextensionof thepresentworktothenewHEVCcompressionstandard.
References
1.Simpson,W.:VideooverIP,2ndedn.FocalPress(2008)
2.Develder,C.,Lambert,P.,VanLancker,W.,Moens,S.,VandeWalle,R.,Nelis,J., Verslype,D.,Latr,S.,Staelens,N.,Vercammen,N.,Vermeulen,B.,Masschelein, B.,VanLeeuwen,T.,Macq,J.-F.,Struyve,K.,DeTurck,F.,Dhoedt,B.:Delivering scalablevideowithQoStothehome.TelecommunicationSystems49,129–148 (2012)
3.VanWallendael,G.,VanLancker,W.,DeCock,J.,Lambert,P.,Macq,J.-F.,Van deWalle,R.:FastchannelswitchingbasedonSVCinIPTVenvironments.IEEE TransactionsonBroadcasting58,57–65(2012)
4.Vetro,A.,etal.:Videotranscodingarchitecturesandtechniques:anoverview. IEEESignalProc.Magazine20,18–29(2003)
5.DeCock,J.,Notebaert,S.,Lambert,P.,VandeWalle,R.:Motion-refinedrewriting ofH.264/AVC-codedvideotoSVCstreams.JournalofVisualCommunicationand ImageRepresentation22(5),391–400(2011)
6.Hait,N.,Malah,D.:Model-BasedTransratingofH.264CodedVideo.IEEETrans. onCircuitsandSystemsforVideoTechnology19(8),1129–1142(2009)
7.Wu,Z.,Yu,H.,Tang,B.,Chen,C.W.:AdaptiveInitialQuantizationParameterDeterminationforH.264/AVCVideoTranscoding.IEEETrans.onBroadcasting58(2),277–284(2012)
8.DeCock,J.,Notebaert,S.,Lambert,P.,DeWolf,K.,VandeWalle,R.:LowComplexitySNRTranscodingforH.264/AVC.In:InternationalConferenceon Communication,InternetandInformationTechnology,pp.728–731.ACTAPress, St-Thomas(2006)
9.DeCock,J.,Notebaert,S.,Lambert,P.,VandeWalle,R.:ArchitecturesforFast TranscodingofH.264/AVCtoQuality-ScalableSVCStream.IEEETransactions onMultimedia11,7(2009)
10.Deknudt,C.,Corlay,P.,Bacquet,A.S.,Zwingelstein-Colin,M.,Coudoux,F.-X.: ReducedComplexityH.264/AVCTransratingbasedonFrequencySelectivityfor High-DefinitionStreams.IEEETrans.onConsumerElectronics56(4),2430–2437 (2010)
11.Richardson,I.:TheH.264AdvancedVideoCompressionStandard,2ndedn.Wiley &Sons(2010)
12.Lefol,D.,Bull,D.,Canagarajah,N.,Redmill,D.:AnEfficientComplexityScalable VideoTranscoderwithModeRefinement.SignalProcessing:ImageCommunication22(4),421–433(2007)
13.Lefol,D.,Bull,D.,Canagarajah,N.:PerformanceEvaluationofTranscodingAlgorithmsforH.264.IEEETrans.onConsumerElectronics52(1),215–222(2006)
14.ITU-TandISO/IECJTC1,JVT-T201r2,JointDraft7ofSVCAmendment,Revision2(2006)
15.Deknudt,C.:Miseenœuvred’architecturesdetranscodagevidoH.264/AVCet SVC:application`alatransmissionoptimis´eedelavid´eoHauteD´efinition.PhD dissertation,UniversityofValenciennes(2011)
