Change Detection and Image Time Series Analysis 1
Unsupervised Methods
Coordinated by
Abdourrahmane M. Atto
Francesca Bovolo
Lorenzo Bruzzone
First published 2021 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the undermentioned address:
ISTE Ltd
27-37 St George’s Road
John Wiley & Sons, Inc.
111 River Street London SW19 4EU Hoboken, NJ 07030 UK USA
www.iste.co.uk
www.wiley.com
© ISTE Ltd 2021
The rights of Abdourrahmane M. Atto, Francesca Bovolo and Lorenzo Bruzzone to be identified as the authors of this work have been asserted by them in accordance with the Copyright, Designs and Patents Act 1988.
Library of Congress Control Number: 2021941648
British Library Cataloguing-in-Publication Data
A CIP record for this book is available from the British Library
ISBN 978-1-78945-056-9
ERC code:
PE1 Mathematics
PE1_18 Scientific computing and data processing
PE10 Earth System Science
PE10_3 Climatology and climate change
PE10_4 Terrestrial ecology, land cover change
PE10_14 Earth observations from space/remote sensing
AbdourrahmaneM.ATTO ,FrancescaB
ListofNotations ..................................
Chapter1.UnsupervisedChangeDetectioninMultitemporal RemoteSensingImages ............................ 1
SicongL IU ,FrancescaB OVOLO ,LorenzoB RUZZONE ,QianD U andXiaohuaT ONG
1.1.Introduction..................................1
1.2.Unsupervisedchangedetectioninmultispectralimages.........3
1.2.1.Relatedconcepts.............................3
1.2.2.Openissuesandchallenges.......................7
1.2.3.Spectral–spatialunsupervisedCDtechniques.............7
1.3.Unsupervisedmulticlasschangedetectionapproachesbasedon modelingspectral–spatialinformation......................9
1.3.1.Sequentialspectralchangevectoranalysis(S2 CVA).........9
1.3.2.Multiscalemorphologicalcompressedchangevectoranalysis...11
1.3.3.Superpixel-levelcompressedchangevectoranalysis.........15
1.4.Datasetdescriptionandexperimentalsetup................18
1.4.1.Datasetdescription............................18
1.4.2.Experimentalsetup............................22
1.5.Resultsanddiscussion............................24
1.5.1.ResultsontheXuzhoudataset.....................24
1.5.2.ResultsontheIndonesiatsunamidataset...............24
1.6.Conclusion..................................28 1.7.Acknowledgements..............................29 1.8.References...................................29
Chapter2.ChangeDetectioninTimeSeriesofPolarimetric SARImages 35
KnutC ONRADSEN ,HenningS KRIVER ,MortonJ.C ANTY andAllanA.N IELSEN
2.1.Introduction..................................35
2.1.1.Theproblem...............................36
2.1.2.Importantconceptsillustratedbymeansofthegamma distribution....................................39
2.2.Testtheoryandmatrixordering.......................45
2.2.1.TestforequalityoftwocomplexWishartdistributions........45
2.2.2.Testforequalityofk-complexWishartdistributions.........47
2.2.3.Theblockdiagonalcase.........................49
2.2.4.TheLoewnerorder............................52
2.3.Thebasicchangedetectionalgorithm...................53
2.4.Applications..................................55
2.4.1.Visualizingchanges...........................58
2.4.2.Fieldwisechangedetection.......................59
2.4.3.DirectionalchangesusingtheLoewnerordering...........62
2.4.4.Softwareavailability...........................65
2.5.References...................................70
Chapter3.AnOverviewofCovariance-basedChangeDetection MethodologiesinMultivariateSARImageTimeSeries ........ 73
AmmarM IAN ,GuillaumeG INOLHAC ,Jean-PhilippeOVARLEZ , ArnaudB RELOY andFrédéricPASCAL
3.1.Introduction..................................73
3.2.Datasetdescription..............................76
3.3.StatisticalmodelingofSARimages....................77
3.3.1.Thedata..................................77
3.3.2.Gaussianmodel..............................77
3.3.3.Non-Gaussianmodeling.........................83
3.4.Dissimilaritymeasures............................84
3.4.1.Problemformulation...........................84
3.4.2.Hypothesistestingstatistics.......................85
3.4.3.Information-theoreticmeasures.....................87
3.4.4.Riemanniangeometrydistances....................89
3.4.5.Optimaltransport.............................90
3.4.6.Summary.................................91
3.4.7.ResultsofchangedetectorsontheUAVSARdataset.........91
3.5.Changedetectionbasedonstructuredcovariances............94
3.5.1.Low-rankGaussianchangedetector..................96
3.5.2.Low-rankcompoundGaussianchangedetector............97
3.5.3.Resultsoflow-rankchangedetectorsontheUAVSARdataset...100
3.6.Conclusion..................................102
3.7.References...................................103
Chapter4.UnsupervisedFunctionalInformationClustering inExtremeEnvironmentsfromFilterBanksand RelativeEntropy 109
AbdourrahmaneM.ATTO ,FatimaK ARBOU ,SophieG IFFARD -ROISIN andLionelB OMBRUN
4.1.Introduction..................................109
4.2.Parametricmodelingofconvnetfeatures..................110
4.3.Anomalydetectioninimagetimeseries..................113
4.4.Functionalimagetimeseriesclustering..................119
4.5.Conclusion..................................123
4.6.References...................................123
Chapter5.ThresholdsandDistancestoBetterDetectWetSnow overMountainswithSentinel-1ImageTimeSeries ........... 127 FatimaK ARBOU ,GuillaumeJAMES ,PhilippeD URAND andAbdourrahmaneM.ATTO
5.1.Introduction..................................127
5.2.Testareaanddata...............................129
5.3.WetsnowdetectionusingSentinel-1....................129
5.4.Metricstodetectwetsnow.........................133
5.5.Discussion...................................138
5.6.Conclusion..................................143
5.7.Acknowledgements..............................143
5.8.References...................................143
Chapter6.FractionalFieldImageTimeSeriesModeling andApplicationtoCycloneTracking .................... 145 AbdourrahmaneM.ATTO ,AluísioP INHEIRO ,GuillaumeG INOLHAC andPedroM ORETTIN
6.1.Introduction..................................145
6.2.Randomfieldmodelofacyclonetexture.................148
6.2.1.Cyclonetexturefeature.........................149
6.2.2.Wavelet-basedpowerspectraldensitiesandcyclonefields......150
6.2.3.Fractionalspectralpowerdecaymodel.................153
6.3.Cyclonefieldeyedetectionandtracking..................157
6.3.1.Cycloneeyedetection..........................157
6.3.2.Dynamicfractalfieldeyetracking...................158
6.4.Cyclonefieldintensityevolutionprediction................159
6.5.Discussion...................................161
6.6.Acknowledgements..............................163
6.7.References...................................163
Chapter7.GraphofCharacteristicPointsforTextureTracking: ApplicationtoChangeDetectionandGlacierFlowMeasurement fromSARImages 167 Minh-TanP HAM andGrégoireM ERCIER
7.1.Introduction..................................167
7.2.Texturerepresentationandcharacterizationusinglocalextrema....169
7.2.1.Motivationandapproach........................169
7.2.2.LocalextremakeypointswithinSARimages.............172
7.3.Unsupervisedchangedetection.......................175
7.3.1.Proposedframework...........................175
7.3.2.Weightedgraphconstructionfromkeypoints.............176
7.3.3.Changemeasure(CM)generation...................178
7.4.Experimentalstudy..............................179
7.4.1.Datadescriptionandevaluationcriteria................179
7.4.2.Changedetectionresults.........................181
7.4.3.Sensitivitytoparameters.........................185
7.4.4.ComparisonwiththeNLMmodel...................188
7.4.5.Analysisofthealgorithmcomplexity.................191
7.5.Applicationtoglacierflowmeasurement.................192
7.5.1.Proposedmethod.............................193
7.5.2.Results...................................194
7.6.Conclusion..................................196
7.7.References...................................197
Chapter8.MultitemporalAnalysisofSentinel-1/2Images forLandUseMonitoringatRegionalScale 201 AndreaG ARZELLI andClaudiaZ OPPETTI
8.1.Introduction..................................201
8.2.Proposedmethod...............................203
8.2.1.Testsiteanddata.............................206
8.3.SARprocessing................................209
8.4.Opticalprocessing..............................215
8.5.Combinationlayer..............................217
8.6.Results.....................................219
8.7.Conclusion..................................220
8.8.References...................................221
Chapter9.StatisticalDifferenceModelsforChangeDetection inMultispectralImages 223
MassimoZ ANETTI ,FrancescaB OVOLO andLorenzoB RUZZONE
9.1.Introduction..................................223
9.2.Overviewofthechangedetectionproblem................225
9.2.1.Changedetectionmethodsformultispectralimages.........227
9.2.2.Challengesaddressedinthischapter..................230
9.3.TheRayleigh–Ricemixturemodelforthemagnitudeofthe differenceimage..................................231
9.3.1.Magnitudeimagestatisticalmixturemodel..............231
9.3.2.Bayesiandecision............................233
9.3.3.Numericalapproachtoparameterestimation.............234
9.4.Acompoundmulticlassstatisticalmodelofthedifferenceimage....239
9.4.1.Differenceimagestatisticalmixturemodel..............240
9.4.2.Magnitudeimagestatisticalmixturemodel..............245
9.4.3.Bayesiandecision............................248
9.4.4.Numericalapproachtoparameterestimation.............249
9.5.Experimentalresults.............................253
9.5.1.Datasetdescription............................253
9.5.2.Experimentalsetup............................256
9.5.3.Test1:Two-classRayleigh–Ricemixturemodel...........256
9.5.4.Test2:MulticlassRicianmixturemodel................260
9.6.Conclusion..................................266
9.7.References...................................267
Preface
AbdourrahmaneM.ATTO 1 ,FrancescaB OVOLO 2 andLorenzoB RUZZONE 3
1 UniversitySavoieMontBlanc,Annecy,France
2 FondazioneBrunoKessler,Trento,Italy
3 UniversityofTrento,Italy
ThisbookispartoftheISTE-Wiley“SCIENCES ”Encyclopediaandbelongsto the Image fieldofthe EngineeringandSystems department.The Image fieldcovers theentireprocessingchainfromacquisitiontointerpretationbyanalyzingthedata providedbyvariousimagingsystems.Thisfieldissplitintosevensubjects,including RemoteSensingImagery (RSI).TheheadsofthissubjectareEmmanuelTrouvéand AvikBhattacharya.Inthissubject,weproposeaseriesofbooksthatportraydiverse andcomprehensivetopicsinadvancedremote-sensingimagesandtheirapplication for EarthObservation (EO).Therehasbeenanincreasingdemandformonitoringand predictingourplanet’sevolutiononalocal,regionalandglobalscale.Hence,over thepastfewdecades,airborne,space-borne andground-basedplatformswithactive andpassivesensorsacquireimagesthatmeasureseveralfeaturesatvariousspatialand temporalresolutions.
RSIhasbecomeabroadmultidisciplinarydomainattractingscientistsacrossthe diversefieldsofscienceandengineering.TheaimofthebooksproposedinthisRSI seriesistopresentthestate-of-the-artandavailablescientificknowledgeaboutthe primarysourcesofimagesacquiredbyopticalandradarsensors.Thebookscover theprocessingmethodsdevelopedbythesignalandimageprocessingcommunityto extractusefulinformationforend-usersforanextensiverangeofEOapplicationsin naturalresources.
Inthisproject,eachRSIbookfocusesongeneraltopicssuchaschange detection,surfacedisplacementmeasurement,targetdetection,modelinversionand
ChangeDetectionandImageTimeSeriesAnalysis1, coordinatedbyAbdourrahmaneM.ATTO ,FrancescaB OVOLO andLorenzoB RUZZONE ©ISTELtd2021.
dataassimilation.ThisfirstbookoftheRSIseriesisdedicatedto ChangeDetection andImageTimeSeriesAnalysis.Itpresentsmethodsdevelopedtodetectchanges andanalyzetheirtemporalevolutionsusingopticaland/orsyntheticapertureradar (SAR)imagesindiversesettings(e.g.imagepairs,imagetimeseries).According tothenumerousworksandapplicationsinthisdomain,thisbookisdividedinto twovolumes,dedicatedto unsupervised and supervised approaches,respectively. Unsupervisedmethodsrequirelittletonoexpert-basedinformationtoresolvea problem,whereasthecontraryholdstrue,especiallyformethodsthataresupervised inthesenseofprovidingawideamountoflabeledtrainingdatatothemethod,before testingthismethod.
Volume1:Unsupervisedmethods
Asignificantpartofthisbookisdedicatedtoawiderangeofunsupervised methods.Thefirstchapterprovidesaninsightintothemotivationsofthisbehaviorand introducestwounsupervisedapproachestomultiple-changedetectioninbitemporal multispectralimages.Chapters2and3introducetheconceptofchangedetection intimeseriesandpostulateitinthecontextofstatisticalanalysisofcovariance matrices.Theformerchapterfocusesonadirectionalanalysisformultiple-change detectionandexercisesonatimeseriesofSARpolarimetricdata.Thelatterfocuses onlocalanalysisforbinarychangedetectionandproposesseveralcovariancematrix estimatorsandtheircorrespondinginformation-theoreticmeasuresformultivariate SARdata.Thelastfourchaptersfocusmoreonapplications.Chapter4addresses functionalrepresentations(waveletsandconvolutionalneuralnetworkfilters)for featureextractioninanunsupervisedapproach.Itproposesanomalydetection andfunctionalevolutionclusteringfromthisframeworkbyusingrelativeentropy informationextractedfromSARdatadecomposition.Chapter5dealswiththe selectionofmetricsthataresensitive tosnowstatevariationinthecontextof thecryosphere,withafocusonmountainareas.Metricssuchascross-correlation ratiosandHausdorffdistanceareanalyzedwithrespecttooptimalreferenceimages toidentifyoptimalthresholdingstrategiesforthedetectionofwetsnowbyusing Sentinel-1imagetimeseries.Chapter6presentstimeseriesanalysisinthecontextof spatio-temporalforecastingandmonitoring fast-movingmeteorologicaleventssuch ascyclones.Theapplicationbenefitsfromthefusionofremotesensingdataunderthe fractionaldynamicfieldassumptiononthecyclonebehavior.Chapter7proposesan analysisbasedoncharacteristicpointsfortexturemodelingwithgraphtheory.Such anapproachovercomesissuesarisingfromlarge-sizedenseneighborhoodsthataffect spatialcontext-basedapproaches.Theapplicationproposedinthischapterconcerns glacierflowmeasurementinbitemporalim ages.Chapter8focusesondetecting newland-covertypesbyclassification-basedchangedetectionorfeature/pixel-based changedetection.Monitoringtheconstructionofnewbuildingsinurbanandsuburban scenariosatalargeregionalscalebymeansofSentinel-1and-2imagesisconsidered asanapplication.Chapter9focusesonthestatisticalmodelingofclassesinthe
differenceimageandderivesfromscratchamulticlassmodelforitinthecontext ofchangevectoranalysis.
Volume2:Supervisedmethods
Thesecondvolumeofthisbookisdedicatedtosupervisedmethods.Chapter1of thisvolumeaddressesthefusionofmultisensor,multiresolutionandmultitemporal data.Thischapterreviewsrecentadvancesintheliteratureandproposestwo supervisedMarkovrandomfield-basedsolutions:onereliesonaquadtreeandthe secondoneisspecificallydesignedtodealwithmultimission,multifrequencyand multiresolutiontimeseries. Chapter2providesanoverviewofpixel-basedmethods fortimeseriesclassificationfromtheearliestshallow-learningmethodstothemost recentdeeplearning-basedapproaches.Thischapteralsoincludesbestpracticesfor referencedatapreparationandmanagement,whicharecrucialtasksinsupervised methods.Chapter3focusesonveryhighspatialresolutiondatatimeseriesandtheuse ofsemanticinformationformodelingspatio-temporalevolutionpatterns.Chapter4 focusesonthechallengesofdensetimeseriesanalysis,includingpre-processing aspectsandataxonomyofexistingmethodologies.Finally,sincetheevaluationof alearningsystemcanbesubjecttomultipleconsiderations,Chapters5and6propose extensiveevaluationsofthemethodologiesusedtoproduceearthquake-induced changemaps,withanemphasisontheirstrengthsandshortcomings(Chapter5) andthedeeplearningsystemsinthecontextofmulticlassmultilabelchange-of-state classificationonglacier observations(Chapter6).
Thisbookcoversbothmethodologicalandapplicationtopics.Fromthe methodologicalviewpoint,contributionsareprovidedwithrespecttofeature extractionandalargenumberofevaluationmetricsforchangedetection,classification andforecastingissues.Analysishasbeen performedinbothbitemporalimagesand timeseries,illustratingbothunsupervised andsupervisedmethodsandconsidering bothbinary-andmulticlassoutputs.Severalapplicationsarementionedinthe chapters,includingagriculture,urbanareasandcryosphereanalysis,amongothers. Thisbookprovidesadeepinsightintotheevolutionofchangedetectionandtime seriesanalysisinthestate-of-the-art,aswellasanoverviewofthemostrecent developments.
July2021
UnsupervisedChangeDetection inMultitemporalRemote SensingImages
SicongL IU 1 ,FrancescaB OVOLO 2 ,LorenzoB RUZZONE 3 , QianD U 4 andXiaohuaT ONG 1
1 TongjiUniversity,Shanghai,China
2 FondazioneBrunoKessler,Trento,Italy
3 UniversityofTrento,Italy
4 MississippiStateUniversity,Starkville,USA
1.1.Introduction
Remotesensingsatelliteshaveagreatpotentialtorecurrentlymonitorthe dynamicchangesoftheEarth’ssurfaceinawidegeographicalarea,andcontribute substantiallytoourcurrentunderstanding oftheland-coverandland-usechanges (BruzzoneandBovolo2013;Song etal.2018;Liu etal.2019c).Scientifically understandinglandchangesisalsoessentialforanalyzingenvironmentalevolution andanthropicphenomena,especiallywhenstudyingtheglobalchangeanditsimpact onhumansociety.Thankstothesatellite revisitproperty, bothlong-term(e.g. yearly)andshort-term(e.g.daily)satelliteobservations produceahugeamountof multitemporalimagesinthedataarchive(Liu etal.2015).Basedontheanalysisof multitemporaldata,land-coverchangescanbeautomaticallydiscoveredanddetected, whereknowledgeofchangescanalsobeacquired.Thisbecomesanimportant andcomplementarywayofoptimizingthetraditional insitu investigation,which
ChangeDetectionandImageTimeSeriesAnalysis1, coordinatedbyAbdourrahmaneM.ATTO ,FrancescaB OVOLO andLorenzoB RUZZONE . ©ISTELtd2021.
Change Detection and Image Time Series Analysis 1: Unsupervised Methods, First Edition. Abdourrahmane M. Atto; Francesca Bovolo and Lorenzo Bruzzone © ISTE Ltd 2021. Published by ISTE Ltd and John Wiley & Sons, Inc.
isoftenverycostlyandlabor-intensive.Inparticular,insomecasessuchasin anaturaldisasterscenario,itisverydifficultorevenimpossibletoconductfield investigations.Changedetection(CD)istheprocessofidentifyingchangeand no-changeregionsontheEarth’ssurfacebyanalyzingimagesacquiredfromthesame geographicalareaatdifferenttimes(Singh1989;Coppin etal.2004;Lu etal.2004; Liu etal.2019c).Therefore,automaticandrobusttechniquesneedtobedesignedto effectivelydiscover,describeanddetectchangesthatoccurinmultitemporalremote sensingimages.Inthepastdecades,CDhas becomeanincreasinglyactiveresearch fieldandhasbeenwidelyappliedinvariousremotesensingapplications,suchas deforestation,disasterevaluation,urbanexpansionevolutionandenvironmentand ecosystemmonitoring(BovoloandBruzzone2007a;Bouziani etal.2010;Du etal 2012;Khan etal.2017;Liu etal.2020b).
Basically,CDtechniquesaredevelopedbasedonspecificremotesensingsatellite sensors.Intheliterature,manyexcellentarticlesfocusedonthediscussionofCD problemsindifferenttypesofsatellitesensors:forexample,CDinmultispectral images(Lu etal.2004;BanandYousif2016),SARimages(BanandYousif2016) andhyperspectralimages(Liu etal.2019c),aswellasinLidardata(Okyay etal. 2019).Amongdifferentsensorsmountedon theEOsatellites,multispectralscanners canacquireimageswithbothhighspatialresolutionandwidespatialcoverage.In thepastdecades,duetodataavailability,multispectralremotesensingimagessuchas LandsatandSentinelserialscontributedthemaindatasourcefortheEarth’ssurface monitoringandCDapplications(Du etal.2013;Liu etal.2020a).However,with theincreasinghighqualityandspatialresolutioninnewmultispectralsensorimages, especiallyforveryhighresolution(VHR)images,itisnecessarytodesignadvanced CDtechniquesthatcandealwithmorecomplexchangepatternspresentedinamore complexCDscenario.
Inrecentyears,manyCDmethodshavebeen developedformultispectralimages, mostofwhichfocusonimprovingtheautomation,accuracyandapplicabilityofCD (Leichtle etal.2017;Liu etal.2017a,2019b;Wang etal.2018;Saha etal.2019; Wei etal.2019).Ingeneral,accordingtotheautomationdegree,theycanbegrouped intothreemaincategories:supervised,semi-supervisedandunsupervisedmethods. Usually,supervisedCDapproacheshavebetterperformancewithhigheraccuracyby takingadvantageofcertainrobustsupervisedclassifiers(Wang etal.2018).However, theirimplementationreliesontheavailabilityofgroundreferencedata,whichis oftendifficulttocollectinmostpracticalcases.Semi-supervisedCDapproachesstart fromlimitedtrainingsamplesorpartialpriorknowledgelearnedfromthesingle-time image,wheretheactivelearningortransfer learningalgorithmcanusuallybeapplied toincreasethesamplerepresentation(Liu etal.2017b,2019a;Zhang etal.2018;Tong etal.2020).Incontrast,unsupervisedapproacheshavehigherautomationwithout relyingontheavailabilityofgroundreferencedataorpriorknowledge(Liu etal. 2017a,2019b;Saha etal.2019).Therefore,theanalysisintheunsupervisedCD caseismainlydata-drivenandisactuallymorechallengingthantheothertwotasks.
However,fromapracticalapplicationpointofview,itisdefinitelymoreattractivedue toitssimplicityandhighautomation.
Inthischapter,wefocusontheunsupervisedCDprobleminmultitemporal multispectralimages.Inparticular,weinvestigateandanalyzethespectral–spatial changerepresentationforaddressingtheimportantmulticlassCDproblem.To thisend,twoapproachesaredeveloped,includingamulti-scalemorphological compressedchangevectoranalysisandasuperpixel-levelmulticlassCD.Bytaking advantageofthespectralandspatialjointanalysisofchangeinformation,both approachesshowhigherperformancethanthecomparedstate-of-the-artmethods. Experimentalresultsobtainedfromtworealmultispectraldatasetsconfirmedthe effectivenessoftheproposedapproaches intermsofhigheraccuracyandefficiency ofCD.
Therestofthischapterisorganizedasfollows.Section1.2pointsoutthe keyconceptsandchallengesinunsupervisedCD,andespeciallyreviewsthe currentdevelopmentofspectral–spatialunsupervisedCDtechniques.Section1.3 describesthetwoproposedunsupervisedmulticlassCDapproachesindetail.Dataset descriptionandexperimentalsetupareprovidedinsection1.4.Experimentalresults anddiscussionsarepresentinsection1.5.Finally,section1.6drawstheconclusionof thischapter.
1.2.Unsupervisedchangedetectioninmultispectralimages
1.2.1.
Relatedconcepts
DependingonthepurposeofunsupervisedCDtasks,twomaincategoriesof methodsaredefined: binarychangedetection and multiclasschangedetection.The formeraimstoseparateonlythechangeandno-changeclasses,whereasthelatter detectschangesanddistinguishesdifferentclasseswithinthechangedpixels.Inthis chapter,weconsiderthelatter,whichismoreattractivebutchallenginginpractical CDapplications.NotethatintheunsupervisedCDcase,nogroundtruthorprior knowledgeisavailable,thusthedata-drivenCDprocessismorepreferablethan themodel-drivenprocess.Therefore,themulticlassdiscriminationrepresentsthe inter-changedifferenceassociatedwithspecificland-coverclasstransitions,whereas thedetailed“from–to”informationisabsent,makingitessentiallydifferentfromthe supervisedcase.
Ingeneral,theunsupervisedCDprocessincludesthefollowingmainsteps: (1)multitemporaldatapre-processing;(2)featuregenerationandselection;(3)change indexconstruction;(4)CDalgorithmdesign;(5)performanceevaluation.Themain componentsofanunsupervisedCDareshowninFigure1.1.Eachstepisbriefly describedanddiscussedasfollows.
Performance Evaluation
Change Detection Algorithm Design
• Accuracy Index
• Error Matrix
• ROC • AUC • Time Cost
Change Index Construction
• Change Number Estimation
• Thresholding
• Clustering
• Deep Networks
Feature Generation and Selection
• Differencing
• Ratioing
• Distance/Similarity Measurement • Transformation
• Spectral Features
• Spectral Indices
• Spatial Features
• Object Features •
Figure1.1. ThemaintechnicalcomponentsofanunsupervisedCDprocess
Multitemporal Data Pre-processing
• Calibration
• Radiometric & Atmospheric Corrections
• Image Enhancement
• Co-registration
Multitemporaldatapre-processing:inthisstep,differentoperationssuchas calibration,bandstriperepair(ifany),radiometricandatmosphericcorrections,image enhancementandimage-to-imageco-registrationareusuallyconductedinorderto generatehigh-qualitypre-processedmultitemporalimagesforCDinthenextsteps.In particular,ahighprecisionofco-registrationisthecoreoperationforasuccessfulCD, whichmaysignificantlyaffecttheCDperformanceduetothepresenceofremaining residualerrors.
Featuregenerationandselection:featuresextractedfromoriginalmultitemporal imagesarethecriticalcarrierforrepresentingdifferentcharacteristicsofobjectsin thesingle-timeimageandtheirvariationsinthetemporaldomain.Featuressuch asoriginalspectralbands,spectralindices (e.g.NormalizedDifferenceVegetation Index–NDVI,ModifiedNormalizedDifferenceWaterIndex–MNDWI,Index-based Built-upIndex–IBI)andtextures(e.g.mean,contrast,homogeneity)derivedfrom originalbandscanbeconsideredinCD.Inaddition,spatialfeaturesgeneratedfrom multispectralbandssuchaswavelettransformation(CelikandMa2011),Gabor filtering(Li etal.2015),morphologicalfiltering(Falco etal.2013),etc.,provide importantmulti-scalegeometricinformationaboutimageobjectstoimprovethe changerepresentation.Recently,deeplearning-basedCDapproacheshaveshown greatpotentialinextractingmorehigh-leveldeepfeatures,whichrepresentsapopular directioninCDresearch(Mou etal.2019;Saha etal.2019).
Changeindexconstruction:thechangeindexrepresentsthetemporalvariations extractedfrommultitemporalimagefeatures.Itcanbeconstructedbasedondifferent operatorsandalgorithms,suchasunivariateimagedifferencing(BruzzoneandPrieto 2000a),changevectoranalysis(CVA)(BovoloandBruzzone2007b),ratioing(Bazi etal.2005),distanceorsimilaritymeasures(Du etal.2012),etc.Transformation approachessuchasiterativereweighted multivariatealterationdetection(IR-MAD) (Nielsen2007),principalcomponentsanalysis(PCA)anditskernelversion(Nielsen andCanty2008;Celik2009),independentcomponentanalysis(ICA)(Liu etal. 2012),arealsodesignedtotransformthechangeinformationfromtheoriginal dataspaceintoaprojectedfeaturespace. However,acarefulselectionofspecific componentsrepresentinguser-interestedchangesisrequired.Thisisoftenvery difficultinanunsupervisedCDcasewithoutpriorknowledgeabouttheconsidered studyareaanddataset,whichmaylimittheautomationdegreeoftheCDapplication. Forasummaryoftherelatedmethodsforconstructingdifferenttypesofchangeindex, readerscanrefertothepaperbyBovoloandBruzzone(2015).
Changedetectionalgorithmdesign:unlikethesupervisedandsemi-supervised CDmethodsthatrelyontheavailablereferencesamples,unsupervisedCDalgorithms focusmoreontheautomationandaccuracy.Thus,basically,mostoftheunsupervised CDapproachesaredata-drivenbyanalyzingthemultitemporaldataitself.Withinthis context,forbinaryCD,ifweconsideragivenchangeindexgeneratedintheprevious step,forexample,themagnitudeofdifferencingimage,automaticthresholdingsuch
asempiricalsegmentation(BruzzoneandPrieto2000b),Kittler–Illingworth(KI) (Bazi etal.2005),Otsu(1979)andBayesian-basedexpectation–maximization(EM) (BruzzoneandPrieto2000a)areallsimplebuteffectivealgorithmsproposedinthe literature.However,thesuccessfuluseofsuchmethodsdependsontheassumptionof acertaindatadistributionsuchasGaussianorRayleigh–Ricemixture(Bovolo etal 2012;Zanetti etal.2015),whereawrongestimationmayleadtomanydetection errors.Onthecontrary,clusteringalgorithmssuchask-means,fuzzyc-meansand Gustafson–Kesselclustering(GKC)havebeenusedtoaddressthebinaryCDproblem (Celik2009;Ghosh etal.2011),whicharedistribution-freebutrequireaspecific settingtoavoidunstableperformance,suchastheaccuracydecreaseduetorandom initialization.
ForamulticlassCDcase,theunsupervisedtaskbecomesmorecomplexsince severalsub-problemsshouldbesolvedsimultaneously,includingthebinarychange andno-changeseparation,thenumberofmulticlasschangeestimationandthe multiclasschangediscrimination(Liu etal.2019c).Inparticular,amongmany solutions,werecalltheclassicalmultipleCDtechnique–changevectoranalysis (CVA)(Malila1980).Itwasdesignedtoanalyzepossiblemultiplechangesinpairs ofbitemporalimagebands.AtheoreticaldefinitionwasgiventotheoriginalCVA approachinthepolardomaintoprovideamoreclearmathematicalexplanationto CVA(BovoloandBruzzone2007b).However,itstillhasalimitation,i.e.onlyapart ofallpossiblechangescanbedetectedsinceonlytwoselectedbandsareconsidered ineachimplementation.Ifmorespectralchannelsareconsidered,itbecomes verydifficulttosimultaneouslymodelandvisualizemultidimensionalchanges.To breakthisconstraint,acompressedchangevectoranalysis(C2 VA)approachwas proposed,whichsuccessfullyextendedtheoriginalCVAtoatwo-dimensional(2D) representationofthemulti-bandproblem(Bovolo etal.2012).Otherworksinthe literaturedevelopeddifferentvariationsofCVA.Forexample,amodifiedCVA wasdevelopedtodeterminethemagnitudethresholdanddirectionbycombining single-dateimageclassificationresults(Chen etal.2003).Animprovedthresholding approachonchangemagnitudewasdesignedtooptimizethebinaryseparationon eachspecificchangeclass(BovoloandBruzzone2011).Ahierarchicalversionof C2 VAwithanadaptiveandsequentialprojectionofspectralchangevectors(SCVs) ateachlevelofthehierarchywasproposed todetectmultiplechangesinbitemporal hyperspectralimages(Liu etal.2015).Inthischapter,wealsoexplorethepotential capabilityofC2 VAandextenditfromthespectral–spatialpointofview.
Performanceevaluation:similartothesupervisedCDmethods,unsupervised binaryandmulticlassCDapproachescanusuallybeassessedaccordingtothe detectionaccuracyorerrorindex,suchasoverallaccuracy(OA),Kappacoefficient (Kappa),omissionerrors(OE )andcommissionerrors(CE ),receiveroperating characteristic(ROC )curveandareaunderthecurve(AUC )value.Inthiscase,the accuracyevaluationusuallyreliesonthemanuallyinterpretedchangereferencemap.
Notethatsuchareferencemapisonlyusedforaccuracyevaluation,whichisnot consideredastrainingdataasinthesupervisedcase.Inaddition,thecomputational timecostisalsoanotherimportantindicatorthatreflectstheautomationandefficiency ofunsupervisedmethods.
1.2.2. Openissuesandchallenges
ThecurrentdevelopmentofunsupervisedCDtechniquesformultispectralremote sensingimageshashadgreatsuccessinmanypracticalapplications.However,there arestillopenissuesandchallengesthatdeservetobefurtheranalyzed,whichinclude butarenotlimitedtothefollowing:
1)ahigh-precisionmultitemporalpre-processingprocedure,forexample, co-registrationtechniques;
2)multitemporaldataqualityimprovement duetobadimagingconditions,such assystemnoise,cloudcontaminationandseasonalspectralvariations;
3)advancedtechniquesforcorrectlyestimatingtherealnumberofmulticlass changesinimagescenarios;
4)spectral–spatialmodelingofchangetargetstoenhancetheoriginalpixel-wise spectralrepresentation;
5)robustandefficientCDapproachinan unsupervisedfashion,especiallyfora largecomplexCDscene;
6)changefeaturerepresentationbytakingadvantageofbothmachinelearningand deeplearningtechniques.
1.2.3. Spectral–spatialunsupervisedCDtechniques
DespitethesuccessofaforementionedCDmethods,especiallytheCVA-based methods,theymainlyfocusonthespectralchangesin eachindividualpixeloralocal neighborhood(Bovolo2009;Bovolo etal.2012;Liu etal.2015).Thegeometrical characteristicsofchangetargetsarenotfullymodeledandpreserved.Thismay increasetheambiguityduetoabnormalspectralvariationsinisolatedpixelsand errors(e.g.co-registrationerrors),leadingtothepresenceofomissionandcommission errors,especiallywhendealingwithVHRimages.Inthiscase,traditionalpixel-based CDmethodsmaylosetheireffectivenesssincetheyweredevelopedunderthe assumptionthatpixelsarespatiallyindependent.However,formultispectralimages incomplexurbanscenarios,challengingissuesmayariseduetothelimitedspectral representation;thus,thesameclassofobjectsmayhavedifferentspectra,ordifferent objectsmayhavethesameorverysimilarspectra.Thismaysignificantlyincreasethe detectiondifficulty,especiallywhenconsideringthemulticlassCDtask.
Toaddresstheaboveproblemsinpixel-basedCD(PBCD)techniques, spectral–spatialjointanalysisandobject-basedCD(OBCD)methodsaremainstream techniquesproposedintheliterature.For theformer,morphologicalfilters(i.e. self-dualreconstructionfiltersandalternatingsequentialfilters)werecombinedwith CVAforbinaryCDinVHRimages(Mura etal.2008).However,aslidingwindow (i.e.structuringelement(SE))forfilteringshouldbefixedatagivenlevel;thus,itis notrobustformultilevelimplementation.Morphologicalattributeprofiles(APs)were appliedtoextractstructure-relatedgeometricalfeatureswithinthescenefromeach dateofpanchromaticimages(Falco etal.2013).Itincludesamultilevelextraction ofconnectedregionsinthesceneatdifferentscales.Buildingchangeinformation basedonthedifferenceinthemultitemporalmorphologicalbuildingindex(MBI) atthefeatureanddecisionlevelwasconsideredfordetectingbuildingchanges inVHRimages(Huang etal.2014).Aspectral–spatialbandexpansionapproach wasdevelopedtoenhancethechangerepresentationinmultispectralimageswith limitedbands,whereadditionalbandsweregeneratedfrombothspectralandspatial viewpoints(Liu etal.2019b).
ForOBCDmethods,fourmaincategoriesexist:image-object,class-object, multitemporal-objectandhybridCD(Chen etal.2012;Hussain etal.2013).An object-basedCDapproachwasdesignedin(LiuandDu2010),whichanalyzes differentspectralandtexturefeaturesextractedduringthesegmentationprocessfrom twoindependenttimeimages.Asuperpixelsegmentationwasappliedtostacked bitemporalimages,andseveralderivedfeatureswereusedtodescribechangesin thedifferenceimageaccordingtothesupervisedclassification(Wu etal.2012). Anobjected-basedmethodwasdesignedtocreateobjectsineachsingle-timeimage accordingtosegmentationandthengenerateddifferentrepresentativefeatures(Wang etal.2018).AweightedDempster–Shafertheory(wDST)fusionOBCDmethodwas proposedbycombiningmultiplePBCDresults,whichcanautomaticallycalculate andassignacertaintyweightforeachobjectofthePBCDresultwhileconsidering thestabilityofanobject(Han etal.2020).However,theselectionoftheoptimal segmentationscaleisstillanopenissueandwasmainlyimplementedbasedonthe empiricalanalysisinOBCDmethods(Kaszta etal.2016).Moreover,mostofthe aboveexistingworkfocusedonsolvingabinaryCDproblem,andveryfewwere designedfordealingwiththemorechallengingunsupervisedmulticlassCDcase(Liu etal.2019b).
Accordingly,thefollowingproblemsshouldbefurtheranalyzed:(1)howspatial neighboringpixelshaveanimpactonunsupervisedchangerepresentationand detection;(2)howtoeffectivelymodelstructuralandgeometricinformationof changetargetstoenhancechangerepresentation;(3)howtointegratemultiscaleand multidimensionalchangedescriptionstoincreasechangeseparability;(4)howto adaptivelyfindasuitablesegmentationscaleintheunsupervisedOBCDapproach. Inthischapter,weinvestigatetheseissuesanddevelopnewspectral–spatialCD approachesinmultitemporalmultispectralimages.
1.3.Unsupervisedmulticlasschangedetectionapproachesbasedon modelingspectral–spatialinformation
1.3.1. Sequentialspectralchangevectoranalysis(S2 CVA)
Here,wefirstrecallapopularstate-of-the-artunsupervisedmulticlassCDmethod, S2 CVA.Thetwoproposedapproachesaredesignedbasedonit.However,itis importanttonotethattheS2 CVAthatispixel-wiseonlyreliesontheanalysis ofspectralinformation.Originally,fromthestandardC2 VA(Bovolo etal.2012), S2 CVAwasspeciallyproposedformulticlassCDinbitemporalhyperspectralimages accordingtoahierarchicalanalysis.Itallowsthevisualizationanddetectionof multiplechangesbyconsideringallspectralchannels,withoutneglectinganyband.A compressed2Dpolardomainisbuiltbasedontheconstructionoftwochangevariables (Liu etal.2015),i.e.magnitude ρ anddirection θ .Moreparticularly,themagnitude ρ istheEuclideancompressionofSCVs.Itmeasuresthespectralbrightnessofchanges. Thedirection θ isbuiltbasedonthespectralangledistance(SAD)(Keshava2004), whichpointsoutdifferentchangeswithrespecttothevariancesinspectralresponse foragivenpixel.However,notethatthe“from–to”classtransitioninformationisnot explainedduetotheunsupervisednatureofS2 CVA.Morespecifically,thechange variables ρ and θ aredefinedas:
where X b D isthe b-th(b=1,...,B)componentofthedifferencing(SCVs)image XD , whichwascalculatedontheco-registeredimages X1 and X2 ,and B isthenumber ofimagebands(i.e.thedimensionalityofSCVs). r b isthe b-thcomponentofan adaptivereferencevector r.InthestandardC2 VA, r isdefinedasaunitconstantvector r “ “1{?B,..., 1{?B ‰ (Bovolo etal.2012).InS2 CVA,itisimprovedasthefirst eigenvectorofthecovariancematrix A of XD (Liu etal.2015):
“ cov pXD q“ E ”p XD
where E rXD s istheexpectationof XD .Theeigen-decompositionof A canbe representedas: A ¨ V “ V ¨ W [1.4] where W isadiagonalmatrixwitheigenvaluesbeingsortedinadescendingorder (i.e. λ1 ą λ2 ą ...λB )inthediagonal,and V isthematrixofeigenvectors(i.e.
V “rv1 , v2 , v3 ,..., vB s).Thereferencevector r isdefinedasthefirsteigenvector v1 correspondingtothelargesteigenvalue λ1 ,i.e. r “ v1 .Notethatthisresultsin animprovedchangerepresentationbyprojectingtheSCVsintoareferencedirection thatmaximizesthevarianceofthemeasurement,whilepreservingthediscriminative informationofdifferentchanges.
Figure1.2. 2D polarchangerepresentationdomainintheC2 VAmethod
Thena2Dcompressedpolardomain D isdefinedbasedon ρ and θ :
where ρmax isthemaximumvalueof ρ.Asemicirclescattergram(seeFigure1.2)is usedforvisualizingmultiplechangesin D.Regionsofthesemicircle SCn (which representstheunchangedpixels)andthesemiannulus SAc (whichrepresentsthe changedpixels)aremathematicallydefinedas:
Undertheassumptionthat X1 and X2 areradiometricallyandgeometrically corrected,inthe2Dpolardomain,theunchangedSCVsresultinlowmagnitude valuesclosetozeroandarethusdistributedwithin SCn ,whereasthechangedSCVs arerepresentedin SAc havinghighermagnitudes.Differentspectralbehaviorsof theSCVsreflectedonthedirectionvariable,henceleadingtotherepresentationof multiclasschangesineachsectorof SAc .TheconsideredmulticlassCDproblemis solvedbydefiningathreshold Tρ alongthemagnitude ρ toseparatetheunchanged andchangedSCVs(i.e.associatedwith SCn and SAc ,respectively),andtoseparate multiplechanges(C1 ,...,CK )alongthedirection θ bysettingmultiplethresholds Tθ,k (k =1,..., K -1)in SAc .Notethatahierarchicalanalysisisimplementedtodiscover
anddetectallpossiblechanges(bothstrongandsubtlechanges)inhyperspectral imagesduetocomplexchangerepresentationsinthehighdimensionalityof hyperspectralimages.However,fewerlevelsmaybeobtainedwhenonlyafew spectralbandsareconsidered,asinthemultispectralcase.
1.3.2. Multiscalemorphologicalcompressedchangevectoranalysis
Inthissection,weintroduceaproposedmultiscalemorphologicalcompressed changevectoranalysis(M2 C2 VA)method.Itaimstoinvestigateaproperway tointegratemultiscalespectral–spatialchangeinformation,especiallytoimprove multiclasschangerepresentationanddiscriminationinC2 VA.Theproposed approachconsistsofthreemainsteps:1)SCVreconstructionbasedonmultiscale morphologicalprocessing;2)multiscalechangeinformationensemble;3)multiclass changerepresentationanddiscrimination.Theblockschemeoftheproposedapproach isshowninFigure1.3.
1.3.2.1. SCVreconstructionbasedonmultiscalemorphologicalprocessing
InthestandardC2 VA,aSCVindicatesapixelthatisunchangedorhasapossible kindofchangeaccordingtoitsspecificsignatureandconstructedchangevariables (i.e. ρ and θ ).However,theoriginalSCVsmaycontainabnormalspectralvariations andnoises,whichmayleadtoahighnumberofomissionandcommissionerrors.To addressthisproblem,themorphologicalprofile(MP)isappliedtobettermodeland preservethegeometricalstructureofchangetargets.Itisdefinedasasequenceof mathematicalclosingandopeningoperationsontheimagewithdifferentstructural element(SE)sizes.Inparticular,openingbyreconstruction(OR )andclosingby reconstruction(CR )(Benediktsson etal.2005)foragray-levelimage f aredefined as:
where i istheradiusoftheSE.Here, δ i p¨q and εi p¨q arethedilationanderosion operations,respectively. Rδ and Rε arethegeodesicreconstructionbydilationand erosion,respectively.Inparticular,thecomponentsofMP,i.e. OR and CR ,areable tosuppressbrighteranddarkerregions,respectively,thataresmallerthanthemoving SE,whilepreservingthegeometricalcharacteristicsofaregionlargerthantheSE (DallaMura etal.2010).Smallisolatedobjectsaremergedintoasurroundinglocal backgroundwhilethemainstructureiskept.Duetothefactthatdifferentimage objectsusuallyhavedifferentsizes,multiscalerepresentationallowsustoexplore differenthypotheticalspatialdomainsbyusingarangeofSEsizes,inordertoobtain thebestresponsefordifferentstructures(Mura etal.2008).
Figure1.3.
Forthe B-dimensionalSCVs XD ,atagivenscale i,its OR and CR arealso B-dimensional:
b Pr1,B s,i Pr1,N s
Notethateither O i R pXD q or C i R pXD q canbeusedasaninputforthedetector (e.g.S2 CVA),butambiguitiesmayariseduetotheselectionofaspecificoperator (i.e. OR or CR )anditsconsequentialeffects(i.e.suppressionofbrighterordarker objects).Thejointuseof OR and CR islikelytobemorereliable.Inthiswork, thefour-connectedneighborhoodwasconsidered,andthemarkerandmaskimage representedthedilationresultandtheoriginalinputband(for OR )(orcomplement imageoftheoriginalinputbandfor CR ),respectively.Adiskshapewasselected fortheSE,whichhasbeendemonstratedtobearobustshapeindifferentscenarios (Benediktsson etal.2005;Mura etal.2008).ThesizeoftheSE i wasincreasedfrom 1to6,inordertoimplementamultiscaleanalysisusingthereconstructedSCVs.
1.3.2.2. Multiscalechangeinformationensemble
ByincreasingthesizeoftheSE,changeobjectscanbemodeledatdifferent scales,whileexploringtheinteractionwiththesurroundingregionstopreservemore geometricaldetails.Accordingly,amorecomprehensivedescriptionofchangeobjects isobtainedthroughmultiscaleanalysis.Therefore,amultiscaleensembleisconducted onthereconstructedSCVs.Let OC i R pXD q bethestackingofthereconstructedSCVs (i.e. O i R p XD q and C i R pXD q )atagivensize i,havingadimensionalityof2ˆB.Itis definedas:
Then,theextendedSCV S ru,v s isdefinedasanintegrationofsequential OC i R p XD q,withlowerandupperboundsequalto u and v ,respectively:
Consequently,anextendedSCVfeaturesetwith2ˆB ˆM dimensionalityisbuilt, where M isthelengthofcomponentsinthesequence,i.e. M “ v u ` 1.Itisworth notingthat S ru,v s extendsthechangerepresentationalongthespectraldirection,as wellasconsideringthemultiscalespatialinformationintheensembleprocess.Then, S ru,v s isusedastheinputforthedetector.
Superpixel-Level Spectral Change Representation S up erp i xe lL eve l S pec t ra
Image Differencing X D
ChangeRepresentation
Optimal Segmentation Scale Determination O p ti ma l S e gme n t a ti on
Figure1.4. Blockschemeoftheproposedsuperpixel-levelmulticlassCDapproach
1.3.2.3. Multiclasschangerepresentationanddiscrimination
Theaimofthisstepistovisualizeanddiscriminatemulticlasschangespresentin thereconstructedSCVs.Tothisend,theS2 CVAdetectorintroducedinsection1.3.1 isappliedonthe S ru,v s.Notethatthedetectorexploitsnotonlyspectralvariations, butalsospatialvariationsrepresentedinthereconstructedSCVcomponents.Itisalso worthnotingthatthe2Dpolarscattergramprojectsmulticlasschangeinformation fromtheconsideredhigh-dimensionalreconstructedSCVsintoalow-dimensional (i.e.2D)featurespace,whichislossyandambiguousonthetypeofchanges.However, themostsignificantdiscriminativeinformationofdifferentchangesispreserved.
Insteadofusingthresholdingtosegmentthebinaryandmultipleclassesinthe variables ρ and θ ,thesimplebuteffectiveclustering,i.e. k -means,isusedfor generatingthefinalCDmap,whichdoesnotrelyonanyspecificdatadistribution. Thisisduetothefactthatchangedandunchangedpixelsin ρ andmulticlasschanges in θ donotalwaysfollowaGaussianmixturedistribution(Zanetti etal.2015).Thus, forthebinaryCDstep(i.e.separatingtwoclasses),thenumberofclusters kρ isequal to2,andforthemulticlassCDstep,thenumber kθ isdefinedasthenumberofchanges observedinthe2Dpolarscattergram.
1.3.3.
Superpixel-levelcompressedchangevectoranalysis
Inthissection,weproposeanunsupervisedsuperpixel-levelcompressedchange vectoranalysis(SPC2 VA)approachformulticlassCD.Thetraditionalpixel-level spectralchangeanalysisisconvertedintothe superpixellevel.Therefore,thespectral changerepresentationandidentificationareregularizedandenhancedunderthe superpixelconstraints.Theblockschemeoftheproposedapproachisshownin Figure1.4.
1.3.3.1.
Superpixel-levelspectralchangerepresentation
TheoriginalSCVsfocusonthespectralvariationrepresentationfromeach individualpixel,thusignoringspatialcorrelationwithneighboringpixelsandthelocal spectralhomogeneityassociatedwithrealland-coverobjects.Thismayleadtothe commissionandomissionerrors,andadecreaseintheoveralldetectionaccuracy. Superpixelsegmentationcancaptureimageredundancyandgenerateconvenient primitivestocomputerepresentativefeatures,whilereducingthecomplexityof thesubsequentprocessingandanalysis.Inthiswork,weusedthesimplelinear iterativeclustering(SLIC)algorithm(Achanta etal.2012)asthecorealgorithm forgeneratingsuperpixelsegments.Comparedtotheotherpopularsegmentation methods,SLICoffersabetterperformanceinboundaryadherenceandgenerates superpixelsefficientlyunderthesamehardwareconditions.Moreover,SLICis
memoryefficient,andonlyrequiresthestorageofthedistancefromeachpixelto itsnearestclustercenter.Mostimportantly,itcanbesmoothlyintegratedwithin theproposedmethod,especiallyfromthepixeltosuperpixel-levelspectralchange representationanddetection,whichdrivesaproperalgorithmutilization.
ThegeneralideaoftheSLICalgorithmistofindsmallregionalclustersby consideringtheirlocalhomogeneity(Achanta etal.2012).Thekeystepistocalculate thedistance d thatimplementsameasurementfromthespectral–spatialpointofview. Let dcolor and dxy bethespectralandspatialdistancesbetweentwogivenpixels α and β ,respectively,definedas:
Here, pL,A,B qT denotestheCIELABcolorspacevalues,with L beingthecolor lightnessand A and B representingcolorvaluesalongred-greenandblue-yellow axes,respectively. px,y qT denotesthecoordinatesofagivenpixel.Afinalweighted distancemeasure dαβ canbedefinedas:
where s isthewidthofgrids.Itcontrolsthesizeofcreatedsuperpixels,i.e.thegreater the s,thelargerthesuperpixels.Aroughlyequal-sizedgridintervalcanbedefinedas s “ apZ {N q,where Z isthetotalnumberofpixelsand N isthedesirednumber ofsuperpixels.Inreality,therealnumberofgeneratedsuperpixels(definedas N 1 ) mightbeslightlydifferentfrom N .Theparameter m controlstherelativeimportance betweenthecolorsimilarityandthespatialproximity.Thegreaterthe m,thegreater theemphasisonspatialproximityandthecompactnessofageneratedsuperpixel.A regular m valuecanbedefinedwithintherangeof[1,40].Formoredetails,readers canrefertothepaperbyAchanta etal.(2012).
Inordertoenhancethespectralvariationsduetothelimitedbandsinmultispectral images,principalcomponentanalysis(PCA)isappliedtotheoriginalSCVs.This strengthensthechangerepresentationandextendsthefeaturespace.Thefirstthree principalcomponents(i.e.PCs)areusedintheSLICalgorithmtogeneratethe segments(i.e.superpixels)withtheidentifiedboundaries.ThenoriginalSCVsare stackedwiththePCstocreateanenhancedfeatureset(denotedasSCVs-PC).Note thatnormalizationisconductedonSCVs-PCbandstomakedatadynamicrange consistent.Finally,ameanoperationisappliedoneachsegmentintheSCVs-PC
UnsupervisedChangeDetectioninMultitemporalRemoteSensingImages17 bands,wherethemeanvectorisusedtoreplacetheoriginalSCVs-PCvectors,in ordertoachievetheenhancedspectralchangerepresentationatthesuperpixellevel. Notethat,bydoingthis,thechangeinformationisconcentratedwithspectral–spatial coherenceandthecomputationalcostislargelyreducedwhencomparedwiththe originalpixel-wiseprocessing.
1.3.3.2. Determinationoftheoptimalsegmentationscale
Asmentionedpreviously,thenumberofsuperpixels N andthecompactness factor m needtobedeterminedintheSLICalgorithm.Notethatinpractical implementations,theparameter m impactslessthan N onthesegmentationresults. Therefore,aftermultipletrials,wefixed m =30inthiswork.Theonlyfocusis onthedeterminationoftheoptimalsegmentationscaleparameter N .Tothisend, anunsupervisedstrategyisappliedbasedontheanalysisoftheglobalentropy. Notethatafterthemeanoperationoneachsuperpixel,thetextureinformationin thesegmentswillberelativelysuppressed,whichmayhaveaninfluenceonthe followingCDperformance.Themainideaoftheusedcriterionistoevaluatethe informationmaintainedinthesuperpixel-levelsegmentedimageinheritedfromthe originalpixel-levelimage.Thus,theone-dimensionalimageentropy(GlobalEntropy, GE)(Han etal.2008)iscalculatedbasedonmulti-scalesegmentationresults,where theoptimalsegmentationscaleisdeterminedbyanalyzingthechangeofGEvalues:
where n denotesthegrayleveland p “ppk qk“1,2,...,n containsthehistogramcounts ofthefirstthreebandsof Y 1 .Itisworthnotingthatwiththeincreasing N,GEvalues areexpectedtoincreasecontinuouslyandapproachthevalueoftheoriginalimage. ThelogarithmicfunctionisthenusedtofittheGEresultstoestimatethethreshold fortheoptimalsegmentationscale.Adetaileddescriptionofthisstepisprovidedin Table1.1.
Step1:Initialize N ,whichisapproximatedasthesmalleroneintherowsandcolumnsof theinputimage,andthesegmentationscaleintervalissetapproximatelyequalto(row+ column)/20.
Step2:CalculatetheGEvalueofeachsegmentedimageunderdifferentsearchingscales.
Step3:FitthelogarithmicfunctionontheobtainedGEresultsandcalculatethegradient.
Step4:EstimatetheoptimalsegmentationscalebyanalyzingthegradientofGEvalues, wheretheconvergencethreshold TGE isdefinedapproximatelyequalto100/(row+ column)basedontheinputimage.
Table1.1. DeterminationoftheoptimalsegmentationscalebasedonGEanalysis
