Camera-BasedTechniques
2.8
Part1
Thefirststepinvideofiredetection(VFD)istoapplyabackgroundsubtractionalgorithmtoextractmovingregionsinthevideo.Thenthedetected regionsareanalyzedtemporallytobeclassifiedintermsofflickeringcharacteristics.Markovmodelsandfrequencydomaintechniquescanbeusedto identifyiftheflickeringcharacteristicsbelongtoflames.Inthenextstep, spatialanalysisisperformedtocheckfortheirregularitiesthatareusedto identifyflames.
Contents Part1 3 2.1 VFDinVisible/VisualSpectralRange 6 2.1.1 ColorDetection 10 2.1.2 MovingObjectDetection 11 2.1.3 MotionandFlickerAnalysisUsingFourierandWaveletTransforms 12 2.1.4 SpatialWaveletColorVariationandAnalysis 14 2.1.5 DynamicTextureandPatternAnalysis 15
SpatiotemporalNormalizedCovarianceDescriptors 16 2.3 ClassificationTechniques 19 2.4 EvaluationofVisibleRangeVFDMethods 19 2.5 VFDinIRSpectralRange 22 2.6 WildfireSmokeDetectionUsingVisibleRangeCameras 23 2.7 WildfireDetectionfromMovingAerialPlatforms 25 References 26 Part2 31
2.2
WildfireDetectionwithPTZCamerasUsingPanoramicBackgrounds 31 2.8.1 PanoramaGeneration 32 2.8.2 WildfireDetectionAlgorithm 36 2.8.3 ExperimentalResults 40 2.8.4 Conclusion 42 References 45
3 MethodsandTechniquesforFireDetection © 2016ElsevierLtd. http://dx.doi.org/10.1016/B978-0-12-802399-0.00002-8 Allrightsreserved.
Anothermethodistoextractfeaturesfromthemovingregionsand useclassifierswhoaretrainedofflinewithvideosoffireandfalsealarm sources.Itisalsopossibletouseac tivelearningalgorithmswhichare updatedonlinetoclassifyflameregions.Themostimportantproblem withvisiblerangefiredetectionsisthefalsealarms.Fire-coloredmoving regionscanbedifficulttodistinguishfromtheactualflames.Firedetectionalgorithmsaregenerallydevelopedforstationarycameras.When movingcamerasareused,itbecomesdifficulttodetecttheflickering characteristicsofflames.Detectionoffireusingamovingcameraisa futureresearchproblem.Anotherapp licationofvideo-basedfiredetectionissmokedetectionforearlywildfirewarningsystems.Forestsare usuallymonitoredusingPTZcamerasth atscanrecordedpresetpositions inaspecificorder.Thesecamerascan monitorlargerdistancesthanusual visiblerangecameras.Inwildfiredetectionapplications,smokebecomes visiblebeforetheflames;thereforeitmakessensetofocusonsmoke detectionforthesesystems.
Withthedecreasedcostofinfraredsensors,itbecamepossibletouselongandshort-waveinfraredcamerasforflamedetection.Sinceinfrared(thermal) camerasformimageswhoseintensitydependonthetemperatureofthe objects,theycouldbeusedtoreducemostofthefalsealarms.Inmostcases, thetemperatureofflamesishigherthanthesurroundingenvironmentandIR camerascansuccessfullydetecttheflickeringflames.Initially,infraredflame detectionalgorithmsprocessednearinfraredimages(NIR)toverifytheexistenceofflames.Morerecentmethodsstartedtouseshort-andlong-wave infrared(SWIR,LWIR)thermalcamerasforfireandflamedetection.
ThischapterisbasedontheVFDsurveypaperin[1].RecentlyproposedVFDtechniquesareviablealternativesorcomplementstoexisting firedetectiontechniquesandhaveshowntobeusefulinsolvingseveral problemsrelatedtothetraditionalsensors.Conventionalsensorsaregenerallylimitedtoindoorsandarenotapplicableinlargeopenspacessuchas shoppingcenters,airports,carparks,andforests.Theyrequireacloseproximitytothefireandmostofthemcannotprovideadditionalinformation aboutfirelocation,dimension,etc.Oneofthemainlimitationsofcommerciallyavailablefirealarmsystemsisthatitmaytakealongtimeforcarbonparticlesandsmoketoreachthe“point”detector.Thisiscalledthe “transportdelay.”Itisourbeliefthatvideoanalysiscanbeappliedinconditionsinwhichconventionalmethodsfail.VFDhasthepotentialtodetect thefirefromadistanceinlargeopenspacesbecausecamerascanmonitor
4 MethodsandTechniquesforFireDetection
“volumes.”Asaresult,VFDdoesnothavethetransportandthreshold delayfromwhichthetraditional“point”sensorssuffer.Assoonassmoke orflamesoccurinoneofthecameraviews,itispossibletodetectfire immediately.Weallknowthathumanbeingscandetectanuncontrolled fireusingtheireyesandvisionsystems,butaspointedoutabove,itisnot easytoreplicatehumanintelligence.
Theresearchinthisdomainwasstartedinthelatenineties.Mostofthe VFDarticlesavailableintheliteratureareinfluencedbythenotionof “weak”ArtificialIntelligence(AI)frameworkwhichwasfirstintroduced byHubertL.Dreyfusinhiscritiqueofthe“generalized”AI[3,4].Dreyfus presentssolidphilosophicalandscientificargumentsonwhythesearchfor “generalized”AIisfutile[5].Therefore,eachspecificproblemincluding VFDfireshouldbeaddressedasanindividualengineeringproblemwhich hasitsowncharacteristics[6].Itispossibletoapproximatelymodelthefire behaviorinvideousingvarioussignalandimageprocessingmethodsand automaticallydetectfirebasedontheinformationextractedfromvideo. However,thecurrentsystemssufferfromfalsealarmsbecauseofmodeling andtraininginaccuracies.
CurrentlyavailableVFDalgorithmsmainlyfocusonthedetectionand analysisofsmokeandflamesinconsecutivevideoimages.Inearlyarticles, mainlyflamedetectionwasinvestigated.Recently,thesmokedetection problemisalsoconsidered.Thereasonforthiscanbefoundinthefact thatsmokespreadsfasterandinmostcaseswilloccurmuchfasterinthe fieldofviewofthecameras.Inwildfireapplications,itmaynotevenbe possibletoobserveflamesforalongtime.Themajorityofthestate-ofthe-artdetectiontechniquesfocusesonthecolorandshapecharacteristics, togetherwiththetemporalbehaviorofsmokeandflames.However,due tothevariabilityofshape,motion,transparency,colors,andpatternsof smokeandflames,manyoftheexistingVFDapproachesarestillvulnerable tofalsealarms.Duetonoise,shadows,illuminationchanges,andother visualartifactsinrecordedvideosequences,developingareliabledetection systemisachallengetotheimageprocessingandcomputervision community.
Withtoday’stechnology,itisnotpossibletohaveafullyreliableVFD systemwithoutahumanoperator.However,currentsystemsareinvaluable toolsforsurveillanceoperators.Itisalsoourstrongbeliefthatcombining multi-modalvideoinformationusingbothvisibleandinfrared(IR)technologywillleadtohigherdetectionaccuracy.Eachsensortypehasitsown
5 Camera-basedtechniques
specificlimitations,whichcanbecompensatedbyothertypesofsensors. Although,itwouldbedesirabletodevelopafiredetectionsystemwhich couldoperateontheexistingclosedcircuittelevision(CCTV)equipment withoutintroducinganyadditionalcost.However,thecostofusingmultiplevideosensorsdoesnotoutweighthebenefitofmulti-modalfireanalysis.ThefactthatIRmanufacturersalsoensureadecreaseinthesensorcost inthenearfuturefullyopensthedoortomulti-modalvideoanalysis.VFD camerascanalsobeusedtoextractusefulrelatedinformation,suchasthe presenceofpeoplecaughtinthefire,firesize,firegrowth,smoke direction,etc.
VFDsystemscanbeclassifiedintovarioussubcategoriesaccordingto (i)thespectralrangeofthecameraused, (ii)thepurpose(flameorsmokedetection), (iii)therangeofthesystem. Thereareoverlapsbetweenthecategoriesabove.
2.1VFDINVISIBLE/VISUALSPECTRALRANGE
Overthelastyears,thenumberofpapersaboutvisualfiredetectioninthe computervisionliteraturehasgrownexponentially[2].Asis,thisrelatively newsubjectinvisionresearchisinfullprogressandhasalreadyproduced promisingresults.However,thisisnotacompletelysolvedproblem,aswith mostcomputervisionproblems.Behaviorofsmokeandflamesofanuncontrolledfiredifferswithdistanceandillumination.Furthermore,camerasare notcolorand/orspectralmeasurementdevices.Theyhavedifferentsensors andcolorandilluminationbalancingalgorithms.Theymayproducedifferentimagesandvideoforthesamescenebecauseoftheirinternalsettingsand algorithms.
Inthissection,achronologicaloverviewofthestate-of-the-art(ie,acollectionoffrequentlyreferencedpapersonshortrange[<100m])firedetectionmethodsispresentedinthetablesbelow.Foreachofthesepapers,we investigatedtheunderlyingalgorithmsandcheckedtheappropriatetechniques.Inthefollowing,wediscusseachofthesedetectiontechniques andanalyzetheiruseinthelistedpapers.
6 MethodsandTechniquesforFireDetection
State-of-the-art:underlyingtechniques(PART1:2002-2007).
Paper Color detection Moving object detection Flicker/ energy (wavelet) analysis Spatial difference analysis Dynamic texture/ Pattern analysis Disorder analysisSubblocking Training (models, NN,SVM, …) Cleanup postprocessing Localization/ analysis Flame detection Smoke detection Phillips[7]RGB XX XX X GomezRodriguez [8] XX X X GomezRodriguez [9] XX X X Chen[10]RGB/ HSI XX X X Liu[11]HSV X X X Marbach[12]YUV X X X Toreyin[13]RGBXXX X Toreyin[14]YUVXX X X Celik[15]YCbCr/ RGB XX Xu[16]XX X XX
Xiong [18]
State-of-the-art:underlyingtechniques(PART2:2007-2009).
[20]
[21]
[23]
[25]
[26]
Paper Color detection Moving object detection Flicker/ energy (wavelet) analysis Spatial difference analysis Dynamic texture/ pattern analysis Disorder analysisSubblocking Training (models, NN,SVM, …) Cleanup postprocessing Localization/ analysis Flame detection Smoke detection Celik[17]RGBXXXXX
XXXX
XX XX Calderara
RGBXX XX X Piccinini
RGBXX X X Yuan[22]RGBX XX X Borges
RGB X X Qi[24]RGB/ HSV XX X X Yasmin
RGB/ HSI XX XX X
XX XX
Lee[19]RGBX
Gubbi
State-of-the-art:underlyingtechniques(PART3:2010-2011).
Paper Color detection Moving object detection Flicker/ energy (wavelet) analysis Spatial difference analysis Dynamic texture/ Pattern analysis Disorder analysisSubblocking Training (models, NN,SVM, …) Cleanup postprocessing Localization/ analysis Flame detection Smoke detection
HSI XXXX
XXXXXX
X X XX Ko[30]RGBXX X X GonzalezGonzalez
XX X Borges[32]RGB X X X X Van Hamme [33] HSV X XX X Celik[34]CIE L*a*b* XX X X X Yuan[35]X XX Rossi[36]YUV/ RGB XXXX
Chen[27]RGB/
Gunay[28]RGB/ HSI
Kolesov[29]X
[31]
2.1.1ColorDetection
ColordetectionwasoneofthefirstdetectiontechniquesusedinVFDandis stillusedinalmostalldetectionmethods.Themajorityofthecolor-based approachesinVFDmakeuseofRGBcolorspace,sometimesincombinationwithHSI/HSVsaturation[10,24,27,28].Themainreasonforusing RGBisthatalmostallvisiblerangecamerashavesensorsdetectingvideo inRGBformatandthereistheobviousspectralcontentassociatedwiththis colorspace.ItisreportedthatRGBvaluesofflamepixelsareintheredyellowcolorrangeindicatedbytherule(R > G > B)asshownin Fig.2.1. Similarly,insmokepixels,R,G,andBvaluesareveryclosetoeachother. Morecomplexsystemsuserule-basedtechniquessuchasGaussiansmoothed colorhistograms[7],statisticallygeneratedcolormodels[15],andblending functions[20].Itisobviousthatcolorcannotbeusedbyitselftodetectfire becauseofthevariabilityincolor,density,lighting,andbackground. However,thecolorinformationcanbeusedasapartofamoresophisticated system.Forexample,chrominancedecreaseisusedinsmokedetection
Figure2.1 Colordetection:smokeregionpixelshavecolorvaluesthatareclosetoeach other.Pixelsofflameregionslieinthered-yellowrangeofRGBcolorspacewith R > G > B.
10 MethodsandTechniquesforFireDetection
schemesofRefs.[14,2].Luminancevalueofsmokeregionsshouldbehigh formostsmokesources.Ontheotherhand,thechrominancevaluesshould beverylow.
TheconditionsinYUVcolorspaceareasfollows:
Condition1: Y > TY
Condition2: U 128 jj < TU & jV 128j < TV . where Y, U,and V aretheluminanceandchrominancevaluesofaparticular pixel,respectively.Theluminancecomponent Y takesvaluesintherange [0,255]inan8-bitquantizedimageandthemeanvaluesofchrominance channels U and V areincreasedto128sothattheyalsotakevaluesbetween 0and255.Thethresholds TY, TU,and TV areexperimentallydetermined[37].
2.1.2MovingObjectDetection
MovingobjectdetectionisalsowidelyusedinVFDbecauseflamesand smokearemovingobjects.Todetermineifthemotionisduetosmoke oranordinarymovingobject,furtheranalysisofmovingregionsinvideo isnecessary.
Well-knownmovingobjectdetectionalgorithmsarebackground(BG) subtractionmethods[16,21,18,14,13,17,20,22,27,28,30,34],temporal differencing[19],andopticalflowanalysis[9,8,29].Theycanallbeused aspartofaVFDsystem.
Inbackgroundsubtractionmethods,itisassumedthatthecameraisstationary.In Fig.2.2,abackgroundsubtraction-basedmotiondetection
Dynamic BG model BG subtraction Fire Fire+moving object Figure2.2 Movingobjectdetection:backgroundsubtractionusingdynamicbackground model. 11 Camera-basedtechniques
exampleisshownusingthedynamicbackgroundmodelproposedbyCollinsetal.[38].ThisGaussianMixtureModel-basedapproachmodelwas usedinmanyofthearticleslistedintablesabove.
SomeoftheearlyVFDarticlessimplyclassifiedfire-coloredmoving objectsasfirebutthisapproachleadstomanyfalsealarms,becausefalling leavesinautumnorfire-coloredordinaryobjects,etc.,mayallbeincorrectly classifiedasfire.Furtheranalysisofmotioninvideoisneededtoachieve moreaccuratesystems.
2.1.3MotionandFlickerAnalysisUsingFourierandWavelet Transforms
Asitiswellknown,flamesflickerinuncontrolledfires,thereforeflicker detection[24,18,12,13,27,28,30]invideoandwavelet-domainsignal energyanalysis[21,14,20,26,31,39]canbeusedtodistinguishordinary objectsfromfire.Thesemethodsfocusonthetemporalbehaviorofflames andsmoke.Asaresult,flamecoloredpixelsappearanddisappearattheedges ofturbulentflames.Theresearchin[16,18]showsexperimentallythatthe flickerfrequencyofturbulentflamesisaround10Hzandthatitisnotgreatly affectedbytheburningmaterialandtheburner.Asaresult,usingfrequency analysistodifferentiateflamesfromothermovingobjectsisproposed.However,anuncontrolledfireinitsearlystageexhibitsatransitiontochaosdue tothefactthatthecombustionprocessconsistsofnonlinearinstabilities whichresultintransitiontochaoticbehaviorviaintermittency[40–43]. Consequently,turbulentflamescanbecharacterizedasachaoticwide-band frequencyactivity.Therefore,itisnotpossibletoobserveasingleflickering frequencyinthelightspectrumduetoanuncontrolledfire.ThisphenomenonwasobservedbyindependentresearchersworkingonVFDand methodswereproposedaccordingly[14,44,27].Similarly,itisnotpossible totalkaboutaspecificflickerfrequencyforsmoke,butweclearlyobservea time-varyingmeanderingbehaviorinuncontrolledfires.Therefore,smoke flickerdetectiondoesnotseemtobeaveryreliabletechnique,butitcanbe usedaspartofamulti-featurealgorithmfusingvariousvisioncluesforsmoke detection.TemporalFourieranalysiscanstillbeusedtodetectflickering flames,butwebelievethatthereisnoneedtodetectspecifically10Hz. AnincreaseinFourierdomainenergyin510Hzisanindicatorofflames. Thetemporalbehaviorofsmokecanbeexploitedbywaveletdomain energyanalysis.Assmokegraduallysoftenstheedgesinanimage,Toreyin etal.[14]foundtheenergyvariationbetweenbackgroundandcurrent imageasacluetodetectthepresenceofsmoke.Inordertodetecttheenergy
12 MethodsandTechniquesforFireDetection
decreaseinedgesoftheimage,theyusetheDiscreteWaveletTransform (DWT).TheDWTisamulti-resolutionsignaldecompositionmethod obtainedbyconvolvingtheintensityimagewithfilterbanks.Astandard halfbandfilterbankproducesfourwaveletsubimages:theso-calledlowlowversionoftheoriginalimage Ct,andthehorizontal,vertical,anddiagonalhighfrequencybandimages Ht, Vt,and Dt.Thehigh-bandenergyfrom subimages Ht, Vt,and Dt isevaluatedbydividingtheimage It inblocks bk of arbitrarysizeasfollows:
Sincecontributionofedgesaremoresignificantinhighbandwavelet imagescomparedtoflatareasoftheimage,itispossibletodetectsmoke usingthedecreasein E(I t, b k).Astheenergyvalueofaspecificblock variessignificantlyovertimeinthe presenceofsmoke,temporalanalysis oftheratiobetweenthecurrentinputframewaveletenergyandthe backgroundimagewaveletenergyisusedtodetectthesmokeasshown in Fig.2.3
Figure2.3 DWT-basedvideosmokedetection:Whenthereissmoke,theratiobetween theinputframewaveletenergyandtheBGwaveletenergydecreasesandshowsahigh degreeofdisorder.
EIt , bk ðÞ¼ Xi, j 2bk H 2 t i, j ðÞ + V 2 t i, j ðÞ + D2 t i, j ðÞ (2.1)
13 Camera-basedtechniques
2.1.4SpatialWaveletColorVariationandAnalysis
Flamesofanuncontrolledfirehavevaryingcolorsevenwithinasmallarea. Spatialcolordifferenceanalysis[24,13,28,32]focusesonthischaracteristic. Usingrangefilters[24],variance/histogramanalysis[32],orspatialwavelet analysis[13,28],thespatialcolorvariationsinpixelvaluesareanalyzedto distinguishordinaryfire-coloredobjectsfromuncontrolledfires.In Fig.2.4, theconceptofspatialdifferenceanalysisisfurtherexplainedbymeansofa histogram-basedapproach,whichfocusesonthestandarddeviationofthe greencolorband.ItwasobservedbyQiandEbert[24]thatthiscolorband isthemostdiscriminativebandforrecognizingthespatialcolorvariationof flames.Thiscanalsobeseenbyanalyzingthehistograms.Greenpixelvalues varymorethanredandbluevalues.Ifthestandarddeviationofthegreen colorbandexceeds tσ ¼ 50(:Borges[32])inatypicalcolorvideotheregion islabeledasacandidateregionforaflame.Forsmokedetection,ontheother hand,experimentsrevealedthatthesetechniquesarenotalwaysapplicable becausesmokeregionsoftendonotshowashighspatialcolorvariationas flameregions.Furthermore,texturedsmoke-coloredmovingobjectsare difficulttodistinguishfromsmokeandcancausefalsedetections.Ingeneral, smokeinanuncontrolledfireisgrayanditreducesthecolorvariationinthe background.Therefore,inYUVcolorspaceweexpecttohavereductionin thedynamicrangeofchrominancecolorcomponentsUandVafterthe appearanceofsmokeintheviewingrangeofcamera.
Figure2.4 Spatialdifferenceanalysis:incaseofflames,thestandarddeviation σ G ofthe greencolorbandoftheflameregionexceeds tσ ¼ 50(:Borges[32]).
14 MethodsandTechniquesforFireDetection
2.1.5DynamicTextureandPatternAnalysis
Adynamictextureorpatterninvideo,suchassmoke,flames,water,and leavesinthewind,canbesimplydefinedasatexturewithmotion [45,46](ie,aspatiallyandtime-varyingvisualpatternthatformsanimage sequenceorpartofanimagesequencewithacertaintemporalstationarity) [47].Althoughdynamictexturesareeasilyobservedbyhumaneyes,theyare difficulttodiscernusingcomputervisionmethodsasthespatiallocationand extentofdynamictexturescanvarywithtimeandtheycanbepartiallytransparent.Somedynamictextureandpatternanalysismethodsinvideo [29,33,35]arecloselyrelatedtospatialdifferenceanalysis.Recently,these techniqueshavealsobeenappliedtotheflameandsmokedetectionproblem [46].Currently,awidevarietyofmethodsincludinggeometric,modelbased,statistical,andmotion-basedtechniquesareusedfordynamictexture detection[48–50].
In Fig2.5,dynamictexturedetectionandsegmentationexamplesare shown,usingvideoclipsfromtheDynTexdynamictextureandBilkentdatabases[51,52,50,47].Contoursofdynamictextureregions(eg,fire,water,and steam)areshowninthisfigure.Dynamicregionsinvideoseemtobesegmentedverywell.However,duetothehighcomputationalcost,thesegeneraltechniquesarenotusedinpracticalfiredetectionalgorithmswhichshould runonlow-costcomputers,FPGAs,ordigitalsignalprocessors.Iffuture developmentsincomputersandgraphicsacceleratorscouldlowerthecomputationalcost,dynamictexturedetectionmethodsmaybeincorporatedinto thecurrentlyavailableVFDsystemstoachievemorereliablesystems.
Ordinarymovingobjectsinvideo,suchaswalkingpeople,haveapretty stableoralmostperiodicboundaryovertime.Ontheotherhand,uncontrolledflameandsmokeregionsexhibitchaoticboundarycontours.Therefore,disorderanalysisofboundarycontoursofamovingobjectisusefulfor firedetection.Someexamplesoffrequentlyusedmetricsarerandomnessof areasize[23,32],boundaryroughness[14,11,28,32],andboundaryarea
Figure2.5 Dynamictexturedetection:contoursofdetecteddynamictextureregions areshowninthefigure(ResultsfromDYNTEXandBilkentdatabases [51,53]).
15 Camera-basedtechniques
disorder[18].Althoughthosemetricsdifferindefinition,theoutcomeof eachofthemisalmostidentical.Inthesmokedetectordevelopedby Verstocketal.[2],disorderanalysisoftheBoundaryAreaRoughness (BAR)isused,whichisdeterminedbyrelatingtheperimeteroftheregion tothesquarerootofthearea(Fig.2.6).AnothertechniqueisthehistogrambasedorientationaccumulationbyYuan[22].Thistechniquealsoproduces gooddisorderdetectionresults,butitiscomputationallymorecomplex thantheformermethods.Relatedtothedisorderanalysisisthegrowing ofsmokeandflameregionsintheearlystageofafire.In[31,34],thegrowth rateoftheregion-of-interestisusedasafeatureparameterforfiredetection. Comparedtodisordermetrics,however,growthanalysisislesseffectivein detectingthesmoke,especiallyinwildfiredetection.Thisisbecausethe smokeregionappearstogrowveryslowlyinwildfireswhentheyareviewed fromlongdistances.Furthermore,anordinaryobjectmaybeapproaching thecamera.
2.2SPATIOTEMPORALNORMALIZEDCOVARIANCE DESCRIPTORS
Arecentapproachwhichcombinescolorandspatiotemporalinformationby regioncovariancedescriptorsisusedinEuropeanCommissionfundedFP-7 FIRESENSEproject[54–56].Themethodisbasedonanalyzingthespatiotemporalblocks.Theblocksareobtainedbydividingthefire-andsmokecoloredregionsinto3Dregionsthatoverlapintime.Classificationofthe featuresisperformedonlyatthetemporalboundariesofblocksinsteadof
16 MethodsandTechniquesforFireDetection
Figure2.6 Boundaryarearoughnessofconsecutiveflameregions.
performingitateachframe.Thisreducesthecomputationalcomplexityof themethod.
CovariancedescriptorsareproposedbyTuzel,Porikli,andMeertobe usedinobjectdetectionandtextureclassificationproblems[54,55].In [57,75,76]temporallyextendednormalizedcovariancedescriptorstoextract featuresfromvideosequencesareproposed.
Temporallyextendednormalizedcovariancedescriptorsaredesignedto describespatiotemporalvideoblocks.Let I(i, j, n)betheintensityof(i, j)th pixelofthe nthimageframeofaspatiotemporalblockinvideo.Thepropertyparametersdefinedinequationsbelowareusedtoformacovariance matrixrepresentingspatialinformation.Inadditiontospatialparameters, temporalderivatives, It and Itt areintroducedwhicharethefirstandsecond derivativesofintensitywithrespecttotime,respectively.Byaddingthese twofeaturestothepreviouspropertyset,normalizedcovariancedescriptors canbeusedtodefinespatiotemporalblocksinvideo.
Forflamedetection:
Ri, j , n ¼ Redi, j , n ðÞ; (2.2) Gi, j , n ¼ Greeni, j , n ðÞ; (2.3) Bi, j , n ¼ Bluei, j , n ðÞ; (2.4) Ii, j , n ¼ Intensityi, j , n ðÞ; (2.5) Ixi, j , n ¼ @ Intensityi, j , n ðÞ @ i ; (2.6) Iyi, j , n ¼ @ Intensityi, j , n ðÞ @ j ; (2.7) Ixxi, j , n ¼ @ 2 Intensityi, j , n ðÞ @ i2 ; (2.8) Iyyi, j , n ¼ @ 2 Intensityi, j , n ðÞ @ j 2 ; (2.9) Iti, j , n ¼ @ Intensityi, j , n ðÞ @ n ; (2.10) and Itti, j , n ¼ @ 2 Intensityi, j , n ðÞ @ n2 (2.11) 17 Camera-basedtechniques
Inordertoimprovethedetectionperformance,themajorityofthearticlesintheliteratureuseacombinationofthefirefeatureextractionmethods describedabove.Dependingonthefire/environmentalcharacteristics,one combinationoffeatureswilloutperformtheother,andviceversa.In Section2.4,wedescribeanadaptivefusionmethodcombiningtheresultsof variousfiredetectionmethodsinanonlinemanner.
Itshouldbepointedoutthatarticlesintheliteratureandthosewhichare referencedinthisstate-of-the-artreviewindicatethatordinaryvisiblerange camera-baseddetectionsystemspromisegoodfiredetectionresults.However, theystillsufferfromasignificantamountofmisseddetectionsandfalsealarms inpracticalsituations,asinothercomputervisionproblems[5,6].Themain causeoftheseproblemsisthefactthatvisualdetectionisoftensubjecttoconstraintsregardingthesceneunderinvestigation(eg,changingenvironmental conditions,differentcameraparameters,andcolorsettingsandillumination). Itisalsoimpossibletocomparethearticleswitheachotheranddeterminethe bestone.Thisisbecausetheyusedifferenttraininganddatasets.
Yi, j , n ¼ Luminancei, j , n ðÞ; (2.12) Ui
j
n
ChrominanceUi
j , n
Vi, j , n ¼ ChrominanceVi, j , n ðÞ; (2.14) Ii, j , n ¼ Intensityi, j , n ðÞ; (2.15) Ixi, j , n ¼ @ Intensityi, j , n ðÞ @ i ; (2.16) Iyi, j , n ¼ @ Intensityi, j , n ðÞ @ j ; (2.17) Ixxi, j , n ¼ @ 2 Intensityi, j , n ðÞ @ i2 ; (2.18) Iyyi, j , n ¼ @ 2 Intensityi, j , n ðÞ @ j 2 ; (2.19) Iti, j , n ¼ @ Intensityi, j , n ðÞ @ n ; (2.20) Itti, j , n ¼ @ 2 Intensityi, j , n ðÞ @ n2 (2.21)
Forsmokedetection:
,
,
¼
,
ðÞ; (2.13)
18 MethodsandTechniquesforFireDetection
Adatasetoffireandnon-firevideosisavailabletotheresearchcommunity inEuropeanCommissionfundedFIRESENSEprojectwebpage[56].These testvideoswereusedfortrainingandtestingpurposesofthesmokeandflame detectionalgorithmsdevelopedwithintheFIRESENSEproject.Thus,afair comparisonofthealgorithmsdevelopedbyindividualpartnerscouldbeconducted.Thetestdatabaseincludes27testand29trainingsequencesofvisible spectrumrecordingsofflamescenes,15testand27trainingsequencesofvisiblespectrumrecordingsofsmokescenes,and22testand27training sequencesofvisiblespectrumrecordingsofforestsmokescenes.Thisdatabase iscurrentlyavailabletoregisteredusersoftheFIRESENSEwebsite[Reference:FIRESENSEprojectFileRepository, http://www.firesense.eu,2012].
2.3CLASSIFICATIONTECHNIQUES
Apopularapproachfortheclassificationofthemulti-dimensionalfeature vectorsobtainedfromeachcandidateflameorsmokeblobisSVMclassification,typicallywithRadialBasisFunction(RBF)kernels.Alargenumber offramesoffireandnon-firevideosequencesneedtobeusedfortraining theseSVMclassifiers;otherwisethenumberoffalsealarms(falsepositivesor truenegatives)maybesignificantlyincreased.
OtherclassificationmethodsincludetheAdaBoostmethod[22],neural networks[29,35],Bayesianclassifiers[30,32],Markovmodels[28,33],and rule-basedclassification[58].
Asinanyvideoprocessingmethod,morphologicaloperations,subblocking,andclean-uppost-processing,suchasmedian-filtering,areused asanintegralpartofanyVFDsystem[21,22,25,20,26,33,36,59].
2.4EVALUATIONOFVISIBLERANGEVFDMETHODS
AnevaluationofdifferentvisiblerangeVFDmethodsispresentedin Table2.1. Table2.1 summarizescomparativedetectionresultsforthesmoke andflamedetectionalgorithmbyVerstockt[2](Method1),acombination oftheflamedetectionmethodbyCeliketal.[60]andthesmokedetection byToreyinetal.[14](Method2)andacombinationofthefeature-based flamedetectionmethodbyBorgesetal.[23]andthesmokedetection methodbyXiongetal.[18](Method3).Amongvariousalgorithms, Verstockt’smethodisarelativelyrecentone,whereasflamedetection methodsbyCelikandBorgesandthesmokedetectionmethodsbyToreyin andXiongarecommonlyreferencedmethodsintheliterature.
19 Camera-basedtechniques
Table2.1 AnevaluationofdifferentvisiblerangeVFDmethods
(#frames) #Fireframes groundtruth
#Detectedfireframes#FalsepositiveframesDetectionrate* Method Method Method 123123123 Paperfire(1550)956 897922874917220.930.950.89 Carfire(2043)1415 12931224103738130.910.860.73 Movingpeople(886)0 50285028 Woodfire(592)522 510489504179160.940.920.93 Bunsenburner(115)98 5953320000.600.540.34 Movingcar(332) 0 0131101311 Strawfire(938)721 6796986731621120.920.930.92 Smoke/fogmachine (1733) 923 834654789934520.890.670.80 Poolfire(2260)1844 1665163416180000.900.890.88
20 MethodsandTechniquesforFireDetection
Videosequence
*Detectionrate ¼ (#detectedfireframes #falsealarms)/#fireframes.
Testsequencesusedforperformanceevaluationarecapturedindifferent environmentsundervariousconditions.Snapshotsfromtestvideosarepresentedin Fig.2.7.Inordertoobjectivelyevaluatethedetectionresultsof differentmethods,the“detectionrate”metric[2,61]isused,whichiscomparabletotheevaluationmethodsusedbyCeliketal.[60]andToreyinetal. [13].Thedetectionrateequalstheratioofthenumberofcorrectlydetected framesasfire(ie,thedetectedframesasfireminusthenumberoffalsely detectedframes)tothenumberofframeswithfireinthemanuallycreated groundtruthframes.Asresultsindicate,thedetectionperformancesofdifferentmethodsarecomparablewitheachother.
ComparisonofthesmokeandflamedetectionmethodbyVerstockt[2] (Method1),thecombinedmethodbasedontheflamedetectorbyCelik etal.[60]andthesmokedetectordescribedinToreyinetal.[14](Method 2),andcombinationofthefeature-basedflamedetectionmethodbyBorges etal.[23]andthesmokedetectionmethodbyXiongetal.[18](Method3).
21 Camera-basedtechniques
Figure2.7 Snapshotsfromtestsequenceswithandwithoutfire.
2.5VFDINIRSPECTRALRANGE
Whenthereisverylittleornovisiblelight,orthecoloroftheobjecttobe detectedissimilartothebackground,IRimagingsystemsprovidesolutions [62–68].AlthoughthereisanincreasingtrendinIRcamera-basedintelligentvideoanalysis,thereareveryfewpapersintheareaofIR-basedfire detection[64–68].ThisismainlyduetothehighcostofIRimagingsystems comparedtoordinarycameras.ManufacturerspredictthatIRcameraprices willgodowninthenearfuture.Therefore,weexpectthatthenumberof IRimagingapplicationswillincreasesignificantly[63].Long-waveInfrared (8-12micronrange)camerasarethemostwidelyavailablecamerasonthe market.Long-waveInfrared(LWIR)lightgoesthroughsmoke,thereforeit iseasytodetectsmokeusingLWIRimagingsystems.Nevertheless,results fromexistingworkalreadyensurethefeasibilityofIRcamerasforflame detection.
Owrutskyetal.[64]workedintheNIRspectralrangeandcomparedthe globalluminosity L,whichisthesumofthepixelintensitiesofthecurrent frame,toareferenceluminosity Lb andathreshold Lth.Ifthereareanumber ofconsecutiveframeswhere L exceedsthepersistencecriterion Lb + Lth,the systemgoesintoanalarmstage.Althoughthisfairlysimplealgorithmseems toproducegoodresultsinthereportedexperiments,itslimitedconstraints doraisequestionsaboutitsapplicabilityinlargeandopenuncontrolledpublicplacesanditwillprobablyproducemanyfalsealarmstohotmoving objects,suchascarsandhumanbeings.AlthoughthecostofNIRcameras isnothigh,theirimagingrangesareshortercomparedtovisiblerangecamerasandotherIRcameras.
Toreyinetal.[65]detectflamesinLWIRbysearchingforbrightlookingmovingobjectswithrapidtime-varyingcontours.Awavelet domainanalysisofthe1D-curverepresentationofthecontoursisusedto detectthehighfrequencynatureoftheboundaryofafireregion.Inaddition,thetemporalbehavioroftheregionisanalyzedusingaHiddenMarkov Model(HMM).Thecombinationofbothspatialandtemporalcluesseems moreappropriatethantheluminosityapproachand,accordingtothe authorstheirapproachgreatlyreducesfalsealarmscausedbyordinarybright movingobjects.Asimilarcombinationoftemporalandspatialfeaturesisalso usedbyBoschetal.[66].Hotspots(ie,candidateflameregions)aredetected byautomatichistogram-basedimagethresholding.Byanalyzingtheintensity,signature,andorientationoftheseresultinghotobjects’regions,discriminationbetweenflamesandotherobjectsismade.Verstocketal.[2]
22 MethodsandTechniquesforFireDetection
alsoproposedanIR-basedfiredetectorwhichmainlyfollowsthelatter feature-basedstrategy,butcontrarytoBoschetal.’swork[66]adynamic backgroundsubtractionmethodisused,whichaimsatcopingwiththe time-varyingcharacteristicsofdynamicscenes.
Tosumup,itshouldbepointedoutthatitisnotstraightforwardto detectfiresusingIRcameras.NoteverybrightobjectinIRvideoisasource ofwildfire.ItisimportanttomentionthatIRimaginghasitsownspecific limitations,suchasthermalreflections,IRblocking,andthermal-distance problems.Insomesituations,IR-baseddetectionwillperformbetterthan visibleVFD,butunderothercircumstances,visibleVFDcanimproveIR flamedetection.Thisisduetothefactthatsmokeappearsearlierand becomesvisiblefromlongdistancesinatypicaluncontrolledfire.Flames andburningobjectsmaynotbeintheviewingrangeoftheIRcamera. Assuch,higherdetectionaccuracieswithlowerfalsealarmratescanbe achievedbycombiningmulti-spectrumvideoinformation.Variousimage fusionmethodsmaybeemployedforthispurpose[69,70].Clearly,eachsensortypehasitsownspecificlimitations,whichcanonlybecompensatedby othertypesofsensors.
2.6WILDFIRESMOKEDETECTIONUSINGVISIBLERANGE CAMERAS
Aspointedoutintheprevioussection,smokeisclearlyvisiblefromlong distancesinwildfiresandforestfires.Inmostcases,flamesarehindered bytrees.Therefore,IRimagingsystemsmaynotprovidesolutionsforearly firedetectioninwildfiresbutordinaryvisiblerangecamerascandetect smokefromlongdistances.
Therearemanymethodsintheliteratureforwildfiresmokedetection [37,71,77].ThemethoddevelopedforFIRESENSE[56]projectcombines rule-basedandlearning-basedmethods.Therearefivemainalgorithmsused inthemethod.
ThefirstalgorithmusesdoubleIIR-basedbackgroundsubtractionand doublebackgroundstofindslow-movingregions.Duringtheinitialphases ofwildfire,smokeappearstomoveslowlywhenviewedfromadistance. Thisobservationisusedtoseparateslowmovingsmokeregionsfromother fastmovingobjects.
ThesecondalgorithmusesthresholdsinYUVcolorspacetoextract smokecoloredregions.Smokeisassumedtohavegray-to-whitecolorduringtheinitialstagesofwildfirecausedbytheburningofvegetation.
23 Camera-basedtechniques
