METHODSAND TECHNIQUESFOR FIREDETECTION
AcademicPressisanimprintofElsevier
125LondonWall,London,EC2Y5AS,UK
525BStreet,Suite1800,SanDiego,CA92101-4495,USA 50HampshireStreet,5thFloor,Cambridge,MA02139,USA TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UK
Copyright © 2016ElsevierLtd.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyany means,electronicormechanical,includingphotocopying,recording,oranyinformation storageandretrievalsystem,withoutpermissioninwritingfromthepublisher.Details onhowtoseekpermission,furtherinformationaboutthePublisher’spermissionspolicies andourarrangementswithorganizationssuchastheCopyrightClearanceCenterand theCopyrightLicensingAgency,canbefoundatourwebsite: www.elsevier.com/ permissions
Thisbookandtheindividualcontributionscontainedinitareprotectedunder copyrightbythePublisher(otherthanasmaybenotedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchand experiencebroadenourunderstanding,changesinresearchmethods,professional practices,ormedicaltreatmentmaybecomenecessary.
Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledge inevaluatingandusinganyinformation,methods,compounds,orexperimentsdescribed herein.Inusingsuchinformationormethodstheyshouldbemindfuloftheirownsafety andthesafetyofothers,includingpartiesforwhomtheyhaveaprofessionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors, assumeanyliabilityforanyinjuryand/ordamagetopersonsorpropertyasamatterof productsliability,negligenceorotherwise,orfromanyuseoroperationofanymethods, products,instructions,orideascontainedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN:978-0-12-802399-0
ForinformationonallAcademicPresspublications visitourwebsiteat http://store.elsevier.com/
Publisher: JoeHayton
AcquisitionEditor: TimPitts
EditorialProjectManager: CharlotteKent
ProductionProjectManager: JasonMitchell
Designer: MariaIne ˆ sCruz
broadlyinsignalprocessingandpatternrecognitionwithapplicationsto image/videoanalysis,andcommunicationsystems.Hisresearchisfocused ondevelopingnovelalgorithmstoanalyzeandcompresssignalsfromamultitudeofsensorssuchasvisible/infra-red/hyperspectralcameras,microphones,passiveinfra-redsensors,vibrationsensors,andspectrumsensors forwirelesscommunications.
StevenVerstockt receivedhismaster’sdegreeininformaticsfromGhent Universityin2003.Followinghisstudiesinappliedinformatics,hebegan teachingMultimediacoursesatHogeschoolGentandattheendof2007, hejoinedtheELITLaboftheUniversityCollegeWest-Flandersasa researcher.In2008,hestartedaPhDonvideofireanalysisattheMultimedia LaboftheDepartmentofElectronicsandInformationSystemsofGhent University—iMinds(Belgium).Since2012,hehasworkedasapostdoctoral researcherinthislabfocusingonmulti-sensorfireanalysis.InOctober2015, hewasappointedatenuretrackpositionasassistantprofessorinMultimedia atthesamelab.
analysisoftheflameflickerexistinginherentlyinuncontrolledfires.The PIRsensorsarecommonlyusedforoccupancydetectionpurposesinbuildings.Utilizingtechniquesandmethodspresentedin Chapter3,theymay turnintouncontrolledfiredetectorsaswell.
Currentmethodsandtechniquesusedformulti-sensorfireanalysisare describedin Chapter4.Methodsin Chapter4 areaimedatestimating theoriginandgrowthoffires,ratherthandetectingthem.Modelingfire behaviorhasimportantbenefitsinfirefightingandmitigation,andisessential inassessingtheriskofescalation.Techniquesin Chapter4 focusonmultimodal/multi-sensoranalysisoffirecharacteristics,suchasflameand smokespread.
SurveillancecamerasandPIR-basedmotionsensorsarewidelyusedin modernbuildings.Itisnowpossibletousethemforfireandsmokedetectionbyanalyzingthevideoandsignalsthattheygenerate.Itisourhopethat themethodsandtechniquesdiscussedinthisbookwillleadtosaferbuildings andlivingenvironmentsinthenearfuture.
Camera-BasedTechniques
2.8
Part1
Thefirststepinvideofiredetection(VFD)istoapplyabackgroundsubtractionalgorithmtoextractmovingregionsinthevideo.Thenthedetected regionsareanalyzedtemporallytobeclassifiedintermsofflickeringcharacteristics.Markovmodelsandfrequencydomaintechniquescanbeusedto identifyiftheflickeringcharacteristicsbelongtoflames.Inthenextstep, spatialanalysisisperformedtocheckfortheirregularitiesthatareusedto identifyflames.
Anothermethodistoextractfeaturesfromthemovingregionsand useclassifierswhoaretrainedofflinewithvideosoffireandfalsealarm sources.Itisalsopossibletouseac tivelearningalgorithmswhichare updatedonlinetoclassifyflameregions.Themostimportantproblem withvisiblerangefiredetectionsisthefalsealarms.Fire-coloredmoving regionscanbedifficulttodistinguishfromtheactualflames.Firedetectionalgorithmsaregenerallydevelopedforstationarycameras.When movingcamerasareused,itbecomesdifficulttodetecttheflickering characteristicsofflames.Detectionoffireusingamovingcameraisa futureresearchproblem.Anotherapp licationofvideo-basedfiredetectionissmokedetectionforearlywildfirewarningsystems.Forestsare usuallymonitoredusingPTZcamerasth atscanrecordedpresetpositions inaspecificorder.Thesecamerascan monitorlargerdistancesthanusual visiblerangecameras.Inwildfiredetectionapplications,smokebecomes visiblebeforetheflames;thereforeitmakessensetofocusonsmoke detectionforthesesystems.
Withthedecreasedcostofinfraredsensors,itbecamepossibletouselongandshort-waveinfraredcamerasforflamedetection.Sinceinfrared(thermal) camerasformimageswhoseintensitydependonthetemperatureofthe objects,theycouldbeusedtoreducemostofthefalsealarms.Inmostcases, thetemperatureofflamesishigherthanthesurroundingenvironmentandIR camerascansuccessfullydetecttheflickeringflames.Initially,infraredflame detectionalgorithmsprocessednearinfraredimages(NIR)toverifytheexistenceofflames.Morerecentmethodsstartedtouseshort-andlong-wave infrared(SWIR,LWIR)thermalcamerasforfireandflamedetection.
ThischapterisbasedontheVFDsurveypaperin[1].RecentlyproposedVFDtechniquesareviablealternativesorcomplementstoexisting firedetectiontechniquesandhaveshowntobeusefulinsolvingseveral problemsrelatedtothetraditionalsensors.Conventionalsensorsaregenerallylimitedtoindoorsandarenotapplicableinlargeopenspacessuchas shoppingcenters,airports,carparks,andforests.Theyrequireacloseproximitytothefireandmostofthemcannotprovideadditionalinformation aboutfirelocation,dimension,etc.Oneofthemainlimitationsofcommerciallyavailablefirealarmsystemsisthatitmaytakealongtimeforcarbonparticlesandsmoketoreachthe“point”detector.Thisiscalledthe “transportdelay.”Itisourbeliefthatvideoanalysiscanbeappliedinconditionsinwhichconventionalmethodsfail.VFDhasthepotentialtodetect thefirefromadistanceinlargeopenspacesbecausecamerascanmonitor
“volumes.”Asaresult,VFDdoesnothavethetransportandthreshold delayfromwhichthetraditional“point”sensorssuffer.Assoonassmoke orflamesoccurinoneofthecameraviews,itispossibletodetectfire immediately.Weallknowthathumanbeingscandetectanuncontrolled fireusingtheireyesandvisionsystems,butaspointedoutabove,itisnot easytoreplicatehumanintelligence.
Theresearchinthisdomainwasstartedinthelatenineties.Mostofthe VFDarticlesavailableintheliteratureareinfluencedbythenotionof “weak”ArtificialIntelligence(AI)frameworkwhichwasfirstintroduced byHubertL.Dreyfusinhiscritiqueofthe“generalized”AI[3,4].Dreyfus presentssolidphilosophicalandscientificargumentsonwhythesearchfor “generalized”AIisfutile[5].Therefore,eachspecificproblemincluding VFDfireshouldbeaddressedasanindividualengineeringproblemwhich hasitsowncharacteristics[6].Itispossibletoapproximatelymodelthefire behaviorinvideousingvarioussignalandimageprocessingmethodsand automaticallydetectfirebasedontheinformationextractedfromvideo. However,thecurrentsystemssufferfromfalsealarmsbecauseofmodeling andtraininginaccuracies.
CurrentlyavailableVFDalgorithmsmainlyfocusonthedetectionand analysisofsmokeandflamesinconsecutivevideoimages.Inearlyarticles, mainlyflamedetectionwasinvestigated.Recently,thesmokedetection problemisalsoconsidered.Thereasonforthiscanbefoundinthefact thatsmokespreadsfasterandinmostcaseswilloccurmuchfasterinthe fieldofviewofthecameras.Inwildfireapplications,itmaynotevenbe possibletoobserveflamesforalongtime.Themajorityofthestate-ofthe-artdetectiontechniquesfocusesonthecolorandshapecharacteristics, togetherwiththetemporalbehaviorofsmokeandflames.However,due tothevariabilityofshape,motion,transparency,colors,andpatternsof smokeandflames,manyoftheexistingVFDapproachesarestillvulnerable tofalsealarms.Duetonoise,shadows,illuminationchanges,andother visualartifactsinrecordedvideosequences,developingareliabledetection systemisachallengetotheimageprocessingandcomputervision community.
Withtoday’stechnology,itisnotpossibletohaveafullyreliableVFD systemwithoutahumanoperator.However,currentsystemsareinvaluable toolsforsurveillanceoperators.Itisalsoourstrongbeliefthatcombining multi-modalvideoinformationusingbothvisibleandinfrared(IR)technologywillleadtohigherdetectionaccuracy.Eachsensortypehasitsown
specificlimitations,whichcanbecompensatedbyothertypesofsensors. Although,itwouldbedesirabletodevelopafiredetectionsystemwhich couldoperateontheexistingclosedcircuittelevision(CCTV)equipment withoutintroducinganyadditionalcost.However,thecostofusingmultiplevideosensorsdoesnotoutweighthebenefitofmulti-modalfireanalysis.ThefactthatIRmanufacturersalsoensureadecreaseinthesensorcost inthenearfuturefullyopensthedoortomulti-modalvideoanalysis.VFD camerascanalsobeusedtoextractusefulrelatedinformation,suchasthe presenceofpeoplecaughtinthefire,firesize,firegrowth,smoke direction,etc.
VFDsystemscanbeclassifiedintovarioussubcategoriesaccordingto (i)thespectralrangeofthecameraused, (ii)thepurpose(flameorsmokedetection), (iii)therangeofthesystem. Thereareoverlapsbetweenthecategoriesabove.
2.1VFDINVISIBLE/VISUALSPECTRALRANGE
Overthelastyears,thenumberofpapersaboutvisualfiredetectioninthe computervisionliteraturehasgrownexponentially[2].Asis,thisrelatively newsubjectinvisionresearchisinfullprogressandhasalreadyproduced promisingresults.However,thisisnotacompletelysolvedproblem,aswith mostcomputervisionproblems.Behaviorofsmokeandflamesofanuncontrolledfiredifferswithdistanceandillumination.Furthermore,camerasare notcolorand/orspectralmeasurementdevices.Theyhavedifferentsensors andcolorandilluminationbalancingalgorithms.Theymayproducedifferentimagesandvideoforthesamescenebecauseoftheirinternalsettingsand algorithms.
Inthissection,achronologicaloverviewofthestate-of-the-art(ie,acollectionoffrequentlyreferencedpapersonshortrange[<100m])firedetectionmethodsispresentedinthetablesbelow.Foreachofthesepapers,we investigatedtheunderlyingalgorithmsandcheckedtheappropriatetechniques.Inthefollowing,wediscusseachofthesedetectiontechniques andanalyzetheiruseinthelistedpapers.
State-of-the-art:underlyingtechniques(PART1:2002-2007).
2.1.1ColorDetection
ColordetectionwasoneofthefirstdetectiontechniquesusedinVFDandis stillusedinalmostalldetectionmethods.Themajorityofthecolor-based approachesinVFDmakeuseofRGBcolorspace,sometimesincombinationwithHSI/HSVsaturation[10,24,27,28].Themainreasonforusing RGBisthatalmostallvisiblerangecamerashavesensorsdetectingvideo inRGBformatandthereistheobviousspectralcontentassociatedwiththis colorspace.ItisreportedthatRGBvaluesofflamepixelsareintheredyellowcolorrangeindicatedbytherule(R > G > B)asshownin Fig.2.1. Similarly,insmokepixels,R,G,andBvaluesareveryclosetoeachother. Morecomplexsystemsuserule-basedtechniquessuchasGaussiansmoothed colorhistograms[7],statisticallygeneratedcolormodels[15],andblending functions[20].Itisobviousthatcolorcannotbeusedbyitselftodetectfire becauseofthevariabilityincolor,density,lighting,andbackground. However,thecolorinformationcanbeusedasapartofamoresophisticated system.Forexample,chrominancedecreaseisusedinsmokedetection
Figure2.1 Colordetection:smokeregionpixelshavecolorvaluesthatareclosetoeach other.Pixelsofflameregionslieinthered-yellowrangeofRGBcolorspacewith R > G > B.
decreaseinedgesoftheimage,theyusetheDiscreteWaveletTransform (DWT).TheDWTisamulti-resolutionsignaldecompositionmethod obtainedbyconvolvingtheintensityimagewithfilterbanks.Astandard halfbandfilterbankproducesfourwaveletsubimages:theso-calledlowlowversionoftheoriginalimage Ct,andthehorizontal,vertical,anddiagonalhighfrequencybandimages Ht, Vt,and Dt.Thehigh-bandenergyfrom subimages Ht, Vt,and Dt isevaluatedbydividingtheimage It inblocks bk of arbitrarysizeasfollows:
Sincecontributionofedgesaremoresignificantinhighbandwavelet imagescomparedtoflatareasoftheimage,itispossibletodetectsmoke usingthedecreasein E(I t, b k).Astheenergyvalueofaspecificblock variessignificantlyovertimeinthe presenceofsmoke,temporalanalysis oftheratiobetweenthecurrentinputframewaveletenergyandthe backgroundimagewaveletenergyisusedtodetectthesmokeasshown in Fig.2.3
Figure2.3 DWT-basedvideosmokedetection:Whenthereissmoke,theratiobetween theinputframewaveletenergyandtheBGwaveletenergydecreasesandshowsahigh degreeofdisorder.
2.1.4SpatialWaveletColorVariationandAnalysis
Flamesofanuncontrolledfirehavevaryingcolorsevenwithinasmallarea. Spatialcolordifferenceanalysis[24,13,28,32]focusesonthischaracteristic. Usingrangefilters[24],variance/histogramanalysis[32],orspatialwavelet analysis[13,28],thespatialcolorvariationsinpixelvaluesareanalyzedto distinguishordinaryfire-coloredobjectsfromuncontrolledfires.In Fig.2.4, theconceptofspatialdifferenceanalysisisfurtherexplainedbymeansofa histogram-basedapproach,whichfocusesonthestandarddeviationofthe greencolorband.ItwasobservedbyQiandEbert[24]thatthiscolorband isthemostdiscriminativebandforrecognizingthespatialcolorvariationof flames.Thiscanalsobeseenbyanalyzingthehistograms.Greenpixelvalues varymorethanredandbluevalues.Ifthestandarddeviationofthegreen colorbandexceeds tσ ¼ 50(:Borges[32])inatypicalcolorvideotheregion islabeledasacandidateregionforaflame.Forsmokedetection,ontheother hand,experimentsrevealedthatthesetechniquesarenotalwaysapplicable becausesmokeregionsoftendonotshowashighspatialcolorvariationas flameregions.Furthermore,texturedsmoke-coloredmovingobjectsare difficulttodistinguishfromsmokeandcancausefalsedetections.Ingeneral, smokeinanuncontrolledfireisgrayanditreducesthecolorvariationinthe background.Therefore,inYUVcolorspaceweexpecttohavereductionin thedynamicrangeofchrominancecolorcomponentsUandVafterthe appearanceofsmokeintheviewingrangeofcamera.
Figure2.4 Spatialdifferenceanalysis:incaseofflames,thestandarddeviation σ G ofthe greencolorbandoftheflameregionexceeds tσ ¼ 50(:Borges[32]).
2.1.5DynamicTextureandPatternAnalysis
Adynamictextureorpatterninvideo,suchassmoke,flames,water,and leavesinthewind,canbesimplydefinedasatexturewithmotion [45,46](ie,aspatiallyandtime-varyingvisualpatternthatformsanimage sequenceorpartofanimagesequencewithacertaintemporalstationarity) [47].Althoughdynamictexturesareeasilyobservedbyhumaneyes,theyare difficulttodiscernusingcomputervisionmethodsasthespatiallocationand extentofdynamictexturescanvarywithtimeandtheycanbepartiallytransparent.Somedynamictextureandpatternanalysismethodsinvideo [29,33,35]arecloselyrelatedtospatialdifferenceanalysis.Recently,these techniqueshavealsobeenappliedtotheflameandsmokedetectionproblem [46].Currently,awidevarietyofmethodsincludinggeometric,modelbased,statistical,andmotion-basedtechniquesareusedfordynamictexture detection[48–50].
In Fig2.5,dynamictexturedetectionandsegmentationexamplesare shown,usingvideoclipsfromtheDynTexdynamictextureandBilkentdatabases[51,52,50,47].Contoursofdynamictextureregions(eg,fire,water,and steam)areshowninthisfigure.Dynamicregionsinvideoseemtobesegmentedverywell.However,duetothehighcomputationalcost,thesegeneraltechniquesarenotusedinpracticalfiredetectionalgorithmswhichshould runonlow-costcomputers,FPGAs,ordigitalsignalprocessors.Iffuture developmentsincomputersandgraphicsacceleratorscouldlowerthecomputationalcost,dynamictexturedetectionmethodsmaybeincorporatedinto thecurrentlyavailableVFDsystemstoachievemorereliablesystems.
Ordinarymovingobjectsinvideo,suchaswalkingpeople,haveapretty stableoralmostperiodicboundaryovertime.Ontheotherhand,uncontrolledflameandsmokeregionsexhibitchaoticboundarycontours.Therefore,disorderanalysisofboundarycontoursofamovingobjectisusefulfor firedetection.Someexamplesoffrequentlyusedmetricsarerandomnessof areasize[23,32],boundaryroughness[14,11,28,32],andboundaryarea
Figure2.5 Dynamictexturedetection:contoursofdetecteddynamictextureregions areshowninthefigure(ResultsfromDYNTEXandBilkentdatabases [51,53]).
disorder[18].Althoughthosemetricsdifferindefinition,theoutcomeof eachofthemisalmostidentical.Inthesmokedetectordevelopedby Verstocketal.[2],disorderanalysisoftheBoundaryAreaRoughness (BAR)isused,whichisdeterminedbyrelatingtheperimeteroftheregion tothesquarerootofthearea(Fig.2.6).AnothertechniqueisthehistogrambasedorientationaccumulationbyYuan[22].Thistechniquealsoproduces gooddisorderdetectionresults,butitiscomputationallymorecomplex thantheformermethods.Relatedtothedisorderanalysisisthegrowing ofsmokeandflameregionsintheearlystageofafire.In[31,34],thegrowth rateoftheregion-of-interestisusedasafeatureparameterforfiredetection. Comparedtodisordermetrics,however,growthanalysisislesseffectivein detectingthesmoke,especiallyinwildfiredetection.Thisisbecausethe smokeregionappearstogrowveryslowlyinwildfireswhentheyareviewed fromlongdistances.Furthermore,anordinaryobjectmaybeapproaching thecamera.
2.2SPATIOTEMPORALNORMALIZEDCOVARIANCE DESCRIPTORS
Arecentapproachwhichcombinescolorandspatiotemporalinformationby regioncovariancedescriptorsisusedinEuropeanCommissionfundedFP-7 FIRESENSEproject[54–56].Themethodisbasedonanalyzingthespatiotemporalblocks.Theblocksareobtainedbydividingthefire-andsmokecoloredregionsinto3Dregionsthatoverlapintime.Classificationofthe featuresisperformedonlyatthetemporalboundariesofblocksinsteadof
Figure2.6 Boundaryarearoughnessofconsecutiveflameregions.
performingitateachframe.Thisreducesthecomputationalcomplexityof themethod.
CovariancedescriptorsareproposedbyTuzel,Porikli,andMeertobe usedinobjectdetectionandtextureclassificationproblems[54,55].In [57,75,76]temporallyextendednormalizedcovariancedescriptorstoextract featuresfromvideosequencesareproposed.
Temporallyextendednormalizedcovariancedescriptorsaredesignedto describespatiotemporalvideoblocks.Let I(i, j, n)betheintensityof(i, j)th pixelofthe nthimageframeofaspatiotemporalblockinvideo.Thepropertyparametersdefinedinequationsbelowareusedtoformacovariance matrixrepresentingspatialinformation.Inadditiontospatialparameters, temporalderivatives, It and Itt areintroducedwhicharethefirstandsecond derivativesofintensitywithrespecttotime,respectively.Byaddingthese twofeaturestothepreviouspropertyset,normalizedcovariancedescriptors canbeusedtodefinespatiotemporalblocksinvideo.
Forflamedetection:
Table2.1 AnevaluationofdifferentvisiblerangeVFDmethods
(#frames) #Fireframes groundtruth
