AI for Games THIRDEDITION
IanMillington
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2019 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed on acid-free paper
Version Date: 20190221
International Standard Book Number-13: 978-1-138-48397-2 (Hardback)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www. copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Names: Millington, Ian, author.
Title: AI for games / Ian Millington.
Other titles: AI for games
Description: Third edition. | Boca Raton : Taylor & Francis, a CRC title, part of the Taylor & Francis imprint, a member of the Taylor & Francis Group, the academic division of T&F Informa, plc, [2019] | Includes bibliographical references.
Identifiers: LCCN 2018041305 | ISBN 9781138483972 (hardback : acid-free paper)
Subjects: LCSH: Computer games--Programming. | Computer animation. | Artificial intelligence.
Classification: LCC QA76.76.C672 M549 2019 | DDC 794.8/1525--dc23
LC record available at https://lccn.loc.gov/2018041305
Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com
and the CRC Press Web site at http://www.crcpress.com
for Daniel myfavoritegamedesigner
3.3.12PathFollowing..............................
3.3.13Separation................................
3.9.6LookWhereYou’reGoing........................
4.4.5Non-TranslationalProblems.......................
4.4.6CostFunctions..............................
4.7OtherIdeasinPathfinding
5.4.6ReusingTrees..............................
5.4.7LimitationsofBehaviorTrees......................
5.7Goal-OrientedBehavior
5.9.6OtherThingsAreBlackboardSystems.................
7.1.2Intra-BehaviorLearning.........................
7.1.3Inter-BehaviorLearning.........................
7.1.4AWarning................................
7.1.5Over-Learning..............................
7.1.6TheZooofLearningAlgorithms....................
7.1.7TheBalanceofEffort...........................
7.2ParameterModification
7.2.1TheParameterLandscape........................
7.2.3ExtensionstoBasicHillClimbing....................
7.3.4N-Grams.................................
7.3.5WindowSize...............................
7.3.6HierarchicalN-Grams..........................
7.7.1TheProblem...............................
7.7.7TailoringParameters...........................
7.7.8WeaknessesandRealisticApplications.................
7.7.9OtherIdeasinReinforcementLearning.................
9.2.1TheStaticEvaluationFunction.....................
9.2.2Minimaxing...............................
9.2.3TheMinimaxingAlgorithm.......................
9.2.4Negamaxing...............................
9.9.2Real-TimeAIinaTurn-BasedGame..................
PARTI AIandGames
INTRODUCTION
Gamedevelopment livesinitsowntechnicalworld.Ithasitsownidioms,skills,and challenges.That’soneofthereasonsgamesaresomuchfuntoworkon.Eachgame hasitsownrules,itsownaesthetic,itsowntrade-offs,andthehardwareitwillrunonkeeps changing.There’sareasonablygoodchanceyouwillbethefirstpersontomeetandbeata newprogrammingchallenge.
Despitenumerouseffortstostandardizegamedevelopment,inlinewiththerestofthe softwareindustry(effortsthatgobackatleast25years),thestyleofprogramminginagame isstillratherunique.Thereisafocusonspeed,butitdiffersfromreal-timeprogramming forembeddedorcontrolapplications.Thereisafocusoncleveralgorithms,butitdoesn’t sharethesamerigorasdatabaseserverengineering.Itdrawstechniquesfromahugerange ofdifferentsources,butalmostwithoutexceptionmodifiesthembeyondresemblance.And, toaddanextralayerofintrigue,developersmaketheirmodificationsindifferentways,often underextremetimepressure,andtailoredentirelytothegameathand,leavingalgorithms unrecognizablefromstudiotostudioorprojecttoproject.
Asexcitingandchallengingasthismaybe,itmakesitdifficultfornewdevelopersto gettheinformationtheyneed.Twentyyearsago,itwasalmostimpossibletogetholdof informationabouttechniquesandalgorithmsthatrealdevelopersusedintheirgames.There wasanatmosphereofsecrecy,evenalchemy,aboutthecodingtechniquesintopstudios.Then cametheInternetandanever-growingrangeofwebsites,alongwithbooks,conferences,and periodicals.Itisnoweasierthanevertoteachyourselfnewtechniquesingamedevelopment.
Thisbookisdesignedtohelpyoumasteroneelementofgamedevelopment:artificial intelligence(AI).TherehavebeenmanyarticlespublishedaboutdifferentaspectsofgameAI: websitesonparticulartechniques,compilationsinbookform,someintroductorytexts,and
plentyoflecturesatdevelopmentconferences.Butthisbookwasdesignedtocoveritall,asa coherentwhole.
InmycareerIhavedevelopedmanyAImodulesforlotsofdifferentgenresofgames.I developedAImiddlewaretoolsthathavealotofnewresearchandclevercontent.Ihave workedonresearchanddevelopmentfornext-generationAI,andIgettodoalotwithsome veryclevertechnologies.However,throughoutthisbookItriedtoresistthetemptationto passoffhowIthinkitshouldbedoneashowitisdone.Myaimhasbeentotellitlikeitis(or forthosetechnologiesImentionthatarestillemerging,totellyouhowmostpeopleagreeit willbe).
ThemeatofthisbookcoversawiderangeoftechniquesforgameAI.Someofthemare barelytechniques,morelikeageneralapproachordevelopmentstyle.Somearefull-blown algorithms,readytoimplementinareusableway.Andothersareshallowintroductionsto hugefieldsthatwouldfillmultiplebooksontheirown.Inthesecases,I’vetriedtogiveenough ofanoverviewforyoutounderstandhowandwhyanapproachmaybeuseful(ornot).
Thisbookisintendedforawiderangeofreaders:fromhobbyistsorstudentslooking togetasolidunderstandingofgameAIthroughtoprofessionalswhoneedacomprehensive referencetotechniquestheymaynothaveusedbefore.Iwillassumeyoucanprogram(though Iwillnotassumeanyparticularlanguage—seeChapter13foradiscussionofimplementation languages),andIwillassumeyouhavehigh-schoollevelmathematicalknowledge.
Beforewegetintothetechniquesthemselves,thischapterintroducesAI,itshistory,and thewayitisused.We’lllookatamodelofAItohelpfitthetechniquestogether,andI’llgive somebackgroundonhowtherestofthebookisstructured.
1.1 WHATISAI?
Artificialintelligenceisaboutmakingcomputersabletoperformthethinkingtasksthathumansandanimalsarecapableof.
Wecanprogramcomputerstohavesuperhumanabilitiesinsolvingmanyproblems: arithmetic,sorting,searching,andsoon.SomeoftheseproblemswereoriginallyconsideredAIproblems,butastheyhavebeensolvedinmoreandmorecomprehensiveways,they haveslippedoutofthedomainofAIdevelopers.
Buttherearemanythingsthatcomputersaren’tgoodatwhichwefindtrivial:recognizing familiarfaces,speakingourownlanguage,decidingwhattodonext,andbeingcreative.These arethedomainofAI:tryingtoworkoutwhatkindsofalgorithmsareneededtodisplaythese properties.
OftenthedividinglinebetweenAIandnot-AIismerelydifficulty:thingswecan’tdo requireAI,thingswecanaretricksandmath.Itistemptingtogetintoadiscussionofwhat is‘real’AI,totrydefining‘intelligence’,‘consciousness’or‘thought’.Inmyexperience,itisan impossibletask,largelyirrelevanttothebusinessofmakinggames.
Inacademia,someAIresearchersaremotivatedbythosephilosophicalquestions:understandingthenatureofthoughtandthenatureofintelligenceandbuildingsoftwaretomodel
howthinkingmightwork.Othersaremotivatedbypsychology:understandingthemechanicsofthehumanbrainandmentalprocesses.Andyetothersaremotivatedbyengineering: buildingalgorithmstoperformhuman-liketasks.Thisthreefolddistinctionisattheheartof academicAI,andthedifferentconcernsareresponsiblefordifferentsubfieldsofthesubject.
Asgamesdevelopers,wearepracticalfolks;interestedinonlytheengineeringside.We buildalgorithmsthatmakegamecharactersappearhumanoranimal-like.Developershave alwaysdrawnfromacademicresearch,wherethatresearchhelpsthemgetthejobdone,and ignoredtherest.
ItisworthtakingaquickoverviewoftheAIworkdoneinacademiatogetasenseofwhat existsinthesubjectandwhatmightbeworthplagiarizing.
1.1.1 ACADEMICAI
Totellthestory,IwilldivideacademicAIintothreeperiods:theearlydays,thesymbolicera, andthenaturalcomputingandstatisticalera.Thisisagrossoversimplification,ofcourse, andtheyalloverlaptosomeextent,butIfinditausefuldistinction.Foracollectionofpapers whichgiveamuchmorenuancedandfleshedoutaccount,see[38].
TheEarlyDays
Theearlydaysincludethetimebeforecomputers,wherephilosophyofmindoccasionally madeforaysintoAIwithsuchquestionsas:“Whatproducesthought?”“Couldyougive lifetoaninanimateobject?”“Whatisthedifferencebetweenacadaverandthehumanit previouslywas?”Tangentialtothiswasthepopulartasteinautomata,mechanicalrobots, fromthe18thcenturyonward.Intricateclockworkmodelswerecreatedthatdisplayedthe kindofanimated,animal-orhuman-likebehaviorsthatwenowemploygameartiststocreate inamodelingpackage(foranacademichistoryanddiscussion,see[23],andforanin-depth lookattwopianoplayingexamples,see[74]).
Inthewareffortofthe1940s,theneedtobreakenemycodes[8]andtoperformthecalculationsrequiredforatomicwarfaremotivatedthedevelopmentofthefirstprogrammable computers[46].Giventhatthesemachineswerebeingusedtoperformcalculationsthat wouldotherwisebedonebyaperson,itwasnaturalforprogrammerstobeinterestedinAI. Severalcomputingpioneers(suchasTuring,vonNeumann,andShannon)werealsopioneers inearlyAI.Turing,inparticular,hasbecomeanadoptedfathertothefield,asaresultofa philosophicalpaperhepublishedin1950[70].
TheSymbolicEra
Fromthelate1950sthroughtotheearly1980sthemainthrustofAIresearchwas“symbolic” systems.Asymbolicsystemisoneinwhichthealgorithmisdividedintotwocomponents:
asetofknowledge(representedassymbolssuchaswords,numbers,sentences,orpictures) andareasoningalgorithmthatmanipulatesthosesymbolstocreatenewcombinationsthat hopefullyrepresentproblemsolutionsornewknowledge.
Anexpertsystem,oneofthepurestexpressionsofthisapproach,isamongthemostfamousAItechniques.IftodayalltheAIheadlinestalkabout“deeplearning,”inthe1980s, theynamedropped“expertsystems.”Anexpertsystemhasalargedatabaseofknowledge anditappliesacollectionofrulestodrawconclusionsortodiscovernewthings(see Section5.8).Othersymbolicapproachesapplicabletogamesincludeblackboardarchitectures (5.9),pathfinding(Chapter4),decisiontrees(Section5.2)andstatemachines(Section5.3). Theseandmanymorearedescribedinthisbook.
Acommonfeatureofsymbolicsystemsisatrade-off:whensolvingaproblemthemore knowledgeyouhave,thelessworkyouneedtodoinreasoning.Often,reasoningalgorithms consistofsearching:tryingdifferentpossibilitiestogetthebestresult.Thisleadsustothe goldenruleofAI,thatwewillseeinvariousformsthroughoutthisbook:
Searchandknowledgeareintrinsicallylinked.Themoreknowledgeyouhave, thelesssearchingforanansweryouneed;themoresearchyoucando(i.e.,the fasteryoucansearch),thelessknowledgeyouneed.
ItwassuggestedbyresearchersNewellandSimonin1976thatknowledgeinfusedsearch (knownas‘heuristicsearch’)isthewayallintelligentbehaviorarises[45].Unfortunately, despiteitshavingseveralsolidandimportantfeatures,thistheoryhasbeenlargelydiscredited asanaccountofallintelligence.Nevertheless,manypeoplewitharecenteducationinAIare notawarethat,asanengineeringtrade-off,knowledgeversussearchisunavoidable.Recent workonthemathematicsofproblemsolvinghasprovedthistheoretically[76].Atapractical level,AIengineershavealwaysknownit.
TheNaturalComputing/StatisticalEra
Throughthe1980sandintotheearly1990s,therewasanincreasingfrustrationwithsymbolic approaches.Thefrustrationcamefromvariousdirections.
Fromanengineeringpointofview,theearlysuccessesonsimpleproblemsdidn’tseemto scaletomoredifficultproblems.Forexample:itseemedeasytodevelopAIthatunderstood (orappearedtounderstand)simplesentences,butdevelopinganunderstandingofafullhumanlanguageseemednonearer.Thiswascompoundedbyhype:whenAItoutedas‘thenext bigthing’failedtoliveuptoitsbilling,confidenceinthewholesectorcrashed.
Therewasalsoaninfluentialphilosophicalargumentthatsymbolicapproachesweren’t biologicallyplausible.Youcan’tunderstandhowahumanbeingplansaroutebyusinga symbolicroute-planningalgorithmanymorethanyoucanunderstandhowhumanmuscles workbystudyingaforklifttruck.
Theeffectwasamovetowardnaturalcomputing:techniquesinspiredbybiologyorother naturalsystems.Thesetechniquesincludeneuralnetworks,geneticalgorithms,andsimulated
annealing.1 Manynaturalcomputingtechniqueshavebeenaroundforalongtime.Neural networks,forexample,predatethesymbolicera;theywerefirstsuggestedin1943[39].
Butinthe1980sthroughtotheearly2000stheyreceivedthebulkoftheresearcheffort. WhenIbeganmyPhDinartificialintelligenceinthe90s,itwasdifficulttofindresearchplaces inExpertSystems,forexample.Istudiedgeneticalgorithms;mostofmypeerswereworking onneuralnetworks.
Despiteitsoriginasacorrelatetobiology,AIresearchheavilyappliedmathematics,particularlyprobabilityandstatistics,tounderstandingandoptimizingnaturalcomputingtechniques.Theabilitytohandlealltheuncertaintyandmessinessofreal-worlddata,incontrast tothecleanandrigidboundariesofthesymbolicapproaches,ledtothedevelopmentofawide rangeofotherprobabilistictechniques,suchasBayesnets,support-vectormachines(SVMs), andGaussianprocesses.
ThebiggestchangeinAIinthelastdecadehasnotcomefromabreakthroughinacademia. WearelivinginatimewhenAIisagainbackinthenewspapers:selfdrivingcars,deepfakes, worldchampionGoprograms,andhomevirtualassistants.Thisistheeraofdeeplearning. Thoughmanyacademicinnovationsareused,thesesystemsarestillfundamentallypowered byneuralnetworks,nowmadepracticalbytheincreaseincomputingpower.
Engineering
Thoughnewspaperheadlinesandhigh-profileapplicationshaveflourishedinthelastfive years,AIhasbeenakeytechnologyrelevanttosolvingreal-worldproblemsfordecades.Navigationsystemsincars,jobschedulinginfactories,voicerecognitionanddictation,andlargescalesearchareallmorethan20yearsold.Google’ssearchtechnology,forexample,haslong beenunderpinnedbyAI.ItisnocoincidencethatPeterNorvigisbothaDirectorofResearch atGoogleandtheco-author(alongwithhisformergraduateadviser,professorStuartRussell) ofthecanonicalundergraduatetextbookformodernacademicAI[54].
Whensomethingishot,itistemptingtoassumeitistheonlythingthatmatters.When naturalcomputingtechniquestookcenterstage,therewasatendencytoassumethatsymbolic approachesweredead.Similarly,withtalkofdeeplearningeverywhere,youmightbeforgiven forthinkingthatiswhatshouldbeused.
Butwealwayscomebacktothesametrade-off:searchvsknowledge.Deeplearningisthe ultimateinthecomputeintensivesearch,AlphaGoZero(thethirditerationoftheAlphaGo software,[60])wasgivenveryminimalknowledgeoftherulesofthegame,butextraordinary amountsofprocessingtimetotrydifferentstrategiesandlearnthebest.Ontheotherhand, acharacterthatneedstouseahealthpackwheninjuredcanbetoldthatexplicitly: IFinjuredTHENusehealthpack
1 Thoughold,agoodintroductiontoGeneticAlgorithmscanbefoundin[41],andSimulatedAnnealingin[34].In theircurrentvogueasDeepLearning,therearemanymorecontemporaryresourcesonneuralnetworks.Foragentle andshallowintroduction,see[21],foramorecomprehensiveguide,[18].
Nosearchrequired.
Theonlywayanyalgorithmcanoutperformanotheriseithertoconsumemoreprocessing power(moresearch),ortobeoptimizedtowardaspecificsetofproblems(moreknowledge oftheproblem).
Inpractice,engineersworkfrombothsides.Avoicerecognitionprogram,forexample, convertstheinputsignalsusingknownformulaeintoaformatwheretheneuralnetworkcan decodeit.Theresultsarethenfedthroughaseriesofsymbolicalgorithmsthatlookatwords fromadictionaryandthewaywordsarecombinedinthelanguage.Astatisticalalgorithm optimizingtheorderofaproductionlinewillhavetherulesaboutproductionencodedinto itsstructure,soitcan’tpossiblysuggestanillegaltimetable:theknowledgeisusedtoreduce theamountofsearchrequired.
Unfortunately,gamesareusuallydesignedtorunonconsumerhardware.AndwhileAIis important,graphicshavealwaystakenthemajorityoftheprocessingpower.Thisseemsinno dangerofchanging.ForAIdesignedtorunonthedeviceduringthegame,lowcomputation /highknowledgeapproachesareoftentheclearwinners.Andtheseareveryoftensymbolic: approachespioneeredinacademiainthe70sand80s.
We’lllookatseveralstatisticalcomputingtechniquesinthisbook,usefulforspecificproblems.ButIhaveenoughexperiencetoknowthatforgamestheyareoftenunnecessary:the sameeffectcanoftenbeachievedbetter,faster,andwithmorecontrolusingasimplerapproach.Thishasn’tchangedsignificantlysincethefirsteditionofthisbookin2004.OverwhelminglytheAIusedingamesisstillsymbolictechnology.
1.1.2 GAMEAI
Pac-Man[140]wasthefirstgamemanypeoplerememberplayingwithfledglingAI.Uptothat pointtherehadbeen Pong cloneswithopponent-controlledbats(followingtheballupand down)andcountlessshootersinthe SpaceInvaders mold.ButPac-Manhaddefiniteenemy charactersthatseemedtoconspireagainstyou,movedaroundtheleveljustasyoudid,and madelifetough.
Pac-ManreliedonaverysimpleAItechnique:astatemachine(whichI’llcoverlaterin Chapter5).Eachofthefourmonsters(latercalled ghosts afteradisastrouslyflickeringport totheAtari2600)occupiedoneofthreestates:chasing,scattering(headingforthecornersat specifictimeintervals),andfrightened(whenPac-Maneatsapowerup).Foreachstatethey chooseatileastheirtarget,andturntowarditateachjunction.Inchasemode,eachghost choosesthetargetaccordingtoaslightlydifferenthard-codedrule,givingthemtheirpersonalities.AcompletebreakdownofthePac-Mangamemechanics,includingtheAIalgorithms, canbefoundat[49].
GameAIdidn’tchangemuchuntilthemid-1990s.Mostcomputer-controlledcharacters priortothenwereaboutassophisticatedasaPac-Manghost.
Takeaclassiclike GoldenAxe [176]eightyearslater.Enemycharactersstoodstill(or walkedbackandforwardashortdistance)untiltheplayergotclosetothem,whereuponthey
Another random document with no related content on Scribd:
prohibition against accepting unsolicited donations from donors in such states who approach us with offers to donate.
International donations are gratefully accepted, but we cannot make any statements concerning tax treatment of donations received from outside the United States. U.S. laws alone swamp our small staff.
Please check the Project Gutenberg web pages for current donation methods and addresses. Donations are accepted in a number of other ways including checks, online payments and credit card donations. To donate, please visit: www.gutenberg.org/donate.
Section 5. General Information About Project Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project Gutenberg™ concept of a library of electronic works that could be freely shared with anyone. For forty years, he produced and distributed Project Gutenberg™ eBooks with only a loose network of volunteer support.
Project Gutenberg™ eBooks are often created from several printed editions, all of which are confirmed as not protected by copyright in the U.S. unless a copyright notice is included. Thus, we do not necessarily keep eBooks in compliance with any particular paper edition.
Most people start at our website which has the main PG search facility: www.gutenberg.org.
This website includes information about Project Gutenberg™, including how to make donations to the Project Gutenberg Literary Archive Foundation, how to help produce our new eBooks, and how to subscribe to our email newsletter to hear about new eBooks.
