https://ebookmass.com/product/python-for-scientists-3rd-
Instant digital products (PDF, ePub, MOBI) ready for you
Download now and discover formats that fit your needs...
Prototyping Python Dashboards for Scientists and Engineers Padraig Houlahan
https://ebookmass.com/product/prototyping-python-dashboards-forscientists-and-engineers-padraig-houlahan/
ebookmass.com
Applied Numerical Methods with Python for Engineers and Scientists Steven C. Chapra
https://ebookmass.com/product/applied-numerical-methods-with-pythonfor-engineers-and-scientists-steven-c-chapra/
ebookmass.com
Prototyping Python Dashboards for Scientists and Engineers: Build and Deploy a Complete Dashboard with Python 1st Edition Padraig Houlahan
https://ebookmass.com/product/prototyping-python-dashboards-forscientists-and-engineers-build-and-deploy-a-complete-dashboard-withpython-1st-edition-padraig-houlahan/
ebookmass.com
The New Negro: The Life of Alain Locke Jeffrey C. Stewart
https://ebookmass.com/product/the-new-negro-the-life-of-alain-lockejeffrey-c-stewart/
ebookmass.com
https://ebookmass.com/product/bad-bachelors-bundle-stefanie-london/
ebookmass.com
Recalibrating the Relevance of Adult Neurogenesis Jason S. Snyder
https://ebookmass.com/product/recalibrating-the-relevance-of-adultneurogenesis-jason-s-snyder/
ebookmass.com
Cursed Shadows (Silvercrest Academy Book 3) Kc Kean
https://ebookmass.com/product/cursed-shadows-silvercrest-academybook-3-kc-kean/
ebookmass.com
The Palgrave Handbook of Canada and International Affairs Robert W. Murray
https://ebookmass.com/product/the-palgrave-handbook-of-canada-andinternational-affairs-robert-w-murray/
ebookmass.com
Dirty Secret Love (Heartwood Lake Secret Billionaires Book 2) Claudia Burgoa
https://ebookmass.com/product/dirty-secret-love-heartwood-lake-secretbillionaires-book-2-claudia-burgoa/
ebookmass.com
https://ebookmass.com/product/nursing-for-wellness-in-older-adultsebook-pdf-version/
ebookmass.com
PythonforScientists
ThirdEdition
ThethirdeditionofthispracticalintroductiontoPythonhasbeenthoroughlyupdated, withallcodemigratedtoPython3andmadeavailableasJupyternotebooks.Thenotebooksareavailableonlinewithexecutableversionsofallthebook’scontent(andmore).
ThetextstartswithadetailedintroductiontothebasicsofthePythonlanguage, withoutassuminganypriorknowledge.Buildinguponeachother,themostimportant Pythonpackagesfornumericalmath(NumPy),symbolicmath(SymPy),andplotting (Matplotlib)areintroduced,withbrandnewchapterscoveringnumericalmethods (SciPy)anddatahandling(Pandas).Furthernewmaterialincludesguidelinesforwriting efficientPythoncodeandpublishingcodeforotherusers.
Simpleandconcisecodeexamples,revisedforcompatibilitywithPython3,guide thereaderandsupportthelearningprocessthroughoutthebook.Readersfromthequantitativesciences,whatevertheirbackground,willbeabletoquicklyacquiretheskills neededforusingPythoneffectively.
johnm . stewart wasEmeritusReaderinGravitationalPhysicsattheUniversityof Cambridge,andaLifeFellowatKing’sCollege,Cambridge,beforehisdeathin2016. Hewastheauthorof Non-equilibriumRelativisticKineticTheory (Springer,1971)and AdvancedGeneralRelativity (Cambridge,1991),andhetranslatedandeditedHans Stephani’s GeneralRelativity (Cambridge,1990).
michaelmommert isAssistantProfessorforComputerVisionattheUniversityofSt. Gallen,Switzerland,wherehecombinescomputervisionandEarthobservationtoimplementefficientlearningmethodsforawiderangeofusecases.Before,hewasasolar systemastronomerandactivelywrotescientificopen-sourcecodeforthiscommunity.
PythonforScientists
ThirdEdition
JOHNM.STEWART
UniversityofCambridge
MICHAELMOMMERT
UniversityofSt.Gallen,Switzerland
ShaftesburyRoad,CambridgeCB28EA,UnitedKingdom OneLibertyPlaza,20thFloor,NewYork,NY10006,USA 477WilliamstownRoad,PortMelbourne,VIC3207,Australia
314–321,3rdFloor,Plot 3,SplendorForum,JasolaDistrictCentre,NewDelhi–110025,India
103 PenangRoad,#05–06/07,VisioncrestCommercial,Singapore238467
CambridgeUniversityPressispartofCambridgeUniversityPress&Assessment, adepartmentoftheUniversityofCambridge.
WesharetheUniversity’smissiontocontributetosocietythroughthepursuitof education,learningandresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle: www.cambridge.org/9781009014809
DOI: 10.1017/9781009029728
FirstandSecondeditions © JohnM.Stewart2014,2017
Thirdedition © CambridgeUniversityPress&Assessment2023
Thispublicationisincopyright.Subjecttostatutoryexceptionandtotheprovisions ofrelevantcollectivelicensingagreements,noreproductionofanypartmaytake placewithoutthewrittenpermissionofCambridgeUniversityPress&Assessment.
Firstpublished2014 Secondedition2017 Thirdedition2023
PrintedintheUnitedKingdombyCPIGroupLtd,CroydonCR04YY
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary ACataloging-in-PublicationdatarecordforthisbookisavailablefromtheLibraryofCongress
ISBN978-1-009-01480-9 Paperback
Additionalresourcesforthispublicationat www.cambridge.org/9781009014809
CambridgeUniversityPress&Assessmenthasnoresponsibilityforthepersistence oraccuracyofURLsforexternalorthird-partyinternetwebsitesreferredtointhis publicationanddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain, accurateorappropriate.
3.5.7Strings
3.5.8Dictionaries
3.5.9Sets
3.8.3KeywordArguments
3.8.4ArbitraryNumberofPositionalArguments
3.8.5ArbitraryNumberofKeywordArguments
3.8.6AnonymousFunctions
4.2WorkingwithArrays
4.6Polynomials
4.6.1ConvertingDatatoCoe
4.7.3SolvingLinearSystemsofEquations
4.8.1TextFileInput
4.8.2BinaryFileInput
4.9.1MaskedArrays
5.3.1IntegratingoverFunctions
5.3.2IntegratingoverSampledValues
5.4.1LocalUnivariateOptimization
5.4.2LocalMultivariateOptimization
5.4.3FunctionFitting
5.4.4RootSearch
5.5NumericalInterpolation
5.5.1UnivariateInterpolation
5.7.1UnivariateContinuousProbabilityDistributions
5.7.2MultivariateandDiscreteProbabilityDistributions
5.7.3CorrelationTests
5.7.4DistributionTests
5.8OrdinaryDifferentialEquations
5.8.1InitialValueProblems
5.8.2BoundaryValueProblems
5.9SciKits:AWholeNewWorld
5.10References
6 Matplotlib:Plotting
6.1GettingStarted:SimpleFigures
6.1.1Frontends
6.1.2Backends
6.1.3ASimpleFigure
6.2Object-OrientedMatplotlib
6.3.1FigureSize
6.3.2AxisRangeandScaling
6.3.3Ticks
6.3.4Grid
6.3.5Legend
6.3.6Transparency
6.3.7TextandAnnotations
6.3.8MathematicalFormulae
6.3.9Colors
6.4CartesianPlots
6.4.1LinePlots
6.4.2ScatterPlots
6.4.3ErrorBars
6.4.4PlottingFilledAreas
6.4.5BarPlots
6.5PolarPlots
6.6PlottingImages
6.7ContourPlots
6.8CompoundFigures
6.9MultidimensionalVisualization
6.9.1TheReductiontoTwoDimensions
6.9.23DPlots
6.10References
7 SymPy:SymbolicMath
7.1SymbolsandFunctions
7.2ConversionsfromPythontoSymPyandViceVersa
7.3MatricesandVectors
7.4SomeElementaryCalculus
7.4.3SeriesandLimits
7.5Equality,SymbolicEqualityandSimplification
7.6SolvingEquations
7.6.1EquationswithOneIndependentVariable
7.6.2LinearEquationswithMorethanOneIndependentVariable
7.6.3MoreGeneralEquations
8 Pandas:DataHandling
8.2.1AxisLabelsandIndexes
8.2.2AccessingData
8.2.3ModifyingData
8.2.4DealingwithMissingData
8.3SpecificTypesofData
8.3.1CategoricalData
9 PerformancePython
9.1HowtoWriteEfficientPythonCode
9.1.1Measuringperformance
9.1.2OptimizationStartsbeforeCoding
9.1.3OptimizingBasicPython
9.1.4NumPy
9.2Parallelization
9.2.1Multithreading
9.2.2Multiprocessing
9.4References
Preface
Iboughtacopyof PythonforScientists ataconferenceboothin2016,lookingforan affordableandeasilyreadabletextbookforaPythoncourseIwasteachingatthattime. Iwasintriguedbyhowstraightforwardlyevencomplexthingswereexplainedinthis book.Itwasaperfectmatchformycourseandmystudents,despitemyimpressionthat thebookwasratherheavilyfocusedonmathematicalapplications.
Ithasbeenfiveyearssincethesecondeditionof PythonforScientists wasreleased. Thisisalongtimeinthelifecycleofaprogramminglanguagethatisstillunderactive development.Itwasdefinitelytimeforanupdate.
Unfortunately,Johnisnolongerwithustoprovidethisupdatehimself.Instead,Iwas honoredthatthistaskwasofferedtome,andIcouldnotdecline.
BesidesupdatingthePythoncodeexamplesshowninthebook,Itooktheopportunity toalsoupdatethecontentofthebookwiththegoalofmakingitaccessibletoabroader audienceofscientists,especiallythosewithaquantitativefocusintheirwork.Thisincludesamorein-depthdiscussionofnumericalmathematicswithNumPy(Chapter 4) andSciPy(Chapter 5),plottingcapabilitieswithMatplotlib(Chapter 6),and,forthe firsttime,datahandlingwithPandas(Chapter 8),performancecomputingwithPython (Chapter 9),andanoutlineofsoftwaredevelopmenttechniquesthatareusefultoscientists(Chapter 10).However,inordertokeepthebookreasonablyshortandaffordable, othercontent,suchasthedetailedtreatmentofordinaryandpartialdifferentialequations,hadtobesignificantlyshortenedorremovedaltogether–Pythonpackagesfor dealingwithsuchproblemsexist,buttheirdiscussionisbeyondthescopeofthisbeginnerbook.
Isincerelyhopethethirdeditionof PythonforScientists willbeausefulcompanionon yourlongjourneytobecomingascientificprogrammer.
MichaelMommert St.Gallen,November2022
1 Introduction
1.1PythonforScientists
Thetitleofthisbookis PythonforScientists,butwhatdoesthatmean?Thedictionary defines“Python”aseither(a)anonvenomoussnakefromAsiaorSaharanAfricaor(b) acomputerprogramminglanguage,anditisthesecondoptionthatisintendedhere.By “scientist,”wemeananyonewhousesquantitativemodelseithertoobtainconclusions byprocessingprecollectedexperimentaldataortomodelpotentiallyobservableresults fromamoreabstracttheory,andwhoasks“whatif?”WhatifIanalyzethedataina differentway?WhatifIchangethemodel?
Giventhesteadyprogressinthedevelopmentofevermorecomplexexperimentsthat exploretheinnerworkingsofnatureandgeneratevastamountsofdata,aswellasthe necessitytodescribetheseobservationswithcomplex(nonlinear)theoreticalmodels, theuseofcomputerstoanswerthesequestionsismandatory.Luckily,advancesincomputerhardwareandsoftwaredevelopmentmeanthatimmenseamountsofdataorcomplexmodelscanbeprocessedatincreasinglyrapidspeeds.Itmightseemagiventhat suitablesoftwarewillalsobeavailablesothatthe“whatif”questionscanbeanswered readily.However,thisturnsoutnotalwaystobethecase.Aquickpragmaticreason isthatwhilethereisahugemarketforhardwareimprovements,scientistsformavery smallfractionofitandsothereislittlefinancialincentivetoimprovescientificsoftware.Butforscientists,specialized,yetversatile,softwaretoolsarekeytounraveling complexproblems.
Beforewediscusswhattypesofscientificsoftwareareavailable,itisimportantto notethatallcomputersoftwarecomesinoneoftwotypes:proprietaryoropen-source. Proprietarysoftware issuppliedbyacommercialfirm.Suchorganizationshaveboth topaywagesandtaxesandtoprovideareturnfortheirshareholders.Therefore,they havetochargerealmoneyfortheirproducts,and,inordertoprotecttheirassetsfrom theircompetitors,theydonottellthecustomerhowtheirsoftwareworks.Thustheend usershavelittlechanceofbeingabletoadaptoroptimizetheproductfortheirownuse.
Sincewagesandtaxesarerecurrentexpenditures,thecompanyneedstoissuefrequent charged-forupdatesandimprovements(the Danegeldeffect).
Open-sourcesoftware,ontheotherhand,isavailableforfree.Itisusuallydeveloped bycomputer-literateindividuals,oftenworkingforuniversitiesorsimilarorganizations, whoprovidetheservicefortheircolleagues.Itisdistributedsubjecttoanti-copyright licenses,whichgivenobodytherighttocopyrightitortouseitforcommercialgain. Conventionaleconomicsmightsuggestthatthegamutofopen-sourcesoftwareshould beinferiortoitsproprietarycounterpart,orelsethecommercialorganizationswould losetheirmarket.Asweshallsee,thisisnotnecessarilythecase.
Nextweneedtodifferentiatebetweentwodifferenttypesofscientificsoftware.Theeasiestapproachtoextractinginsightfromdataormodelingobservationsutilizesprebuilt softwaretools,whichwerefertoas“scientificsoftwaretools.”Proprietaryexamples includesoftwaretoolsandpackageslikeMatlab,Mathematica,IDL,Tableau,oreven Excelandopen-sourceequivalentslikeR,Octave,SciLab,andLibreOffice.Someof thesetoolsprovidegraphicaluserinterfaces(GUIs)enablingtheusertointeractwith thesoftwareinanefficientandintuitiveway.Typically,suchtoolsworkwellforstandardtasks,buttheydoofferonlyalimiteddegreeofflexibility,makingithardifnot impossibletoadaptthesepackagestosolvesometasktheywerenotdesignedfor.Other softwaretoolsprovidemoreflexibilitythroughtheirownidiosyncraticprogramming languageinwhichproblemsareenteredintoauserinterface.Afteracoherentgroup ofstatements,oftenjustanindividualstatement,hasbeentyped,thesoftwarewrites equivalentcorelanguagecodeandcompilesitonthefly.Thuserrorsand/orresultscan bereportedbacktotheuserimmediately.Suchtoolsarecalled“interpreters”astheyinterpretcodeonthefly,thusofferingahigherdegreeofflexibilitycomparedtosoftware toolswithshinyGUIs.
Onamorebasiclevel,theaforementionedsoftwaretoolsareimplementedina programminglanguage,whichisasomewhatlimitedsubsetofhumanlanguageinwhich sequencesofinstructionsarewritten,usuallybyhumans,tobereadandunderstoodby computers.Themostcommonlanguagesarecapableofexpressingverysophisticated mathematicalconcepts,albeitoftenwithasteeplearningcurve.Althoughamyriadof programminglanguagesexist,onlyahandfulhavebeenwidelyacceptedandadopted forscientificapplications.Historically,thisincludesCandFortran,aswellastheirdescendants.Inthecaseoftheseso-called compiledlanguages,compilerstranslatecode writtenbyhumansintomachinecodethatcanbeoptimizedforspeedandthenprocessed.Assuch,theyareratherlikeFormula1racingcars.Thebestofthemarecapable ofbreathtakinglyfastperformance,butdrivingthemisnotintuitiveandrequiresagreat dealoftrainingandexperience.Thisexperienceisadditionallycomplicatedbythefact thatcompilersforthesamelanguagearenotnecessarilycompatibleandneedtobesupplementedbylargelibrariestoprovidefunctionalityforseeminglybasicfunctionality. Sinceallscientificsoftwaretoolsarebuiltuponcompiledprogramminglanguages, whynotsimplywriteyourowntools?Well,aracingcarisnotusuallythebestchoice foratriptothesupermarket,wherespeedisnotofparamountimportance.Similarly,
compiledlanguagesarenotalwaysidealforquicklytryingoutnewideasorwriting shortscriptstosupportyouinyourdailywork.Thus,fortheintendedreadersofthis book,thedirectuseofcompilersislikelytobeunattractive,unlesstheiruseismandatory.Wethereforelookattheothertypeofprogramminglanguage,theso-called interpretedlanguages,whichincludethepreviouslymentionedscientifictoolsbasedon interpreters.Interpretedlanguageslackthespeedofcompiledlanguages,buttheytypicallyaremuchmoreintuitiveandeasiertolearn.
Letussummarizeourposition.Thereareprebuiltsoftwaretools,someofwhichare proprietaryandsomeofwhichareopen-sourcesoftware,thatprovidevariousdegrees offlexibility(interpreterstypicallyoffermoreflexibilitythantoolsthatfeatureGUIs) andusuallyfocusonspecifictasks.Onamorebasiclevel,therearetraditionalcompiled languagesfornumericsthatareverygeneral,veryfast,ratherdifficulttolearn,anddo notinteractreadilywithgraphicaloralgebraicprocesses.Finally,thereareinterpreted languagesthataretypicallymucheasiertolearnthancompiledlanguagesandoffera largedegreeofflexibilitybutarelessperformant.
So,whatpropertiesshouldanidealscientificsoftwarehave?Ashortlistmightcontain: amatureprogramminglanguagethatisbotheasytounderstandandhasextensive expressiveability,
integrationofalgebraic,numerical,andgraphicalfunctions,andtheoptiontoimport functionalityfromanalmostendlesslistofsupplementallibraries, theabilitytogeneratenumericalalgorithmsrunningwithspeedswithinanorderof magnitudeofthefastestofthosegeneratedbycompiledlanguages, auserinterfacewithadequateon-linehelpanddecentdocumentation, anextensiverangeoftextbooksfromwhichthecuriousreadercandevelopgreater understandingoftheconcepts, open-sourcesoftware,freelyavailable, implementationonallstandardplatforms,e.g.,Linux/Unix,MacOS,Windows. aconcisepackage,andthusimplementableonevenmodesthardware.
Youmighthaveguessedit:wearetalkingaboutPythonhere.
In1991,GuidovanRossumcreatedPythonasanopen-source,platform-independent, generalpurposeprogramminglanguage.Itisbasicallyaverysimplelanguagesurroundedbyanenormouslibraryofadd-onpackagesforalmostanyusecaseimaginable.Pythonisextremelyversatile:itcanbeusedtobuildcomplexsoftwaretoolsoras ascriptinglanguagetoquicklygetsometaskdone.Thisversatilityhasbothensuredits adoptionbypowerusersandledtotheassemblyofalargecommunityofdevelopers. ThesepropertiesmakePythonaverypowerfultoolforscientistsintheirdailyworkand wehopethatthisbookwillhelpyoumasterthistool.
1.3AboutThisBook
ThepurposeofthisintentionallyshortbookistointroducethePythonprogramming languageandtoprovideanoverviewofscientificallyrelevantpackagesandhowthey canbeutilized.Thisbookiswrittenforfirst-semesterstudentsandfacultymembers, graduatestudentsandemeriti,high-schoolstudentsandpost-docs–orsimplyforeveryonewhoisinterestedinusingPythonforscientificanalysis.
However,thisbookbynomeansclaimstobeacompleteintroductiontoPython.We leavethecomprehensivetreatmentofPythonandallitsdetailstootherswhohave donethiswithgreatsuccess(see,e.g., Lutz, 2013).Wehavequitedeliberatelypreferredbrevityandsimplicityoverencyclopediccoverageinordertogettheinquisitive readerupandrunningassoonaspossible.
Furthermore,thisbookwillnotserveasthe“NumericalRecipesforPython,”meaning thatwewillnotexplainmethodsandalgorithmsindetail:wewillsimplyshowcasehow theycanbeusedandappliedtoscientificproblems.Foranin-depthdiscussionofthese algorithms,werefertothereal NumericalRecipes – Pressetal. (2007) andallfollowing releasesthatwereadaptedtodifferentprogramminglanguages–aswellasotherworks. Giventhedynamicenvironmentofsoftwaredevelopment,detailsonspecificpackages arebestretrievedfromonlinedocumentationandreferencewebsites.Wewillprovide references,links,andpointersinordertoguideinterestedreaderstotheappropriate places.InordertoenableaneasyentryintotheworldofPython,weprovideallcode snippetspresentedinthisbookintheformofJupyterNotebooksontheCoCalccloud computingplatform.TheseNotebookscanbeaccessed,run,andmodifiedonlinefora moreinteractivelearningexperience.
Weaimtoleavethereaderwithawell-foundedframeworktohandlemanybasic,and notsobasic,tasks,aswellastheskillsettofindtheirownwayintheworldofscientific programmingandPython.
1.4References
PrintResources
Lutz,Mark. LearningPython:PowerfulObject-OrientedProgramming.O’ReillyMedia,2013.
Press,WilliamH,etal. NumericalRecipes:TheArtofScientificComputing.3rded., CambridgeUniversityPress,2007.
AboutPython
Pythoniscurrentlythemostpopularprogramminglanguageamongscientistsandother programmers.Thereareanumberofreasonsleadingtoitspopularityandfame,especiallyamongyoungerresearchers.ThischapterintroducesthePythonprogramming languageandprovidesanoverviewonhowtoinstallandusethelanguagemostefficiently.
2.1WhatIsPython?
Pythonisageneral-purposeprogramminglanguagethatisextremelyversatileandrelativelyeasytolearn.Itisconsideredahigh-levelprogramminglanguage,meaningthat theusertypicallywillnothavetodealwithsometypicalhousekeepingtaskswhendesigningcode.Thisisdifferentfromother(especiallycompiled)languagesthatheavily relyontheusertodothesetasksproperly.Pythonisdesignedinsuchawayastohelp theusertowriteeasilyreadablecodebyfollowingsimpleguidelines.ButPythonalso implementspowerfulprogrammingparadigms:itcanbeusedasanobject-oriented,procedural,andfunctionalprogramminglanguage,dependingonyourneedsandusecase. ThusPythoncombinesthesimplicityofascriptinglanguagewithadvancedconcepts thataretypicallycharacteristicforcompiledlanguages.Someofthesefeatures–which wewillintroduceindetailinChapter 3 –includedynamictyping,built-inobjecttypes andothertools,automaticmemorymanagementandgarbagecollection,aswellasthe availabilityofaplethoraofadd-onandthird-partypackagesforawiderangeofuse cases.Despiteitsapparentsimplicity,thesefeaturesmakePythonaverycompetitive, powerful,andflexibleprogramminglanguage.
Mostimportantly,Pythonisopen-sourceandassuchfreelyavailabletoeveryone.We detailinSection 2.2 howtoobtainandinstallPythononyourcomputer.
Basedonvariousrecentreportsandstatistics,Pythoniscurrentlythemostpopular programminglanguageamongresearchersandprofessionalsoftwaredevelopersfora widerangeofapplicationsandproblems.Thispopularitylargelystemsfromtheease oflearningPython,aswellastheavailabilityofalargenumberofadd-onpackages thatsupplementbasicPythonandprovideeasyaccesstotasksthatwouldotherwisebe cumbersometoimplement.
Butthereisalsoadownside:Pythonisaninterpretedlanguage,whichmakesitslower thancompiledlanguages.However,Pythonprovidessomeremediesforthisissueaswe willseeinChapter 9.
Forresearchers,Pythonoffersalargerangeofwell-maintainedopen-sourcepackages, manyofwhicharerelatedtooratleastbasedontheSciPyecosystem.SciPycontains packagesforscientificcomputing,mathematics,andengineeringapplications.Despite beingthebackboneofmanyPythonapplications,SciPyiscompletelyopen-sourceand fundedinsomepartthroughNumFocus,anonprofitorganizationsupportingthedevelopmentofscientificPythonpackages.Wewillgettoknowsomeofthepackagesthat arepartoftheSciPyuniverseinChapters 4, 5,and 8.
2.1.1ABriefHistoryofPython
ThePythonprogramminglanguagewasconceivedbyGuidovanRossum,aDutchcomputerscientist,inthe1980s.Hestartedtheimplementationin1989asahobbyproject overtheChristmasholidays.Thefirstreleasebecameavailablein1991andPython1.0 wasreleasedin1994;Python2.0becameavailablein2000.Withagrowinguserbase, thedevelopmentteamalsostartedtogrowandgraduallyallthefeaturesthatweappreciateaboutthislanguagewereimplemented.Python3.0wasreleasedin2008,which brokethebackwardscompatibilitywithPython2.xduetosomedesigndecisions.The existenceoftwoversionsofPythonthatwereincompatiblewitheachothergenerated someconfusion,especiallywithinexperiencedusers.However,supportforPython2.x endedin2020,leavingPython3.xastheonlysupportedversionofPython.Theexample codeshowninthisbookandtheaccompanyingJupyterNotebooks(seeSection 2.4.2) arebasedonPythonversion3.9.12,butthisshouldnotmatterasfutureversionsshould becompatiblewiththatone.
VanRossumisconsideredtheprincipalauthorofPythonandhasplayedacentralrole initsdevelopmentuntil2018.Since2001,thePythonSoftwareFoundation,anonprofit organizationfocusingonthedevelopmentofthecorePythondistribution,managing intellectualrights,andorganizingdeveloperconferences,hasplayedanincreasingly importantroleintheproject.Majordesigndecisionswithintheprojectaremadebya five-personsteeringcouncilanddocumentedinPythonEnhancementProtocols(PEPs). PEPsmainlydiscusstechnicalproposalsanddecisions,butwewillbrieflylookattwo PEPsthatdirectlyaffectusers:theZenofPython(PEP20,Section 2.1.2)andthePython StyleGuide(PEP8,Section 3.13).
Wewouldalsoliketonotethatin2012,NumFOCUSwasfoundedasanonprofitorganizationthatsupportsthedevelopmentofawiderangeofscientificPythonpackages including,butnotlimitedto,NumPy(seeChapter 4),SciPy(seeChapter 5),Matplotlib (seeChapter 6),SymPy(seeChapter 7),Pandas(seeChapter 8),ProjectJupyter,and IPython.ThesupportthroughNumFOCUSfortheseprojectsincludesfundingthatis basedondonationstoNumFOCUS;formostoftheseopen-sourceprojects,donations aretheironlysourceoffunding.
OnedetailwehaveskippedsofariswhyVanRossumnamedhisnewprogramming languageafterasnake.Well,hedidnot.PythonisactuallynamedaftertheBBCcomedy TVshow MontyPython’sFlyingCircus,ofwhichVanRossumisahugefan.Incaseyou werewondering,thisisalsothereasonwhythewords“spam”and“eggs”areoftentimes usedasmetasyntacticvariablesinPythonexamplecodeinareferencetotheirfamous “Spam”sketchfrom1970.
2.1.2TheZenofPython
TheZenofPythonisanattempttosummarizeVanRossum’sguidingprinciplesforthe designofPythoninto20aphorisms,only19ofwhichhavebeenwrittendown.These guidingprinciplesareveryconciseanddistillmanyfeaturesofPythonintoafewwords.
TheZenofPythonissoimportantthatitisactuallypublished(PEP20)anditscontent isliterallybuiltintothePythonlanguageandcanbeaccessedasfollows:
import this
Beautifulisbetterthanugly. Explicitisbetterthanimplicit. Simpleisbetterthancomplex. Complexisbetterthancomplicated. Flatisbetterthannested. Sparseisbetterthandense. Readabilitycounts.
Specialcasesaren’tspecialenoughtobreaktherules. Althoughpracticalitybeatspurity. Errorsshouldneverpasssilently. Unlessexplicitlysilenced. Inthefaceofambiguity,refusethetemptationtoguess. Thereshouldbeone andpreferablyonlyone obviouswayto doit.
Althoughthatwaymaynotbeobviousatfirstunlessyou’reDutch. Nowisbetterthannever.
Althoughneverisoftenbetterthan *right* now. Iftheimplementationishardtoexplain,it’sabadidea. Iftheimplementationiseasytoexplain,itmaybeagoodidea. Namespacesareonehonkinggreatidea let’sdomoreofthose!
PleasenotethattheseguidelinesfocusonthedesignofthePythonprogramminglanguage,notnecessarilythedesignofcodewritteninPython.Nevertheless,youarefreeto followtheseguidelineswhenwritingyourowncodetocreatetruly pythonic code.The term pythonic isoftenusedwithinthePythoncommunitytorefertocodethatfollows theguidingprinciplesmentionedhere.
Theseguidingprinciplesarenumerousandsomeofthemmightnotbeimmediately cleartothereader,especiallyifyouarenewtoPythonprogramming.WewouldsummarizethemostimportantPythonconceptsasfollows.
Simplicity Simplecodeiseasiertowriteandread;itimprovesreadability,shareability, andmaintainability,andthereforehelpsyouandothersintheshorttermand longterm.
Readability Itisnicetowritecodeascompactaspossible,butifwritingcompact coderequiressometricksthatarehardtounderstand,youmightpreferamore extensiveimplementationthatprovidesbetterreadability.Why?Imaginethat yourfutureselftriestomodifysomecodethatyouwroteyearsago.Ifyour codeiswell-readable,youwillprobablyhavefewerproblemsunderstanding whattheindividuallinesofcodedo.
Explicitness Wewillexplainthisideawithanexample.Consideryouarewritingcode thatisabletoreaddatafromdifferentfileformats.Adecisionyouhaveto makeisthefollowing:willyoucreateasinglefunctionthatisabletoread allthedifferentfileformats,ordoyoucreateanumberofindividualfunctions, eachofwhichisabletoreadonlyasinglefileformat?The pythonic waywould bethelatter:eachfunctionthatyoucreatewillexplicitlybeabletodealwith onlyasinglefileformatincontrasttoasinglefunctionthatimplicitlydeals withallfileformats.Whyisthissolutionfavored?Generally,explicitcodeis easiertounderstandandlesspronetoconfusion.
Naturally,theseconceptsareentangledandcloselyrelatedtoeachother.However,there isnoneedtomemorizetheseconcepts.Youwillinternalizethoseconceptsthatare relevanttoyoubywritingcodeandreadingcodewrittenbyothers.And,ofcourse, nobodycanforceyoutofollowtheseprinciplesinyourowncoding;butwehopethat thissectionprovidesyouabetterunderstandingofthePythonprogramminglanguage anditsdesign.
2.2InstallingPython
Dependingontheoperatingsystemyouareusing,thereareseveralwaystoinstall Pythononyourcomputer,someofwhicharesimplerthanothers.Theeasiestandat thesametimesafestwaytoinstallPythonistousetheAnacondaenvironmentasdetailedbelow.
Alternatively,youcanalsoinstallPythonfromscratchonyourcomputer–unlessitis alreadyinstalled.Inthelattercase,youshouldbecarefulnottointerferewiththenative Pythonalreadyinstalledasitmightberequiredbyyouroperatingsystem.Thisprocess mightbeabitmorecomplicated,buttherearedetailedinstallationguidesforalloperatingsystemsavailableonline.Tobeonthesafeside,werecommendtheinstallationof Anaconda,whichcomeswithConda,atooltosetupandutilizevirtualenvironments,
inordertopreventinterferencewithotherversionsofPythonthatmightbeinstalledon yourcomputer.OncePythonisinstalled,additionalpackagescanalsobeinstalledusing Condaandthe PackageinstallerforPython,pip.
2.2.1AnacondaandConda
AnacondaisaPythondistributionpackagefordatascienceandmachinelearningapplications.Despitethisspecialization,theAnacondaIndividualEdition(alsoknownasthe “AnacondaDistribution”)constitutesasolidbasisforanyscientificPythoninstallation.
TheAnacondaDistributionisprovidedandmaintainedbyAnacondaInc.(previously knownasContinuumAnalytics).Despitebeingafor-profitcompany,AnacondaInc. distributestheAnacondaIndividualEditionforfree.
InstallingAnaconda
InstallingAnacondaissimpleandstraightforward.Allthatisrequiredistodownload therespectiveAnacondaIndividualEditioninstaller(seeSection 2.6)foryouroperating systemandrunit.Theinstallerwillwalkyouthroughtheinstallationprocess.Notethat youwillneedtoagreetotheAnacondalicenseagreement.Attheendoftheinstallationroutine,youwillbeaskedwhethertomakeAnacondaPythonyourdefaultPython version,whichisagoodideainmostcases.IfyounowstartthePythoninterpreter(see Section 2.4.1),youwillbegreetedbyAnacondaPython.Congratulations,youhave successfullyinstalledAnacondaPythononyourcomputer.
Conda
OneadvantageofusingAnacondaistheavailabilityofConda,anopen-sourcepackageandenvironmentmanagerthatwasoriginallydevelopedbyAnacondaInc.,buthas subsequentlybeenreleasedseparatelyunderanopen-sourcelicense.Although,fora beginner,thesimpleinstallationprocessforAnacondaPythonismostlikelyitsmost importantfeature,Condaalsosolvestwoproblemsinthebackground.Asapackage manager,itallowsyoutoeasilyinstallPythonpackageswithasinglecommandon yourcommandline,e.g., condainstallnumpy
AlmostallmajorPythonpackagesareavailablethroughConda.Packagesareavailable throughConda-Forge(seeSection 2.6),aGitHub(seeSection 10.3.1)organizationthat containsrepositoriesof“Condarecipes”forawiderangeofpackages.Conda-Forge containsmoredetailedinformationonhowtoinstallpackagesthroughConda,aswell asalistofallpackagesthatareavailablethroughConda.
Asanenvironmentmanager,Condaallowsyoutodefinedifferentenvironments,each ofwhichcanhaveitsownPythoninstallation.Althoughthisisanadvancedfeatureand becomesimportantwhenyouaredealingwithspecificversionsofyourPythonpackages,thereisstillsomebenefitforthePythonbeginner.Someoperatingsystemsuse
nativePythoninstallationtoruncrucialservices;meddlingwiththesePythoninstallationscanseriouslyharmyoursystem.Bydefault,Anacondacreatesa base environment fortheuser.Sincethisenvironmentisindependentfromyoursystem,thereisnodanger inmeddlingwithyoursystemPythoninstallation.ThususingAnacondaissaferthan usingyoursystemPythoninstallation.
ItisnotcomplicatedtodefinenewCondaenvironmentsandtoswitchbetweenthem. However,duetotheadvancednatureofdealingwithdifferentenvironments,werefer totheCondadocumentationtolearnmoreabouthowtodothis.
2.2.2PipandPyPI
PrettymuchallPythonpackagesareregisteredwiththe PythonPackageIndex,PyPI, whichenablestheeasydistributionofthesepackages.InstallingpackagesfromPyPIis veryeasyusingthepippackagemanager,whichcomeswithmostPythoninstallations, e.g.,
pipinstallnumpy
EverybodycanpublishtheircodeviaPyPI;inSection 10.3.2 wewillshowhowthiscan beachieved.SincePyPIistheofficialrepositoryofPythonpackages,prettymuchall availablepackagesareinstallableusingpip.
PiporConda?
AfterlearningaboutCondaandpipyoumightbeconfusedwhichofthesetoolsyou shouldusetoinstallPythonpackages.Theshortansweris,inmostcasesitdoesnot matter.Especiallyforbeginners,itisperfectlyfineandtypicallyalsosafetoinstall packagesusingpip.PipistypicallyfasterthanCondaininstallingpackages.
Thisfasterinstallationprocesscomesata(small)pricethatwon’tmattertomostusers. ThepriceisthatCondaisgenerallysaferininstallingnewpackages.BeforeCondainstallsanewpackage,itwillchecktheversionnumbersofallpackagesthatarealready installedinyourcurrentCondaenvironmentanditwillcheckwhetherthesepackages inthepresentversionsarecompatiblewiththenewpackageandviceversa.Pipsimplycheckswhethertheversionsoftheinstalledpackagesarecompatiblewiththenew package–anditwillupdatethealreadypresentpackages,tomakethemcompatible withthenewpackage.However,pipdisregardsthattheremightberequirementsby otherpackagesthatwillbreakbyupdatingtheseexistingpackages.Asaresult,pipmay breakpackagesthatwerepreviouslyinstalled.
Thishappensveryrarely,sincemostPythonpackagesarecompatibleovermanydifferentversions.However,inthecaseofquicklydevelopingprojectsitismandatoryto usespecificversionsofpackages.Inthosecases,itismuchsafertouseCondatoinstall newpackages.Formostotherusers,especiallyonthebeginnerlevel,thereshouldbe nomajorissues.
2.3HowPythonWorks
InChapter 1,wealreadyintroducedcompiledandinterpretedprogramminglanguages. Asabriefreminder,compiledlanguagestakethecodewrittenbytheuserinsome high-levelprogramminglanguageandtranslateitintomachine-readablecodethatis writtentoanexecutablefile.Interpretedlanguages,ontheotherhand,donotrequirethe high-levelcodeprovidedbytheusertobecompiled.Instead,the interpreter readsthe codeinchunksandtranslatesthemsequentiallyintosomeless-basickindofmachinereadable bytecode thatisdirectlyexecuted.Asyoucanimagine,compiledlanguages performfasterthaninterpretedlanguages,sincethe compiler alreadydoesthehardwork totranslateusercodetoefficientmachine-readablecode,whereasaninterpreterhasto dothisontheflyinalessefficientway.
Thefollowingsectionswilldetailhowtodirectlyprovidecodetotheinterpreterin differentways.
2.4HowtoUsePython
TherearedifferentwaystousePython,themostimportantofwhichwewillintroduce inthefollowingsections.Whichoftheseoptionsyoushouldusedependsonyourpreferencesandtheproblemyouaretryingtosolve.
Intheremainderofthisbook,weassumethatyouareusingJupyterNotebooks.This choiceismainlydrivenbytheopportunitytopublishallcodeelementsfromthisbook asreadilyaccessibleJupyterNotebooks.YoucanruntheseNotebooks(aswellasyour ownNotebooks)onlineinthecloud,orlocallyonyourcomputerasdetailedbelow. However,wewouldliketopointoutthatitisnotarequirementforthereadertouse theseNotebooksinordertofollowthisbookinanyway.Feelfreetousewhichever interfacetoPythonyoufeelmostcomfortablewith.
2.4.1ThePythonInterpreter
TheeasiestwaytousePythonistorunitsinterpreterininteractivemode.Onmost operatingsystems,thisisdonebysimplytyping python intoaterminalorpowershell window.Oncestarted,youcantypePythoncommandsandstatementsintotheinterpreter,whicharethenexecutedlinebyline(orblockbyblockifyouuseindentation). Whilethismightbeusefultoquicklytrysomethingout,itisnotreallysuitedtowrite longscriptsorothermoreorlesscomplexpiecesofcode.Theinterpreteralsoprovides onlyabareminimumintermsofsupportandusability.
ThePythoninterpreteralsooffersadifferentwaytorunPythoncodethatismuchbetter suitedforrunninglongerpiecesofcode.Insteadofwritingyourcodelinebylineinto theinterpreter,youcansimplywriteyourcodeintoanordinarytextfileandpassthat
filetotheinterpreterinyourterminalwindoworonthecommandline.Youcangive yourcodefileanynameyouwant,butbyconvention,youshouldusethefileending “.py.”Youcanusethemostbasictexteditorforthispurpose:Emacs,Vim,Nanoor GeditonLinux,TextEditorSublimeonaMac,orNotePadonWindows.Itisimportant thattheresultingPythoncodefiledoesnotcontainanyfancyformatting,justclean text.
Forexample,youcouldcreateafilenamed“hello.py”withthefollowingsingleline ofcontent:
print(’HelloWorld!’)
Youcanthenrunthisscriptinaterminalwindoworpowershellbyusing pythonhello.py
MakesurethatPythonisproperlyinstalledonyoursystem(seeSection 2.2)andthat yourunthiscommandinthesamedirectorywherethe hello.py fileresides.Ifsuccessful,theoutputthatyoureceiveshouldlooklikethis:
HelloWorld!
AndthisisyourfirstPythonprogram!
2.4.2IPythonandJupyter
IPython(InteractivePython)isanarchitectureforinteractivecomputingwithPython: itcanbeconsideredasthePythoninterpreteronsteroids.TheIPythoninterpreterhas beendesignedandwrittenbyscientistswiththeaimofofferingveryfastexploration andconstructionofcodewithminimaltypingeffort,andofferingappropriate,even maximal,on-screenhelpwhenrequired.Itfurthersupportsintrospection(theabilityto examinethepropertiesofanyPythonobjectatruntime),tabcompletion(autocompletionsupportduringtypingwhenhittingtheTabkey),history(IPythonstorescommands thatareenteredandtheirresults,bothofwhichcanbeaccessedatruntime),aswell assupportforparallelcomputing.Mostimportantly,IPythonincludesabrowser-based Notebookinterfacewithavisuallyappealingnotebook-likeappearance.
ThefirstversionofIPythonwaspublishedin2001.ProjectJupyterevolvedfromIPython around2014asanonprofit,open-sourceprojecttosupportinteractivedatascience andscientificcomputing.TheNotebookinterfacewassubsequentlyoutsourcedfrom IPythonandimplementedaspartofJupyter,whereitwasperfectedandextendedin differentways.Mostnotably,JupyterNotebooksarelanguageagnosticandcanbeused withdifferentprogramminglanguagesusingso-calledkernels.ThePythonkernelis providedandstillmaintainedbytheIPythonproject.
Figure2.1 AnewlycreatedJupyterNotebookcontainingasingle,emptycodecell.
ThefollowingsectionsintroducethemostimportantfeaturesofJupyter.
JupyterNotebooks
ThemostrelevantfeatureofJupyterforyouwillmostlikelybethe JupyterNotebook, whichisanenhancedversionoftheIPythonNotebook.Allprogrammingexamples arepresentedintheformofJupyterNotebooksandimitatetheirappearance(seeFigure 2.1).Furthermore,allcodeelementsshowninthisbookareavailableasJupyter Notebooksonlineat CoCalc,andalsoat www.cambridge.org/9781009014809
JupyterNotebooksaredocumentsthatconsistofdistinctcellsthatcancontainandrun code,formattedtext,mathematicalequations,andothermedia.Notebooksarerunin yourbrowserthroughaserverthatiseitherhostedlocallyonyourcomputerorinthe cloud(seeSection 2.4.4).
TostartaJupyterNotebookserverlocally,yousimplyhavetorun jupyternotebook
inaterminalwindoworpowershell.Thiswillrunaserverinthebackgroundthatis typicallyaccessiblethrough http://localhost:8888 (youneedtotypethisintothe URLfieldofyourbrowsertoaccesstheNotebookserver).Youwillseealistoffilesand directorieslocatedinthedirectoryyoustartedtheserverfrom.Fromhereyoucannavigateyourfilesystem,andopenexistingJupyterNotebooksorcreatenewones.Notebookfilesusethefilenameending .ipynb,indicatingthattheyareusingtheIPython kernel.ToopenaNotebook,simplyclickonthefileandyouwillseesomethingthat lookslikeFigure 2.1.
CloudserviceshostingJupyterNotebookserversareadifferentavenuethatallowyouto utilizeNotebookswithoutthe(minor)hassleofhavingtoinstallthenecessarysoftware onyourcomputer.Asaresult,Notebooksthatrunoncloudservices(seeSection 2.4.4) mightlookalittlebitdifferenttowhatisshowninFigure 2.1,butrestassuredthatthey canbeusedinthesamewayasdescribedinthisbook.
Notebooksconsistof cells thatareeither codecells thatcontainPythoncodeor markdowncells thatcontaintextorothermediautilizingthemarkdownlanguage. Markdown isalightweightmarkuplanguage(punintended)thatenablesyoutoquickly formattextandevensupportsLaTeXinlinemath.AmarkdowncheatsheetcontainingsomeformattingbasicsisprovidedinTable 2.1 foryourconvenience;formore
