ProbabilityandStatistics
CarnegieMellonUniversity MarkJ.Schervish
CarnegieMellonUniversity
Addison-Wesley
BostonColumbusIndianapolisNewYorkSanFranciscoUpperSaddleRiver AmsterdamCapeTownDubaiLondonMadridMilanMunichParisMontr ´ ealToronto DelhiMexicoCitySaoPauloSydneyHongKongSeoulSingaporeTaipeiTokyo
EditorinChief: DeirdreLynch
AcquisitionsEditor: ChristopherCummings
AssociateContentEditors: LeahGoldberg,DanaJonesBettez
AssociateEditor: ChristinaLepre
SeniorManagingEditor: KarenWernholm
ProductionProjectManager: PattyBergin
CoverDesigner: HeatherScott
DesignManager: AndreaNix
SeniorMarketingManager: AlexGay
MarketingAssistant: KathleenDeChavez
SeniorAuthorSupport/TechnologySpecialist: JoeVetere
RightsandPermissionsAdvisor: MichaelJoyce
ManufacturingManager: CarolMelville
ProjectManagement,Composition: WindfallSoftware,usingZzTEX
CoverPhoto: Shutterstock/©MarilynVolan
Theprogramsandapplicationspresentedinthisbookhavebeenincludedfortheirinstructionalvalue.Theyhavebeentestedwithcare,butarenotguaranteedforanyparticular purpose.Thepublisherdoesnotofferanywarrantiesorrepresentations,nordoesitaccept anyliabilitieswithrespecttotheprogramsorapplications.
Manyofthedesignationsusedbymanufacturersandsellerstodistinguishtheirproductsare claimedastrademarks.Wherethosedesignationsappearinthisbook,andPearsonEducation wasawareofatrademarkclaim,thedesignationshavebeenprintedininitialcapsorallcaps.
LibraryofCongressCataloging-in-PublicationData DeGroot,MorrisH.,1931–1989.
Probabilityandstatistics/MorrisH.DeGroot,MarkJ.Schervish.—4thed. p.cm.
ISBN978-0-321-50046-5
1.Probabilities—Textbooks.2.Mathematicalstatistics—Textbooks. I.Schervish,MarkJ.II.Title.
QA273.D352012
519.2—dc22
Copyright©2012,2002PearsonEducation,Inc.
2010001486
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem, ortransmitted,inanyformorbyanymeans,electronic,mechanical,photocopying,recording, orotherwise,withoutthepriorwrittenpermissionofthepublisher.PrintedintheUnited StatesofAmerica.Forinformationonobtainingpermissionforuseofmaterialinthiswork, pleasesubmitawrittenrequesttoPearsonEducation,Inc.,RightsandContractsDepartment, 75ArlingtonStreet,Suite300,Boston,MA02116,faxyourrequestto617-848-7047,ore-mail athttp://www.pearsoned.com/legal/permissions.htm.
123456789 10—EB—1413121110
www.pearsonhighered.com
ISBN10:0-321-50046-6
ISBN13:978-0-321-50046-5
This page intentionally left blank
Prefacexi
1 IntroductiontoProbability 1
1.1 TheHistoryofProbability1
1.2 InterpretationsofProbability2
1.3 ExperimentsandEvents5
1.4 SetTheory6
1.5 TheDefinitionofProbability16
1.6 FiniteSampleSpaces22
1.7 CountingMethods25
1.8 CombinatorialMethods32
1.9 MultinomialCoefficients42
1.10 TheProbabilityofaUnionofEvents46
1.11 StatisticalSwindles51
1.12 SupplementaryExercises53
2 ConditionalProbability 55
2.1 TheDefinitionofConditionalProbability55
2.2 IndependentEvents66
2.3 Bayes’Theorem76
2.4 TheGambler’sRuinProblem86
2.5 SupplementaryExercises90
3 RandomVariablesandDistributions 93
3.1 RandomVariablesandDiscreteDistributions93
3.2 ContinuousDistributions100
3.3 TheCumulativeDistributionFunction107
3.4 BivariateDistributions118
3.5 MarginalDistributions130
3.6 ConditionalDistributions141
3.7 MultivariateDistributions152
3.8 FunctionsofaRandomVariable167
3.9 FunctionsofTwoorMoreRandomVariables175
3.10 MarkovChains188
3.11 SupplementaryExercises202
4 Expectation 207
4.1 TheExpectationofaRandomVariable207
4.2 PropertiesofExpectations217
4.3 Variance225
4.4 Moments234
4.5 TheMeanandtheMedian241
4.6 CovarianceandCorrelation248
4.7 ConditionalExpectation256
4.8 Utility265
4.9 SupplementaryExercises272
5 SpecialDistributions 275
5.1 Introduction275
5.2 TheBernoulliandBinomialDistributions275
5.3 TheHypergeometricDistributions281
5.4 ThePoissonDistributions287
5.5 TheNegativeBinomialDistributions297
5.6 TheNormalDistributions302
5.7 TheGammaDistributions316
5.8 TheBetaDistributions327
5.9 TheMultinomialDistributions333
5.10 TheBivariateNormalDistributions337
5.11 SupplementaryExercises345
6 LargeRandomSamples 347
6.1 Introduction347
6.2 TheLawofLargeNumbers348
6.3 TheCentralLimitTheorem360
6.4 TheCorrectionforContinuity371
6.5 SupplementaryExercises375
7 Estimation 376
7.1 StatisticalInference376
7.2 PriorandPosteriorDistributions385
7.3 ConjugatePriorDistributions394
7.4 BayesEstimators408
7.5 MaximumLikelihoodEstimators417
7.6 PropertiesofMaximumLikelihoodEstimators426
7.7 SufficientStatistics443
7.8 JointlySufficientStatistics449
7.9 ImprovinganEstimator455
7.10 SupplementaryExercises461
8 SamplingDistributionsofEstimators 464
8.1 TheSamplingDistributionofaStatistic464
8.2 TheChi-SquareDistributions469
8.3 JointDistributionoftheSampleMeanandSampleVariance473
8.4 The t Distributions480
8.5 ConfidenceIntervals485
8.6 BayesianAnalysisofSamplesfromaNormalDistribution495
8.7 UnbiasedEstimators506
8.8 FisherInformation514
8.9 SupplementaryExercises528
9 TestingHypotheses 530
9.1 ProblemsofTestingHypotheses530
9.2 TestingSimpleHypotheses550
9.3 UniformlyMostPowerfulTests559
9.4 Two-SidedAlternatives567
9.5 The t Test576
9.6 ComparingtheMeansofTwoNormalDistributions587
9.7 The F Distributions597
9.8 BayesTestProcedures605
9.9 FoundationalIssues617
9.10 SupplementaryExercises621
10 CategoricalDataandNonparametricMethods 624
10.1 TestsofGoodness-of-Fit624
10.2 Goodness-of-FitforCompositeHypotheses633
10.3 ContingencyTables641
10.4 TestsofHomogeneity647
10.5 Simpson’sParadox653
10.6 Kolmogorov-SmirnovTests657
Contents
10.7 RobustEstimation666
10.8 SignandRankTests678
10.9 SupplementaryExercises686
11
LinearStatisticalModels 689
11.1 TheMethodofLeastSquares689
11.2 Regression698
11.3 StatisticalInferenceinSimpleLinearRegression707
11.4 BayesianInferenceinSimpleLinearRegression729
11.5 TheGeneralLinearModelandMultipleRegression736
11.6 AnalysisofVariance754
11.7 TheTwo-WayLayout763
11.8 TheTwo-WayLayoutwithReplications772
11.9 SupplementaryExercises783
12 Simulation 787
12.1 WhatIsSimulation?787
12.2 WhyIsSimulationUseful?791
12.3 SimulatingSpecificDistributions804
12.4 ImportanceSampling816
12.5 MarkovChainMonteCarlo823
12.6 TheBootstrap839
12.7 SupplementaryExercises850
Tables853
AnswerstoOdd-NumberedExercises865 References879 Index885
ChangestotheFourthEdition
. Ihavereorganizedmanymainresultsthatwereincludedinthebodyofthe textbylabelingthemastheoremsinordertofacilitatestudentsinfindingand referencingtheseresults.
. Ihavepulledtheimportantdefintionsandassumptionsoutofthebodyofthe textandlabeledthemassuchsothattheystandoutbetter.
. Whenanewtopicisintroduced,Iintroduceitwithamotivatingexamplebefore delvingintothemathematicalformalities.ThenIreturntotheexampleto illustratethenewlyintroducedmaterial.
. Imovedthematerialonthelawoflargenumbersandthecentrallimittheorem toanewChapter6.Itseemedmorenaturaltodealwiththemainlarge-sample resultstogether.
. ImovedthesectiononMarkovchainsintoChapter3.EverytimeIcoverthis materialwithmyownstudents,Istumbleovernotbeingabletorefertorandom variables,distributions,andconditionaldistributions.Ihaveactuallypostponed thismaterialuntilafterintroducingdistributions,andthengonebacktocover Markovchains.Ifeelthatthetimehascometoplaceitinamorenatural location.IalsoaddedsomematerialonstationarydistributionsofMarkov chains.
. Ihavemovedthelengthyproofsofseveraltheoremstotheendsoftheir respectivesectionsinordertoimprovetheflowofthepresentationofideas.
. IrewroteSection7.1tomaketheintroductiontoinferenceclearer.
. IrewroteSection9.1asamorecompleteintroductiontohypothesistesting, includinglikelihoodratiotests.Forinstructorsnotinterestedinthemoremathematicaltheoryofhypothesistesting,itshouldnowbeeasiertoskipfrom Section9.1directlytoSection9.5.
Someotherchangesthatreaderswillnotice:
. Ihavereplacedthenotationinwhichtheintersectionoftwosets A and B had beenrepresented AB withthemorepopular A ∩ B .Theoldnotation,although mathematicallysound,seemedabitarcaneforatextatthislevel.
. IaddedthestatementsofStirling’sformulaandJensen’sinequality.
. Imovedthelawoftotalprobabilityandthediscussionofpartitionsofasample spacefromSection2.3toSection2.1.
. Idefinethecumulativedistributionfunction(c.d.f.)asthepreferednameof whatusedtobecalledonlythedistributionfunction(d.f.).
. IaddedsomediscussionofhistogramsinChapters3and6.
. IrearrangedthetopicsinSections3.8and3.9sothatsimplefunctionsofrandom variablesappearfirstandthegeneralformulationsappearattheendtomake iteasierforinstructorswhowanttoavoidsomeofthemoremathematically challengingparts.
. Iemphasizedtheclosenessofahypergeometricdistributionwithalargenumberofavailableitemstoabinomialdistribution.
. IgaveabriefintroductiontoChernoffbounds.Thesearebecomingincreasingly importantincomputerscience,andtheirderivationrequiresonlymaterialthat isalreadyinthetext.
. Ichangedthedefinitionofconfidenceintervaltorefertotherandominterval ratherthantheobservedinterval.Thismakesstatementslesscumbersome,and itcorrespondstomoremodernusage.
. IaddedabriefdiscussionofthemethodofmomentsinSection7.6.
. IaddedbriefintroductionstoNewton’smethodandtheEMalgorithmin Chapter7.
. Iintroducedtheconceptofpivotalquantitytofacilitateconstructionofconfidenceintervalsingeneral.
. Iaddedthestatementofthelarge-sampledistributionofthelikelihoodratio teststatistic.Ithenusedthisasanalternativewaytotestthenullhypothesis thattwonormalmeansareequalwhenitisnotassumedthatthevariancesare equal.
. ImovedtheBonferroniinequalityintothemaintext(Chapter1)andlater (Chapter11)useditasawaytoconstructsimultaneoustestsandconfidence intervals.
HowtoUseThisBook
Thetextissomewhatlongforcompletecoverageinaone-yearcourseattheundergraduatelevelandisdesignedsothatinstructorscanmakechoicesaboutwhichtopics aremostimportanttocoverandwhichcanbeleftformorein-depthstudy.Asanexample,manyinstructorswishtodeemphasizetheclassicalcountingargumentsthat aredetailedinSections1.7–1.9.Aninstructorwhoonlywantsenoughinformation tobeabletocoverthebinomialand/ormultinomialdistributionscansafelydiscussonlythedefinitionsandtheoremsonpermutations,combinations,andpossibly multinomialcoefficients.Justmakesurethatthestudentsrealizewhatthesevalues count,otherwisetheassociateddistributionswillmakenosense.Thevariousexamplesinthesesectionsarehelpful,butnotnecessary,forunderstandingtheimportant distributions.AnotherexampleisSection3.9onfunctionsoftwoormorerandom variables.TheuseofJacobiansforgeneralmultivariatetransformationsmightbe moremathematicsthantheinstructorsofsomeundergraduatecoursesarewilling tocover.Theentiresectioncouldbeskippedwithoutcausingproblemslaterinthe course,butsomeofthemorestraightforwardcasesearlyinthesection(suchasconvolution)mightbeworthintroducing.ThematerialinSections9.2–9.4onoptimal testsinone-parameterfamiliesisprettymathematics,butitisofinterestprimarily tograduatestudentswhorequireaverydeepunderstandingofhypothesistesting theory.TherestofChapter9coverseverythingthatanundergraduatecoursereally needs.
Inadditiontothetext,thepublisherhasan Instructor’sSolutionsManual, availablefordownloadfromtheInstructorResourceCenteratwww.pearsonhighered .com/irc,whichincludessomespecificadviceaboutmanyofthesectionsofthetext. Ihavetaughtayear-longprobabilityandstatisticssequencefromearliereditionsof thistextforagroupofmathematicallywell-trainedjuniorsandseniors.Inthefirst semester,Icoveredwhatwasintheearliereditionbutisnowinthefirstfivechapters(includingthematerialonMarkovchains)andpartsofChapter6.Inthesecond semester,IcoveredtherestofthenewChapter6,Chapters7–9,Sections11.1–11.5, andChapter12.Ihavealsotaughtaone-semesterprobabilityandrandomprocesses
courseforengineersandcomputerscientists.Icoveredwhatwasintheoldedition andisnowinChapters1–6and12,includingMarkovchains,butnotJacobians.This lattercoursedidnotemphasizemathematicalderivationtothesameextentasthe courseformathematicsstudents.
Anumberofsectionsaredesignatedwithanasterisk(*).Thisindicatesthat latersectionsdonotrelymateriallyonthematerialinthatsection.Thisdesignation isnotintendedtosuggestthatinstructorsskipthesesections.Skippingoneofthese sectionswillnotcausethestudentstomissdefinitionsorresultsthattheywillneed later.Thesectionsare2.4,3.10,4.8,7.7,7.8,7.9,8.6,8.8,9.2,9.3,9.4,9.8,9.9,10.6, 10.7,10.8,11.4,11.7,11.8,and12.5.Asidefromcross-referencesbetweensections withinthislist,occasionalmaterialfromelsewhereinthetextdoesreferbackto someofthesectionsinthislist.Eachofthedependenciesisquiteminor,however. MostofthedependenciesinvolvereferencesfromChapter12backtooneofthe optionalsections.Thereasonforthisisthattheoptionalsectionsaddresssomeof themoredifficultmaterial,andsimulationismostusefulforsolvingthosedifficult problemsthatcannotbesolvedanalytically.Exceptforpassingreferencesthathelp putmaterialintocontext,thedependenciesareasfollows:
. Thesampledistributionfunction(Section10.6)isreintroducedduringthe discussionofthebootstrapinSection12.6.Thesampledistributionfunction isalsoausefultoolfordisplayingsimulationresults.Itcouldbeintroducedas earlyasExample12.3.7simplybycoveringthefirstsubsectionofSection10.6.
. Thematerialonrobustestimation(Section10.7)isrevisitedinsomesimulation exercisesinSection12.2(Exercises4,5,7,and8).
. Example12.3.4makesreferencetothematerialontwo-wayanalysisofvariance (Sections11.7and11.8).
Supplements
Thetextisaccompaniedbythefollowingsupplementarymaterial:
. Instructor’sSolutionsManual containsfullyworkedsolutionstoallexercises inthetext.AvailablefordownloadfromtheInstructorResourceCenterat www.pearsonhighered.com/irc.
. StudentSolutionsManual containsfullyworkedsolutionstoalloddexercisesin thetext.AvailableforpurchasefromMyPearsonStoreatwww.mypearsonstore .com.(ISBN-13:978-0-321-71598-2;ISBN-10:0-321-71598-5)
Acknowledgments
TherearemanypeoplethatIwanttothankfortheirhelpandencouragementduring thisrevision.Firstandforemost,IwanttothankMarilynDeGrootandMorrie’s childrenforgivingmethechancetoreviseMorrie’smasterpiece.
Iamindebtedtothemanyreaders,reviewers,colleagues,staff,andpeople atAddison-Wesleywhosehelpandcommentshavestrengthenedthisedition.The reviewerswere:
AndreAdler,IllinoisInstituteofTechnology;E.N.Barron,LoyolaUniversity;Brian Blank,WashingtonUniversityinSt.Louis;IndranilChakraborty,UniversityofOklahoma;DanielChambers,BostonCollege;RitaChattopadhyay,EasternMichigan University;StephenA.Chiappari,SantaClaraUniversity;Sheng-KaiChang,Wayne StateUniversity;JustinCorvino,LafayetteCollege;MichaelEvans,Universityof
Toronto;DougFrank,IndianaUniversityofPennsylvania;AndaGadidov,KennesawStateUniversity;LynGeisler,Randolph–MaconCollege;PremGoel,Ohio StateUniversity;SusanHerring,SonomaStateUniversity;PawelHitczenko,Drexel University;LifangHsu,LeMoyneCollege;Wei-MinHuang,LehighUniversity; SyedKirmani,UniversityofNorthernIowa;MichaelLavine,DukeUniversity;Rich Levine,SanDiegoStateUniversity;JohnLiukkonen,TulaneUniversity;Sergio Loch,GrandViewCollege;RosaMatzkin,NorthwesternUniversity;TerryMcConnell,SyracuseUniversity;Hans-GeorgMueller,UniversityofCalifornia–Davis; RobertMyers,BethelCollege;MarioPeruggia,TheOhioStateUniversity;Stefan Ralescu,QueensUniversity;KrishnamurthiRavishankar,SUNYNewPaltz;Diane Saphire,TrinityUniversity;StevenSepanski,SaginawValleyStateUniversity;HenSiongTan,PennsylvaniaUniversity;KanapathiThiru,UniversityofAlaska;KennethTroske,JohnsHopkinsUniversity;JohnVanNess,UniversityofTexasatDallas;YehudaVardi,RutgersUniversity;YelenaVaynberg,WayneStateUniversity; JosephVerducci,OhioStateUniversity;MahbobehVezveai,KentStateUniversity; BraniVidakovic,DukeUniversity;KarinVorwerk,WestfieldStateCollege;Bette Warren,EasternMichiganUniversity;CalvinL.Williams,ClemsonUniversity;Lori Wolff,UniversityofMississippi.
ThepersonwhocheckedtheaccuracyofthebookwasAndaGadidov,KennesawStateUniversity.IwouldalsoliketothankmycolleaguesatCarnegieMellon University,especiallyAnthonyBrockwell,JoelGreenhouse,JohnLehoczky,Heidi Sestrich,andValerieVentura.
ThepeopleatAddison-Wesleyandotherorganizationsthathelpedproduce thebookwerePaulAnagnostopoulos,PattyBergin,DanaJonesBettez,Chris Cummings,KathleenDeChavez,AlexGay,LeahGoldberg,KarenHartpence,and ChristinaLepre.
IfIleftanyoneout,itwasunintentional,andIapologize.Errorsinevitablyarise inanyprojectlikethis(meaningaprojectinwhichIaminvolved).Forthisreason, Ishallpostinformationaboutthebook,includingalistofcorrections,onmyWeb page,http://www.stat.cmu.edu/~mark/,assoonasthebookispublished.Readersare encouragedtosendmeanyerrorsthattheydiscover.
MarkJ.Schervish
October2010
Introductionto Probability
1.1TheHistoryofProbability
Theuseofprobabilitytomeasureuncertaintyandvariabilitydatesbackhundreds ofyears.Probabilityhasfoundapplicationinareasasdiverseasmedicine,gambling,weatherforecasting,andthelaw.
Theconceptsofchanceanduncertaintyareasoldascivilizationitself.Peoplehave alwayshadtocopewithuncertaintyabouttheweather,theirfoodsupply,andother aspectsoftheirenvironment,andhavestriventoreducethisuncertaintyandits effects.Eventheideaofgamblinghasalonghistory.Byabouttheyear3500 b.c., gamesofchanceplayedwithboneobjectsthatcouldbeconsideredprecursorsof dicewereapparentlyhighlydevelopedinEgyptandelsewhere.Cubicaldicewith markingsvirtuallyidenticaltothoseonmoderndicehavebeenfoundinEgyptian tombsdatingfrom2000 b.c. Weknowthatgamblingwithdicehasbeenpopularever sincethattimeandplayedanimportantpartintheearlydevelopmentofprobability theory.
Itisgenerallybelievedthatthemathematicaltheoryofprobabilitywasstartedby theFrenchmathematiciansBlaisePascal(1623–1662)andPierreFermat(1601–1665) whentheysucceededinderivingexactprobabilitiesforcertaingamblingproblems involvingdice.Someoftheproblemsthattheysolvedhadbeenoutstandingforabout 300years.However,numericalprobabilitiesofvariousdicecombinationshadbeen calculatedpreviouslybyGirolamoCardano(1501–1576)andGalileoGalilei(1564–1642).
Thetheoryofprobabilityhasbeendevelopedsteadilysincetheseventeenth centuryandhasbeenwidelyappliedindiversefieldsofstudy.Today,probability theoryisanimportanttoolinmostareasofengineering,science,andmanagement. Manyresearchworkersareactivelyengagedinthediscoveryandestablishmentof newapplicationsofprobabilityinfieldssuchasmedicine,meteorology,photography fromsatellites,marketing,earthquakeprediction,humanbehavior,thedesignof computersystems,finance,genetics,andlaw.Inmanylegalproceedingsinvolving antitrustviolationsoremploymentdiscrimination,bothsideswillpresentprobability andstatisticalcalculationstohelpsupporttheircases.
References
TheancienthistoryofgamblingandtheoriginsofthemathematicaltheoryofprobabilityarediscussedbyDavid(1988),Ore(1960),Stigler(1986),andTodhunter (1865).
Someintroductorybooksonprobabilitytheory,whichdiscussmanyofthesame topicsthatwillbestudiedinthisbook,areFeller(1968);Hoel,Port,andStone(1971); Meyer(1970);andOlkin,Gleser,andDerman(1980).Otherintroductorybooks, whichdiscussbothprobabilitytheoryandstatisticsataboutthesamelevelasthey willbediscussedinthisbook,areBrunk(1975);Devore(1999);Fraser(1976);Hogg andTanis(1997);KempthorneandFolks(1971);LarsenandMarx(2001);Larson (1974);Lindgren(1976);MillerandMiller(1999);Mood,Graybill,andBoes(1974); Rice(1995);andWackerly,Mendenhall,andSchaeffer(2008).
1.2InterpretationsofProbability
Thissectiondescribesthreecommonoperationalinterpretationsofprobability. Althoughtheinterpretationsmayseemincompatible,itisfortunatethatthecalculusofprobability(thesubjectmatterofthefirstsixchaptersofthisbook)applies equallywellnomatterwhichinterpretationoneprefers.
Inadditiontothemanyformalapplicationsofprobabilitytheory,theconceptof probabilityentersoureverydaylifeandconversation.Weoftenhearandusesuch expressionsas“Itprobablywillraintomorrowafternoon,”“Itisverylikelythat theplanewillarrivelate,”or“Thechancesaregoodthathewillbeabletojoinus fordinnerthisevening.”Eachoftheseexpressionsisbasedontheconceptofthe probability,orthelikelihood,thatsomespecificeventwilloccur.
Despitethefactthattheconceptofprobabilityissuchacommonandnatural partofourexperience,nosinglescientificinterpretationoftheterm probability is acceptedbyallstatisticians,philosophers,andotherauthorities.Throughtheyears, eachinterpretationofprobabilitythathasbeenproposedbysomeauthoritieshas beencriticizedbyothers.Indeed,thetruemeaningofprobabilityisstillahighly controversialsubjectandisinvolvedinmanycurrentphilosophicaldiscussionspertainingtothefoundationsofstatistics.Threedifferentinterpretationsofprobability willbedescribedhere.Eachoftheseinterpretationscanbeveryusefulinapplying probabilitytheorytopracticalproblems.
TheFrequencyInterpretationofProbability
Inmanyproblems,theprobabilitythatsomespecificoutcomeofaprocesswillbe obtainedcanbeinterpretedtomeanthe relativefrequency withwhichthatoutcome wouldbeobtainediftheprocesswererepeatedalargenumberoftimesundersimilar conditions.Forexample,theprobabilityofobtainingaheadwhenacoinistossedis consideredtobe1/2becausetherelativefrequencyofheadsshouldbeapproximately 1/2whenthecoinistossedalargenumberoftimesundersimilarconditions.Inother words,itisassumedthattheproportionoftossesonwhichaheadisobtainedwould beapproximately1/2.
Ofcourse,theconditionsmentionedinthisexamplearetoovaguetoserveasthe basisforascientificdefinitionofprobability.First,a“largenumber”oftossesofthe coinisspecified,butthereisnodefiniteindicationofanactualnumberthatwould
beconsideredlargeenough.Second,itisstatedthatthecoinshouldbetossedeach time“undersimilarconditions,”buttheseconditionsarenotdescribedprecisely.The conditionsunderwhichthecoinistossedmustnotbecompletelyidenticalforeach tossbecausetheoutcomeswouldthenbethesame,andtherewouldbeeitherall headsoralltails.Infact,askilledpersoncantossacoinintotheairrepeatedlyand catchitinsuchawaythataheadisobtainedonalmosteverytoss.Hence,thetosses mustnotbecompletelycontrolledbutmusthavesome“random”features.
Furthermore,itisstatedthattherelativefrequencyofheadsshouldbe“approximately1/2,”butnolimitisspecifiedforthepermissiblevariationfrom1/2.Ifacoin weretossed1,000,000times,wewouldnotexpecttoobtainexactly500,000heads. Indeed,wewouldbeextremelysurprisedifweobtainedexactly500,000heads.On theotherhand,neitherwouldweexpectthenumberofheadstobeveryfarfrom 500,000.Itwouldbedesirabletobeabletomakeaprecisestatementofthelikelihoodsofthedifferentpossiblenumbersofheads,buttheselikelihoodswouldof necessitydependontheveryconceptofprobabilitythatwearetryingtodefine.
Anothershortcomingofthefrequencyinterpretationofprobabilityisthatit appliesonlytoaprobleminwhichtherecanbe,atleastinprinciple,alargenumberof similarrepetitionsofacertainprocess.Manyimportantproblemsarenotofthistype. Forexample,thefrequencyinterpretationofprobabilitycannotbeapplieddirectly totheprobabilitythataspecificacquaintancewillgetmarriedwithinthenexttwo yearsortotheprobabilitythataparticularmedicalresearchprojectwillleadtothe developmentofanewtreatmentforacertaindiseasewithinaspecifiedperiodoftime.
TheClassicalInterpretationofProbability
Theclassicalinterpretationofprobabilityisbasedontheconceptof equallylikely outcomes.Forexample,whenacoinistossed,therearetwopossibleoutcomes:a headoratail.Ifitmaybeassumedthattheseoutcomesareequallylikelytooccur, thentheymusthavethesameprobability.Sincethesumoftheprobabilitiesmust be1,boththeprobabilityofaheadandtheprobabilityofatailmustbe1/2.More generally,iftheoutcomeofsomeprocessmustbeoneof n differentoutcomes,and ifthese n outcomesareequallylikelytooccur,thentheprobabilityofeachoutcome is1/n.
Twobasicdifficultiesarisewhenanattemptismadetodevelopaformaldefinitionofprobabilityfromtheclassicalinterpretation.First,theconceptofequally likelyoutcomesisessentiallybasedontheconceptofprobabilitythatwearetrying todefine.Thestatementthattwopossibleoutcomesareequallylikelytooccuristhe sameasthestatementthattwooutcomeshavethesameprobability.Second,nosystematicmethodisgivenforassigningprobabilitiestooutcomesthatarenotassumed tobeequallylikely.Whenacoinistossed,orawell-balanceddieisrolled,oracardis chosenfromawell-shuffleddeckofcards,thedifferentpossibleoutcomescanusually beregardedasequallylikelybecauseofthenatureoftheprocess.However,whenthe problemistoguesswhetheranacquaintancewillgetmarriedorwhetheraresearch projectwillbesuccessful,thepossibleoutcomeswouldnottypicallybeconsidered tobeequallylikely,andadifferentmethodisneededforassigningprobabilitiesto theseoutcomes.
TheSubjectiveInterpretationofProbability
Accordingtothesubjective,orpersonal,interpretationofprobability,theprobability thatapersonassignstoapossibleoutcomeofsomeprocessrepresentsherown
judgmentofthelikelihoodthattheoutcomewillbeobtained.Thisjudgmentwillbe basedoneachperson’sbeliefsandinformationabouttheprocess.Anotherperson, whomayhavedifferentbeliefsordifferentinformation,mayassignadifferent probabilitytothesameoutcome.Forthisreason,itisappropriatetospeakofa certainperson’s subjectiveprobability ofanoutcome,ratherthantospeakofthe trueprobability ofthatoutcome.
Asanillustrationofthisinterpretation,supposethatacoinistobetossedonce. Apersonwithnospecialinformationaboutthecoinorthewayinwhichitistossed mightregardaheadandatailtobeequallylikelyoutcomes.Thatpersonwould thenassignasubjectiveprobabilityof1/2tothepossibilityofobtainingahead.The personwhoisactuallytossingthecoin,however,mightfeelthataheadismuch morelikelytobeobtainedthanatail.Inorderthatpeopleingeneralmaybeable toassignsubjectiveprobabilitiestotheoutcomes,theymustexpressthestrengthof theirbeliefinnumericalterms.Suppose,forexample,thattheyregardthelikelihood ofobtainingaheadtobethesameasthelikelihoodofobtainingaredcardwhenone cardischosenfromawell-shuffleddeckcontainingfourredcardsandoneblackcard. Becausethosepeoplewouldassignaprobabilityof4/5tothepossibilityofobtaining aredcard,theyshouldalsoassignaprobabilityof4/5tothepossibilityofobtaining aheadwhenthecoinistossed.
Thissubjectiveinterpretationofprobabilitycanbeformalized.Ingeneral,if people’sjudgmentsoftherelativelikelihoodsofvariouscombinationsofoutcomes satisfycertainconditionsofconsistency,thenitcanbeshownthattheirsubjective probabilitiesofthedifferentpossibleeventscanbeuniquelydetermined.However, therearetwodifficultieswiththesubjectiveinterpretation.First,therequirement thataperson’sjudgmentsoftherelativelikelihoodsofaninfinitenumberofevents becompletelyconsistentandfreefromcontradictionsdoesnotseemtobehumanly attainable,unlessapersonissimplywillingtoadoptacollectionofjudgmentsknown tobeconsistent.Second,thesubjectiveinterpretationprovidesno“objective”basis fortwoormorescientistsworkingtogethertoreachacommonevaluationofthe stateofknowledgeinsomescientificareaofcommoninterest.
Ontheotherhand,recognitionofthesubjectiveinterpretationofprobability hasthesalutaryeffectofemphasizingsomeofthesubjectiveaspectsofscience.A particularscientist’sevaluationoftheprobabilityofsomeuncertainoutcomemust ultimatelybethatperson’sownevaluationbasedonalltheevidenceavailable.This evaluationmaywellbebasedinpartonthefrequencyinterpretationofprobability, sincethescientistmaytakeintoaccounttherelativefrequencyofoccurrenceofthis outcomeorsimilaroutcomesinthepast.Itmayalsobebasedinpartontheclassical interpretationofprobability,sincethescientistmaytakeintoaccountthetotalnumberofpossibleoutcomesthatareconsideredequallylikelytooccur.Nevertheless, thefinalassignmentofnumericalprobabilitiesistheresponsibilityofthescientist herself.
Thesubjectivenatureofscienceisalsorevealedintheactualproblemthata particularscientistchoosestostudyfromtheclassofproblemsthatmighthave beenchosen,intheexperimentsthatareselectedincarryingoutthisstudy,and intheconclusionsdrawnfromtheexperimentaldata.Themathematicaltheoryof probabilityandstatisticscanplayanimportantpartinthesechoices,decisions,and conclusions.
Note:TheTheoryofProbabilityDoesNotDependonInterpretation. ThemathematicaltheoryofprobabilityisdevelopedandpresentedinChapters1–6ofthis bookwithoutregardtothecontroversysurroundingthedifferentinterpretationsof
thetermprobability.Thistheoryiscorrectandcanbeusefullyapplied,regardlessof whichinterpretationofprobabilityisusedinaparticularproblem.Thetheoriesand techniquesthatwillbepresentedinthisbookhaveservedasvaluableguidesand toolsinalmostallaspectsofthedesignandanalysisofeffectiveexperimentation.
1.3ExperimentsandEvents
Probabilitywillbethewaythatwequantifyhowlikelysomethingistooccur(in thesenseofoneoftheinterpretationsinSec.1.2).Inthissection,wegiveexamples ofthetypesofsituationsinwhichprobabilitywillbeused.
TypesofExperiments
Thetheoryofprobabilitypertainstothevariouspossibleoutcomesthatmightbe obtainedandthepossibleeventsthatmightoccurwhenanexperimentisperformed.
ExperimentandEvent.An experiment isanyprocess,realorhypothetical,inwhich thepossibleoutcomescanbeidentifiedaheadoftime.An event isawell-definedset ofpossibleoutcomesoftheexperiment.
Thebreadthofthisdefinitionallowsustocallalmostanyimaginableprocessan experimentwhetherornotitsoutcomewilleverbeknown.Theprobabilityofeach eventwillbeourwayofsayinghowlikelyitisthattheoutcomeoftheexperimentis intheevent.Noteverysetofpossibleoutcomeswillbecalledanevent.Weshallbe morespecificaboutwhichsubsetscountaseventsinSec.1.4.
Probabilitywillbemostusefulwhenappliedtoarealexperimentinwhichthe outcomeisnotknowninadvance,buttherearemanyhypotheticalexperimentsthat provideusefultoolsformodelingrealexperiments.Acommontypeofhypothetical experimentisrepeatingawell-definedtaskinfinitelyoftenundersimilarconditions. Someexamplesofexperimentsandspecificeventsaregivennext.Ineachexample, thewordsfollowing“theprobabilitythat”describetheeventofinterest.
1.Inanexperimentinwhichacoinistobetossed10times,theexperimentermight wanttodeterminetheprobabilitythatatleastfourheadswillbeobtained.
2.Inanexperimentinwhichasampleof1000transistorsistobeselectedfrom alargeshipmentofsimilaritemsandeachselecteditemistobeinspected,a personmightwanttodeterminetheprobabilitythatnotmorethanoneofthe selectedtransistorswillbedefective.
3.Inanexperimentinwhichtheairtemperatureatacertainlocationistobe observedeverydayatnoonfor90successivedays,apersonmightwantto determinetheprobabilitythattheaveragetemperatureduringthisperiodwill belessthansomespecifiedvalue.
4.FrominformationrelatingtothelifeofThomasJefferson,apersonmightwant todeterminetheprobabilitythatJeffersonwasbornintheyear1741.
5.Inevaluatinganindustrialresearchanddevelopmentprojectatacertaintime, apersonmightwanttodeterminetheprobabilitythattheprojectwillresult inthesuccessfuldevelopmentofanewproductwithinaspecifiednumberof months.
Definition 1.4.1
TheMathematicalTheoryofProbability
AswasexplainedinSec.1.2,thereiscontroversyinregardtothepropermeaning andinterpretationofsomeoftheprobabilitiesthatareassignedtotheoutcomes ofmanyexperiments.However,onceprobabilitieshavebeenassignedtosome simpleoutcomesinanexperiment,thereiscompleteagreementamongallauthorities thatthemathematicaltheoryofprobabilityprovidestheappropriatemethodology forthefurtherstudyoftheseprobabilities.Almostallworkinthemathematical theoryofprobability,fromthemostelementarytextbookstothemostadvanced research,hasbeenrelatedtothefollowingtwoproblems:(i)methodsfordetermining theprobabilitiesofcertaineventsfromthespecifiedprobabilitiesofeachpossible outcomeofanexperimentand(ii)methodsforrevisingtheprobabilitiesofevents whenadditionalrelevantinformationisobtained.
Thesemethodsarebasedonstandardmathematicaltechniques.Thepurposeof thefirstsixchaptersofthisbookistopresentthesetechniques,which,together,form themathematicaltheoryofprobability.
1.4SetTheory
Thissectiondevelopstheformalmathematicalmodelforevents,namely,thetheory ofsets.Severalimportantconceptsareintroduced,namely,element,subset,empty set,intersection,union,complement,anddisjointsets.
TheSampleSpace
SampleSpace.Thecollectionofallpossibleoutcomesofanexperimentiscalledthe samplespace oftheexperiment.
Thesamplespaceofanexperimentcanbethoughtofasa set ,orcollection,of differentpossibleoutcomes;andeachoutcomecanbethoughtofasa point ,oran element ,inthesamplespace.Similarly,eventscanbethoughtofas subsets ofthe samplespace.
Example 1.4.1
RollingaDie.Whenasix-sideddieisrolled,thesamplespacecanberegardedas containingthesixnumbers1, 2, 3, 4, 5, 6,eachrepresentingapossiblesideofthedie thatshowsaftertheroll.Symbolically,wewrite
S ={1, 2, 3, 4, 5, 6}
Oneevent A isthatanevennumberisobtained,anditcanberepresentedasthe subset A ={2, 4, 6}.Theevent B thatanumbergreaterthan2isobtainedisdefined bythesubset B ={3, 4, 5, 6}
Becausewecaninterpretoutcomesaselementsofasetandeventsassubsets ofaset,thelanguageandconceptsofsettheoryprovideanaturalcontextforthe developmentofprobabilitytheory.Thebasicideasandnotationofsettheorywill nowbereviewed.
Condition 1
RelationsofSetTheory
Let S denotethesamplespaceofsomeexperiment.Theneachpossibleoutcome s oftheexperimentissaidtobeamemberofthespace S ,ortobelongtothespace S . Thestatementthat s isamemberof S isdenotedsymbolicallybytherelation s ∈ S . Whenanexperimenthasbeenperformedandwesaythatsomeevent E has occurred,wemeantwoequivalentthings.Oneisthattheoutcomeoftheexperiment satisfiedtheconditionsthatspecifiedthatevent E .Theotheristhattheoutcome, consideredasapointinthesamplespace,isanelementof E . Tobeprecise,weshouldsaywhichsetsofoutcomescorrespondtoeventsasdefinedabove.Inmanyapplications,suchasExample1.4.1,itwillbeclearwhichsetsof outcomesshouldcorrespondtoevents.Inotherapplications(suchasExample1.4.5 cominguplater),therearetoomanysetsavailabletohavethemallbeevents.Ideally,wewouldliketohavethelargestpossiblecollectionofsetscalledeventssothat wehavethebroadestpossibleapplicabilityofourprobabilitycalculations.However, whenthesamplespaceistoolarge(asinExample1.4.5)thetheoryofprobability simplywillnotextendtothecollectionofallsubsetsofthesamplespace.Wewould prefernottodwellonthispointfortworeasons.First,acarefulhandlingrequires mathematicaldetailsthatinterferewithaninitialunderstandingoftheimportant concepts,andsecond,thepracticalimplicationsfortheresultsinthistextareminimal.Inordertobemathematicallycorrectwithoutimposinganundueburdenon thereader,wenotethefollowing.Inordertobeabletodoalloftheprobabilitycalculationsthatwemightfindinteresting,therearethreesimpleconditionsthatmust bemetbythecollectionofsetsthatwecallevents.Ineveryproblemthatweseein thistext,thereexistsacollectionofsetsthatincludesallthesetsthatwewillneedto discussandthatsatisfiesthethreeconditions,andthereadershouldassumethatsuch acollectionhasbeenchosenastheevents.Forasamplespace S withonlyfinitely manyoutcomes,thecollectionofallsubsetsof S satisfiestheconditions,asthereader canshowinExercise12inthissection.
Thefirstofthethreeconditionscanbestatedimmediately.
Thesamplespace S mustbeanevent.
Thatis,wemustincludethesamplespace S inourcollectionofevents.Theothertwo conditionswillappearlaterinthissectionbecausetheyrequireadditionaldefinitions. Condition2isonpage9,andCondition3isonpage10.
Definition 1.4.2
Theorem 1.4.1
Example 1.4.2
Containment.Itissaidthataset Aiscontainedin anotherset B ifeveryelement oftheset A alsobelongstotheset B .Thisrelationbetweentwoeventsisexpressed symbolicallybytheexpression A ⊂ B ,whichistheset-theoreticexpressionforsaying that A isasubsetof B .Equivalently,if A ⊂ B ,wemaysaythat B contains A andmay write B ⊃ A.
Forevents,tosaythat A ⊂ B meansthatif A occursthensodoes B . Theproofofthefollowingresultisstraightforwardandisomitted.
,and
,
and B ⊂ C ,then A ⊂ C
RollingaDie.InExample1.4.1,supposethat A istheeventthatanevennumber isobtainedand C istheeventthatanumbergreaterthan1isobtained.Since A ={2, 4, 6} and C ={2, 3, 4, 5, 6},itfollowsthat A ⊂ C
Definition 1.4.3
Theorem 1.4.2
TheEmptySet Someeventsareimpossible.Forexample,whenadieisrolled,it isimpossibletoobtainanegativenumber.Hence,theeventthatanegativenumber willbeobtainedisdefinedbythesubsetof S thatcontainsnooutcomes.
EmptySet.Thesubsetof S thatcontainsnoelementsiscalledthe emptyset ,or null set ,anditisdenotedbythesymbol ∅.
Intermsofevents,theemptysetisanyeventthatcannotoccur.
Let A beanevent.Then ∅⊂ A.
Proof Let A beanarbitraryevent.Sincetheemptyset ∅ containsnopoints,itis logicallycorrecttosaythateverypointbelongingto ∅ alsobelongsto A,or ∅⊂ A.
FiniteandInfiniteSets Somesetscontainonlyfinitelymanyelements,whileothers haveinfinitelymanyelements.Therearetwosizesofinfinitesetsthatweneedto distinguish.
Definition 1.4.4
Definition 1.4.5
Countable/Uncountable.Aninfiniteset A is countable ifthereisaone-to-onecorrespondencebetweentheelementsof A andthesetofnaturalnumbers {1, 2, 3,...}.A setis uncountable ifitisneitherfinitenorcountable.Ifwesaythatasethas atmost countablymany elements,wemeanthatthesetiseitherfiniteorcountable.
Examplesofcountablyinfinitesetsincludetheintegers,theevenintegers,theodd integers,theprimenumbers,andanyinfinitesequence.Eachofthesecanbeput inone-to-onecorrespondencewiththenaturalnumbers.Forexample,thefollowing function f putstheintegersinone-to-onecorrespondencewiththenaturalnumbers:
f(n) = n 1 2 if n isodd, n 2 if n iseven.
Everyinfinitesequenceofdistinctitemsisacountableset,asitsindexingputsitin one-to-onecorrespondencewiththenaturalnumbers.Examplesofuncountablesets includetherealnumbers,thepositivereals,thenumbersintheinterval[0, 1],andthe setofallorderedpairsofrealnumbers.Anargumenttoshowthattherealnumbers areuncountableappearsattheendofthissection.Everysubsetoftheintegershas atmostcountablymanyelements.
OperationsofSetTheory
Complement.The complement ofaset A isdefinedtobethesetthatcontainsall elementsofthesamplespace S that donot belongto A.Thenotationforthe complementof A is Ac
Intermsofevents,theevent Ac istheeventthat A doesnotoccur.
Example 1.4.3
RollingaDie.InExample1.4.1,supposeagainthat A istheeventthatanevennumber isrolled;then Ac ={1, 3, 5} istheeventthatanoddnumberisrolled.
Wecannowstatethesecondconditionthatwerequireofthecollectionofevents.
Theevent Ac
Condition 2 If A isanevent,then Ac isalsoanevent.
Thatis,foreachset A ofoutcomesthatwecallanevent,wemustalsocallits complement Ac anevent.
Agenericversionoftherelationshipbetween A and Ac issketchedinFig.1.1. Asketchofthistypeiscalleda Venndiagram Somepropertiesofthecomplementarestatedwithoutproofinthenextresult.
Theorem 1.4.3
Definition 1.4.6
Let A beanevent.Then (Ac )c = A, ∅c = S,S c =∅
Theemptyevent ∅ isanevent.
UnionofTwoSets.If A and B areanytwosets,the union of A and B isdefinedtobe thesetcontainingalloutcomesthatbelongto A alone,to B alone,ortoboth A and B .Thenotationfortheunionof A and B is A ∪ B
Theset A ∪ B issketchedinFig.1.2.Intermsofevents, A ∪ B istheeventthateither A or B orbothoccur.
Theunionhasthefollowingpropertieswhoseproofsarelefttothereader.
Theorem 1.4.4
Definition 1.4.7
Forallsets A and B , A ∪ B = B ∪ A,A ∪ A = A,A ∪ Ac = S, A ∪∅= A,A ∪ S = S. Furthermore,if A ⊂ B ,then A ∪ B = B
Theconceptofunionextendstomorethantwosets.
UnionofManySets.The union of n sets A1,...,An isdefinedtobethesetthat containsalloutcomesthatbelongtoatleastoneofthese n sets.Thenotationforthis unioniseitherofthefollowing:
Figure1.1
Similarly,the union ofaninfinitesequenceofsets A1,A2 ,... isthesetthatcontains alloutcomesthatbelongtoatleastoneoftheeventsinthesequence.Theinfinite unionisdenotedby ∞ i =1 Ai .
Intermsofevents,theunionofacollectionofeventsistheeventthatatleast oneoftheeventsinthecollectionoccurs.
Wecannowstatethefinalconditionthatwerequireforthecollectionofsets thatwecallevents.
Condition 3 If A1,A2 ,... isacountablecollectionofevents,then ∞ i =1 Ai isalsoanevent.
Inotherwords,ifwechoosetocalleachsetofoutcomesinsomecountablecollection anevent,wearerequiredtocalltheirunionaneventalso.Wedo not requirethat theunionofanarbitrarycollectionofeventsbeanevent.Tobeclear,let I bean arbitrarysetthatweusetoindexageneralcollectionofevents {Ai : i ∈ I }.Theunion oftheeventsinthiscollectionisthesetofoutcomesthatareinatleastoneofthe eventsinthecollection.Thenotationforthisunionis i ∈I Ai .Wedonotrequire that i ∈I Ai beaneventunless I iscountable.
Condition3referstoacountablecollectionofevents.Wecanprovethatthe conditionalsoappliestoeveryfinitecollectionofevents.
Theorem 1.4.5
Theorem 1.4.6
Definition 1.4.8
Theunionofafinitenumberofevents A1,...,An isanevent.
Proof Foreach m = n + 1,n + 2,...,define Am =∅.Because ∅ isanevent,wenow haveacountablecollection A1,A2 ,... ofevents.ItfollowsfromCondition3that ∞ m=1 Am isanevent.Butitiseasytoseethat ∞ m=1 Am = n m=1 Am
Theunionofthreeevents A, B ,and C canbeconstructedeitherdirectlyfromthe definitionof A ∪ B ∪ C orbyfirstevaluatingtheunionofanytwooftheeventsand thenformingtheunionofthiscombinationofeventsandthethirdevent.Inother words,thefollowingresultistrue.
AssociativeProperty.Foreverythreeevents A, B ,and C ,thefollowingassociative relationsaresatisfied:
∪ B ∪ C = (A ∪ B) ∪ C = A ∪ (B ∪ C).
IntersectionofTwoSets.If A and B areanytwosets,the intersection of A and B is definedtobethesetthatcontainsalloutcomesthatbelong bothtoAandtoB.The notationfortheintersectionof A and B is A ∩ B
Theset A ∩ B issketchedinaVenndiagraminFig.1.3.Intermsofevents, A ∩ B is theeventthatboth A and B occur.
TheproofofthefirstpartofthenextresultfollowsfromExercise3inthissection. Therestoftheproofisstraightforward.
Figure1.3 Theset A ∩ B
Theorem 1.4.7
Definition 1.4.9
If A and B areevents,thensois A ∩ B .Forallevents A and B , A ∩ B = B ∩ A,A ∩ A = A,A ∩ Ac =∅, A ∩∅=∅,A ∩ S = A.
Furthermore,if A ⊂ B ,then A ∩ B = A
Theconceptofintersectionextendstomorethantwosets.
IntersectionofManySets.The intersection of n sets A1,...,An isdefinedtobethe setthatcontainstheelementsthatarecommontoallthese n sets.Thenotationfor thisintersectionis A1 ∩ A2 ∩ ∩ An or n i =1 Ai .Similarnotationsareusedforthe intersectionofaninfinitesequenceofsetsorfortheintersectionofanarbitrary collectionofsets.
Intermsofevents,theintersectionofacollectionofeventsistheeventthatevery eventinthecollectionoccurs.
Thefollowingresultconcerningtheintersectionofthreeeventsisstraightforwardtoprove.
Theorem 1.4.8
Definition 1.4.10
Figure1.4 Partitionof S determinedbythree events A1, A2 , A3
AssociativeProperty.Foreverythreeevents A, B ,and C ,thefollowingassociative relationsaresatisfied:
Disjoint/MutuallyExclusive.Itissaidthattwosets A and B are disjoint ,or mutually exclusive,if A and B havenooutcomesincommon,thatis,if A ∩ B =∅.Thesets A1,...,An orthesets A1,A2 ,... aredisjointifforevery i = j ,wehavethat Ai and Aj aredisjoint,thatis, Ai ∩ Aj =∅ forall i = j .Theeventsinanarbitrarycollection aredisjointifnotwoeventsinthecollectionhaveanyoutcomesincommon.
Intermsofevents, A and B aredisjointiftheycannotbothoccur.
Asanillustrationoftheseconcepts,aVenndiagramforthreeevents A1, A2 ,and A3 ispresentedinFig.1.4.Thisdiagramindicatesthatthevariousintersectionsof A1, A2 ,and A3 andtheircomplementswillpartitionthesamplespace S intoeight disjointsubsets.
