An Introduction to Numerical Methods and Analysis
Third Edition
JamesF.Epperson
MathematicalReviews,AmericanMathematicalSociety
Thisthirdeditionfirstpublished2021 ©2021JohnWiley&Sons,Inc.
EditionHistory
JohnWileyandSons,Inc.(2e,2014)
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.
The right of James F. Epperson to be identified as the author of this work has been asserted in accordance with law.
RegisteredOffice
JohnWiley&Sons,Inc.,111RiverStreet,Hoboken,NJ07030,USA
EditorialOffice
111RiverStreet,Hoboken,NJ07030,USA
Fordetailsofourglobaleditorialoffices,customerservices,andmoreinformationaboutWiley productsvisitusatwww.wiley.com.
Wileyalsopublishesitsbooksinavarietyofelectronicformatsandbyprint-on-demand.Some contentthatappearsinstandardprintversionsofthisbookmaynotbeavailableinotherformats.
LimitofLiability/DisclaimerofWarranty
Whilethepublisherandauthorshaveusedtheirbesteffortsinpreparingthiswork,theymakeno representationsorwarrantieswithrespecttotheaccuracyorcompletenessofthecontentsofthis workandspecificallydisclaimallwarranties,includingwithoutlimitationanyimpliedwarrantiesof merchantabilityorfitnessforaparticularpurpose.Nowarrantymaybecreatedorextendedbysales representatives,writtensalesmaterialsorpromotionalstatementsforthiswork.Thefactthatan organization,website,orproductisreferredtointhisworkasacitationand/orpotentialsourceof furtherinformationdoesnotmeanthatthepublisherandauthorsendorsetheinformationorservices theorganization,website,orproductmayprovideorrecommendationsitmaymake.Thisworkis soldwiththeunderstandingthatthepublisherisnotengagedinrenderingprofessionalservices.The adviceandstrategiescontainedhereinmaynotbesuitableforyoursituation.Youshouldconsultwith aspecialistwhereappropriate.Further,readersshouldbeawarethatwebsiteslistedinthisworkmay havechangedordisappearedbetweenwhenthisworkwaswrittenandwhenitisread.Neitherthe publishernorauthorsshallbeliableforanylossofprofitoranyothercommercialdamages, includingbutnotlimitedtospecial,incidental,consequential,orotherdamages.
Library of Congress Cataloging-in-Publication Data Applied for ISBN: 9781119604532
Cover design by Wiley
Setin9/11ptNimbusRomNo9LbyStraive,Chennai,India 10987654321
3.4Application:DivisionUsingNewton’sMethod
3.10RootsofPolynomials(Part1)
3.11SpecialTopicsinRoot-findingMethods
4.4Application:Muller’sMethodandInverseQuadraticInterpolation119 4.5Application:MoreApproximationstotheDerivative
5.1AReviewoftheDefiniteIntegral
5.2ImprovingtheTrapezoidRule
5.3Simpson’sRuleandDegreeofPrecision
5.4TheMidpointRule
5.5Application:Stirling’sFormula
5.6GaussianQuadrature
6.1TheInitialValueProblem—Background
6.3AnalysisofEuler’sMethod
6.5SingleStepMethods—Runge-Kutta
6.7StabilityIssues
6.8ApplicationtoSystemsofEquations
7NumericalMethodsfortheSolutionofSystemsofEquations
7.1LinearAlgebraReview
7.2LinearSystemsandGaussianElimination
7.3OperationCounts
7.4The LU Factorization
7.5Perturbation,ConditioningandStability
7.6SPDMatricesandtheCholeskyDecomposition
7.7Application:NumericalSolutionofLinearLeastSquaresProblems236
7.8SparseandStructuredMatrices
7.9IterativeMethodsforLinearSystems–ABriefSurvey
7.10NonlinearSystems:Newton’sMethodandRelatedIdeas
7.11Application:NumericalSolutionofNonlinearBVP’s
8ApproximateSolutionoftheAlgebraicEigenvalueProblem
8.1EigenvalueReview
8.4BisectionandInertiatoComputeEigenvaluesofSymmetricMatrices253 8.5AnOverviewofthe QR Iteration
9ASurveyofNumericalMethods
10AnIntroductiontoSpectralMethods
PrefacetotheSolutionsManualfortheThirdEdition
Thismanualiswrittenforinstructors,notstudents.Itincludesworkedsolutionsformany (roughly75%)oftheproblemsinthetext.ForthecomputationalexercisesIhavegiventhe outputgeneratedbymyprogram,orsometimesaprogramlisting.Mostoftheprogramming wasdoneinMATLAB,someinFORTRAN.(TheauthoriswellawarethatFORTRAN isarchaic,butthereisalotof“legacycode"inFORTRAN,andtheauthorbelievesthere isvalueinlearninganewlanguage,evenanarchaicone.)Whenthetexthasaseriesof exercisesthatareobviouslysimilarandhavesimilarsolutions,thensometimesonlyoneof theseproblemshasaworkedsolutionincluded.Whencomputationalresultsareaskedfor aseriesofsimilarfunctionsorproblems,onlyasubsetofsolutionsarereported,largely forthesakeofbrevity.Someexercisesthatsimplyaskthestudenttoperformastraightforwardcomputationareskipped.Exercisesthatrepeatthesamecomputationbutwitha differentmethodarealsooftenskipped,asareexercisesthataskthestudentto“verify”a straight-forwardcomputation.
Someoftheexercisesweredesignedtobeopen-endedandalmost“essay-like.”For theseexercises,theonlysolutiontypicallyprovidedisashorthintorbriefoutlineofthe kindofdiscussionanticipatedbytheauthor.
Inmanyexercisesthestudentneedstoconstructanupperboundonaderivativeofsome functioninordertodeterminehowsmallaparameterhastobetoachieveadesiredlevelof accuracy.Formanyofthesolutionsthiswasdoneusingacomputeralgebrapackageand thedetailsarenotgiven.
Studentswhoacquireacopyofthismanualinordertoobtainworkedsolutionsto homeworkproblemsshouldbeawarethatnoneofthesolutionsaregiveninenoughdetail toearnfullcreditfromaninstructor.
Theauthorfreelyadmitsthepotentialforerrorinanyofthesesolutions,especiallysince manyoftheexercisesweremodifiedafterthefinalversionofthetextwassubmittedtothe publisherandbecausetheorderingoftheexerciseswaschangedbetweeneditions.While wetriedtomakealltheappropriatecorrections,thepossibilityoferrorisstillpresent,and undoubtedlytheauthor’sresponsibility.
Becausemuchofthemanualwasconstructedbydoing“copy-and-paste”fromthefiles forthetext,theenumerationofmanytablesandfigureswillbedifferent.Ihavetriedto notewhatthenumberisinthetext,butcertainlymayhavemissedsomeinstances.
Suggestionsfornewexercisesandcorrectionstothesesolutionsareverywelcome. Contacttheauthorat jfe@ams.org or jfepperson@gmail.com
Differences from the text
The text itself went through a copy-editing process after this manual was completed. As was to be expected, the wording of several problems was slightly changed. None of these changes should affect the problem in terms of what is expected of students; the vast majority of the changes were to replace “previous problem” (a bad habit of mine) with “Problem X.Y” (which I should have done on my own, in the first place). Some punctuation was also changed. The point of adding this note is to explain the textual differences which might be noticed between the text and this manual. If something needs clarification, please contact me at the above email.
CHAPTER1
INTRODUCTORYCONCEPTSAND CALCULUSREVIEW
1.1BASICTOOLSOFCALCULUS
Exercises:
1.Showthatthethird-orderTaylorpolynomialfor
2.Whatisthethird-orderTaylorpolynomialfor f(x)= √x +1,about x0 =0?
,is
Solution: Wehave f(x0)=1 and f ′(x)= 1 2(x +1)1/2 ,f ′′(x)= 1 4(x +1)3/2 ,f ′′′(x)= 3 8(x +1)5/2 ,
sothat f ′(0)=1/2, f ′′(0)= 1/4, f ′′′ =3/8.Therefore
′
3(x)= f
1 2 x 2
′′(0)+ 1 6 x 3f ′′′(x) =1+ x(1/2)+ 1 2 x 2( 1/4)+ 1 6 x 3(3/8) =1 (1/2)x (1/8)x 2 +(1/16)x 3 .
3.Whatisthesixth-orderTaylorpolynomialfor f(x)= √1+ x2,using x0 =0?Hint: Considerthepreviousproblem.
4.Giventhat R(x)= |x|6 6! eξ for x ∈ [ 1, 1],where ξ isbetween x and 0,findanupperboundfor |R|,validfor all x ∈ [ 1, 1],thatisindependentof x and ξ.
5.Repeattheabove,butthistimerequirethattheupperboundbevalidonlyforall x ∈ [ 1 2 , 1 2 ]
Solution: Theonlysignificantdifferenceistheintroductionofafactorof 26 inthe denominator: |R(x)|≤ √e 26 × 720 =3 6 × 10 5
6.Giventhat
for x ∈ [ 1 2 , 1 2 ],where ξ isbetween x and 0,findanupperboundfor |R|,validfor all x ∈ [ 1 2 , 1 2 ],thatisindependentof x and ξ.
7.UseaTaylorpolynomialtofindanapproximatevaluefor √e thatisaccurateto within 10 3
Solution: Therearetwowaystodothis.Wecanapproximate f(x)= ex anduse x =1/2,orwecanapproximate g(x)= √x anduse x = e.Inaddition,wecanbe conventionalandtake x0 =0,orwecantake x0 =0 inordertospeedconvergence. Themoststraightforwardapproach(inmyopinion)istouseaTaylorpolynomialfor ex about x0 =0.Theremainderafter k termsis
Wequicklyhavethat
andalittleplayingwithacalculatorshowsthat
but
Sowewoulduse
Tofourteendigits, √e =1 64872127070013,andtheerroris 2 84 × 10 4,much smallerthanrequired.
8.Whatisthefourth-orderTaylorpolynomialfor f(x)=1/(x +1),about x0 =0?
Solution: Wehave f(0)=1 and
, sothat f ′(0)= 1, f ′′(0)=2, f ′′′ = 6,f ′′′′(0)=24.Thus,
9.Whatisthefourth-orderTaylorpolynomialfor f(x)=1/x,about x0 =1?
10.FindtheTaylorpolynomialofthird-orderfor sin x,using:
(a) x0 = π/6
Solution: Wehave f(x0)= 1 2 ,f ′(x
(b) x0 = π/4; (c) x0 = π/2
11.Foreachfunctionbelowconstructthethird-orderTaylorpolynomialapproximation, using x0 =0,andthenestimatetheerrorbycomputinganupperboundonthe remainder,overthegiveninterval.
(a) f(x)= e x , x ∈ [0, 1];
(b) f(x)=ln(1+ x), x ∈ [ 1, 1];
(c) f(x)=sin x, x ∈ [0,π];
(d) f(x)=ln(1+ x), x ∈ [ 1/2, 1/2];
(e) f(x)=1/(x +1), x ∈ [ 1/2, 1/2].
Solution:
(a)Thepolynomialis p3(x)=1 x + 1 2 x 2 1 6 x 3 , withremainder
3(x)= 1 24 x 4 e ξ
Thiscanbeboundedabove,forall x ∈ [0, 1],by
(b)Thepolynomialis
3(x)= x 1 2 x 2 + 1 3 x 3 , withremainder
3(x)= 1 4 x 4 1 (1+ ξ)4
We can’t boundthisforall x ∈ [ 1, 1],becauseofthepotentialdivisionby zero.
(c)Thepolynomialis
3(x)= x 1 6 x 3 , withremainder
Thiscanbeboundedabove,forall x ∈ [0,π],by
(d)Thepolynomialisthesameasin(b),ofcourse, p3(x)= x 1 2 x 2 + 1 3 x 3 , withremainder R3(x)= 1 4 x 4 1 (1+ ξ)4
Forall x ∈ [ 1/2, 1/2] thiscanbeboundedby
≤
(e)Thepolynomialis p3(x)=1 x + x 2 x 3 , withremainder
R3(x)= x 4 1 (1+ ξ)5 .
Thiscanbeboundedabove,forall x ∈ [ 1/2, 1/2],by |R3(x)|≤ (1/2)4 1 (1 1/2)5 =2
Obviously,thisisnotanespeciallygoodapproximation.
12.ConstructaTaylorpolynomialapproximationthatisaccuratetowithin 10 3,over theindicatedinterval,foreachofthefollowingfunctions,using x0 =0
(a) f(x)=sin x, x ∈ [0,π];
(b) f(x)= e x , x ∈ [0, 1];
(c) f(x)=ln(1+ x), x ∈ [ 1/2, 1/2];
(d) f(x)=1/(x +1), x ∈ [ 1/2, 1/2];
(e) f(x)=ln(1+ x), x ∈ [ 1, 1]
Solution:
(a)Theremainderhereis
Rn(x)= ( 1)n+1 (2n +1)! x 2n+1 cos c, for c ∈ [0,π].Therefore,wehave |Rn
Simplemanipulationswithacalculatorthenshowthat
x∈[0,π] |R6(x)|≤ 0 4663028067 × 10 3 but max x∈[0,π] |R5(x)|≤ 0 7370430958 × 10 2
ThereforethedesiredTaylorpolynomialis
(b)Theremainderhereis
n(x)= ( 1)n+1 (n +1)! xn+1e c , for c ∈ [0, 1].Therefore,wehave |Rn(x)|≤ 1 (n +1)! |x|n+1 ≤ 1 (n +1)! .
Simplemanipulationswithacalculatorthenshowthat
x∈[0,1] |R6(x)|≤ 0 0001984126984
but
x∈[0,1] |R5(x)|≤ 0 1388888889 × 10 2
ThereforethedesiredTaylorpolynomialis
(c) f(x)=ln(1+ x), x ∈ [0, 3/4]
(d) Solution: Theremainderisnow |Rn(x)|≤ (1/2)n+1 (n +1) , and n =8 makestheerrorsmallenough.
(e) f(x)=ln(1+ x), x ∈ [0, 1/2].
13.Repeattheabove,thistimewithadesiredaccuracyof 10 6
14.Since π 4 =arctan1,
wecanestimate π byestimating arctan1.HowmanytermsareneededintheGregory seriesforthearctangenttoapproximate π to100decimalplaces?1,000?Hint:Use theerrortermintheGregoryseriestopredictwhentheerrorgetssufficientlysmall.
Solution: TheremainderintheGregoryseriesapproximationis
sotoget100decimalplacesofaccuracyfor x =1,werequire |
thus,wehavetotake n ≥ (10100 3)/2 terms.For1,000placesofaccuracywe thereforeneed n ≥ (101000 3)/2 terms.
Obviously,thisisnotthebestprocedureforcomputingmanydigitsof π!
15.Elementarytrigonometrycanbeusedtoshowthat arctan(1/239)=4arctan(1/5) arctan(1)
Thisformulawasdevelopedin1706bytheEnglishastronomerJohnMachin.Use thistodevelopamoreefficientalgorithmforcomputing π.Howmanytermsare neededtoget100digitsofaccuracywiththisform?Howmanytermsareneeded toget1,000digits?Historicalnote:Until1961,thiswasthebasisforthemost commonlyusedmethodforcomputing π tohighaccuracy.
Solution: WenowhavetwoGregoryseries,thuscomplicatingtheproblemabit. Wehave π =4arctan(1)=16arctan(1/5) 4arctan(1/239)
Define pm,n ≈ π astheapproximationgeneratedbyusingan m termGregoryseries toapproximate arctan(1/5) andan n termGregoryseriesfor arctan(1/239).Then wehave
, where Rk istheremainderintheGregoryseries.Therefore,
Tofinishtheproblemwehavetoapportiontheerrorbetweenthetwoseries,which introducessomearbitrarinessintotheproblem.Ifwerequirethattheybeequally accurate,thenwehavethat
Usingpropertiesoflogarithms,thesebecome log(2m +3)+(2m +3)log5 ≥ log16 log ϵ and log(2n +3)+(2n +3)log239 ≥ log4 log ϵ.
For ϵ =(1/2) × 10 100,thesearesatisfiedfor m =70, n =20.For ϵ = (1/2) × 10 1000,weget m =712, n =209.Changingtheapportionmentofthe errordoesn’tchangetheresultsbymuchatall.
16.In1896,avariationonMachin’sformulawasfound:
arctan(1/239)=arctan(1) 6arctan(1/8) 2arctan(1/57), andthisbegantobeusedin1961tocompute π tohighaccuracy.Howmanyterms areneededwhenusingthisexpansiontoget100digitsof π?1,000digits?
Solution: Wenowhavethreeseriestoworkwith,whichcomplicatesmattersonly slightlymorecomparedtothepreviousproblem.Ifwedefine pk,m,n ≈ π basedon π =4arctan(1)=24arctan(1/8)+8arctan(1/57)+4arctan(1/239), taking k termsintheseriesfor arctan(1/8), m termsintheseriesfor arctan(1/57), and n termsintheseriesfor arctan(1/239),thenweareledtotheinequalities
log(2k +3)+(2k +3)log8 ≥ log24 log ϵ,
log(2m +3)+(2m +3)log57 ≥ log8 log ϵ, and
log(2n +3)+(2n +3)log239 ≥ log4 log ϵ.
For ϵ =(1/3) × 10 100,weget k =54, m =27,and n =19;for ϵ =(1/3) × 10 1000 weget k =552, m =283,and n =209
Note:Inbothoftheseproblemsaslightlymoreinvolvedtreatmentoftheerrormight leadtofewertermsbeingrequired.
17.WhatistheTaylorpolynomialoforder3for f(x)= x4 +1,using x0 =0?
Solution: Thisisverydirect: f ′(x)=4x 3,f ′′(x)=12x 2,f ′′′(x)=24x,
sothat p3(x)=1+ x(0)+ 1 2 x 2(0)+ 1 6 x 3(0)=1
18.WhatistheTaylorpolynomialoforder4for f(x)= x4 +1,using x0 =0?Simplify asmuchaspossible.
19.WhatistheTaylorpolynomialoforder2for f(x)= x3 + x,using x0 =1?
20.WhatistheTaylorpolynomialoforder3for f(x)= x3 + x,using x0 =1?Simplify asmuchaspossible.
Solution: Wenotethat f ′′′(1)=6,sowehave(usingthesolutionfromtheprevious problem) p4(x)=3x 2 2x +1+ 1 6 (x 1)3(6)= x 3 + x.
ThepolynomialisitsownTaylorpolynomial.
21.Let p(x) beanarbitrarypolynomialofdegreelessthanorequalto n.Whatisits Taylorpolynomialofdegree n,aboutanarbitrary x0?
22.TheFresnelintegralsaredefinedas C(x)= x 0 cos(πt2/2)dt, and S(x)= x 0 sin(πt2/2)dt.
UseTaylorexpansionstofindapproximationsto C(x) and S(x) thatare 10 4 accurateforall x with |x|≤ 1 2 .Hint:Substitute x = πt2/2 intotheTaylor expansionsforthecosineandsine.
Solution: Wewillshowtheworkforthecaseof S(x),only.Wehave S(x)= x 0 sin(πt2/2)dt = x 0 pn(t2)dt + x 0 Rn(t2)dt.
Lookingmorecarefullyattheremainderterm,weseethatitisgivenby rn(x)= ± x 0 t2(2n+3) (2n +3)! cos ξdt.
Therefore, |rn(x)|≤ 1/2 0 t2(2n+3) (2n +3)! dt = (1/2)4n+7 (4n +7)(2n +3)!
Alittleeffortwithacalculatorshowsthatthisislessthan 10 4 for n ≥ 1;therefore thepolynomialis
23.UsetheIntegralMeanValueTheoremtoshowthatthe“pointwise”form(1.3)of theTaylorremainder(usuallycalledthe Lagrange form)followsfromthe“integral” form(1.2)(usuallycalledthe Cauchy form).
24.ForeachfunctioninProblem11,usetheMeanValueTheoremtofindavalue M suchthat
isvalidforall x1, x2 intheintervalusedinProblem11.
Solution: Thisamountstofindinganupperboundon |f ′| overtheintervalgiven. Theanswersareasgivenbelow.
(a) f(x)= e x , x ∈ [0, 1]; M ≤ 1
(b) f(x)=ln(1+ x), x ∈ [ 1, 1]; M isunbounded,since f ′(x)=1/(1+ x) and x = 1 ispossible.
(c) f(x)=sin x, x ∈ [0,π]; M ≤ 1
(d) f(x)=ln(1+ x), x ∈ [ 1/2, 1/2]; M ≤ 2.
(e) f(x)=1/(x +1), x ∈ [ 1/2, 1/2]. M ≤ 4.
25.Afunctioniscalled monotone onanintervalifitsderivativeisstrictlypositiveor strictlynegativeontheinterval.Suppose f iscontinuousandmonotoneonthe interval [a,b],and f(a)f(b) < 0;provethatthereisexactlyonevalue α ∈ [a,b] suchthat f(α) =0.
Solution: Since f iscontinuousontheinterval [a,b] and f(a)f(b) < 0,theIntermediateValueTheoremguaranteesthatthereisapoint c where f(c)=0,i.e., thereisatleastoneroot.Supposenowthatthereexistsasecondroot, γ.Then f(c)= f(γ)=0.BytheMeanValueTheorem,then,thereisapoint ξ between c and γ suchthat
f ′(ξ)= f(γ) f(c) γ c =0.
Butthisviolatesthehypothesisthat f ismonotone,sinceamonotonefunction musthaveaderivativethatisstrictlypositiveorstrictlynegative.Thuswehavea contradiction,thustherecannotexistthesecondroot. Averyacceptableargumentcanbemadebyappealingtoagraphofthefunction.
26.FinishtheproofoftheIntegralMeanValueTheorem(Theorem1.5)bywritingup theargumentinthecasethat g isnegative.
INTRODUCTORYCONCEPTSANDCALCULUSREVIEW
Solution: Allthatisrequiredistoobservethatif g isnegative,thenwehave
and
Theproofiscompletedasinthetext.
27.ProveTheorem1.6,providingalldetails.
28.Let ck > 0,begiven, 1 ≤ k ≤ n,andlet xk ∈ [a,b], 1 ≤ k ≤ n.Then,usethe DiscreteAverageValueTheoremtoprovethat,foranyfunction f ∈ C([a,b]),
forsome ξ ∈ [a,b]
Solution: Wecan’tapplytheDiscreteAverageValueTheoremtotheproblemasit isposedoriginally,sowehavetomanipulateabit.Define
then
andnowwecanapplytheDiscreteAverageValueTheoremtofinishtheproblem.
29.Discuss,inyourownwords,whetherornotthefollowingstatementistrue:“The Taylorpolynomialofdegree n isthebestpolynomialapproximationofdegree n to thegivenfunctionnearthepoint x0.”
NOTATION
Exercises:
1.UseTaylor’sTheoremtoshowthat ex =1+ x + O(x2) for x sufficientlysmall.
2.UseTaylor’sTheoremtoshowthat 1 cos x x = 1 2 x + O(x3) for x sufficientlysmall.
Solution: WecanexpandthecosineinaTaylorseriesas
Ifwesubstitutethisinto (1 cos x)/x andsimplify,weget 1
sothatwehave
/24.Therefore,
3.UseTaylor’sTheoremtoshowthat
) for x sufficientlysmall.
Solution: Wehave,fromTaylor’sTheorem,with x0 =0,
forsome ξ between 0 and x.Since
forall x sufficientlysmall,theresultfollows.Forexample,wehave
| forall x ∈ [ 1/2, 1/2].
4.UseTaylor’sTheoremtoshowthat
for x sufficientlysmall.
Solution: Thistime,Taylor’sTheoremgivesusthat (1+ x) 1 =1
x 3/(1+
)4 forsome ξ between 0 and x.Thus,forall x suchthat |x|≤ m, (1+ x) 1 (1 x + x 2) = x 3/(1+ ξ)4 ≤|x|3/(1 m)4 = C|x|3 , where C =1/(1 m)4 .
5.Showthat
6.Recallthesummationformula
Usethistoprovethat n k=0 r k = 1 1 r + O(rn+1).
Hint:Whatisthe definition ofthe O notation?
7.Usetheaboveresulttoshowthat10terms(k =9)areallthatisneededtocompute
towithin 10 4 absoluteaccuracy.
Solution: Theremainderinthe9termpartialsumis
8.Recallthesummationformula
Usethistoshowthat n k=1 k = 1 2 n 2 + O(n).
9.StateandprovetheversionofTheorem1.7whichdealswithrelationshipsofthe form x = xn + O(β(n)).
Solution: Thetheoremstatementmightbesomethinglikethefollowing:
Theorem: Let x = xn + O(β(n)) and y = yn + O(γ(n)),with bβ(n) >γ(n) for all n sufficientlylarge.Then x + y = xn + yn + O(β(n)+ γ(n)), x + y = xn + yn + O(β(n)), Ax = Axn + O(β(n))
Inthelastequation, A isanarbitraryconstant,independentof n Theproofparallelstheoneinthetextalmostperfectly,andsoisomitted.
10.Usethedefinitionof O toshowthatif y = yh + O(hp),then hy = hyh + O(hp+1)
11.Showthatif an = O(np) and bn = O(nq),then anbn = O(np+q)
Solution: Wehave
and |bn|≤ Cb|nq|
Thesefollowfromthedefinitionofthe O notation.Therefore,
whichimpliesthat anbn = O(np+q).
12.Supposethat y = yh + O(β(h)) and z = zh + O(β(h)),for h sufficientlysmall. Doesitfollowthat y z = yh zh (for h sufficientlysmall)?
13.Showthat
forall h sufficientlysmall.Hint:Expand f(x ± h) outtothefourthorderterms.
Solution: Thisisastraight-forwardmanipulationwiththeTaylorexpansions
and
Addthetwoexpansionstoget
Nowsolvefor f ′′(x).
14.Explain,inyourownwords,whyitisnecessarythattheconstant C in(1.8)be independentof h
Exercises:
1.Ineachproblembelow, A istheexactvalue,and Ah isanapproximationto A.Find theabsoluteerrorandtherelativeerror.
(a) A = π, Ah =22/7;
(b) A = e, Ah =2.71828;
(c) A = 1 6 , Ah =0.1667;
(d) A = 1 6 , Ah =0 1666
Solution:
(a)Abs.error ≤ 1 265 × 10 3,rel.error ≤ 4 025 × 10 4;
(b)Abs.error ≤ 1.828 × 10 6,rel.error ≤ 6.72 × 10 7;
(c)Abs.error ≤ 3.334 × 10 5,rel.error ≤ 2.000 × 10 4;
(d)Abs.error ≤ 6 667 × 10 5,rel.error ≤ 4 × 10 4
2.Performtheindicatedcomputationsineachofthreeways:(i)Exactly;(ii)Using three-digitdecimalarithmetic,withchopping;(iii)Usingthree-digitdecimalarithmetic,withrounding.Forbothapproximations,computetheabsoluteerrorandthe relativeerror.
(a) 1 6 + 1 10 ;
(b) 1 6 × 1 10 ;
(c) 1 9 + 1 7 + 1 6 ;
(d) 1 7 + 1 6 + 1 9 .
3.Foreachfunctionbelowexplainwhyanaiveconstructionwillbesusceptibleto significantroundingerror(for x nearcertainvalues),andexplainhowtoavoidthis error.
(a) f(x)=(√x +9 3)x 1;
(b) f(x)= x 1(1 cos x);
(c) f(x)=(1 x) 1(ln x sin πx);
(d) f(x)=(cos(π + x) cos π)x 1;
(e) f(x)=(e1+x e1 x)(2x) 1 .
Solution: Ineachcase,thefunctionissusceptibletosubtractivecancellationwhich willbeamplifiedbydivisionbyasmallnumber.Thewaytoavoidtheproblemisto useaTaylorexpansiontomakethesubtractionanddivisionbothexplicitoperations. Forinstance,in(a),wewouldwrite
f(x)=((3+(1/6)x (1/216)x 2 +O(x 3)) 3)x 1 =(1/6) (1/216)x+O(x 2).
Togetgreateraccuracy,takemoretermsintheTaylorexpansion.
4.For f(x)=(ex 1)/x,howmanytermsinaTaylorexpansionareneededtoget singleprecisionaccuracy(7decimaldigits)forall x ∈ [0, 1 2 ]?Howmanytermsare neededfordoubleprecisionaccuracy(14decimaldigits)overthissamerange?
5.Usingsingleprecisionarithmetic,only,carryouteachofthefollowingcomputations, usingfirsttheformontheleftsideoftheequalssign,thenusingtheformontheright side,andcomparethetworesults.Commentonwhatyougetinlightofthematerial in 1.3. (a) (x + ϵ)3 1= x3 +3x2ϵ +3xϵ2 + ϵ3 1, x =1 0, ϵ =0 000001 (b) b + √b2 2c =2c( b √b2 2c) 1 , b =1, 000, c = π.
Solution: “Singleprecision”means6or7decimaldigits,sothepointoftheproblem istodothecomputationsusing6or7digits.
(a)UsingMATLAB’s single commandontheauthor’slaptop(runningMATLAB R2019b),weget (x + ϵ)3 1=3 000002999797857 × 10 6 but x 3 +3x 2 ϵ +3xϵ 2 + ϵ 3 1=3 000003000019902 × 10 6 §
(b)Usingthesamesoftwareandhardware,weget
but
003141597588407
Whatisinterestingishowmodernhardwareandsoftwarehavedramatically improvedtheresultshere.Earliereditions,whichrelieduponresultsusing FORTRANorConalate1990sSunworkstation,showedmuchmoreofa difference.
6.Considerthesum
where m =2×105.Againusingonlysingleprecision,computethistwoways:First, bysummingintheorderindicatedintheformula;second,bysumming backwards, i.e.,startingwiththe k =200, 000 termandendingwiththe k =0 term.Compare yourresultsandcommentuponthem.
7.(a)Usingthecomputerofyourchoice,findthreevalues a, b,and c,suchthat (a + b)+ c = a +(b + c)
(b)Repeatforyourfavoritecalculatorapp.
(c)Dothisforsingleprecisioninyourpreferredcomputingenvironment.
Solution: (a)Thekeyissueistogetanapproximationtothemachineepsilon, thentake a =1, b = c =(2/3)u orsomethingsimilar.Thiswillguaranteethat (a + b)+ c = a but a +(b + c) >a.Thereisanadditionalissue,inthatMATLAB alwaysroundsunformattedoutput,sotoseethatyougotadifferentresultyouhave touse(ugh!) fprintf toprintoutenoughdigits.Onmylaptop,Iwasabletouse
a =1
b =1 101642356786233 × 10 16
c =1.101642356786233 × 10 16
andthen fprintf toldmethat
D =(a + b)+ c =1.00000000000000000, E = a +(b + c)=1.00000000000000022.
Itisaninterestingaspectofthehistoryofthisbook,thatwhenthisexercisewas firstwritten(“alongtimeago,inacomputationalenvironmentfar,far,away”), actualphysicalcalculatorswerestillcommonplace(asopposedtosmartphone/tablet apps).OnanelderlySharpcalculator,circa1997,theauthorfoundthat a =1, b =4 × 10 10,and c =4 × 10 10 worked.Usingascientificcalculatorapponhis phone,theauthorfoundthat a =1, b =4 × 10 16,and c =4 × 10 16 worked.
(However,hewasnotabletogetthistoworkontheWindows10calculatorapp.It wouldmakeaninterestingquasi-researchquestiontoexplainwhy.)
(b)UsingMATLAB’s single command(carefully),Iused a =1 b =3.9572964 × 10 8 c = b.
Then,
8.Assumeweareusing3-digitdecimalarithmetic.For ϵ =0.0001, a1 =5,compute
1 for a0 equaltoeachof 1, 2,and 3.Comment.
9.Let ϵ ≤ u.Explain,inyourownwords,whythecomputation
1 ispotentiallyrifewithroundingerror.(Assumethat a0 and a1 areofcomparable size.)Hint:Seepreviousproblem.
Solution: Thisisjustageneralizationofthepreviousproblem.If ϵ issmallenough, then a2 willbeindependentof a0
10.Usingthecomputerandlanguageofyourchoice,writeaprogramtoestimatethe machineepsilon.
Solution: Therearelotsofwaystodothis.Thebasicideaistoaddasmallnumber to1,andthenchecktoseeiftheresultisdifferentfromone,otherwisecontinueon. Onepossiblesolutionisthefollowing:
Algorithm1.1
Computationofthemachineepsilon. x=1.e-10; fork=1:6000 y=1+x; ify<=1 disp(‘macheps=’) disp(x) break end x=x*.99; end x
Thisproduces(ontheauthor’slaptop) u =1.101642356786233 × 10 16.Ifwe changetheinitial x to0.5,anddecrementbyafactorof2eachstep,weget u = 1.110223024625157 × 10 16,which,beinglarger,isabetterestimate.(Why?)
11.Wecancompute e x usingTaylorpolynomialsintwoways,eitherusing
orusing
Discuss,inyourownwords,whichapproachismoreaccurate.Inparticular,which oneismore(orless)susceptibletoroundingerror?
Solution: Becauseofthealternatingsignsinthefirstapproach,thereissomeconcern aboutsubtractivecancellationwhenitisused.
12.Whatisthemachineepsilonforacomputerthatusesbinaryarithmetic,24bitsfor thefraction,androunds?Whatifitchops?
Solution: Recallthatthemachineepsilonisthe largest number x suchthatthe computerreturns 1+ x = x.Wethereforeneedtofindthelargestnumber x thatcan berepresentedwith24binarydigitssuchthat 1+ x,whenroundedto24bits,isstill equalto 1.Thisisperhapsbestdonebyexplicitlywritingouttheadditioninbinary notation.Wehave
1+ x =1 000000000000000000000002
+0.00000000000000000000000 dddddddddddddddddddddddd2.
Ifthemachinechops,thenwecansetallofthe d valuesto 1 andthecomputerwill stillreturn 1+ x =1;ifthemachinerounds,thenweneedtomakethefirstdigita zero.Thus,thedesiredvaluesare
13.Whatisthemachineepsilonforacomputerthatuses octal (base8)arithmetic, assumingitretains8octoldigitsinthefraction?
1.5ABRIEFHISTORYOFSCIENTIFICCOMPUTING
(Noexercisesinthissection.)
