Solution Manual For Geophysical Data Analysis and Inverse Theory with MATLAB® and Python 5th Edition

Solutions to Problems Geophysical Data Analysis and Inverse Theory with MATLAB® and Python Fifth Edition 2.1 Suppose that you determine the masses of 100 objects by weighing the first, then weighing the first and second together, and then weighing the rest in triplets: the first, second, and third; the second, third, and fourth; and so forth. (A) Identify the data and model parameters in this problem. How many of each are there? (B) Write down the matrix, 𝐆, in the equation, 𝐝 = 𝐆𝐦, that relates the data to the model parameters. (C) How sparse is 𝐆? What percent of it is zero (A) The 100 masses of the objects are the model parameters. The 100 weights, as determined by a scale, are the data. (B) The data kernel is 100×100 1 0 0 0 0 0 ⋯ 0 1 1 0 0 0 0 ⋯ 0 1 1 1 0 0 0 ⋯ 0 𝐆= 0 1 1 1 0 0 ⋯ 0 0 0 1 1 1 0 ⋯ 0 ⋱ [0 0 0 0 0 1 1 1 ] (C) The data kernel has 10,000 elements, of which about 300 are non-zero, so it is about 9,700/10,000 = 97% zero.

2.2 Suppose that you determine the height of 50 objects by measuring the first, and then stacking the second on top of the first and measuring their combined height, stacking the third on top of the first two and measuring their combined height, and so forth. (A) Identify the data and model parameters in this problem. How many of each are there? (B) Write down the matrix, 𝐆, in the equation, 𝐝 = 𝐆𝐦, that relates the data to the model parameters. (C) How sparse is 𝐆? What percent of it is zero?

(A) The 50 heights of the objects are the model parameters. The 50 heights of the stack, as measured by a ruler, are the data. (B) The data kernel is 50×50 1 1 1 𝐆= 1 1

0 1 1 1 1

0 0 1 1 1

0 0 0 1 1

0 0 0 0 1

0 0 0 0 0

⋯ 0 ⋯ 0 ⋯ 0 ⋯ 0 ⋯ 0 ⋱ [1 1 1 1 1 1 1 1 ] (C) The data kernel has 2500 elements, of which somewhat less than half are non-zero, so it is about 50% zero.

2.3 Write a MATLAB® or Python script to compute 𝐆 in the case of the cubic equation, 𝑇 = 𝑇 = 𝑎 + 𝑏𝑧 + 𝑐𝑧 2 + 𝑑𝑧 3 . Assume that 11 𝑧s are equally spaced from 0 to 10.

See scripts gdasolnma02_03 and gdasolnpy02_03.

2.4 Let the data, 𝐝, be the running average of the model parameters, 𝐦, computed by averaging groups of three neighboring points; that is, 𝑑𝑖 = ⅓𝑚𝑖−1 + ⅓𝑚𝑖 + ⅓𝑚𝑖+1 . (A) What is the matrix, 𝐆, in the equation, 𝐝 = 𝐆𝐦, in this case? (B) What problems arise at the top and bottom rows of the matrix and how can you deal with them? (C) How sparse is 𝐆? What percent of it is zero?

(A) The data kernel is N×M where N=M-4, and has the form 1 1 0 𝐆= 0 3

1 1 0 1 1 1 0 1 1

0 0 1

[0

0 0 0

0

0 0 0 ⋱ 1

⋯ ⋯ ⋯

0 0 0

1

1]

(B) Note that we can regard the running average di=(mi-1,mi,mi+1) as being centered about model parameter mi. However, no datum represents an average centered about model parameters 1, 2, M-1 or M. One could possible deal with them by adding two rows at the top and two at the bottom, such as 1 0 0 0 0 0 ⋯ 0 1/2 1/2 0 0 0 0 ⋯ 0 1/3 1/3 1/3 0 0 0 ⋯ 0 0 1/3 1/3 1/3 0 0 ⋯ 0 𝐆= 0 0 1/3 1/3 1/3 0 ⋯ 0 ⋱ 0 0 0 0 0 1/3 1/3 1/3 0 0 0 0 0 0 1/2 1/2 [ 0 0 0 0 0 0 0 0 ] These rows are not three-point running averages, but at least they are roughly centered about the first and last pairs of model parameters. (C) The sparseness of the data kernel depends upon its size. About 100×(M-3)/M = 100×(1-3/M) percent of its elements are zero.

2.5 Simplify Eq. (2.22) by assuming that there is only one sample, 𝑆1𝑗 , whose composition is measured. Consider the case where the composition of the 𝑃 factors is known, but their proportions in the sample are unknown, and rewrite the equation in the form, 𝐝 = 𝐆𝐦. Hint: You might start by taking the transpose of Eq. (2.22).

The equation becomes 𝐹11 𝑆11 𝐹 𝑆 [ 12 ] = 12 ⋮ ⋮ 𝑆1𝑀 𝐹 [ 1𝑀

𝐹21 𝐹22 ⋮ 𝐹2𝑀

⋯ ⋯ ⋯ ⋯

𝐹𝑝1 𝐶11 𝐹𝑝2 𝐶12 [ ] ⋮ ⋮ 𝐹𝑝𝑀 ] 𝐶1𝑀

The composition of the sample is measured, so they the data. The factor loadings are unknown, so they are the model parameters. The equation is then form d=Gm, as long as we presume that the composition of the factors are known.

3.1 What is the mean and variance of the uniform distribution, 𝑝(𝑑) = 1 on the interval, (0,1)?

The mean and variance are: 1

1

〈𝑑〉 = ∫ 𝑑 d𝑑 = 0 1 2

𝜎 = ∫ (𝑑 − ½) d𝑑 = ∫ 0

1

1 2

0

𝑑2 | =½−0=½ 2 0

(𝑑2

1

𝑑3 𝑑2 𝑑1 1 1 1 4 6 3 1 − 𝑑 + ¼) d𝑑 = | − | + | = − + = − + = 3 0 2 0 4 0 3 2 4 12 12 12 12

This result can be used in script, as in gdasolnma03_01 and gdasolnpy03_031 3.2 Suppose 𝑑 is a Normal random variable with zero mean and unit variance. What is the probability density function of 𝐸 = 𝑑2 ? Hint: Since the sign of 𝑑 gets lost when it is squared, you can assume that 𝑝(𝑑) is one-sided, that is, defined for only 𝑑 ≥ 0 and with twice the amplitude of the usual Normal pdf.

The p.d.f. of a Gaussian-distributed variable with zero mean and unit variance on the interval (0,∞) is twice the corresponding p.d.f. on the (-∞,∞) interval, or

𝑝(𝑒) = √

2 exp (−½𝑒 2 ) 𝜋

Given the function E=e2, or e=E½, we note that (0,∞) on e corresponds to (0,∞) on E. The derivative is de/dE = ½E-½. The p.d.f. for E is then:

𝑑𝑒 2 1 1 𝑝(𝐸) = 𝑝(𝑒(𝐸)) | | = √ exp (−½𝐸) = √ exp (−½𝐸) 𝑑𝐸 𝜋 2√𝐸 2𝜋𝐸

Note that this function has an integrable singularity at E=0.

3.3 Write a script that uses MATLAB's random('Normal',…) function or Python's np.random.normal(…) method to create a column-vector, d, of N=1000 realizations of a Normally-distributed random variable with mean, dbar=4, and standard deviation, sigmad=2.

Count up the number of instances where 𝑑 > 〈𝑑〉 + 2𝜎𝑑 . Is this about the number you expected?

See scripts gda_solnma03_03 and gda_solnpy03_03. I get about 2.5%, which is the right number.

3.4 Suppose that the data are uncorrelated with uniform variance, [cov 𝐝] = 𝜎𝑑2 𝐈, and that the model parameters are linear functions of the data, 𝐦 = 𝐌𝐝, where 𝐌 is a matrix. (A) What 2 property must 𝐌 have for the model parameters to be uncorrelated with uniform variance, 𝜎𝑚 ?

(B) Express this property in terms of the rows of 𝐌.

(A) The usual error propagation formula is [cov m] = M[cov d]MT. We are given the simplification, [cov d]=σd2I, so [cov m] = σd2MMT. This can only be proportional to σm2I when MMT∝I which implies MT∝M-1; that is M is proportional to a unary matrix. (B) The rows of M must be mutually orthogonal unit vectors and they all must have the same length.

3.5 Use the transformation method to compute realizations of the probability density function, 𝑝(𝑚) = 3𝑚2 , on the interval, (0,1), starting from realizations of the uniform distribution, 𝑝(𝑑) = 1. Check your results by plotting a histogram.

We are given p(m)=3m2. The cumulative distribution is P=m3 and its inverse is m=P1/3. A histogram (blue) based on L=10000 realizations were compute using scripts gda_solnma03_05 and gda_solnma03_05. They compare favorably to the true pdf. (red).

p(m)

2 1 0

0

0.2

0.4

0.6

0.8

1

m

4.1 Show that the equations worked out for the straight line problem in Eq. (4.14) have the solution given in Eq. (4.20).

After canceling a factor of 2 and rearranging, the text gives 𝑁

𝑁

𝑁𝑚1 + 𝑚2 ∑ 𝑧𝑖 = ∑ 𝑑𝑖 𝑁

𝑖=1

𝑖=1

𝑁

𝑁

𝑚1 ∑ 𝑧𝑖 + 𝑚2 ∑ 𝑧𝑖2 = ∑ 𝑧𝑖 𝑑𝑖 𝑖=1

𝑖=1

𝑖=1

or in matrix form 𝑁

𝑁

∑ 𝑧𝑖

𝑁

𝑖=1 𝑁

∑ 𝑧𝑖

∑ 𝑧𝑖2

[ 𝑖=1

𝑖=1

𝑁

𝑚1 [𝑚 ] = 2 ]

∑ 𝑑𝑖

𝑚1 or [𝑚 ] = 2

𝑖=1 𝑁

∑ 𝑧𝑖 𝑑𝑖

[ 𝑖=1

−1

𝑁

]

𝑁

𝑁

∑ 𝑧𝑖

∑ 𝑑𝑖

𝑁

𝑖=1 𝑁

𝑖=1 𝑁

∑ 𝑧𝑖

∑ 𝑧𝑖2

[ 𝑖=1

𝑖=1

]

∑ 𝑧𝑖 𝑑𝑖 [ 𝑖=1 ]

The second equation is of the form m=[GTG]-1[GTd] and matches the version given in the text.

4.2 This problem builds on Problem 2.1. Suppose that you determine the masses of 100 objects by weighing the first, then weighing the first and second together, and then weighing the rest in triplets: the first, second, and third; the second, third, and fourth; and so forth.Write a script that (A) randomly assigns masses, 𝑚𝑖𝑡𝑟𝑢𝑒 , to the objects in the range of 0 − 1 kg; (B) builds the appropriate data kernel; 𝐆; (C) creates synthetic observed data, 𝐝𝑜𝑏𝑠 = 𝐆𝐦𝑡𝑟𝑢𝑒 + 𝐧, where 𝐧 is a vector of Normally-distributed random numbers with zero mean and 𝜎𝑑 = 0.01 kg; (D) solves 2 the inverse problem by simple least squares; (E) estimates the variance, 𝜎𝑚 , of each of the 𝑖 estimated model parameters; and (F) counts up the number of estimated model parameters that are within 2𝜎𝑚𝑖 of their true value. (G) Make a plot of 𝜎𝑚𝑖 as a function of the index, 𝑖, of the model parameter. Does it decline, remain constant, or grow?

See scripts gdasolnma03_02 and gdasolnpy04_02. I ran the script several time, getting anywhere from 84 to 100 of the model parameters being within 2𝜎𝑚 of their mean (the expected number being 95). The plot of 𝜎𝑚 is shown below; it grows with the index of the model parameter.

0.08

sigmam

0.06 0.04 0.02 0

0

20

40

60

80

100

index i

4.3 This problem builds on Problem 2.2. Suppose that you determine the height of 50 objects by measuring the first, and then stacking the second on top of the first and measuring their combined height, stacking the third on top of the first two and measuring their combined height, and so forth. Write a script that (A) randomly assigns heights, 𝑚𝑖𝑡𝑟𝑢𝑒 , to the objects in the range of 0 − 1 m; (B) builds the appropriate data kernel, 𝐆; (C) creates synthetic observed data, 𝐝𝑜𝑏𝑠 = 𝐆𝐦𝑡𝑟𝑢𝑒 + 𝐧, where 𝐧 is a vector of Normally-distributed random numbers with zero mean and 𝜎𝑑 = 0.01 m; (D) solves the inverse problem by simple least squares; (E) estimates the variance, 2 𝜎𝑚 , of each of the estimated model parameters; and (F) counts up the number of estimated 𝑖 model parameters that are within 2𝜎𝑚𝑖 of their true value. (G) Make a plot of 𝜎𝑚𝑖 as a function of the index, 𝑖, of the model parameter. Does it decline, remain constant, or grow?

See scripts gdasolnma04_03 and gdasolnpy04_03. I ran the script several time, getting anywhere from 44 to 50 of the model parameters being within 2𝜎𝑚 of their mean (the expected number being 48). The plot of 𝜎𝑚 is shown below. The first point is smallest; all the rest are of equal size.

4.4 This problem builds on Problem 2.3, which considers the cubic equation, 𝑑𝑖𝑜𝑏𝑠 = 𝑚1 + 𝑚2 𝑧𝑖 + 𝑚3 𝑧𝑖2 + 𝑚4 𝑧𝑖3 . Write a script that (A) computes a vector, 𝐳, with 𝑁 = 11 elements equally spaced between 0 and 1; (B) randomly assigns the elements of 𝑚𝑡𝑟𝑢𝑒 in the range of −1 to 1; (C) builds the appropriate data kernel, 𝐆; (D) creates synthetic observed data, 𝐝𝑜𝑏𝑠 = 𝐆𝐦𝑡𝑟𝑢𝑒 + 𝐧, where 𝐧 is a vector of Normally-distributed random numbers with zero mean and 𝜎𝑑 = 0.05; (E) solves the inverse problem by simple least squares; (F) calculates the predicted data, 𝐝𝑝𝑟𝑒 = 𝐆𝐦𝑒𝑠𝑡 ; and (G) plots 𝑑𝑖𝑜𝑏𝑠 and 𝑑𝑖𝑝𝑟𝑒 . Comment upon the results.

See MatLab script gdasolnma04_04 and gdasolnpy04_04. The choice 𝜎𝑚 = 0.05 is a low-noise case, so that the data (red) is usually very well fit (black), as in the figure below.

30

d

20 10 0 0

2

4

6

8

10

z

4.5 This problem builds on Problem 4.4. Modify your solution of Problem 4.4 by adding the constraint that the predicted data pass through the point, (𝑧 ′ , 𝑑′) = (5,0). Comment upon the results.

Here we use 𝐇 = [1, 𝑧′, 𝑧 ′2 , 𝑧 ′3 ]𝑇 and 𝐡 = [𝑑 ′ ]. See scripts gdasolnma04_05 and gdasolnpy04_05. Since the constraint (green) rarely is close to the data (red), the fit (black) is not so good.

0

d

-100 -200 -300 0

2

4

6

8

10

z

4.6 Modify the reflection coefficient example of Section 4.8 to include the prior information that the ratio of compressional to shear wave velocities is exactly 𝑟 = 1.76. (A) Convert the equation,

(𝛼 + ∆𝛼)⁄(𝛽 + ∆𝛽) to a linear relationship involving model parameters, ∆𝛼 ⁄𝛼 and ∆𝛽⁄𝛽 using small number approximations. (B) Apply this prior information to the estimation of the model parameters using Eq. (3.31). Omit calculation of confidence intervals. (B) Compare the results to the case where no prior information is used.

(A) We use the small number approximation (1 + 𝑥)−1 ≈ 1 − 𝑥 to write the constraint (𝛼 + ∆𝑎)⁄(𝛽 + ∆𝛽) = 𝑟, where 𝑟 is specified, as a linear function of ∆𝑎 and ∆𝛽:

𝑟 = (𝛼 + ∆𝑎)⁄(𝛽 + ∆𝛽) = (𝑎⁄𝛽) (1 + ∆𝑎/𝛼)⁄(1 + ∆𝛽/𝛽) = (𝑎⁄𝛽) (1 + ∆𝑎/𝛼)(1 − ∆𝛽/𝛽) so ∆𝑎⁄𝛼 − ∆𝛽⁄𝛽 = 𝑟 𝛽⁄𝑎 − 1. (B) The a priori information 𝐇𝐦 = 𝐡 is [0,1, −1]T = [𝑟 𝛽⁄𝑎 − 1]. (C) Script gda_solnma04_06 and gda_solnpy04_06 give the ratios: prior ratio 1.760000 starting ratio 1.795918 estimated ratio 1.763262

The estimated ratio is not exactly 𝑟 = 1.76, because the prior information was implemented via a linearized approximation. The fit to the data is poorer than in the case with no prior information, because the true data were computed for 𝑟 = 1.79, not 𝑟 = 1.76.

5.1 Consider an underdetermined problem in which each datum is the sum of three neighboring model parameters, that is, 𝑑𝑖𝑜𝑏𝑠 = 𝑚𝑖−1 + 𝑚𝑖 + 𝑚𝑖+1 for 2 ≤ 𝑖 ≤ (𝑀 − 1) and 𝑀 = 100.

Compute and plot both the Dirichlet and Backus-Gilbert model resolution matrices. Use the standard Backus-Gilbert weight function, 𝑤(𝑖, 𝑗) = (𝑖 − 𝑗)2 . Interpret the results.

See scripta gdasolnma05_01 and gdasolnma05_01. The Backus-Gilbert solution (red line) is close to the

m(i)

1

0.5

0

0

20

40

60

80

100

i

The resolution matrix has a central band with elements like [ … 0.00 0.05 0.23 0.33 0.23 0.05 0.00 …] (spread of 0.15) which is only slightly narrower than the input [ … 0.00 0.00 0.33 0.33 0.33 0.00 0.00 … ] (spread of 0.22):

=

This is a case where the resolution cannot be spiked without introducing big side lobes.

5.2 This problem builds upon Problem 5.1. How does the Backus-Gilbert result change if you use the weight function, 𝑤(𝑖, 𝑗) = |𝑖 − 𝑗|½ , which gives less weight to distant sidelobes?

See scripta gdasolnma05_02 and gdasolnpy05_02. The Backus-Gilbert solution (red line) is not as close to the data (blue circles) and in the previous problem, but is still far from the true solution (red circles).

m(i)

1

0.5

0

0

20

40

60

80

100

i

The resolution matrix has a central band with elements like [ … 0.00 0.02 0.03 0.33 0.03 0.02 0.00 …] (spread of 0.025) which is spikier than the input [ … 0.00 0.00 0.33 0.33 0.33 0.00 0.00 … ] (spread of 0.222), but the wrong amplitude. The side-lobes are more diffuse, but actually contain fully two thirds of the amplitude.

=

5.3 This problem is especially difficult. Consider a two-dimensional acoustic tomography problem like the one discussed in Sec. 2.7, consisting of a 20 × 20 rectangular array of pixels, with observations only along rows and columns. (A) Design an appropriate Backus-Gilbert weight function that quantifies the spread of resolution. (B) Write a script that calculates the

model resolution matrix, 𝐑. (C) Plot a few representative rows of 𝐑, but where each row is reorganized into a two-dimensional image, using the same scheme that was applied to the model parameters. Interpret the results. (Hint: You will need to switch back and forth between a 20 × 20 rectangular array of model parameters and a length, 𝑀 = 400, column-vector of model parameters, as in Sec. 5.12).

Suppose that the tomographic grid is Lx by Ly so that there are M=Lx×Ly model parameter. A pixel at a (row, column) of (x, y) maps to model parameter with indice (x-1)*Ly+y. A generalization of the usual spread function for a model parameter at (x0,y0) is just radial distance squared (x-x0)2+(y-y0)2. Three rows of the resolution matrix, for (x0,y0) = (Lx/4, Ly/4), (Lx/2, Ly/2) and (x0,y0) = (3Lx/4, 3Ly/4) are shown below.

Note that the resolving kernel is localized about the target pixel, but is rather broad. (Note that I have written the code so that the power of the radial distance, currently pow=2, can be varied.

It is instructive to try powers of 1 and ½, too). Scripts gdasolnma05_03 and gdasolnma05_03.

6.1 Suppose that a random variable, 𝑚, is defined on the interval, [−1,1]. A reasonable choice for the null probability density function is 𝑝𝑁 (𝑚) = ½, meaning that 𝑚 can be anywhere within the interval with equal probability. (A) Calculate (either analytically or numerically) the information gain, 𝑆, of the prior probability density function, 𝑝𝐴 (𝑚) = ½ + 𝑐𝑚, where 0 < 𝑐 < ½. Note that the larger the constant, 𝑐, the higher the probability that 𝑚 will fall in the positive half of the interval. (B) Make and interpret a plot of 𝑆(𝑐).

The problem states that p(m)=½+cm over pN(m)=½ on the interval (-1, 1), where 0<c<½.

Note that p=pN when c=0, thus we expect the information gain S to be zero in that case. As c is increased, p(m) has its area increasingly concentrated on the right side of the interval. Thus we expect S to increase with c. I evaluate this formula 1

𝑆 = ∫ (½ + 𝑐𝑚) log ( −1

½ + 𝑐𝑚 ) d𝑚 ½

numerically in scripts gdasolnma06_01 and gdasolnma06_01. The results (shown below) seem to agree with expectations:

S(c)

0.2 0.1 0

0

0.05

0.1

0.15

0.2

0.25 c

0.3

0.35

0.4

0.45

0.5

2 6.2 Suppose that a Normal probability density function with mean, 〈𝑚〉, and variance, 𝜎𝑚 , is used to represent prior information about the following types of model parameters: (A) the density of sea water; (B) the shear velocity at 100-km depth in the earth; (C) the O18 to O16 ratio 2 in glacial ice. Propose plausible values of 〈𝑚〉 and 𝜎𝑚 in each case, citing references that justify your values.

Priori values: Wikipedia (http://en.wikipedia.org/wiki/Seawater#Properties) gives the range 1,020 to 1,029 m/kg3 for the density of sea water. Assuming this range to represent 95% of observations, a mean of 1,205 and a standard deviation of 2.5 might be reasonable. However, that properties are strongly depth-dependent, and bimodal, with the surface and abyssal waters being quite different.

Dalton et al’s (Earth Planet. Sci. Lett., 284, 65–75, 2009) map of global shear velocities at 100 km depth indicates a mean of about 4400 m/s with a standard deviation of about 200.

Note, however, that the velocity is highly correlated spatially.

The 018 to O18 ratio is normally stated in terms of δ018, defined as (𝑂18 /𝑂16 ) 𝛿𝑂18 = 1000 × [ 18 16 − 1] (𝑂 /𝑂 )𝑟𝑒𝑓 Grootes and Stuiver (Quaternary Research 53, 277–284, 2000) plot a time series in δ018 from the present to about 90 ka for GISP2 Greenland core, during which time it varies from -32 to -44 per mil. The distribution of values is strongly bimodal, so a Normal pdf is probably a poor representation of the variability. A mean of about 38 per mil with a standard deviation of 4 might be reasonable. However, the mean value is affected by air temperature, which varies geographically from glacier to glacier. Thus, mean at the Vostok site in Antarctica is about 57 (Dansgaard et al, Nature 364, 1993) while the standard deviation is similar to GISP2.

6.3 Suppose that you are fitting a cubic polynomial to data, 𝑑𝑖 = 𝑚1 + 𝑚2 𝑧𝑖 + 𝑚3 𝑧𝑖2 + 𝑚4 𝑧𝑖3 , but have exact prior information that 𝑚1 = 2𝑚2 = 4𝑚3 = 8𝑚4 . Write a script to solve this problem using Eq. (4.67). Set up the problem so that 𝑁 = 50, 0 < 𝑧𝑖 < 1, and 𝑚1𝑡𝑟𝑢𝑒 = 1, and

generate synthetic data with 𝜎𝑑2 = (0.1)2. How well do the data alone (i.e., without the prior information) constrain the ratios between the 𝑚s?

In the plot, the data (red circles) scatter about the true data (green curve). The fit (black curve) to the data looks adequate in both the simple least squares case (left plot) and when the constraints are included (right plot). However, the ratios between the model parameters are way off for simple least squares and has 4exactly the correct (2, 4, 8) pattern in the constrained case: ratios: simple least squares: 4.453122 3.363336 3.222113 ratios: constrained: 2.000000 4.000000 8.000000

1.8

1.8

1.6

1.6

d

2

d

2

1.4

1.4

1.2

1.2

1

1 0

0.2

0.4

0.6

0.8

1

0

0.2

z

0.4

0.6

0.8

1

z

See scripts gdasolnma06_03 and gdasolnpy06_03.

6.4 This problem builds upon Problem 6.3. Suppose that you are fitting a cubic polynomial to data, 𝑑𝑖 = 𝑚1 + 𝑚2 𝑧𝑖 + 𝑚3 𝑧𝑖2 + 𝑚4 𝑧𝑖3 3, but have inexact prior information that 𝑚1 = 2𝑚2 = 4𝑚3 = 8𝑚4 . Write a script to solve this problem using Eq. (6.17). Use a range of values for the 2 variance, 𝜎𝑚 , of the prior information, from very uncertain to very certain. Set up the problem so that so that 𝑁 = 50, 0 < 𝑧𝑖 < 1, and 𝑚1𝑡𝑟𝑢𝑒 = 1, and generate synthetic data with 𝜎𝑑2 = (0.1)2.

How well do the data alone (i.e., without the prior information) constrain the ratios between the 𝑚s?

In the example below I used σm=0.05. In the plot, the data (red circles) scatter about the true data (green curve). The fit (black curve) to the data looks adequate in both the simple least squares case (left plot) and when a priori information is included (right plot). However, the ratios between the model parameters are way off for simple least squares and more or less has the correct (2, 4, 8) pattern when a priori information is included: ratios: simple least squares: 1.033032 -0.708058 0.718958 ratios: generalized least squares: 2.032077 4.017045 7.999425

2

1.8

1.8

1.6

1.6

d

d

2

1.4

1.4

1.2

1.2

1

1 0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

z

0.6

0.8

1

z

Concerning other values of σm, when it large, (e.g. 106) the two fits are the same and when it is very small (e.g. 10-6) the correct ratios are exactly recovered. As far as I can see, there is no downside in using 10-6. See scripts gdasolnma06_04 and gdasolnpy06_04.

6.5 This problem builds upon Problems 6.3 and 6.4. Modify your script in Problem 6.4 in the case where the model of the data fitting a cubic polynomial is thought to be inexact, with a variance, 𝜎𝑔2 = (0.05)2 . How does this modification change your results?

Theory uncertainty can just be added to data uncertainty, 𝜎𝑑2 → 𝜎𝑑2 + 𝜎𝑔2 . Its effect is to increase the relative weight given to the a prioi information. The results (not shown) do not look much different from Problem 6.4. See scripts gdasolnma06_05 and gdasolnpy06_05.

6.6 Write a script to empirically generate the 𝑝(𝐹) probability density function with 𝜈1 = 𝜈2 = 20 by (A) using a random number generator to generate batches of 20 Normally-distributed 2 random numbers with zero mean and unit variance. (B) Calculating 𝜒20 by summing the squares of each batch. (C) Calculating 𝐹 for pairs of batches. (D) Repeat many times, creating a histogram of the resulting 𝐹s, and normalize to unit area to produce an empirical estimate of 𝑝(𝐹). (E) Compare your result with the result of MATLAB® or Python's 𝑝(𝐹) function.

The empirically-derived F pdf. matches the exact result very well:

1

p(F)

0.8 0.6 0.4 0.2 0

0

1

2

3 F

4

5

6

See scripts gdasolnma06_06 and gdasolnpy06_06.

6.7 This problem expands upon Problem 6.6. Suppose that the random numbers in step (A) are drawn from a uniform pdf with zero mean and unit variance, but that 𝐹 is calculated the same as before (let us call it 𝐹0 ). (A) How different is 𝑝(𝐹0 ) from 𝑝(𝐹)? (B) How would treating data with uniformly distributed error as if they were Normally-distributed affect the results of an 𝐹test? Hint: A pdf that is uniform between 𝑎 and 𝑏 has a variance of (𝑏 − 𝑎)2 ⁄12.

For a pdf that is symmetric about the origin, 𝑎 = −𝑏, 𝑏 must satisfy (2𝑏)2 ⁄12 = 1 or 𝑏 = √12/2.

2

2

1.5

1.5

p(F)

p(F)

The empirically-derived p(F’) pdf (right plot) is similar in shape to p(F) (left plot), but is narrower tailed:

1 0.5 0

1 0.5

0

2

4

6

0

0

2

F

4

6

F

See scripts gdasolnma06_07 and gdasolnpy06_07.

7.1 The file, Data/my_random_fields.txt, created by scripts gdama07_01 and (𝐸)

(𝐺)

gdapy07_01, contains 𝑁 = 256 data point (𝑥𝑖 , 𝑦𝑖 , 𝑑𝑖 , 𝑑𝑖 ) that sample two functions, 𝑑 (𝐸) (𝑥, 𝑦) and 𝑑(𝐺) (𝑥, 𝑦) on 0 < 𝑥 < 41 and 0 < 𝑦 < 41. The function, 𝑑 (𝐸) (𝑥, 𝑦), has an exponential auto-covariance; 𝑑 (𝐸) (𝑥, 𝑦) has a Gaussian auto-covariance, both with scale factor, 𝑠 = 5. The scripts, gdama07_03 and gdapy07_03, grid these data using GPR, using the correct scale factor. Run test cases to evaluate the effect of using incorrect scale factors, in the range 0.5 < 𝑠 < 50.

In the Gaussian case, too small a scale factor (e.g. 0.5) leads to a solution that is too rough. It fits the data exactly (with variance controlled by the observational error), but is zero in between data points (with a variance controlled by the prior variance). Too larger a scale factor (e.g.50.0) leads to a solution that has low variance but which is too smooth to fit the data well.

In the figures above, the left column is the reconstructed field, the right is the variance.

The exponential case (not shown) is similar, except that the larger scale factors are not nearly as smooth as in the Gaussian case (and so the fit to the data is somewhat better). See scripts gdasolnma07_01 and gdasolnpy07_01.

7.2 An object moving horizontally in the 𝑢-direction is decelerating due to a resistive force, 𝑓 = −𝑐𝑚𝑣, that is proportional to its mass, 𝑚, and velocity, 𝑣, and a constant, 𝑐 = 0.01. The object starts at position, 𝑢0 = 0, with initial velocity, 𝑣0 = 1. Newton's law implies d𝑢⁄d𝑡 = 𝑣 and d𝑣 ⁄d𝑡 = −𝑐𝑣. (A) What are the dynamical quantities, 𝐙, 𝐪, 𝐃, 𝐬 in Eqs. (7.32-7.33)? (B) Create noise-free data by stepping Eq. (7.33) forward in time (say, with ∆𝑡 = 1). (C) Create noisy data by adding Normally-distributed random noise with zero mean and variance, 𝜎𝑑2 = (5)2 to the

noise-free data. (D) Assuming that the dynamical equation (which is ultra-simplified) has variance, 𝜎𝑠2 = (0.01)2 , use Kalman filtering to estimate 𝑢(𝑡). Comment upon the results.

The dynamical equations d𝑢⁄d𝑡 = 𝑣 and d𝑣 ⁄d𝑡 = −𝑐𝑣 can be written as the combined equation: d 𝑢(𝑡) 0 [ ]=[ 0 d𝑡 𝑣(𝑡)

1 𝑢(𝑡) 0 ][ ]+[ ] −𝑐 𝑣(𝑡) 0

which is of the form d 𝑢(𝑡) 0 1 0 𝐦(𝑡) = 𝐙 𝐦(𝑡) + 𝐪(𝑡) with 𝐦(𝑡) ≡ [ ] and 𝐙 ≡ [ ] and 𝐪(𝑡) ≡ [ ] 𝑣(𝑡) 0 −𝑐 0 d𝑡 d

Then writing 𝐦(𝑛) ≡ 𝐦(𝑡 + (𝑛 − 1)∆𝑡) and approximating d𝑡 𝐦(𝑡) ≈ [𝐦(𝑡) − 𝐦(𝑡)]/∆𝒕, we have 𝐦(𝑛+1) = 𝐦(𝑛) + ∆𝑡𝐙 𝐦(𝑛) + ∆𝑡𝐪(𝒏) = 𝐃 𝐦(𝑛) + 𝐬(𝑛) with 𝐃 ≡ 𝐈 + ∆𝑡𝐙 and 𝐬 (𝑛) ≡ ∆𝑡𝐪(𝑛) So 𝐃=[

1 0

∆𝑡 ] (1 − 𝑐∆𝑡)

0 and 𝐬(𝑛) = [ ] 0

The initial conditions are 𝑢0 0 𝐦(1) = [𝑣 ] = [ ] 0 1 I have interpreted the instruction to mean that 𝑢(𝑡) is observed, but not 𝑣(𝑡). The data kernel is then 𝐆 = [1 0] And [cov (𝑛) 𝐝] = 𝜎𝑑2 is a scalar, as there is only one observation per time step. The forward problem provides the true (numerical) solution (black curves) and the synthetic data (red):

As expected, the velocity decreases with time due to the friction. The Kalman filter estimate of the solution is:

The Kalman solution (black curves) is smoother than the data (red), but much rougher than the true (numerical) solution.

7.3. This problem builds on Problem 7.2. How much better is an estimate of 𝑢(𝑡) when it is estimated by Generalized Least Squares (in which both past and future data influence the present)?

The initial conditions and dynamical equation can be written as: First row: 𝐈𝐦(1) = 𝐦0 Subsequent rows: 𝐈𝐦(𝑛) − 𝐃 𝐦(𝑛−1) = 𝟎 𝑇

When the model parameters are aggregated into a column vector, 𝐦 = [𝐦(1)𝑇 , 𝐦(2)𝑇 , … ] , these equations can be written as a prior information equation, 𝐇𝐦 = 𝐡, that is block bidiagonal (with 2 × 2 blocks): 𝐦0 𝜎02 𝐈 𝐈 𝟎 −𝐃 𝐈 𝜎𝑠2 𝐈 𝐇=[ ] and 𝐡 = [ ] and [cov 𝐡] = 2 𝟎 −𝐃 𝐈 𝜎0𝑠 𝐈 ⋮ ⋱ [ ⋱] The data equation can be written as covariance 𝐆𝐦 = 𝐝, wher the data kernel has 1 × 2 blocks: 𝜎𝑑2 𝑢1 [1 0] 𝑢 [1 0] 𝜎𝑑2 𝐆=[ ] and 𝐝 = [ 2 ] and [cov 𝐝] = 𝑢3 [1 0] 𝜎𝑑2 ⋮ [ ⋱ ⋱] The Generalized Least Squares solution is shown in the following figure.

The error of the two solutions, relative to the true numerical solution, is shown in the following figure (with Kalman in red, GLS in green and the data in black):

The GLS solution has about half the error of the Kalman solution, and is considerable smoother.

I attribute the difference to the affect of including future data; that is, a large swing in the Kalman solution at time 𝑡𝑛 is disallowed in the GLS solution by data from future times, 𝑡 > 𝑡𝑛 .

8.1 What is the general solution to the problem, 𝐆𝐦 = 𝐝, with 1 1 0 0 𝐆=[ ] 0 0 1 1 The top row of G indicates that the sum of m1 and m2 is d1. The bottom row indicates that the sum of m3 and m4 is d2. No information is provided about the difference between m1 and m2 or the difference between m3 and m4. One solution that satisfies the data is m1 =m2=½d1 and m3 =m4=½d2. To this particular m we can add any amount of the vectors for which the sum of the first two elements and the sum of the second two elements are zero, for instance [1, -1, 0, 0]T and [0, 0, 1, -1]T:

𝑑1 1 0 𝑑1 −1 0 𝐦 = ½ [ ] + 𝛼1 [ ] + 𝛼2 [ ] 𝑑2 0 1 𝑑2 0 −1 Here α1 and α2 are arbitrary constants.

8.2. Give some examples of physical problems where the model parameters can be assumed, with reasonable certainty, to fall between lower and upper bounds (and identify the bounds).

The density of a small particle at low pressure, such as a meteorite in space, must be necessarily greater than zero (for density is inherently positive) and is very likely no greater than 22,610 kg/m3 (the density of Osmium, the densest known substance). The temperature of water, such as in a river, is bounded by its freezing and boiling points, 0°C and 100°C, respectively. The components of the velocity vector or a particle are necessarily bounded by ±c, the speed of light (for no particle can travel faster than light).

8.3. Suppose that 𝑀 = 21, model parameters are known to have bounds, −1 ≤ 𝑚𝑖 ≤ 1.

Suppose that the unweighted average of each three adjacent model parameters are observed so that the data kernel has the form 1 1 1 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 𝐆= 0 0 1 1 1 0 0 0 0 ⋱ [0 0 0 0 0 0 1 1 1] Can useful bounds be placed on the weighted average of the three adjacent model parameters, where the weights are [¼, ½, ¼]? Adapt script gdama08_01 or gdapy08_01 to explore this problem.

See scripts gdasolnma08_03 and gdasolnma08_03. The top plot shows M=21 randomly chosen true model parameters, drawn from a uniform distribution on the interval (1,1) (red), the observed data, calculated through d=Gm+n, where n is Gaussian random noise with σd=0.001 (black), and mest, calculated using the minimum length solution. The bottom plot shows the true “localized” average, based on the three point running average with weights (0.25, 0.50, 0.25) (black). The red curves are the bounds on the average, using the constraints that |mi|<1. Note that the bounds closely track the true value.

1

m

0.5 0 -0.5 -1

0

2

4

6

8

10 z

12

14

16

18

20

0

2

4

6

8

10 z

12

14

16

18

20

1

m

0.5 0 -0.5 -1

8.4. Consider an inverse problem with 𝑀 = 8 model parameters, 𝑚𝑖 = 𝑚(𝑧𝑖 ) = sin(𝜋𝑧𝑖 ), where the auxiliary depth variable 𝑧 is in the range, 0 ≤ 𝑧 ≤ 1. (A) Construct a synthetic data set in which the observations are a triplicate set of measurements of 𝑚𝑖 equally spaced depths, all made with an accuracy of 𝜎𝑑 = 0.1, (so that 𝑁 = 3𝑀). (B) Estimate a reference solution using simple least squares. (C) Use squeezing to develop end-member solutions for the case where the solution is represented as a Taylor series and where the end-members represent polynomials with mostly low-order or mostly high-order terms. (D) Show that the solution cannot be squeezed to one in which the low-order terms have negligible amplitude but for which the error is acceptably low.

See scripts gdasolnma08_04 and gdasolnpy08_04. I cooked up weighting factors that squeeze toward (red) and away (green) from high order polynomials:

In the figure below the top frame shows the true solution (black curve), the observed data (black circles) and the simple least squares solution (red curve). The bottom frame shows the ”squeezed to low-order solution” (red) and ”squeezed to high-order solution” (green), which are very similar to each another. Each of the solutions has about 10% more error than the simple least squares solution. m(z)

1 0 -1 0

0.5 z

1

0.5 z

1

m(z)

1 0 -1 0

The figure below shows |𝑚 ̂ 𝐢 | for the low-squeezed (red) and high-squeezed (green) cases. The low-squeezed solution has little high-order energy. The high-squeezed solution adds high-order energy, but does not suppress the low-order energy by much. We conclude that no solution exists that lacks low-order energy and fits the data well.

9.1 This problem builds upon the discussion in Section 8.2. Write a MATLAB® or Python script that uses singular-value decomposition to compute the null vectors associated with the data kernel, 𝐆 = [1,1,1,1]. (A) The null vectors are different from that given in Eq. (8.7). Why? Show that the two sets are equivalent.

See scripts gdasolnma09_01 and gdasolnpy09_01. The natural solution, computed with SVD, gives the expected result that the m1=m2=½d1 and m3=m4=½d2. However, SVD gives a different set of null vectors than I gave in the solution for Problem 8.1. Any linear combination of null vectors is also a null vector, and in this case, adding them together produces my first null vector and subtracting them produces the second.

observed data: 1.000000 3.000000 estimated model parameters: 0.500000 0.500000 1.500000 1.500000 null vector 1: 0.500000 -0.500000 0.500000 -0.500000 null vector 2: 0.500000 -0.500000 -0.500000 0.500000 now compute V1B=V1+V2, V2B=V1-V2 null vector 1B: 1.000000 -1.000000 0.000000 0.000000 null vector 2B: 0.000000 0.000000 1.000000 -1.000000 9.2 Modify the weighted damped least squares “gap-filling” problem of Fig. 4.11 (scripts gdama04_12 and gdapy04_12) so that the prior information is applied only to the part of the solution in the null space. First, transform the weighted problem, 𝐆𝐦 = 𝐝, into an unweighted one, 𝐆′ 𝐦′ = 𝐝′ , using the transformation given by Eq. (9.22). Then, find the minimum-length solution (or the natural solution), 𝐦′𝑒𝑠𝑡 . Finally, transform back to obtain 𝐦𝑒𝑠𝑡 . How different is this solution from the one given in the figure? Compare the two prediction errors. In order to insure that 𝐖𝑒 = 𝐃𝑇 𝐃 has an inverse, which is required by the transformation, you should make 𝐃 square by adding two rows, one at the top and the other at the bottom, that constrain the first and last model parameters to known values.

See scripts gdasolnma09_02 and gdasolnma09_02. In the output below, the data are the red circles, the true model parameters are the red curve and the estimated model parameters are the green line. The transformed model parameters are the second derivatives of the middle M-2 model parameters and the values of the two end model parameters. Both the minimum length and natural solutions give the smallest model parameters that fit the data. Thus the data are fit exactly (the green curve goes through all the red circles), and the model parameters have minimum second derivative and vary very smoothly. The prediction error is exactly zero, in contrast to the case shown in the text, where the error is about 10-4 (small, but not zero).

10

m(z)

5 0 -5 -10

0

10

20

30

40

50 z

60

70

80

90

100

9.3 (A) Compute and plot the null vectors for the data kernel shown in Fig. 9.4A. (B) Suppose that the elements of 𝐦𝑡𝑟𝑢𝑒 are drawn from a uniform pdf between 0 and 1. How large a contribution do the null vectors make to the true solution, 𝐦𝑡𝑟𝑢𝑒 ? (Hint: Write the model parameters as a linear combination of all the eigenvectors, 𝐕, by writing 𝐦𝑡𝑟𝑢𝑒 = 𝐕𝛂,

where 𝛂 are coefficients, and then solving for 𝛂. Then examine the size of the elements of 𝛂.

You may wish to refer to scripts gdama09_05 and gdapy09_05 for the definition of 𝐆.

See scripts gdasolnma09_03 and gdasolnpy09_03. This problem has a null space of dimension M-p=4. The four null vectors are shown below. A typical set of randomly chosen model parameters has about 5% of its energy in the null space, determined by the following procedure: Expand the model parameters in the eigenvectors V by solving m=Va, or a=VTm, were a are coefficients; Compare the power of the last M-p coefficients with the total power, where power is defined as the sum of squared coefficients.

Vn(1)

1 0 -1

Vn(2)

1

1

Vn(3)

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5 z

3

3.5

4

4.5

5

0 -1

0 -1 1

Vn(4)

0

0 -1

9.4 Consider fitting a cubic polynomial, 𝑑𝑖 = 𝑚1 + 𝑚2 𝑧𝑖 + 𝑚3 𝑧𝑖2 + 𝑚4 𝑧𝑖3 , to 𝑁 = 20 data, 𝑑𝑖𝑜𝑏𝑠 , where the 𝑧s are evenly spaced on the interval, (0,1), where 𝑚2 = 𝑚3 = 𝑚4 = 1, and where 𝑚1 is varied from −1 to 1, as described later. (A) Write a script that generates synthetic observed data (including Normally-distributed noise) for a particular choice of 𝑚1 , estimates the model parameters using both simple least squares and nonnegative least squares, plots the observed and predicted data, and outputs the total error. (B) Run a series of test cases for a range of values of 𝑚1 , and comment upon the results.

See scripts gdasolnma09_04 and gdasolnpy09_04. The five cases in the plot below show the observed data (black circles), predicted data using simple least squared (red) and the predicted data using non-negative least squares (green). The different plots are for m1=(-1) (top) to m1=(+1) (bottom) in five equal steps. In the top two cases, the prediction of non-

negative least squares is significantly different than that of simple least squares. The total error of the non-negative least squares (NNLS) case is largest for the m1=(-1) case, while the total error of the simple least squares (SLS) case is constant (see table below). The script also makes a plot of the different m1 values (not shown here).

5 0 -5

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

5 0 -5 5 0 -5 5 0 -5 5 0 -5

errors: SLS 0.038588 NNLS 5.472511 (m1=-1.000000) errors: SLS 0.038588 NNLS 0.838694 (m1=-0.500000) errors: SLS 0.038588 NNLS 0.038588 (m1=0.000000) errors: SLS 0.038588 NNLS 0.038588 (m1=0.500000) errors: SLS 0.038588 NNLS 0.038588 (m1=1.000000)

9.5 (A) Modify the script gdama09_11 (or gdapy09_11) so that the last constraint is 𝑚1 ≥ 1.2. How do the feasible region and the solution change? (B) Modify the script to add a constraint 𝑚2 ≤ 0.2. How do the feasible region and the solution change?

See scripts gdasolnma09_05a, gdasolnma09_05b, gdasolnpy09_05a and gdasolnma09_05b. The left plot shows the original feasible area (brown region) and solution (green dot). The middle plot is case (A). Changing the third constraint has changed both the feasible area and the solution. The right plot is case (B). Adding the constraint has changed the feasible area, but not the solution.

10.1 Solve the best-fitting plane problem of Sec. 4.7 under the 𝐿1 norm and compare the estimated model parameters with those determined under the 𝐿2 norm. How much does the estimated strike and dip of the plane change?

See scripts gdasolnma10_01 and gdasolnpy10_01. The plot shows the L2 (left) and L1 (right) fits, which are very similar, as are the strikes and dips (given below).

L2: strike -135.100334 east of North; dip 42.966535 cw of strike L1: strike -134.899505 east of North; dip 41.902412 cw of strike

10.2 Solve the constrained best-fitting line problem of Sec. 4.21 under the 𝐿1 norm, by these two methods: (A) by considering the point, (𝑧0 , 𝑑0 ), as normal data with very small variance and (B) by explicitly including the constraint as a linear equality constraint within the linear programming problem. Compare the estimated model parameters with those determined under the 𝐿2 norm.

See scripts gdasolnma10_02 and gdasolnpy10_02. The blue curve is the constrained L2 case. Both of the L1 cases give the same solution, which are somewhat different than the L2 case.

I used a ratio of σd’s of 10-2 in case (A), but 10-1 and 10-4 (not shown) produced similar results.

15

d

10

5

0

0

1

2

3

4

5 z

6

7

8

9

10

10.3 This problem builds upon Problem 9.4. Consider fitting a cubic polynomial, 𝑑𝑖 = 𝑚1 + 𝑚2 𝑧𝑖 + 𝑚3 𝑧𝑖2 + 𝑚4 𝑧𝑖3 , to 𝑁 = 20 data, 𝑑𝑖𝑜𝑏𝑠 , where the 𝑧s are evenly spaced on the interval, (0, 1), where 𝑚2 = 𝑚3 = 𝑚4 = 1, and where 𝑚1 is varied from −1 to 1, as described below.

(A) Write a script that generates synthetic observed data (including exponentially-distributed noise) for a particular choice of 𝑚1 , estimates the model parameters under the 𝐿1 norm, with the extra inequality constraint that 𝐦 ≥ 0. (B) Run a series of test cases for a range of values of 𝑚1 and comment upon the results.

See scripts gdasolnma10_03 and gdasolnpy10_03. The five cases in the plot below show the observed data (black circles), predicted data using the L1 norm (red) and the predicted data using non-negative L1 norm (green). The different plots are for m1=(-1) (top) to m1=(+1) (bottom) in five equal steps. In the top two cases, the prediction of non-negative L1 norm case is significantly different than that of the L1 norm case. The total error of the non-negative case (NNL1) is largest for the m1=(-1) case, while the total error of the other case (L1) is constant (see table below). The script also makes a plot of the different m1 values (not shown here).

5 0 -5 5

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 -5 5 0 -5 5 0 -5 5 0 -5

errors: L1 0.678390 NNL1 8.478569 (m1=-1.000000). errors: L1 0.678390 NNL1 3.201491 (m1=-0.500000) errors: L1 0.678390 NNL1 0.770324 (m1=0.000000) errors: L1 0.678390 NNL1 0.678390 (m1=0.500000) errors: L1 0.678390 NNL1 0.678390 (m1=1.000000) 10.4 Consider the problem of finding 𝑀 = 100 model parameters, 𝐦, that minimize ‖𝐞‖22 + 𝜇‖𝐇𝐦‖𝑛𝑛 , where 𝜇 is a small positive number, 𝐞 = 𝐝𝑜𝑏𝑠 − 𝐆𝐦 is a vector of errors, 𝐝𝑜𝑏𝑠 = 𝐆𝐦𝑡𝑟𝑢𝑒 + 𝐧 are the observed data, 𝐧 is Normally distributed noise, and: 𝐦𝑡𝑟𝑢𝑒 = [[1, ⋯ ,1]𝑀/2 [2, ⋯ ,2]𝑀/2 ]𝑇 [1, ⋯ ,1]𝑀/2 [0, ⋯ ,0]𝑀/2 𝑇 𝐆=[ ] [0, ⋯ ,0]𝑀/2 [1, ⋯ ,1]𝑀/2 𝐇 = diag(𝐚) with 𝑎𝑘 = 1 + 0.001𝑘 and 𝑘 = 1 ⋯ 𝑀 The true model parameters, 𝐦𝑡𝑟𝑢𝑒 , are divided into two groups, each of size 𝑀/2, with equal values within each group. The data kernel, G, sums each group separately, to yield 𝑁 = 2 data.

The problem is very underdetermined, as 𝑁 ≪ 𝑀. The matrix, 𝐇, is almost the identity matrix, but weights the model parameters near the top of 𝐦 slightly less than those at the bottom, so that, all other things being equal, the minimization of ‖𝐇𝐦‖𝑛𝑛 selects a model that is squeezed toward the top. (A) Show that the true solution, 𝐦𝑡𝑟𝑢𝑒 , is also the 𝐿2 minimum length solution 𝐦𝑀𝐿 =

𝐆𝑇 [𝐆𝐆𝑇 ]−1 𝐝𝑜𝑏𝑠 . (B) Compute the solution for 𝑛 = 0, 1,2 and plot and interpret the result. (C) Repeat the solution in B but for 𝑎𝑘 = 1 + 0.001(𝑀 − 𝑘 + 1). In what respect is the solution different, and why?

The true model parameters have a constant value of unity in the 1 ≤ 𝑘 ≤ 50 interval and a constant value of 2 in the 50 ≤ 𝑘 ≤ 100 interval. The two data are the sums of the model parameters in the two intervals. Because the minimum length solution tends to spread energy evenly among the model parameters, is very close to the true solution (because the latter is piecewise constant). Because the solution is squeezed to the left, the 𝐿2 , 𝐿1 and 𝐿0 are largest at the left ends of the two intervals, but this size difference is most dramatic with the 𝐿0 solution, which puts all the energy in the left-most values. In all cases, the prediction error 𝐸 is very close to zero. Note that the figure shows the same signal at two different vertical scales. See gdasolnma10_04 and gdasolnma10_04.

10.5 Solve the previous problem, but for 𝐇 = 𝐃, where D is the (𝑀 − 2) × 𝑀 first difference operator: −1 1 0 0 0 0 0 −1 1 0 0 0 𝐃=[ ] ⋱ ⋱ ⋱ ⋱ ⋱ ⋱ 0 0 0 0 −1 1 Show that the model parameter vector becomes piecewise constant as 𝑛 → 0.

When the solution to Problem 10.4 is modified to select a flat solution (that is, one that suppresses the differences between adjacent model parameters), the 𝐿2 , 𝐿1 and 𝐿0 solutions all approximate a piecewise constant function, with the 𝐿0 solution being extremely close to the true solution (which is piecewise constant). See gdasolnma10_05 and 5gdasolnpy10_05.

11.1 Use the principle of maximum likelihood to estimate the mean, 𝜇, of 𝑁 uncorrelated data, each of which is drawn from the same one-sided exponential probability density function, 𝑝(𝑑𝑖 ) = 𝜇 −1 exp(− 𝑑𝑖 ⁄𝜇 ) on the interval, (0, ∞). Suppose all the 𝑁 data are equal to unity.

What is the variance?

The one-sided exponential distribution p(d) has only one parameter, μ, which is its mean. The likel2ihood function is the logarithm of the product ∏𝑖 𝑝(𝑑𝑖 ) or 𝐿 = −𝑁 ln(𝜇) − 𝜇 −1 ∑ 𝑑𝑖 𝑖

Setting its derivative to zero yields

0=

𝜕𝐿 𝑁 1 = − + 𝜇 −2 ∑ 𝑑𝑖 or 𝜇 = ∑ 𝑑𝑖 𝜕𝜇 𝜇 𝑁 𝑖

𝑖

Hence the sample mean is the maximum likelihood estimator of the mean of a one-sided exponential distribution. The variance of the one-sided exponential distribution is just μ2; it cannot be independently varied. So if the data are all equal to unity, both the estimated mean and variance are unity. Thus the estimated variance is not the square of the sample standard deviation, as it would be for a Gaussian (Normal) distribution.

11.2 Experiment with the Newton’s method example of Fig. 11.8, by changing the initial guess, mg, of the solution. Map out the region of the (𝑚1 , 𝑚2 ) plane for which the solution converges to the global minimum.

See scripts gdasolnma11_02 and gdasolnma11_02. The left image below is taken from the text, and shows one run of Newton’s Method overlain on the error surface, shown with the smaller errors in blue. In the right image, the colors represent the quantity 𝐿 = ‖𝐦true − 𝐦est ‖2 ; that is, the Euclidian distance between the true solution and the estimated solution, with deep

blue corresponding to L=0. The (m1, m2) location of each pixel was used as the initial guess in Newton’s method, and the pixel was colored by the L of the final estimate.

Error surface

Distance From True Solution Surface

Note that a roughly rectangular portion of the (m1, m2) plane is deep blue. Any initial guess in this region converges to the true solution. Outside this region, initial guesses converge to other local minima. Roughly speaking, the shapes of the regions of convergence match the shape of the error surface. 𝑚

11.3 Solve the nonlinear inverse problem, 𝑑𝑖 = 𝑚1 𝑧𝑖 2 , with 𝑧𝑖 an auxiliary variable, on the interval, (1,2), and with 𝐦𝑡𝑟𝑢𝑒 = [3,2]𝑇 , using both a linearizing transformation based on taking the logarithm of the equation and by Newton’s method and compare the result. Use noisy synthetic data to test your scripts.

See scripts gdasolnma11_03 and gdasolnpy11_03. The black curve shows the true data for m=[3,1]T. The circles are data, created with additive random noise. The red curve is for the linearized transformation, the green for Newton’s method. They give different, but similar results. Note that the transformation will fail if any of the data are less than zero (e.g. due to noise), while Newton’s method will continue to work.

12

d

10 8 6 4 1

1.1

1.2

1.3

1.4

1.5 z

1.6

1.7

1.8

1.9

2

11.4 Suppose that the 𝑧s in the straight line problem, 𝑑𝑖 = 𝑚1 + 𝑚2 𝑧𝑖 , are considered data, not auxiliary variables. (A) How should the inverse problem be classified? (B) Write a script that solves a test case using an appropriate method.

See scripts gdasolnma11_04 and gdasolnma11_04. (A) The problem is implicit, since the equation cannot be rearranged so that the data and model parameters are on opposite sides of the equation. The problem is also nonlinear, since it cannot be reorganized into the linear form Fx-f=0, when the data and model parameters are grouped into the vector x = [dT, zT, mT]T. Instead, we must write the N nonlinear equations f(x)=m1+m2z-d=0. (B) The matrix F(p) of partial derivatives contains the pieces 𝜕𝑓𝑖 𝜕𝑓𝑖 𝜕𝑓𝑖 𝜕𝑓𝑖 = −𝛿𝑖𝑗 and = 𝑚2 𝛿𝑖𝑗 and = 1 and = 𝑧𝑖 𝜕𝑑𝑗 𝜕𝑧𝑖 𝜕𝑚1 𝜕𝑚2 The iterative solution method described in the text is shown in the figure, below.

2.5 2

1.5

1

0.5

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

The true intercept and slope yield is the back line. The observed data are based on this line, with both z and d values perturbed with Gaussian random noise. The red line is the ordinary least squares solution, which treats the zs as auxiliary variables. The green circles and green line are the results of the iterative method, using the observed data and the least squares model

parameters as the a priori information. The variance of the data match that used in the Gaussian noise, and the variance of the a priori model parameters is taken to be very large. The predicted data (circles) all fall on the green line, but their z values are different than the observed data.

The model parameters are shown in the table below; both the least squares and iterative values are similar to the true value. The error of the final iterative solution, meaning fTf, is very small and much reduced from the initial error. initial error in implicit equation 0.357012 final error in implicit equation 0.000309 mtrue 1.000000 1.000000 mls 0.952060 1.088806 mest 0.903560 1.184209 11.5 A column-vector, 𝐝, is constructed by adding together scaled and shifted versions of vectors 𝐚, 𝐛, and 𝐜. Noisy data, 𝐝𝑜𝑏𝑠 , of length, 𝑁, is observed. The vectors, 𝐚, 𝐛, and 𝐜, also of length 𝑁, are auxiliary variables. They are related by, 𝑑𝑖 = 𝐴𝑎𝑖+𝑝 + 𝐵𝑏𝑖+𝑞 + 𝐶𝑐𝑖+𝑟 , where 𝐴, 𝐵, and 𝐶 are unknown constants and 𝑝, 𝑞, and 𝑟 are unknown positive integers (assume 𝑎𝑖−𝑝 = 0 if 𝑖 + 𝑝 > 𝑁 and similarly for 𝐛 and 𝐜). This problem can be solved using a three-dimensional grid search over 𝑝, 𝑞, and 𝑟, only, as 𝐴, 𝐵, and 𝐶 can be found using least squares once 𝑝, 𝑞, and 𝑟 are specified. Write a script that solves a test case for 𝑁 = 50. Assume that the error is 𝐸 = 𝐞𝑇 𝐞 with 𝑒𝑖 = 𝑑𝑖𝑜𝑏𝑠 − (𝐴𝑎𝑖+𝑝 + 𝐵𝑏𝑖+𝑞 + 𝐶𝑐𝑖+𝑟 ). See scripts gdasolnma11_05 and

gdasolnpy11_06.

a

1 0 -1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

b

1 0 -1

c

1 0 -1

d

1 0 -1

The top three graphs are the hypothetical vectors a, b, c. In the both graph the black curve is the true data, the dots are the observed data, constructed from the true data by adding random noise, and the red is the predicted data, resulting from the grid search. The solution fits the data well.

The true and estimated solutions are shown below: p q r A B C true 5 8 7 0.500000 0.300000 -0.600000 est 5 8 7 0.532258 0.275191 -0.616993 12.1 Modify your code for Problem 11.5 to substitute a Monte Carlo search in place of a grid search. If the modified code any more efficient?

I approached the problem by keeping track of the smallest error yet encountered as a function of iteration number (lack in the plot, below), and comparing it to the minimum error of the grid search (red).

The results are of course variable, but generally Monte Carlo is at least as good as a grid search.

In the case above, the MC error is comparable to the grid search error in after about one fifth of the iterations (although MC has not yet found exactly the right offsets).

12.2 This problem concerns the inverse problem shown in Figs. 11.8 and 12.3. (A) Modify the Newton's method solution script (gdama11_10 or gdapy11_10) to produce a linearized −1

𝑇 estimate of the posterior covariance, [cov 𝐦(𝑝+1) ] ≈ 𝜎𝑑2 [𝐆(𝑝) 𝐆(𝑝) ] , where the posterior

variance, 𝜎𝑑2 , is estimated from the prediction error. (B) Modify the MCMC method solution script (gdama12_03 or gdapy12_03) to estimate the posterior covariance as the sample covariance of the ensemble. Be sure that the prior data covariance matches between the code. Finesse the issue of the MCMC code having prior information by making it very weak, so that it has minimal effect on the solution. (C) The MCMC estimate is probably closer to the true covariance, so use it as the standard to judge the accuracy of the linearized version.

How well does it do?

The difference is variable, as the data changes from simulation to simulation. (I use the same observed data for Newton’s method and MCMC). posterior Cov m (Newtons method) [[ 4.83604267e-05 -4.01780085e-05] [-4.01780085e-05 2.12008475e-03]] ensemble Cov m (MCMC method) [[ 3.73413602e-05 -3.06836800e-05] [-3.06836800e-05 1.56416136e-03]] element-wise percent difference [[29.50901195 30.94260048] [30.94260048 35.54130661]]

Newton’s method does reasonably well, with errors of 30-40% (especially considering that a variance of 30% corresponds to a 15% difference in uncertainty of the model parameters.

12.3 Modify the code for trans-dimensional problem shown in FIG 12.5 (gdama12_05 or gdapy12_05) so that the one-dimensional pdf is a two-sided exponential pdf with the same mean and variance of the original Normal pdf. Does the method perform any better or worse?

True trans-dimensional pdf

Estimate transdimensional pdf

True (red) and estimated (black) 1D portion of trans-dimensional pdf As can be seen from the above plots, the method correctly samples the trans-dimensional pdf, and correctly recovers the 1D exponential pdf.

13.1 Suppose that a set of 𝑁 > 𝑀 samples are represented as 𝐒 = 𝐂𝑝 𝐅𝑝 where the matrix, 𝐅𝑝 contains p<M factors whose values are prescribed (that is, constitute prior information). (A) How can the loadings, 𝐂𝑝 , be determined? (B) Write a script that implements your procedure for the case 𝑀 = 3, 𝑁 = 10, 𝑝 = 2. (C) Make a three-dimensional plot of your results. .

13.1. (A) We first transform S=CF into an equation that looks like Gm=d, by first taking its transpose ST=FTCT and then identifying FT=G, a column of ST as d and a corresponding column of CT as m. The least squares generalized inverse is then (GTG)-1GT=(FFT) -1F and the solution is given by CT = (FFT) -1F ST or C = SFT(FFT) -1. (B) When run with noisy S data and with F and Ctrue chosen to have elements drawn from a uniform distribution on (0,1), the script recovers the loadings up to this error: C: 0.234008 S: 0.030288. The error is, as a expected, a small fraction of the values of C and S. (C) A plot of the elemental composition of the observed samples (black) and estimated samples (red) are shown below. Note that, as expected, the true data scatter about a mixing line and the estimates are on it.

1.5

e3

1

0.5

0 1.5 1

0.5 e1

0

1.5

0.5

1

0

e2

13.2 Write a script that verifies the varimax result given in Eq. (13.13). Use the following steps.

(A) Compute the factors, 𝐟 ′𝐴 and 𝐟 ′𝐵 , for a complete suite of angles, 𝜃, using the rotation 𝐟 ′𝐴 = cos(𝜃) 𝐟 𝐴 + sin(𝜃) 𝐟 𝐵 and 𝐟 ′𝐵 = − sin(𝜃) 𝐟 𝐴 + cos(𝜃) 𝐟 𝐵 (13.16) (B) Compute and plot the variance, (𝜎𝑓2′𝐴 + 𝜎𝑓2′𝐵 ), as a function of angle, 𝜃. (C) Note the angle of the minimum variance and verify that the angle is the one predicted by the varimax formula.

(D) Verify that the factors corresponding to the angle are as stated in Eq. (13.13).

The scripts gdasolnma13_02 and gdasolnpy13_02 gives results: search: angle=45.000000 variance=2.666667 A 1.414214 0.000000 1.414214 0.000000 B 0.000000 -1.414214 0.000000 -1.414214 varimax procedure: angle=45.000000 variance=2.666667

A 1.414214 0.000000 1.414214 0.000000 B 0.000000 -1.414214 0.000000 -1.414214 Note that the results of the search and the varimax formula are identical. The total variance varies with angle as shown by the back curve shown in the accompanying figure. The maximum is marked with a green circle.

variance

5 0 -5

0

10

20

30

40 50 theta, deg

60

70

80

90

13.3 Suppose that a data set represents a function of three spatial dimensions; that is, with samples, 𝑆(𝑡, 𝑥, 𝑦, 𝑧), on an evenly spaced three-dimensional grid. Write a script that unfolds these samples into a matrix, S A 3D cube 𝑎𝑖𝑗𝑘 with indices 1 ≤ 𝑖 ≤ 𝐼, 1 ≤ 𝑗 ≤ 𝐽 and 1 ≤ 𝑘 ≤ 𝐾 can be unfolded into a vector 𝑏𝑚 with indices 1 ≤ 𝑚 ≤ 𝑀 where 𝑀 = 𝐼𝐽𝐾 by constructing index tables that translate back and forth between (𝑖, 𝑗, 𝑘) and 𝑚. See the scripts gdasolnma13_03 and gdasolnpy13_03 for details. In the EOF problem, the four-index quantity 𝑆𝑛𝑖𝑗𝑘 (where 𝑛 is a temporal index and (𝑖, 𝑗, 𝑘) are spatial indices) is converted to a matrix 𝑆𝑛𝑛 using these tables.

14.1 This inverse problem is due to Robert Parker. Consider a radially stratified sphere of radius, 𝑅 = 1, with unknown density, 𝜌(𝑟). Its mass, 𝑀 = 4𝜋 ∫ 𝜌(𝑟) 𝑟 2 d𝑟, and its moment of inertia. 𝐼 = (8𝜋⁄3) ∫ 𝜌(𝑟) 𝑟 4 d𝑟 are measured. What is the best averaging kernel that can be designed that is centered about ½𝑅? As all the integrals involve only rational functions and all the matrices are 2 × 2, the problem can be solved analytically. Do as much of the problem as you can, analytically; do the parts that you cannot, numerically.

The model parameters are continuous m(r)=ρ(r) and there are two data d1=M and d2=I, related via the kernels 𝑔1 (𝑟) = 4𝜋𝑟 2 and 𝑔2 (𝑟) = The u integrals are

8𝜋 4 𝑟 3

1

1

𝑢1 = ∫ 𝑔1 (𝑟) d𝑟 = ∫ 4𝜋𝑟 2 d𝑟 = 0 1

0 1

4𝜋 3 1 4𝜋 𝑟 |0 = ≈ 4.1888 3 3

8𝜋 4 8𝜋 5 1 8𝜋 𝑟 d𝑟 = 𝑟 |0 = ≈ 1.6755 15 15 0 3

𝑢2 = ∫ 𝑔2 (𝑟) d𝑟 = ∫ 0

The S integrals are 1

1

1

𝑆11 = ∫ (𝑟 − ½)2 𝑔1 (𝑟) 𝑔1 (𝑟)d𝑟 = (4𝜋)2 ∫ (𝑟 2 − 𝑟 + ¼) 𝑟 4 d𝑟 = (4𝜋)2 ∫ (𝑟 6 − 𝑟 5 + ¼𝑟 4 ) d𝑟 0

0

0

1 1 1 120 − 140 + 42 = (4𝜋)2 [ − + ] = (4𝜋)2 [ ] 7 6 20 840 352 (𝜋)2 ≈ 0.419 (𝜋)2 = 4.1354 = 840

𝑆12 = 𝑆21 = (4𝜋) (

8𝜋 1 2 8𝜋 1 ) ∫ (𝑟 − 𝑟 + ¼) 𝑟 6 d𝑟 = (4𝜋)( ) ∫ (𝑟 8 − 𝑟 7 + ¼𝑟 6 ) d𝑟 3 0 3 0

= (4𝜋) ( =(

𝑆22 = (

8𝜋 1 1 1 32(𝜋)2 224 − 252 + 72 )[ − + ] = ( )[ ] 3 9 8 28 3 2016

32(𝜋)2 44 1408 (𝜋)2 ≈ 0.233 (𝜋)2 = 2.2996 )[ ]= 3 2016 6048

8𝜋 2 1 2 8𝜋 1 ) ∫ (𝑟 − 𝑟 + ¼) 𝑟 8 d𝑟 = (4𝜋)( ) ∫ (𝑟10 − 𝑟 9 + ¼𝑟 8 ) d𝑟 3 3 0 0

8𝜋 2 1 1 1 64(𝜋)2 360 − 396 + 110 =( ) [ − + ]=( )[ ]= 3 11 10 36 9 3960 (

64(𝜋)2 74 )[ ] ≈ 0.133(𝜋)2 = 1.312 9 3960

g1

I’ve checked these analytic formulas against numerical calculations; they agree. Past this point, there seems little point in working analytically; all one is doing is carrying around horrible fractions.

10 5

g2

0

0.1

0.2

0.3

0.4

0.5 r

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5 r

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5 r

0.6

0.7

0.8

0.9

1

5 0

R

0

2 1 0

The result is R(r) = 0.693 g1(r) -1.136 g2(r), which is plotted above. The result is vaguely localized near the middle of the model. See MatLab script gda_soln11_01.m.

14.2 Suppose that a function, 𝑚(𝑥), is discretized in each of two ways: (A) by dividing it into 𝑀 rectangles, each of height, 𝑚𝑖 , and width, ∆𝑥; and (B) by representing it as a sum of 𝑀 overlapping Gaussians, spaced at regular intervals, ∆𝑥, prescribed variance, 𝜎 2 , and unknown amplitude, 𝑚𝑛′ 𝑀

𝑚(𝑥) = ∑ 𝑚𝑛′ 𝑛=1

∆𝑥 √2𝜋𝜎

exp {−

(𝑥 − 𝑛∆𝑥)2 } 2𝜎 2

Discuss the advantages and disadvantages of each representation. Include the respective effect of damping in the two cases.

The rectangular parameterization is just a time series (say m). The second parameterization (say m’) is the sum of smooth Gaussian functions. The parameterizations are related by m=Tm’, where T is a square matrix built up out of columns which are Gaussians. For σ>Δx, the inverse transformation is pretty close to being singular. The second method builds a function out of smooth basis functions. That’s good if you have a priori information that m(x) is smooth, but bad if you think that m(x) is highly oscillatory. The fact that T-1 is near singular implies that you can’t represent an arbitrary time series in with it. But it also mean that in order to get short wavelength oscillations in m(x), you have to make the elements of m large. Damping m’ will therefore tend to select a smooth m(x), which can be desirable. In the example below, the true model is the black curve. It is measured at only a subset of points (circles). A damped least squares inversion using the first model is red; using the second is green. Note that the second produces a much smoother solution. See MatLab script gda_soln11_02.m.

1

m

0 -1 0

0.1

0.2

0.3

0.4

0.5 x

0.6

0.7

0.8

0.9

1

14.3 How sensitive are the results of tomography to noise in the data, 𝑑(𝑢, 𝜃)? Run experiments with the Fourier slice script (gdama14_02 or gdapy14_02). Try two different types of noise: (A) uncorrelated random noise drawn from a Normal pdf and (B) a few large outliers. Comment upon the results.

A little (5%) random noise added to all the elements of the Radon transform just adds ‘speckle noise’ to the reconstructed image (middle), which is not present in the true image (left).. But a few big outliers at the 100% level produce nasty lines crossing the image. The random transform represents integrals of the image along lines, so it makes sense that the outliers appear as lines crossing the reconstructed image (right). See MatLab scripts gda_soln11_02a.m and gda_soln11_02b.m. reconstructed image

true image

50

50

50

100

100

x

x

x

100

reconstructed image

150

150

150

200

200

200

250

250 50

100

150

200

250

y

250

50

100

150

200

250

y

50

100

150

200

250

y

14.4 Modify the example in Section 14.9 to use a truncated Fourier sine series representation of the model: 𝐾

𝐾 𝑛𝜋𝑥 𝑛𝜋𝑥 (0) (0) (0) 𝑚 (𝑥) = ∑ 𝑚𝑛 sin ( ) and 𝛿𝑚 (𝑥) = ∑ 𝛿𝑚𝑛 sin ( ) 𝐿 𝐿 𝑛=1 𝑛=1 (0) (0) where 𝑚𝑛 and 𝛿𝑚𝑛 are scalar coefficients, 𝐾 = 15 and 0 < 𝑥 < 𝐿. You will need to use the (0)

chain rule

𝛿𝐸 𝛿𝐸 𝛿𝑚 | =( | , | ) 𝛿𝑚𝑛 𝐦(0) 𝛿𝑚 𝐦(0) 𝛿𝑚𝑛 𝐦(0)

The parameterization is 𝑀

𝑀

(0) 𝑚(𝑥) = ∑ 𝑚𝑛 sin(𝑛𝜋𝑥/𝐿) 𝑛=1

(0)

and 𝛿𝑚(𝑥) = ∑ 𝛿𝑚𝑛 sin(𝑛𝜋𝑥/𝐿) 𝑛=1

so the derivative is 𝛿𝑚 𝜕𝑚 = = sin(𝑛𝜋𝑥/𝐿) | | 𝛿𝑚𝑛 𝐦(0) 𝜕𝑚𝑛 (0) 𝐦

and the Frechét derivative of error is then 𝛿𝐸 𝛿𝐸 𝜕𝑚 𝛿𝐸 =( | , | | ) = ∫[ | ] sin(𝑛𝜋𝑥/𝐿) d𝑥 𝛿𝑚𝑛 𝐦(0) 𝛿𝑚 𝐦(0) 𝜕𝑚𝑛 𝐦(0) 𝛿𝑚 𝐦(0)

The key issue is whether this integral should be performed numerically or analytically. The original script, eda11_03.m, has a numerical version of the Frechét derivative in the variable dEdm. Thus, one of the two pieces needed to perform the integral is already in place. The other piece can be created by defining a matrix, say F, such that Fin=sin(nπxi/L). The integral is then Dx*F'*dEdm. This result can now be used in a gradient method to find the value of the mn’s that minimize the error E, as is shown in the script gda_soln11_04a.m.

m(x)

2 0 -2

0

0.1

0.2

0.3

0.4

0.5 x

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5 x

0.6

0.7

0.8

0.9

1

10

20

30

40

50 60 iteration

70

80

90

100

d(x)

5

log10 error, E

0

0 -2 -4

The search converges faster than with the original parameterization, since the model space is smaller. It’s also instructive to run the problem with smaller values of K, where the fit is not as good (not shown).

While the numerical integration adequately solves the problem, analytic integration is both more accurate and faster. We derive it here for comparison. We start by writing the Frechét derivative 𝛿𝐸 = −2ℒ † [𝑑 𝑜𝑏𝑠 (𝑥) − 𝑑(𝑥)] = −2ℒ † 𝑑𝑜𝑏𝑠 (𝑥) + 2ℒ † ℒ 𝑚(𝑥) = 𝐴(𝑥) + 𝐵(𝑥) 𝛿𝑚

as the sum of two functions, A(x) and B(x). Note that only A(x) depends upon the observed data

𝐴(𝑥) = −2ℒ † 𝑑𝑜𝑏𝑠 (𝑥) = 2𝑎

𝐿 𝑑 𝑜𝑏𝑠 𝑑 (𝑥) − 2𝑏 ∫ 𝑑𝑜𝑏𝑠 (𝑥) 𝑑𝑥 𝑑𝑥 𝑥

In order to compute B(x) we first construct ℒ𝑚(𝑥) 𝐾

𝐾

ℒ𝑚(𝑥) = ∑ 𝑚𝑛 ℒsin (𝑛𝜋𝑥/𝐿) = ∑ 𝑚𝑛 {𝑎 𝑛=1

𝑛=1

x d 𝑛𝜋𝑥 𝑛𝜋𝑥′ sin ( ) + 𝑏 ∫ sin ( ) d𝑥′} = d𝑥 𝐿 𝐿 0

𝐾

𝐾

𝑛=1

𝑛=1

𝑛𝜋 𝑛𝜋𝑥 𝐿 𝑛𝜋𝑥 = ∑ 𝑚𝑛 {𝑎 ( ) cos ( ) + 𝑏 ( ) {1 − cos ( )}} = ∑ 𝐺𝑛 (𝑥) 𝑚𝑛 𝐿 𝐿 𝑛𝜋 𝐿

with 𝑛𝜋 𝑛𝜋𝑥 𝐿 𝑛𝜋𝑥 𝐺𝑛 (𝑥) = 𝑎 ( ) cos ( ) + 𝑏 ( ) {1 − cos ( )} 𝐿 𝐿 𝑛𝜋 𝐿

and then construct 𝐵 = 2ℒ † [ℒ𝑚(𝑥)] 𝐾 †

𝐵 = 2ℒ ℒ𝑚(𝑥) = 2 ∑ 𝑚𝑛 𝑓𝑛 (𝑥) 𝑛=1

𝑛𝜋 d 𝑛𝜋𝑥 𝐿 d 𝑛𝜋𝑥 𝑓𝑛 (𝑥) = −𝑎2 ( ) cos ( ) − 𝑎 𝑏 ( ) {1 − cos ( )} 𝐿 d𝑥 𝐿 𝑛𝜋 d𝑥 𝐿 L 𝑛𝜋 L 𝑛𝜋𝑥′ 𝐿 𝑛𝜋𝑥′ +𝑎 𝑏 ( ) ∫ cos ( ) d𝑥′ + 𝑏 2 ( ) ∫ {1 − cos ( )} d𝑥′ 𝐿 x 𝐿 𝑛𝜋 x 𝐿 𝑛𝜋 2 𝑛𝜋𝑥 𝑛𝜋𝑥 = 𝑎2 ( ) sin ( ) − 𝑎𝑏 sin ( ) 𝐿 𝐿 𝐿

𝑛𝜋𝑥 𝐿 𝐿 𝑛𝜋𝑥 −𝑎 𝑏 sin ( ) + 𝑏 2 ( ) {(𝐿 − 𝑥) + ( ) sin ( )} 𝐿 𝑛𝜋 𝑛𝜋 𝐿

We break the derivative 𝜕𝐸 ⁄𝜕𝑚𝑞 into two terms, cq involving A(x) and dq involving B(x) 𝜕𝐸 𝛿𝐸 𝜕𝑚 𝛿𝐸 𝑞𝜋𝑥 𝑞𝜋𝑥 𝑞𝜋𝑥 =( , , sin ( )) = (𝐴, sin ( )) + (𝐵, sin ( )) = 𝑐𝑞 + 𝑑𝑞 )=( 𝜕𝑚𝑞 𝛿𝑚 𝜕𝑚𝑞 𝛿𝑚 𝐿 𝐿 𝐿

The terms cq is 𝑞𝜋𝑥 𝑞𝜋𝑥 𝑞𝜋𝑥 𝑐𝑞 = (𝐴, sin ( )) = (−2ℒ † 𝑑𝑜𝑏𝑠 (𝑥), sin ( )) = (𝑑𝑜𝑏𝑠 (𝑥), −2ℒsin ( )) 𝐿 𝐿 𝐿 𝑞𝜋 𝑞𝜋𝑥 𝐿 𝑞𝜋𝑥 = (𝑑𝑜𝑏𝑠 (𝑥), −2𝑎 ( ) cos ( ) − 2𝑏 ( ) {1 − cos ( )}) = (𝑑𝑜𝑏𝑠 (𝑥), 𝐶𝑞 (𝑥)) 𝐿 𝐿 𝑞𝜋 𝐿

with 𝑞𝜋 𝑞𝜋𝑥 𝐿 𝑞𝜋𝑥 𝐶𝑞 (𝑥) = −2𝑎 ( ) cos ( ) − 2𝑏 ( ) {1 − cos ( )} 𝐿 𝐿 𝑞𝜋 𝐿

The expression for cq cannot be further simplified until 𝑑𝑜𝑏𝑠 (𝑥) is specified. The term dq is 𝐾

𝐾

𝑛=1

𝑛=1

𝑞𝜋𝑥 𝑞𝜋𝑥 𝑑𝑞 = (𝐵, sin ( )) = 2 ∑ 𝑚𝑛 (𝑓𝑛 (𝑥), sin ( )) = 2 ∑ 𝐷𝑛𝑞 𝑚𝑛 𝐿 𝐿

with 𝑞𝜋𝑥 𝐷𝑛𝑞 = (𝑓𝑛 (𝑥), sin ( )) = 𝐿 𝑛𝜋 2 𝑞𝜋𝑥 𝑛𝜋𝑥 𝑞𝜋𝑥 𝑛𝜋𝑥 = 𝑎2 ( ) (sin ( ) , sin ( )) − 𝑎𝑏 (sin ( ) , sin ( )) 𝐿 𝐿 𝐿 𝐿 𝐿 𝑞𝜋𝑥 𝑛𝜋𝑥 𝐿 𝑞𝜋𝑥 𝐿 𝑛𝜋𝑥 −𝑎 𝑏 (sin ( ) , sin ( )) + 𝑏 2 ( ) (sin ( ) , {(𝐿 − 𝑥) + ( ) sin ( )}) 𝐿 𝐿 𝑛𝜋 𝐿 𝑛𝜋 𝐿

We now perform all the inner products analytically. Actually, there are really only two inner products, the inner product of two sine functions and the inner product of a sine function with (Lx). These two inner products are L 𝑞𝜋𝑥 𝑛𝜋𝑥 𝑞𝜋𝑥 𝑛𝜋𝑥 𝐿 π 𝐿 (sin ( ) , sin ( )) = ∫ sin ( ) sin ( ) d𝑥 = ∫ sin(𝑞𝑦) sin(𝑛𝑦)𝑑𝑦 = 𝛿𝑞𝑛 𝐿 𝐿 𝐿 𝐿 𝜋 0 2 0

and

L L 𝑞𝜋𝑥 𝑞𝜋𝑥 𝑞𝜋𝑥 ) , (𝐿 − 𝑥)) = 𝐿 ∫ sin ( ) d𝑥 − ∫ 𝑥 sin ( ) d𝑥 (sin ( 𝐿 𝐿 𝐿 0 0

= −(

𝐿2 𝑞𝜋𝑥 L 𝐿 𝑞𝜋𝑥 L 𝐿2 𝐿2 )] + ( ) [𝑥 cos ( )] = − ( ) [cos(𝑞𝜋) − 1] + ( ) cos(𝑞𝜋) ) [cos ( 𝑞𝜋 𝐿 0 𝑞𝜋 𝐿 0 𝑞𝜋 𝑞𝜋 =(

𝐿2 ) 𝑞𝜋

which yields 𝑞𝜋𝑥 𝐿 𝑛𝜋 2 𝐿 2 𝐿3 𝐷𝑛𝑞 = (𝑓𝑛 (𝑥), sin ( )) = 𝛿𝑞𝑛 [(𝑎2 ( ) + 𝑏 2 ( ) ) − 2𝑎𝑏] + 𝑏 2 𝐿 2 𝐿 𝑛𝜋 𝑛𝑞𝜋 2

(The calculation of these analytic functions is straightforward, but tedious. I checked each step against numerical calculations in order to spot errors, a process which took several hours. Thus, any speed advantage of the method over a strictly numerical approach is negated, except in circumstances where the algorithm was run many thousands of times so as to recoup the development effort).

We put this all together as follows: The vector c and matrix D are not functions of the current best estimate of the coefficients m, so compute them only once, at the top of the script. Note that we must perform an inner product to compute c, which we perform numerically. Then, inside the optimization loop, we compute d=2Dm. The derivative is then 𝜕𝐸 ⁄𝜕𝑚𝑞 = 𝑐𝑞 + 𝑑𝑞 . The gradient method can then be used to improve the estimate of m in the usual way.

m

0.2 0.1 0 0

1

2

3

4

5 x

6

7

8

9

0

1

2

3

4

5 x

6

7

8

9

2

d

1

log10(E/E0)

0

0 -2 -4 100

200

300 iteration

400

500

600

The analytic method is implemented in script gda_soln11_04b.m, with output shown above.

The top plot shows the true model (black), starting model (dashed green) and final model (solid green). Note that the true model is exactly recovered (using noise-free data). The middle plot is the observed data (black), initial predicted data (green dashed) and final predicted data (green).

The bottom row shows the logarithm of the normalized error E/E0 as a function of iteration number. Note that only 600 iterations are needed to achieve an acceptable solution, as compared to tens of thousands in the original unparameterized problem.

14.5 Suppose that the Green function, 𝐹(𝑡, 𝜏), of a particular differential equation, ℒ 𝐹 𝐹(𝑡, 𝜏) = 𝛿(𝑡 − 𝜏), depends only upon time differences; that is, 𝐹(𝑡, 𝜏) = 𝐹(𝑡 − 𝜏). Denote the Green function integral as the linear operator, (ℒ 𝐹 )−1. (A) Use the formula for the inner product to derive (ℒ 𝐹 )−1†. Call the function within this operator, 𝐹 ′ . (B) Suppose the differential equation, ℒ 𝑄† 𝑄(𝑡, 𝜏) = 𝛿(𝑡 − 𝜏), has Green function, 𝑄(𝑡, 𝜏). Denote the Green function integral as the linear operator, (ℒ 𝑄† )−1 . By comparing (ℒ 𝐹 )−1† and (ℒ 𝑄† )−1, establish the relationship between 𝐹 ′ and 𝑄. Explain your results in words.

We know: (A) The time shift invariant Green function F(t, τ)=F(t-τ) solves ℒFF(t, τ)=δ(t-τ) and corresponds to the integral

ℒ F 𝑢(𝑡) = 𝑓(𝑡) and

𝑢(𝑡) = (ℒ 𝐹 )−1 𝑓(𝑡) = ∫ 𝐹(𝑡 − 𝜏) 𝑓(𝜏) d𝜏

This is just convolution by F(t). Compute adjoint by writing down definition of inner product (say with an arbitrary function g(t)) ∫ 𝑔(𝑡) ∫ 𝐹(𝑡 − 𝜏) 𝑓(𝜏) d𝜏 d𝑡 = ∫ 𝑓(𝜏) ∫ 𝐹(𝑡 − 𝜏) 𝑔(𝑡) dt d𝜏 𝑡

𝜏

𝜏

𝑡

interchange variables t and τ ∫ 𝑔(𝑡) ∫ 𝐹(𝑡 − 𝜏) 𝑓(𝜏) d𝜏 d𝑡 = ∫ 𝑓(𝑡) ∫ 𝐹(−(𝑡 − 𝜏)) 𝑔(𝜏) dτ d𝑡 𝑡

𝜏

𝑡

𝜏

Thus (ℒF )-1† is a Green function integral corresponds to convolution by F’(t)=F(-t) ℒ F† 𝑢(𝑡) = 𝑔(𝑡) and

𝑢(𝑡) = ∫ 𝐹(−(𝑡 − 𝜏)) 𝑔(𝜏) dτ = ∫ 𝐹′(𝑡 − 𝜏) 𝑔(𝜏) dτ 𝜏

𝜏

(B) Green function Q(t, τ) solves ℒQ†Q(t, τ)=δ(t-τ) and corresponds to the integral ℒ Q† 𝑢(𝑡) = 𝑔(𝑡) and

𝑢(𝑡) = (ℒ 𝑄† )−1 𝑔(𝑡) = ∫ 𝑄(𝑡, 𝜏) 𝑔(𝜏) d𝜏

B y comparing (ℒF )-1† and (ℒQ† )-1, we find 𝑄(𝑡, 𝜏) = 𝐹′(𝑡 − 𝜏) We have discovered that the adjoint of the Green function of a differential equation is the Green function of the adjoint differential equation In the special case where the Green function of the original differential equation is time shift invariant then the Green function of the adjoint differential equation is the Green function of the original equation backward in time

14.6 The file, prob1406_data.txt, which can be generated by gdama14_11 or gdapy14_11, contains three columns, corresponding to 𝐭, 𝛿𝐦𝑡𝑟𝑢𝑒 and 𝐝𝑡𝑟𝑢𝑒 (all of length, 𝑁) in the temperature problem discussed in Sec. 14.13. The parameters associated with this problem include a source time of zero, 𝑐0 = 0.7, 𝑏 = 0.4, 𝑚0 = 0, and: 𝑀

𝑐1 (𝑡) = ∑ 𝛿𝑚𝑖𝑡𝑟𝑢𝑒 𝛿(𝑡 − 𝑡𝑖 ) 𝑖=1

where 𝑀 = 𝑁. Construct a set of synthetic data, 𝐝𝑜𝑏𝑠 , by adding random noise to 𝐝𝑡𝑟𝑢𝑒 and then use the data kernel in Eq. (11.86) to invert for 𝛿𝐦𝑒𝑠𝑡 . Compare your result with 𝛿𝐦𝑡𝑟𝑢𝑒 . You will need to add prior information of smallness or smoothness to the inversion, in order to prevent amplification of high-frequency noise.

The scripts gdasolnma14_06 and gdasolnpy14_06 reads in the data file, adds random noise to 𝐝true to create synthetic data 𝐝obs , uses the Green function method to solve the reference equation for 𝐮0 , integrates 𝐮0 to yield the corresponding data 𝐝0 , and then computes the perturbation in data δ𝐝 = 𝐝𝑜𝑏𝑠 − 𝐮0 .The data kernel 𝐆 is built column-by-column by evaluating Equation (14.86) for each model parameter.

In MATLAB®, this is performed by: G=zeros(N,M); for i=[1:M] t0 = t(i); dddm = -(b/c0)*(t>0).*(t>t0).*(1-exp(-c0*(t-t0))).*exp(-c0*t0); G(:,i)=dddm; end

and similarly, in Python, by: G=np.zeros((N,M)); for i in range(M): t0 = t[i,0]; p1=-(b/c0)*(t>0.0); p2=(t>t0); p3=(1.0-np.exp(-c0*(t-t0)))*exp(-c0*t0); dddm = np.multiply( np.multiply(p1,p2), p3); G[0:N,i:i+1]=dddm;

Note that the Heaviside function 𝐻(𝑡 − 𝑡0 ) is coded as (t>t0). Solutions 𝛿𝐦 of δ𝐝 = 𝐆𝛿𝐦 are then constructed for two types of prior information: smallness (red) and flatness (green). The strength of the prior information is chosen by trial-and-error. The solutions both match the true model 𝛿𝐦𝑡𝑟𝑢𝑒 fairly well, although in the case of smallness, it contains some high-frequency noise (indicating that the flat solution is to be preferred).

0.1

dm

0.05 0 -0.05 -0.1 0

1

2

3

4 time t

5

6

7

8

9

15.1 Modify the MATLAB® or Python script for the airgun problem (gdama15_02 or gdapy15_02) to compute a shorter filter, 𝑚(𝑡). How short can it be and still do a reasonable job at spiking the airgun pulse?

See scripts gdasolnma15_01 and gdasolnpy15_01. I modified the script to include a for loop over different filter lengths M. I started with M=513, as in the original example, and then decreased it by a factor of two for each subsequent iteration. The results g(t)*ginv(t) are shown below. The filter does fine for M=128 and reasonably well for M=64, but poorly for M<64, where two secondary pulses start to have significant amplitude.

00

20

30

30

0.5 time, t

0.6

0.7

0.8

0.9

1

0.1

0.2

0.3

0.4

0.5 time, t

0.6

0.7

0.8

0.9

1

0.1

0.2

0.3

0.4

0.5 time, t

0.6

0.7

0.8

0.9

1

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.5 0 -0.5

0.5 0 -0.5

0.5 0 -0.5

0

0.5

10

10

0.220 0.3

200.4

30

30

0 -0.5

0.1

60 0

0.5 time, t

0.6

0.7

0.9 20

0.8

1

30

20 40 60 2040 4060 60 0 20 40 40 60 0 60 020 y y y y y See MatLab script gda_soln12_02.m. The results are shown for sources and receivers on

0 20

10

10

20

40

x

x

20

30

R

20

30

50 40

60 -100 0

020 theta

10040 y

0

0

10

10

20

20

30

30

x

0

0

10

R

10

all edges (left), only on the sides (middle) and only on the top and bottom (right). traveltimes estimated model traveltimes estimated model estimated model

0

40

true model with rays

0

or 50it retains 50 of the model but omits them on 50 receivers on the top and bottom 50gdapy15_04) so that the sides. Interpret the60 results.60 60 60

50

30

truetrue model model with rays time, t

40 40 40 4015.2. Modify the MATLAB® or Python script for the tomography problem (gdama15_04

40

60

0.4

x

20

0.3

x

10

0.2

x

M=257 M=128 M=64 M=32

10

0.1

true model true model with rays 0 M=16

0

x

x

0

0.5 0 -0.5

40

40

50

50

60

60

60-1000

0 20 theta

100 40 y

60 0

20

40 y

60

The only-the-sides case correctly places the structures vertically, but smears them horizontally.

The only-the-top-and-bottom case correctly places structures horizontally, but smears them out

vertically. This make sense; in both cases, the structures are being smeared out along the ray paths.

15.3 Modify the MATLAB® or Python script for the 𝐿1 straight-line problem (gdama15_07 or gdapy15_07) to allow for data of different accuracy. Test it against synthetic data, where the first 𝑁/2 data have variance, 𝜎𝐿2 , and the last 𝑁/2 have variance, 𝜎𝑅2 , both for the case where 𝜎𝐿 = 10𝜎𝑅 and 𝜎𝑅 = 10𝜎𝐿 .

5

5

0

0

-5 -1

d

d

See scripts gdasolnma12_03 and gdasolnma12_03.m. The true line has a break in slope at z=0. The accurate half of the data have standard deviation of 0.05 and the inaccurate half, 0.40. In the left figure, the inaccurate side is on the left. In the right figure, it is on the right. In both cases, the weighted L1 fit the accurate side very well, and the inaccurate side only poorly (which is what we expect).

-0.5

0 z

0.5

1

-5 -1

-0.5

0 z

0.5

1

15.4 Modify the MATLAB® or Python script for the earthquake location problem (gdama15_14 or gdapy15_14) to include prior information that all the earthquakes occur near depth, 𝑧𝐴 . Adjust the true locations of the earthquakes to reflect this information. [Hint: See Method Summary 3 in Ch. 17 for instructions on how to implement the prior information. The problem can be solved using biconjucated gradients.]

See MatLab script gda_soln12_024.m. In the plot, below, the true earthquake locations (red) scatter about a depth zA=2.5. The ratio of standard deviation of data to a priori information is 1:100, so the estimate depth (green) are forced very nearly to that a priori depth (left plot).

Their estimated horizontal location, which are controlled by the data, are close to the true values (right plot). I assume that the origin times adjust to compensate for the depth error, but I haven’t checked that this is actually so.

0

10

-2

5

x2

x3

-4 -6

0

-5

-8 -10 10 0-10 x2

-5

0 x1

5

10

-10 -10

-5

0 x1

5

10

15.5 Modify the MATLAB® or Python script for the earthquake location problem (gdama15_14 or gdapy15_14) to use differential P wave travel time data, but no absolute travel time data. Each of these new data is the time difference between the arrival times of two earthquakes observed at a common station. Compare the locations to the results of the absolutearrival time script, in the case where the variance of the differential data is four orders of magnitude smaller than the variance of the absolute data. [Hint: You may have to use a better starting guess than was used in the absolute-arrival time script.]

See MatLab script gda_soln12_024.5. The final locations are show

0

-2

x3

-4

-6

-8

10 0

-10 -10

-5

0

5

10

-10

x2

x1

The results are: absolute: RMS model misfit: x 0.077415 y 0.082640 z 0.170481 t0 0.020014 differential: RMS model misfit: x 0.003352 y 0.003656 z 0.006697 t0 0.098725

The results are a rather hard to compare, because the earthquakes are being randomly generated, and are different for each run, but the spatial locations do seem better for the differential data.

The origin times should be ignored, for the differential times cannot resolve them.

16.1 Show that the differential operator in Eq. (16.12) is self-adjoint.

The adjoint of a matrix operator is the matrix transpose of the adjoint of each of its elements.

That the wave equation operator 𝓛 is self-adjoint (that is, 𝓛 = 𝓛𝐴 ) can be shown by examining its diagonal and off-diagonal elements, separately. The diagonal elements have the necessary property of being self-adjoint, as can be seen for the [𝓛]𝟏𝟏 case: †

𝜕2 𝜕2 𝜕2 𝜕 2 𝜕(𝜆 + 2𝜇) 𝜕 † [𝓛]𝟏𝟏 = (𝜌 2 − (𝜆 + 2𝜇) 2 − 𝜇 2 − 𝜇 2 − 𝜕𝑡

𝜕𝑥

𝜕𝑦

𝜕𝑧

𝜕𝑥

𝜕𝜇 𝜕 𝜕𝜇 𝜕 − − ) 𝜕𝑥 𝜕𝑦 𝜕𝑦 𝜕𝑧 𝜕𝑧

𝜕2 𝜕 𝜕 𝜕 𝜕 𝜕 𝜕 (𝜆 + 2𝜇) = (𝜌 2 − − 𝜇 − 𝜇 ) 𝜕𝑡 𝜕𝑥 𝜕𝑥 𝜕𝑦 𝜕𝑦 𝜕𝑧 𝜕𝑧

†

𝜕2 𝜕 𝜕 𝜕 𝜕 𝜕 𝜕 (𝜆 + 2𝜇) =𝜌 2− − 𝜇 − 𝜇 = [𝓛]𝟏𝟏 𝜕𝑡 𝜕𝑥 𝜕𝑥 𝜕𝑦 𝜕𝑦 𝜕𝑧 𝜕𝑧

Similarly, the off-diagonal elements [𝓛]𝑖𝑗 and [𝓛]𝑗𝑖 have the necessary property of being adjoints of one another. For example, in the [𝓛]12 and [𝓛]21 case: †

[𝓛]†𝟏𝟐 = (−(𝜆 + 𝜇)

𝜕2 𝜕𝜆 𝜕 𝜕𝜇 𝜕 𝜕 𝜕 𝜕 𝜕 † − − ) = (− 𝜆 − 𝜇 ) 𝜕𝑥𝜕𝑦 𝜕𝑥 𝜕𝑦 𝜕𝑦 𝜕𝑥 𝜕𝑥 𝜕𝑦 𝜕𝑦 𝜕𝑥

𝜕 𝜕 𝜕 𝜕 𝜕2 𝜕𝜆 𝜕 𝜕𝜇 𝜕 =− 𝜆 − 𝜇 = −(𝜆 + 𝜇) − − = [𝓛]𝟐𝟏 𝜕𝑦 𝜕𝑥 𝜕𝑥 𝜕𝑦 𝜕𝑥𝜕𝑧 𝜕𝑦 𝜕𝑥 𝜕𝑥 𝜕𝑦 Note that we have employed the adjoint relationship 𝑑 𝑑 † 𝑑 𝑑 ( 𝑓 ) = 𝑓 𝑑𝑥 𝑑𝑦 𝑑𝑦 𝑑𝑥 This relationship is proved by starting with the inner product (𝑣, u) and moving, in sequence, each factor of ℒ to the other side of the inner product: 𝑑 𝑑 𝑑 † 𝑑 𝑑 𝑑 𝑑 𝑑 𝑑 𝑑 (𝑣, 𝑓 𝑢) = ([ ] 𝑣, 𝑓 𝑢) = (− 𝑣, 𝑓 𝑢) = (−𝑓 𝑣, 𝑢) = ( 𝑓 𝑣, 𝑢) 𝑑𝑥 𝑑𝑦 𝑑𝑥 𝑑𝑦 𝑑𝑥 𝑑𝑦 𝑑𝑥 𝑑𝑦 𝑑𝑦 𝑑𝑥

16.2. Modify the seismic migration example (gdama16_03 or gdama16_03) to include 𝑁𝐻 = 5 heterogeneities at locations of your choosing.

In the scripts gdasolnma16_01 and gdasolnpy16_01, heterogeneities were added at the following locations:

MATLAB: % heterogeneity locations Ns = 5; xs = [-0.3, -0.1, 0.0, 0.1, 0.3]'; zs = [0.18, 0.22, 0.25, 0.22, 0.18]'; Python: % heterogeneity locations Ns = 5; xs = gda_cvec(-0.3, -0.1, 0.0, 0.1, 0.3); zs = gda_cvec(0.18, 0.22, 0.25, 0.22, 0.18); The migration process does an excellent job at recovering them.

1

0.5

0 -0.5

0

0.5