Solutions Manual for A First Course in Linear Model Theory, 2nd Edition by Ravishanker, Chi, Dey

Solutions Manual for A First Course in Linear Model Theory, 2nd Edition by Ravishanker, Chi, Dey

Solutions Manual for A First Course in Linear Model Theory, 2nd Edition by Ravishanker, Chi, Dey

Solutions Manual for A First Course in Linear Model Theory, 2nd Edition by Ravishanker, Chi, Dey

Solutions Manual for A First Course in Linear Model Theory, 2e by Nalini Ravishanker, Zhiyi Chi, Dipak Dey (All Chapters) Solutions to Chapter 1

1.1 |a • b| = | − 9| = 9, while a

b =

√ √ 6 22 ∼ = 11.489 > 9.

1.2 To verify the Cauchy–Schwarz inequality, first see that the inequality holds trivially if a and b are zero vectors. We therefore assume that both a and b are nonzero. Let c be the vector c = xa— yb, where x = b′b, and y = a′b. Clearly, c′c ≥ 0. We express c′c in terms of x and y: c′c = (xa − yb)′(xa − yb) = x2a′a − 2xya′b + y2b′b.

Since c′c ≥ 0, and using the definitions of x and y, we see that (b′b)2(a′a) − 2(a′b)2(b′b) + (a′b)2(b′b) ≥ 0 and dividing by b′b in the last inequality, we see that (b′b)(a′a) − (a′b)2 ≥ 0, which verifies the Cauchy–Schwarz inequality.

We use the Cauchy–Schwarz inequality to deduce the triangle inequality, which can be written in an equivalent form a+b

2

≤ ( a + b )2.

The expression on the left is a+b

2

= (a + b) • (a + b) = a • a + 2a • b + b • b = a

2

+ 2a • b + b

2

while the expression on the right is ( a + b )2 = a

2

+2 a

2

b + b .

Comparing these two formulas, we see that the triangle inequality holds if and only if a • b ≤ a b . By Cauchy–Schwarz inequality, |a • b| ≤ a b , so the triangle inequality follows as a consequence of the Cauchy–Schwarz inequality. The converse is also true; i.e., if the triangle inequality holds, then a• b ≤ a b holds for a and for −a, from which Cauchy–Schwarz inequality follows. If equality holds, i.e., if a • b = a b , 2 2 then b = ca, for some scalar c. Hence, a • b = c a , and a b = |c| a . For nonnull a, this implies that c = |c|, so that c ≥ 0. If b /= 0, then b = ca, with c > 0.

1.3 Since x′y = 0 = x′z = y′z, it follows that we m u √ s t solve the e q u a√t i ons a2 + b2 = 1, and 2 2 a − b = 0, for which the solutions are a = ±1/ 2 and b = ±1/ 2.

1.4 Let

,1

2 0 1 0 −2 . ..

V=.0 1 1 −1 −1 1 1

2

Solutions to Chapter 1 Using elementary column transformations C2 − 2C1, and then, C3 − C2, the matrix V becomes , 1 0 0 . 1 −2 0. , V=.0 1 0 −1 −1 0 so that the column rank of V (or the dimension of its column space) is 2 < 3. Therefore, v1, v2 and v3 are linearly dependent. It is easily seen that v1 and v2 are LIN, and that v3 = v2 − 2v1.

1.5 It is easy to see that c1v1 + c2v2 + c3v3 = 0 results in the following three equations, and the only solution is c1 = 0, c2 = 0, and c3 = 0: 2c1+ 8c2− 4c3 = 0 3c1− 6c2+ 3c3 = 0 2c1+ 5c2+ c3 = 0.

1.6 Using elementary transformations, we can show that A is equivalent to the matrix , —3 3 3 0 4 4 0 0 −1

,

so that the columns of A are LIN. 2 1 3 1.7 We can solve the system = c1 + c2 to get c1 = −1 and c2 = 1. Hence, u 3 2 5 is in Span{v1, v2}.

1.8 If v 1 , Σ · · · , vm are linearly dependent, then there are scalars c1, · · · , cm noΣ t all zero, such m that c v = 0.

For any k with c / = 0, we then have v = − i i k k i=1 i/=k (ci/ck)vi, showing property 1 in Result 1.2.2.

Suppose without loss of generality t h aΣ t v1, · · · , vs are linearly dependent and c1, · · · , cs s are constants, not all zero, such that Σ i=1 civi = 0. Let cj = 0 for all j = s + 1, · · · , n. n are not all zero and n Then c1, · · · , c i=1 civ i = 0. Hence v1, · · · , v n are linearly dependent, showing property 2 in Result 1.2.2.

1.9 1. Let S = {v1, · · · , vn }denote a set of nonzero orthogonal vectors, and let u belong to the span of S with u = c1v1 + · · · + cnvn.

For a fixed i = 1, · · · , n, take the inner product of each side with vi. Since vi • vj = 0, i /= j, u • vi = c1(v1 • vi) + · · · + cn(vn • vi) = ci(vi • vi). , Hence, ci = (u• vi) (vi • vi). To verify linear independence, set u = c1v1 +· · ·+cnvn = 0.

This implies that ci = 0, i = 1, · · · , n, which in turn implies LIN of {v1, · · · , vn}.

Solutions to Chapter 1

3

2. By definition, every v ∈ V1 +· · · +V m can be written as v1 + ····· + vm, where vi ∈ Vi, i = 1, . . . , m. Let wi ∈ Vi, i = 1, . . . , m, such that we also have v = w1 + ···· + wm. Then (w1 − v1) + ···· + (wm − vm) = 0. For each i, pre-multiply both sides by (wi − vi)′. For j /= i, since Vi ⊥ Vj, then (wi − vi)′(wj − vj) = 0. As a result, (wi − vi)′(wi − vi) = wi − vi 2 = 0, giving wi = vi. Hence by definition, the sum of the Vi’s is a direct sum.

1.10 Since {v1, ····· , vm} is a basis of V, the vectors are LIN. So from y1 = v1,

yk = vk −

k −1 Σ

y′i vk

yi, k = 2, · · · , m,

i=1

yk /= 0, implying that zk = yk yare well-defined and each has length 1. On the other k hand, for 1 ≤ j < k ≤ m, yj′ yk = y′j vk −

k −1 Σ

y′i vk

y′j yi = −

i=1

j−1 Σ

y′i vk

i=1

Σ y′iv k k−1

yi′yj −

yj′ yi.

i=j+1

If k = 2, then j = 1, and it is straightforward to see that y1′ y2 = 0. Suppose we have shown that for all i < j < k, yi′ yj = 0. Then the above identity shows that for all j < k, yj′ yk = 0. By induction, the yi’s are orthogonal to each other. Then z·1·, · , zm are orthonormal. Since they are LIN from Exercise 1.9, and there are m of them, they form an orthonormal basis of V. 1.11 We haveW∩ (W ⊥∩V) ⊂ W∩W ⊥ = {0 }. From W ⊂ V andW ⊥∩V ⊂ V, W⊕ (W ⊥∩V) ⊂ V . On the other hand, for any v∈ V , there are unique w∈ W and u ∈ W ⊥, such that v = w + u. Since w ∈ V , then u = v− w ∈ V , and so u∈ W ⊥ ∩ V . As a result ). Then V= W ⊕( W⊥ ∩ ).

V ⊂ W ⊕( W⊥ ∩ V V From the paragraph below Definition 1.2.8, dimV = dimW + dim(W ⊥ ∩ V ), completing the proof of property 1. Next, by W ⊥ ∩V ⊂ W ⊥, ( W ⊥ ∩V)⊥ ∩V ⊃ (W ⊥)⊥ ∩V = W ∩V = W. On the other hand, if v ∈ V ⊥ and v ⊥( W ∩),Vthen by property 1, there are unique w ∈ W and u∈ W ⊥ ∩ V such that v = w+u. From assumption, v⊥ u. Meanwhile w ⊥ u. Then u′u = u′(v− w) = 0, ⊥ ⊥ ⊥ ⊥ so u = 0. Then v = w ∈ W. As a result, ( W ∩V ) )∩V =W , ∩V ⊂ W . ThenW ( ∩V showing property 2.

1.12 In order for a matrix A = {aij i, } j = 1, 2, 3, to commute with the matrix B, we require that AB = BA. Computing the product on both sides, and equating them, we see that the conditions are a11 = a22 = a33, a12 = a23, and a21 = a31 = a32 = 0.

1.13 By repeated multiplication, we see that Ak =

ak 0

β , 1

where, β = b(1 + a + · · · + ak−1).

1.14 It is easily verified that the product (A − B)C is equal to Ak − Bk, where, C = Ak−1 + Ak−2B + · · · + ABk−2 + Bk−1.

1.15 Clearly, (A′A)′ = A′(A′)′ = A′A, and (AA′)′ = (A′)′A′ = AA′.

1.16 If A = O, then clearly A′A = O. To show the converse, let the column vectors of A be a1, · · · , an. Since A′A = {a′iaj}, if A′A = O, then for all i = 1, · · · , n, a′iai = 0, giving ai = 0, and so A = O.

4

Solutions to Chapter 1

1.17 (a) From Result 1.3.5, k

tr(A

Σ i=1

k

xix′i) = tr( k

=

Σ i=1

Σ

k

Axix′i) =

Σ

i=1

tr(x′iAxi) =

i=1 k

tr(Axix′i)

Σ (xiAxi).

i=1

(b) From property 4 of Result 1.3.5, it follows that tr(B−1AB) = tr(BB−1A) = tr(A).

1.18 Suppose A = {aij} and B = {bij} are both lower triangular matrices a n d Σ a i i = ai and n a b . bii = bi. Then aij = bij = 0 for all i < j. Now C = AB = {cij} with cij = k=1 ik kj If i > j, then for every k = 1, · · · , n, either i > k or k > j, so aikbkj = 0, giving cij = 0. Then C is lower triangular. If i = j, then aikbkj /= 0 if and only if k = i, so cii = aiibii = anbn. This completes the proof for the lower triangular case. The proof for the upper triangular case is similar. Σm Σ n aijbji = 1.19 Properties 1, 3, 5, and 6, of Result 1.3.5 are obvious. From tr(AB) = j=1 i=1 Σn Σm Σm b a = tr(BA), Property 2 follows.

Property 4 is a ji ij Σ j=1 i=1 i=1 aijbji = nj=1 direct consequence, since we can regard ABC as the product of AB and C, or of A and BC. Both tr(AA′) and tr(A′A) equal the sum of squares of the elements of A, which is nonnegative, and shows property 7, of which property 8 is an immediate consequence.

1.20 If A = {ak5} is lower triangular, then ak5 = 0 for all k < l. Now Mij = {mk5}, where ,a k5 ,, ak+1,5 mk5 = ak,5+1 , , , ak+1,5+1

if k < i, l < j if k ≥ i, l < j if k < i, l ≥ j if k ≥ i, l ≥ j.

If i > j, then for k < l, the second case in the display is not possible, and all the other cases are possible, each one giving value 0. Therefore, mk5 = 0, so that Mij is lower triangular. Furthermore, for any k = j,· · · , i −1, mkk = ak,k+1 = 0, so Mij has at least one diagonal element equal to zero. This verifies the first fact.

From (1.3.1), the ith diagonal element of A−1 is |Mii|/|A|. If the diagonal elements of A are d1, · · · , dn, then the diagonal elements of Mii are d1, · · · , di−1, di+1, · · · , dn. Therefore |Mii| = d1 · · · di−1di+1 · · · dn and |A| = d1 · · · dn. Then the ith diagonal element of A−1 is 1/d i.

1.21 Using Definition 1.3.7, |A| = −8.

1.22 It is easy to verify the given resursive expression, from which we obtain ∆n = (1 + a2 + a4 + · · · + a2n) = [1 − a2(n+1)]/[1 − a4].

1.23 This follows directly from property 7 of Result 1.3.6.

1.24 Using Definition 1.2.7, we can show that the value of this determinant is 1 +

Σn i=1

ai.

1.25 The determinant to be calculated is |(1 − ρ)In + ρJn|. If ρ /= 1, then from 12 of Result 1.3.6, the determinant is (1 − ρ)n|In + ρ(1 − ρ)−11n1′n| = (1 − ρ)n[1 + ρ(1 − ρ)−11′n1n] = (1 − ρ)n[1 + nρ(1 − ρ)−1] = (1 − ρ)n−1[1 + (n − 1)ρ].

Solutions to Chapter 1

5

1.26 We find the RREF(A : I) = (B : C), say. We see that the matrix , 1 2 —1 B = 0 −4 0 0 0 0 has a row of zeroes, so that A is singular.

1.27 If A is n × n and invertible, then there is an n × n matrix B such that AB = I. Then |A||B| = |AB| = |I| = 1, so |A| = / 0.

1.28 We verify Result 1.3.8. The first four properties are straightforward. See Theorem 18.2.8 in Harville (1997) for a proof of the Sherman–Morrison–Woodbury formula. To prove property 6, (A−1a)(b′A−1) 1 ± b′A−1a ab′A−1 ab′A−1ab′A−1 1 ′ − . = I ± ab A ∓ − 1 ± b′A−1a 1 ± b′A−1a

(A ± ab′) A−1 ∓

Let us denote the scalar b′A−1a by c. Hence, ab′A−1ab′A−1 = acb′A−1 = cab′A−1 = (b′A−1a)ab′A−1 so that (A−1a)(b′A−1) 1 ± b′A−1a ab′A−1 ab′A−1 ′ −1 ′ −1 a) − (b A = I ± ab A ∓ 1 ± b′A−1a 1 ± b′A−1a

(A ± ab′)

A−1 ∓

= I.

1.29 (a) Since (I − A(I + BA)−1B)(I + AB) = (I + AB) − A(I + BA)−1(I + BA)B = I + AB − AB = I. Similarly, we can show that (I + AB)(I − A(I + BA)−1B) = I, and it follows that (I + AB)−1 = I − A(I + BA)−1B. (b) Note that we can write aIk + bJk = a(Ik + b Jk) = a(Ik + b 1k1′k). Set b 1k = A and 1′k = B in (a) and use that result to get (aIk + bJk)−1 =

1 a

Ik +

b 1k1′k a

" = 1 Ik − b 1k a a =

a

a

a

−1

kb 1+ a

−1

# 1′k

b ab 1 1 J k = Ik — Ik — Jk . a a(a + kb) a a(a + kb)

1.30 (a) Use Result 1.3.6. Since A is orthogonal, |A|2 = |AA′| = I| = 1, which implies |A| = ±1. (b) This follows from Definition 1.2.9 and Definition 1.3.11. (c) AB(AB)′ = ABB′A′ = AA′ = In. 1.31 Since r(A) = 1, A can be written as ab′ for some a /= 0 and b = / 0. Then by property 12 of Result 1.3.6,| In +A| = 1+b′a = 1+tr(b′a), which by Result 1.3.5 is 1 +tr(ab′) = 1 + tr(A).

6

Solutions to Chapter 1

1.32 It is easy to see that the first two columns of A are LIN, while the third column is the first column times 2 minus the second one. Then the dimension of the column space is 2. We can see that , 1 0 2 0 1 −1 .

RREF(A) = .

0 0 0 0 0 0 so the dimension of the row space of A is also 2. Then dim(N (A)) = 3 − 2 = 1.

1.33 Using row transformations R2 − 3R1, R3 − 5R1, and then R3 − 2R2, A , 1 2 4 3 0 −7 −10 −11 , so that r(A) = 2.

0 0 0 0

'

1.34 Let c be an eigenvalue of A ∈ Rn×n with geometric multiplicity g ≥ 1. Then there are LIN vectors v1, · · · , vg such that Avi = cvi. Expand {v1, · · · , vg} into a basis cIg B . Then {v1, · · · , vn} of Rn and let V = (v1, · · · , vn). Then V−1AV = O A2 (c − λ)Ig B A λI = V−1AV λI = = (c − λ)g|A2 − λIn−g|, showing | − n| | − n| O A2 — λIn−g the algebraic multiplicity of c is at least g.

1.35 SΣu p p o s e vi, wi ∈ Vi, i = 1, . . . , k, such that v1 + · · · + vk = w1 + · · · + wk. Then k i=1 (vi − wi) = 0. Let ui = vi − wi. If not all ui are 0, then let ui1 , · · · , ui m be all those that are not 0. Each uij is an eigenvector corresponding to λij . On the other hand, ui1 + · · · + uim = 0, so ui1 , · · · , uim are linearly dependent. This contradicts 1.3.17. As a result, for all i, vi = wi. From Definition 1.2.8, the sum of V1, · · · , Vk is a direct sum. k 1.36 Let v= / 0 be an eigenvector corresponding to λ, i.e., Av = λv. Then for k ≥ 2, A v = Ak−1(Av) = Ak−1(λv) = λAk−1v. By induction, Akv = λkv, so λk is an eigenvalue of Σp Ak. Note that with A0 = I, this is also true for k = 0. As a result, if f (x) = akxk, k=0 Σp Σp Σp then f (A)v = ( k=0 ak Ak)v = k=0 ak Akv = k=0 akλ kv = f (λ)v. Therefore f (λ) is an eigenvalue of f (A), proving (a). If A is invertible, then Av = λv yields v = λA−1v, and hence A−1v = λ−1v, proving (b). Σn 1.37 Let |λIn − A| = anλi. By Vieta’s formulas, the eigenvalues λ1, · · · , λn have the Σn i=0 Qn property that i=1 λi = −an−1 /an, and i=1 λ i = (−1)na 0/a n. It is easy to see that an = 1, an−1 = −tr(A), and a0 = | − A| = (−1) n | AQ |. Then properties 1 and 2 of Result n 1.3.19 follow. Property 3 follows from |λIn ± A| = i=1 (λ ± λi) with λ = 1. Σ∞ k (k) 1.38 Let A = {aij} be n×n. Denote A = {a ij}. It is clear that the con vergence of k=0 Ak implies Ak → O. To show that the converse is true, denote A = max1≤i,j≤n |aij|.

If Ak → O, we can fix k0, such that Ak0 = b < 1/(2n). Then for m ≥ 2, Σn (mk0−k0) (k0) Amk0 = maxi,j | k=1 aik a kj | ≤ Amk0−k0 · (nb) < Amk0−k0 /2. By inmk0 m−1 duction, A ≤ (1/2) b. Let C = max0≤j<k Aj . Then for k ≥ k0, writk ing k = mk0 + j, m ≥ 1, 0 ≤ j < k0,ΣA∞ = Amk0 Aj ≤ Amk0 (nC) ≤ m −1 k /k 0 k (nCb)(1/2) ≤ (4nC b)(1Σ /2) . Hence k=0 A (absolutely) converges. Let the sum be M. Then M = In + k∞=1 Ak = In + AM, giving (In − A)M = I. Then In − A is nonsingular with inverse M. , 1 1 1 1.39 Let B = A — aI3 = 1 1 1 = J3, which clearly has rank 1. It is easy to verify 1 1 1

Solutions to Chapter 1

7

that solving | B − λI| = (1 − λ)3 −2(1 −λ) + (1 + λ) = 0, we get λ = 3, and λ = 0 (with multiplicity two) as the eigenvalues of b. The eigenvector corresponding to λ = 3 is obtained as v = t(1, 1, 1)′, for arbitrary t /= 0.

1.40 Let λ denote a nonzero eigenvalue of AB. Then, there exists a nonnull vector u such that ABu = λu. Hence, BABu = λBu. Clearly v = Bu is a nonzero vector and BAv = λv, so that λ is an eigenvalue of BA. To show the converse, we just reverse the roles of A and B. 1.41 Since BB−1 = I, and |B−1| = |B|−1, |A − λI| = |B||A − λB−1B||B−1| = |BAB−1 − λI|.

Since A and BAB−1 have the same characteristic polynomial, they have the same eigenvalues. Let Avj = λjvj, where vj /= 0. Then BAB−1(Bvj) = BAvj = λj(Bvj), Note that Bvj /= 0. Then Bvj is an eigenvector of BAB−1 corresponding to the eigenvalue λj.

2 1.42 (a) Given any x and y, (tx − y)′A(tx− y) = (x′Ax)t − 2(x′Ay)t + y′Ay = f (t) is ′ a nonnegative function in t. If x Ax > 0, then f (t) is a quadratic function and its minimum is— (x′Ay)2/(y′Ay) + x′Ax≥ 0, giving (x′Ay)2≤ (x′Ax)(y′Ay). If x′Ax = 0, then from the assumption, x = 0, and in order for f (t) to be nonnegative, there must be x′Ay = 0, again showing the inequality.

To complete the proof of (a), it only remains to show that if (x′Ay)2 = (x′Ax)(y′Ay) and x = 0, then y = cx for some scalar c. However, from the above argument, if the equality holds, then there is c such that f (c) =−(cx y)′A(cx − y) = 0, so by the assumption, y = cx, as desired.

2 2 (b) Let z = A−1y. Then from (a), (x′y)2 = (x′Az)2 = ≤ (x′Ax) (z′Az) 2 ′ −1 2 (xAx) (y A y) . The necessary and sufficient condition of equality follows that from (a).

Solutions to Chapter 2

2.1 We see that A′A is an m × m matrix and P′ P = I =

A′ B′

A

B =

A′A B′A

A′B B′B

=

Im O

O In−m

,

so that A′A = Im. Then (AA′)2 = A(A′A)A′ = AImA′ = AA′, so AA′ is idempotent.

2.2 Since r(A) = r(M) = r and A is r × r, A is nonsingular. Consider Result 2.1.3. By 1 1 using a similar argument, r(M) = r(A) + r(D − CA− B). Then r(A− CA− B) = 0.

1 − Since a matrix has rank 0 if and only if it is O, then D = CA B. 1 1 2.3 By Result 2.1.3, | |A =| P− xx|′ =| P|| 1− x′P− x |, so that (1 − x′P− x)| P| = |P − xx′ |, from which the result follows.

I −B 2.4 The solution to (a) follows directly from Result 2.1.1. By Result 2.1.3, is O I nonsingular. Then by property 5 of Result 1.3.11 followed by Result 2.1.1, the solution to (b) follows:

r

A CA

AB D

A AB I −B CA D O I A 0 =r = r(A) + r(E).

CA E =r

2.5 This is immediately obvious from Example 2.1.3 by seeing that when A2 is an n × 1 matrix, the expression A′2(I − P1)A2 is a scalar.

2.6 By Result 2.1.4, we can write −λIn B

−A = |Ik||(−λIn) − (−A)IkB| Ik = | − λIn + AB| = |AB − λIn|.

Applying the same result again, we have −λIn B

−A = | − λIn||Ik − B(λIn)−1(−A)| = (−λ)n|Ik − λ−1BA| Ik = (−λ)n| − λ−1(−λIk + BA)| = (−λ)n(−λ)−k|BA − λIk| = (−λ)n−k|BA − λIk|.

Hence, (−λ)n−k|BA − λIk| = |AB − λIn|, proving (a). To prove (b), let λ be a nonzero eigenvalue of the matrix AB, so that we have P (λ) =| AB− λIn | = 0, where P (λ) is the characteristic polynomial of AB. Using (a), this implies |AB − λIn| = (−λ)n−k|BA − λIk| = 0, 9

10

Solutions to Chapter 2 i.e., λ is a non-zero eigenvalue of the matrix BA. Hence, each of the k non-zero eigenvalues of AB is an eigenvalue of BA. Further, an inspection of the characteristic polynomial shows that the n× n matrix AB has (n − k) zero eigenvalues. We now prove (c). Let v denote a nonzero eigenvector of AB corresponding to /λ = 0. Then, ABv = λv, or BA(Bv) = λ(Bv), which implies that Bv is an eigenvector of BA corresponding to λ.

2.7 Clearly, r(A) = r(aa′) = 1, so that A has one nonzero eigenvalue. Since Aa = aa′a = (a′a)a, it follows that the nonzero eigenvalue of A is a′a with corresponding eigenvector a (see Definition 1.3.16).

2.8 Note that we can write (A + BC) = A(Ik + A−1BC), so that |A + BC| = |A||Ik + −1 A−1 BC|. Applying Result 2.1.4 to the matrix Ik −A B , we see that its deC In terminant is equal to |Ik||In + CA−1B| = |In||Ik + A−1BC|, so that |A + BC| = |A||In + CA−1B|.

2.9 Let J be the Jordan 1canonical form of A so that A = UJU−1. Then for k ≥ 2, − ) · · · (UJU−1) = UJkU−1. This also holds for k = 0. As Ak = (UJU−1)(UJUΣ Σp Σp p a result, if f (x) = akxk, then f (A) = akAk = akUJkU−1 = k=0 k=0 k=0 Σp U( k=0 ak Jk)U−1 = Uf (J)U −1, so f (A) and f (J) have the same eigenvalues, counting multiplicity. Now J is an upper triangular matrix with diagonal entries equal to the eigenvalues λ1, · · · , λn counting multiplicity. From Result 1.3.3 and induction, Jk is an upper triangular matrix with diagonal entries λk, ·1 · · , λkn, and so f (J) is an upper tria Q ngular matrix with diagonal entries f (λ1), · · · , f (λn). As a result, n |f (J) − λIn| = k=1(f (λk) − λ), yielding that the eigenvalues of f (J), and hence those of f (A) are f (λ1), · · · , f (λn) counting multiplicity. If A is nonsingular, then A−1 = (UJU−1)−1 = UJ−1U−1. From Result 1.3.9, J is upper triangular with diagonal entries 1/λ1, · · · , 1/λn. Then the eigenvalues of A−1 are 1/λ1, · · · , 1/λn, showing (b).

2.10 We can write (A + aa′) = A(Ik + A−1aa′), so that |A + aa′| = |A||Ik + A−1aa′|.

Using Result 2.1.4, and substituting Ik, −A−1a, a′, and 1 for A11, A12, A21, and A22 respectively, we see that |A + aa′| = (1 + a′A−1a)|A|.

2.11 From Result 2.3.4, A = PDP′, where P is an n × n orthogonal matrix, and D = diag(λ1, · · · , λn) with λi being the eigenvalues of A. Denote the orthonormal column vector of P by p1, · · · , pn. Then for any x, ,

p′1x n Σ . . x′Ax = x′PDPx = (x′p1, · · · , x′pn)D . = λi(p′ix)2. i=1 ′ p nx Σn 2 Then x′Ax ≥ λmin i=1(p′x) = λminx′x, with “=” if and only if p′xi = 0 for all p′ i i with λi > λmin. That is, the equality holds if and only if x is an eigenvector corresponding to λmin. Let x1 be an eigenvector corresponding to λmin. Then x′1Ax1/x′1x1 = λmin ≤ x′Ax/x′x with equality if and only if Ax = λminx. The other half the question can be similarly proved.

2.12 (a) By Result 2.3.4, r(A) = r(D), which is equal to the number of nonzero elements of the diagonal matrix D. (b) A 2 = tr(A′A) = tr(A2) = tr(APP′APP′) = tr(P′APP′AP) = tr(D2) = Σn 2 i=1 λ i .

Solutions to Chapter 2 2.13 For any matrix A, since gj ≤ aj for each λj, j = 1, · · · , k, it follows that Σ Σ k aj. Since the characteristic polynomial is of degree n, we must have k j=1

11

Σk j=1 j=1

gj ≤

aj ≤ n.

Σk j=1 Σk Σk By aj = n, from which it follows that j=1 gj = j=1 aj = n. Then, Σ kResult 2.3.4, j=1 (aj − gj) = 0, and aj − gj ≥ 0, j = 1, · · · , k. We must have aj = gj, j = 1, · · · , k.

2.14 Let Q = (q1, · · · , qn). Since Q is nonsingular, the q’s are LIN. Then Q−1AQ = diag(λ1, · · · , λn) if and only if AQ = Q diag(λ1, · · · , λn), which, by comparing the column vectors on both sides, is equivalent to Aqj = λjqj, j = 1, . . . , n, i.e., the q’s are the eigenvectors of A. This shows property 1. To show 2, since P is orthogonal, it is nonsingular. Then the proof immediately follows from property 1.

2.15 (a) Since x′x is a scalar, it is easy to verify that xx′ x′x

′

=

xx′ , x′x

which shows symmetry, while idempotency holds since xx′ x′x

2

=

(x′x)xx′ xx′ xx′xx′ = = . (x′x)2 x′x (x′x)2

(b) xx′ is clearly symmetric; however, it is not idempotent unless x′x = 1.

2.16 Since the column vectors of U are orthonormal, we can construct an n × (n − k) matrix V such that the columns of (U V) form an orthogonal basis for Rn. Then ′

U V A U V =

U′AU U′AV .

V′AU V′AV

By AU = UB, U′AU = U′UB = B, V′AU = V′UB = O, and since A is symmetric, U′AV = (V′AU)′ = (V′UB)′ = O. 2.17 Since A is symmetric and idempotent, we have A = AA′. For any matrix A, we know that AA′ is n.n.d.

2.18 Suppose A is an m× n matrix. For λ/ = 0, let p(λ) =| AA′ − λIm | = ( − λ)m Im | − 1 n 1 ′ n 1 − ′ ′ − λ AA ,| and q(λ) = A| A − λIn | = ( − λ) In| − λ A A = | ( λ) − Im| − λ− AA′ | (using Exercise 2.8). When m = n, p(λ) = q(λ). Hence, the eigenvalues coincide.

2.19 For λ /= 0, let p(λ) =| AB− λIm| = (− λ)m| Im − λ−1AB |, and q(λ) = | BA − λIn| = n 1 ( —λ)n |In −λ−1BA =| ( λ) Exercise 2.8. If m > n, then AB is − Im| λ−− AB , using | singular, and p(0) = 0. If m < n, then BA is singular and q(0) = 0. When m = n, p(0) = |AB |= A| B|| =| BA| = |q(0). This proves (a). To show (b), since λ is an eigenvalue of AB with eigenvector x, we have ABx = λx. Now, BAy = BABx = Bλx = λBx = λy, so that y is the eigenvector of BA corresponding to λ.

2.20 Let p1, · · · , pn ∈ Rn be the orthonormal eigenvectors of A corresponding to its eigenvalues λ1, · · · , λn. For any v with v = 1, let yi = v′pi, i = 1, . . . , n. It follows that ! n n n Σ Σ Σ 2 pip′i yi2 = v′pip′iv = v′ v= v =1 i=1

i=1

i=1

12

Solutions to Chapter 2 and that n Σ

v′Av = v′

! λipip′

i

n Σ

v=

i=1

λiy2i .

i=1

For k = 1, . . . , n, define the vector spaces Pk = Span{p1, · · · , pk}.

Let V be any subspace of dimension n − k + 1, i.e., dim(V) = n − k + 1. Since dim(Pk) + dim(V) = n + 1, there exists one nonzero vector v∈ P k ∩ V . For such v, we have yi = v′pi = 0 for i = k + 1, . . . , n, and therefore, n

v′Av =

Σ

k

2i

λiy =

Σ

i=1

k

λiy ≥ λk 2i

Σ

i=1

y2i = λk,

i=1

which implies that max

v∈V: v =1

v′Av ≥ λk.

Since V is an arbitrary subspace of dimension n − k + 1, we have v Av ≥ λk. max dim(V inf )=n−k+1 v∈V: v =1 ′

To show equality, take V = Span{pk, . . . , pn} (Note dim(V) = n − k + 1), then for any v ∈ V s.t. v = 1, we have yi = v′pi = 0,

i = 1, . . . , k − 1,

and therefore, n

v′Av =

Σ i=k

n

λiy2i ≤ λk

Σ

yi2 = λk.

i=k

This proves the min-max characterization. The max-min characterization can be similarly proved.

2.21 Given that QAQ−1 = D, where D is a diagonal matrix, the result follows by inverting both sides, so that (QAQ−1)−1 = D−1, i.e., QA−1Q−1 = D−1. Σr 2.22 (a) Let λ1, · · · , λr denote the non-zero eigenvalues of A, so that tr(A) = λi. i=1 Σr Σr Σr i=1 λi2 = p ≥ 0. i=1 λ2i ≥ 0, which implies that 2 i=1 λ i = Then, tr(A2) = 2 Σr Σ p p r 2 2 (λ rλ = p — r = p(1 − r). Since p ≥ 0, this Now,0 ≤ i=1 ip — λ) = i=1 λ i − implies that 1 − r ≥ 0, i.e., 0 ≤ p ≤ r, i.e., 0 ≤ tr(A) ≤ r. Σr (b) 2 Use Result 2.3.7. We are given that tr(A) Now, i=1 (λi − λ) = Σr= r λ= r(A). p i = r r = λ = p(1 − r ) = p(1 − r ) = 0, i.e., λ i i=1 r r = 1, i = 1, · · · , r. That is, r eigenvalues of A are equal to 1, while the remaining n — r are zero, so A is idempotent of rank r.

2.23 Suppose that on the contrary, C is nonsingular, i.e., r(C) < q. If the columns of C are linearly dependent, there exists a nonzero q-dimensional vector d such that Cd = 0.

Since B is symmetric and idempotent, (BA2d)′(BA2d) = d′A′2BA2d = d′Cd = 0,

Solutions to Chapter 2

13

which implies that (see Exercise 2.10) BA2d = 0. It follows that A2d = (I − B)A2d = A1(A′1A1)−1A′1A2d = A1e, say. Since the columns of A1 are LIN of the columns of A2, d must be 0. This implies that the columns of C must be LIN. S ince C is symmetric, its rows must be LIN too. Thus, r(C) = q, and C is nonsingular.

2.24 Since P is orthogonal, we have PP′ = In = P′P; also since A is symmetric and idempotent, we have A′ = A, and A2 = A. Now, (P′AP)′ = P′A′(P′)′ = P′AP, and (P′AP)2 = P′APP′AP = P′AP, by the orthogonality of P and the symmetry and idempotency of A. 2.25 To show that A is p.d., we can show that all the principal minors of A are positive, 2 −1 i.e., a11 = 2 > 0; = 1 > 0; and |A| = 3 > 0. To find a matrix B such that −1 1 A = BB′, we see by the sequence of elementary row and column transformations R1/2; R2 + R1, R3 + R1; R3 − R2 that A is reduced to , , 1 −1/2 −1/2 1/2 0 0 RA = 0 1/2 1/2 , where R = 1/2 1 0 .

0 −1 1 0 0 3 , 1/2 Verify that RAR′ = D =

0

0 0 1/2 0 , with |D| = 3/4. Let Q = D−1/2 = 0 3

0 2 √0 0 0 2 0 and let P = QR. Then, PAP′ = I3 and A = P−1(P′)−1, so 0 0 1/ √ 3 , √ 2√ 0√ 0 that B = P−1 = −1/√2 1/√2 √0 . Alternately, we set P (λ) = |A − λI| = 0, 3 −1/ 2 1/ 2 and solve for the eigenvalues as λ1 = 0.3446, λ2 = 1.7892, and λ3 = 4.8662; since all the eigenvalues are positive, A is p.d. The Cholesky decomposition of A, which may be implemented using standard software gives the matrix B. ,√

2.26 The eigenvalues of A are [1 + (n − 1)a] and (1 − a) with multiplicity (n − 1). Since the eigenvalues of a p.d. matrix must be positive, A is p.d. for −1/(n − 1) < a < 1.

2.27 Let the smallest eigenvalue of B be β and the largest eigenvalue of A be α. Then β > 0 because B is p.d. Then, there is λ > 0 such that β > λα. By Exercise 2.11, for any nonzero x, x′(B − λA)x = x′Bx − λx′Ax ≥ β′x − λαx′x > 0. Therefore, B − λA is p.d.

2.28 (a) Setting x = ei gives e′iAei = aii = 0. Then setting x = ei + ej, i = j, gives (ei + ej)′A(ei + ej) = aii + ajj + 2aij = 0, and so aij = 0. Then A = O. (b) If A is p.s.d., there exists an n× n matrix R such that A = RR′. Let b = R′x.

Then 0 = x′Ax = x′R′Rx = b′b, so that b = 0. Then Ax = BR′x = Bb = 0.

14

Solutions to Chapter 2

2.29 Substituting for c, the first term on the right side can be written as [x − (A + B)−1(Aa + Bb)]′(A + B)[x − (A + B)−1(Aa + Bb)] = x′(A + B)x − x′(A + B)(A + B)−1(Aa + Bb) — (Aa + Bb)′(A + B)−1(A + B)x + (Aa + Bb)′(A + B)−1(A + B)(A + B)−1(Aa + Bb) = x′(A + B)x − x′(Aa + Bb) − (Aa + Bb)′x + a′A′(A + B)−1Aa + a′A′(A + B)−1Bb + b′B′(A + B)−1Aa + b′B′(A + B)−1Bb. The second term on the right side is (a − b)′A(A + B)−1B(a − b) = a′A(A + B)−1Ba − a′A(A + B)−1Bb — b′A(A + B)−1Ba + b′A(A + B)−1Bb. Since B(A + B)−1A is symmetric and equal to A(A + B)−1B, adding the two terms gives the right side x′(A + B)x − x′(Aa + Bb)− (Aa + Bb)′x + a′Aa + b′Bb, which is equal to the left side.

2.30 (a) Use property 5 of Result 1.3.8 and simplify. (b) Substitute for (A−1 + C′B−1C)−1 from (a) into the LHS of (b) and simplify, i.e., LHS = {AC′ − AC′(B + CAC′)−1CAC′}B−1 = AC′{In − (B + CAC′)−1CAC′}B−1 = AC′(B + CAC′)−1{(B + CAC′) − CAC′}B−1 = AC′(B + CAC′)−1 = RHS. 2.31 Suppose there exists an n × n nonsingular matrix P such that P−1AiP = Di, i = 1, · · · , k, for some diagonal matrices D1, · · · , Dk. For j /= i = 1, · · · , k, P−1AjAiP = P−1AjPP−1AiP = DjDi = DiDj = P−1AiPP−1AjP = P−1AjAiP, so that AjAi = P(P−1AjAiP)P−1 = P(P−1AiAjP)P−1 = AiAj. 2.32 Suppose r(A) = a, and r(B) = b. Since A is a symmetric, n.n.d. matrix, there exists a nonsingular matrix T such that T′AT = D =

O

O

O

.

F , where E, F and G are respectively a × a, a × (n − a) and F′ G (n − a) × (n − a) matrices. Since B is a n.n.d. matrix with r(B) = b, it follows that T′BT is a n.n.d. matrix with r(T′BT) = b. Suppose r(E) = r, where r ≤ a, and r ≤ b.

Let T′BT =

E

Ia

Solutions to Chapter 2

15

We perform elementary row and column operations on the last r rows and columns of T′BT; suppose we denote them by S′ and S respectively. We reduce T′BT to , Ca O O S′T′BTS = O Ic O O O Id with r(Ca) = r, c = b − r, and d = r − c. We also have S′T′ATS =

Ia

O

O

O

.

Since Ca is a symmetric matrix with r(Ca) = r, there exists an orthogonal matrix Qa such that Q′aCaQa = Da = diag(λ1, ····· , λr, 0, ···· , 0). Define , Qa O O Qa O .

R= O Ic O = O Ir O O Id From the above results, we see that

,

R′S′T′BTSR =

Da O O

O O Ic O O O

= ∆,

Ia O

O O

which is a diagonal matrix. Then, R′S′T′ATSR =

Q′aQa O O O

=

= D,

which is also a diagonal matrix. We have constructed a matrix P = TSR, such that P′AP = D, and P′BP = ∆.

2.33 The result follows directly by using the properties of Kronecker product and trace.

2.34 If possible, let P1 and P2 be two such matrices. Since u is unique, it follows that (P1 − P2)y = O for all y ∈ Rn, so that P1 must be equal to P2.

2.35 Since Ω is symmetric, (y − Xβ)′Ω(y − Xβ) = y′Ωy − 2β′(y′ΩX)′ + β′(X′ΩX)β.

Then by Results 2.7.2 and 2.7.3, ∂(y − Xβ)′Ω(y − Xβ)/∂β = −2(y′ΩX)′ + 2(X′ΩX)β = −2X′Ω(y − Xβ) and also ∂2(y − Xβ)′Ω(y − Xβ)/∂β∂β′ = 2X′ΩX. 1 1 1 2.36 First, from property 6, property 11 follows from (A ⊗ B)(A− ⊗ B− ) = (AA− ) ⊗ 1 (BB− ) = In ⊗ Im = Imn. Next, from Result 2.2.6, A = UJU−1 and B = VKV−1, where U and V are nonsingular, and J and K are Jordan canonical forms of A and B, respectively. Then by properties 6 and 11, A ⊗ B = (U ⊗ V)(J ⊗ K)(U ⊗ V)−1. It is straightforward to check that the diagonal elements of J ⊗ K are exactly all possible pairwise product λiγj of the eigenvalues of A and B, allowing for multiplicity. Then property 9 follows. Finally, since the determinate of a square matrix is the product of all its eigenvalues, counting for multiplicity, property 10 follows from property 9.

Solutions to Chapter 3

3.1 (a) r(A) = 2, and A− =

−1

2 0

.

1 −1 0 , 1/7 2/7 0 3/7 −1/7 0. (b) r(A) = 2, and A− = . . .

0 0 0 0 0 0

3.2 G is a g-inverse of A if and only if it has the form

u′

a

In−1

v

, where u, v are any

vectors in Rn−1 and a ∈ R. 3.3 Given that KAKB−KAK = KAK, and KK = K, it follows that KAK(KB−K)KAK = KAKB−KAK = KAK = B. 3.4 Since GA = AG, for any k ≥ 1, AkGkAk = (AGA)k = Ak. 3.5 We use Result 3.1.9. (a) By property 1 of Result 3.1.9, G′ is also a g-inverse of A′A. The result follows by applying this to property 2 of Result 3.1.9. (b) Transpose both sides of (a); AG is a g-inverse of A′. (c) Transpose (3.1.7). (d) Since AGA′ is invariant to G (by property 3 of Result 3.1.9), AG′A′ = AGA′. (e) (AG′A′)′ = AGA′ = AG′A′ by (d).

3.6 If B−A− is a g-inverse of AB, then by (3.1.1), ABB−A−AB = AB. Premultiply both sides by A− and post-multiply both sides by B−; we see that A−ABB− is idempotent.

3.7 A(A−C)(A−C)′A = AA−CC′(A−)′A = AA−A(A−)′A = A(A−)′A = (AA−A)′ = (A)′ = A. 3.8 From Result 3.1.6, since G is a g-inverse of A, then r(A) ≤ r(G). Now A is a ginverse of G, then r(G) ≤ r(A), so r(A) = r(G). Conversely, from Result 3.1.3, if A = BDC, where B and C are nonsingular and D = diag(Ir, 0) with r = r(A), then I K G = C−1EB−1, where E has the form r . From Result 2.1.3, r(G) = r(E) = L M r + r(M — KL), so if r(G) = r, then M = LK. It is direct to check that EDE = E. As a result, A is a g-inverse of G. Finally, observe that the g-inverse obtained by the algorithm in Result 3.1.2 has the same rank as A. 3.9 If AG is symmetric, denote it by P. We have that PA = A and for any u⊥ C (A), by P = G′A′, Pu = G′A′u = 0. Then, P is the orthogonal projection matrix ontoC (A).

The converse is clear. This proves (a). To show (b), note that the projection matrix is P = A(A′A)−A′. Therefore, AG = P if and only if A[G— (A′A)−A′] = 0. Then, (b) follows from Result 3.2.4.

17

18

Solutions to Chapter 3

3.10 We will show that A having a commuting g-inverse implies (a), which in turn implies (b), which in turns implies that A has a commuting g-inverse.

If AGA = A and AG = GA, then GA2 = A, so r(A) ≤ r(A2). Since it always holds that r(A2) ≤r(A), then (a) follows. Suppose r(A) = r(A2) = r. We first show 2 that C(A) ∩(A) since (A) (A ) and now they have the same N = 0 .{ Indeed, } R ⊂R dimension, the two spaces are equal, so by property 5 of Result 1.3.10, A = CA2 for some matrix C. If v = Au ∈ C(A) ∩ N (A), then v = CA2 u = CAv = 0. By Result 1.3.10, dim[C(A)] + dim[N (A)] = n. Then Rn = C(A) ⊕ N (A). It follows that, if U = (u1, · · · , ur) is a matrix of basis vectors of C(A), and V = (v1, · · · , vn−r) a matrix of basis vectors of N (A), then P = (U, V) is nonsingular. Now each column of AU is in C(A) so it has a unique linear combination of u1, · · · , ur. As a resu lt, AU = UA1 A1 O for some r × r matrix A1. Then, AP = (AU, AV) = (UA1 O) = P O O . Let B = P−1 and note that r(A1) = r so A1 is nonsingular. Then (b) follows. Finally, if (b) holds, then B−1 diag(A1−1 O)B is a commuting g-inverse of A. A shorter proof based on Jordan canonical form that (a) implies (b) is as follows. A = B−1JB, with J = diag(J1, · · · , Ja) and each Jk = {cij} such that cii = a k, ci,i+1 = 1, and all other cij = 0. If ak /= 0, then Jk is nonsingular so r(Jk) = r(J2). If ak = 0, k Σ r(J2k), it then unless J k is 1 × 1, r(Jk2) = r(J k) − 1. Since Σ r(Jk) = r(A) = r(A2) = follows that all the Jk with diagonal entry 0 are 1 × 1, which leads to (b).

3.11 By Result 2.3.4, A = P diag(D, O)P′ where P is an orthogonal matrix and D = dia g(d1, · · · , dr) with di /= 0. For an arbitrary symmetric submatrix C, verify that D−1 O ′ P O C P is a symmetric commuting g-inverse of A. 3.12 AG2x = AG2AG1x = AG1x = x.

3.13 A = CB = (P diag(D1−1, O)P′)(P diag(D1, O)P′) = P diag(Ir, O)P′, where D1 = diag(λ1, · · · , λr). Since p′iP = e′i, where ei is the ith standard basis vector, we have ( e′iP′ = p′i if i ≤ r p′A = e′ diag(I , 0)P′ = . r i i 0 else 3.14 Let Ω = N (A′). Then Ω ⊂ Rm. By Result 2.6.1, PΩ = Im − PΩ⊥ , while by property 2 of Result 1.3.10, Ω⊥ = C(A). The proof then follows from Result 3.1.13.

3.15 This follows from property 2 of Result 3.1.9.

3.16 Let K = (A, B). Then M = K′K. Let G = (A′A)−. From Result 1.3.13, in order to prove that G O −GA′B + S− −B′AG I , O O I ` ˛ ¸ x ` ˛ ¸ x C

D

is a g-inverse of M, it suffices to show that KCK′K + KDK′K = K. Now KCK′ = (A, B)

G O O O

A′ B′

=P

Solutions to Chapter 3

19

where by Result 3.1.9, P = A(A′A)−A′ is the orthogonal projection matrix onto the column space of C(A). Then, KCK′K = P(A, B) = (A, PB) Next, K

−GA′B −GA′B = (A B) = −PB + B = (I − P)B := E, I I

while S = B′B − B′PB = B′(I − P)B = E′E. Then, KDK′ = [(I − P)B][B′(I − P)B]−[(I − P)B]′ = E(E′E)−E′.

Note that E′K = B′(I − P)(AB) = (O, E′E). Then KDK′K = E(E′E)−E′K = (O, E(E′E)−E′E) = (O, E).

Then, KCK′K + KDK′K = (A, B) = K. 3.17 (a) Since A is nonsingular, G is a g-inverse of AB ⇐⇒ ABGAB = AB ⇐⇒ B(GA)B = B ⇐⇒ GA = B− ⇐⇒ G = B−A−1. (b) The solution for (b) is similar.

3.18 We just need to verify that (3.1.1) holds. We have A CA

AB D

=

Ir A Ir C

B +

O O = F + G. O E

Denote L=

A− O , O O

K=

−B E− −C, In−r Ir

Note that FK, KF, GL, and LG are all null. Then, (F + G)(L + K)(F + G) = (FL + GK)(F + G) = FLF + GKG. It is easy to verify that FLF = F and GKG = G, completing the proof.

3.19 Using Result 3.2.2, the system is consistent since r(A) = r(A b) = 3.

3.20 The unique solution is (−1, −4, 2, −1)′.

3.21 From Result 3.2.8, c′x is invariant to the solution to Ax = b if and only if c′ = t′A for some t. The latter is equivalent to c′A−A = c′.

3.22 C(c) ⊂ C(A) and R(b) ⊂ R(A); use Result 3.2.8 3.23 The partial derivatives of the Lagrangian function L(x, λ) with respect to x and λ are ∂L(x, λ)/∂x = 2Vx − 2c + Aλ ∂L(x, λ)/∂λ = A′x − b and lead to the set of equations given in (3.3.6). It is easy to verify that these linear equations in x and λ are consistent. Solving these, we get the constrained minimizer x∗, while λ∗ is the corresponding value of the vector of Lagrangian multipliers.

Solutions to Chapter 4

4.1 By property 8 of Result 1.3.11, r(X′X, X′y) ≥ r(X′X). Also, r(X′X, X′y) = ′ ′ ′ r[X (X, y)] ≤X = r(X X), using properties 4 and 6 of Result 1.3.11. Hence, r(X′X, X′y) = r(X′X).

4.2 (a) This is a direct consequence of property 2 of Result 3.1.9. (b) (XGX′)′ = XG′X′ = XGX′ by property 3 of Result 3.1.9, so that XGX′ and hence I − XGX′ are symmetric. Again by property 2 of Result 3.1.9, we show that XGX′ is idempotent, and it follows that I − XGX′ is idempotent. ΣN ΣN ΣN 4.3 θ^ = 2 i=1 ti2Yi/ i=1t4,i and Var(θ^) = 4σ2/ i=1 t4i. ΣN ΣN ΣN 4.4 v^ = i=1 Siti/ i=1t2,i and Var(v^) = σ2/ i=1 t2i. ^ = (0, 3, 1)′, and σ^2 = 4. The least squares estimates of β1, β2, β3, β1 − β2 and β1 + β3 4.5 β are respectively 0, 3, 1, −3 and 1.

2n1 + n2 n1 − 2n2 4.6 Since X′X = ,^ β and β^ will be uncorrelated if n = 2n .

1 2 1 2 n1 − 2n2 n1 + 4n2 4.7 (a) Compute ′

XX=

By Result 2.1.3,| X′X| = W | ′W x||′kxk since the second term is a scalar.

W′W x′kW

W′xk . x′kxk

− x′kW(W′W)− W′xk ; the | result follows 1

(b) Since x′kxk ≥ [x′kxk − x′kW(W′W)−1W′xk], it follows from (a) that ′ ′ |W W|/X X| = [x′kxk − x′kW(W′W)−1W′xk]−1 ≥ 1/x′kxk. In (4.1.1), let β = (β′W , βk)′, with βW = (β0, · · · , βk−1)′, and ! ^W ) ^W , β^k) Var(β Cov(β 2 ′ −1 ^) = Var(β = σ (X X) . ^ W , β^k ) Cov(β V ar(β^k )

By Result 2.1.3, ^ k) = σ2[x′kxk − x′kW(W′W)−1W′xk]−1 = σ2|W′W|/|X′X|, Var(β ^ k) ≥ σ2(x′kxk)−1 with equality when which implies that Var(β x′kW(W′W)−1W′xk = 0, i.e., columns of W are orthogonal to xk. Σ 2 i 4.8 (a) Var(β^0 ) = σ2/{ X2−NX }. Setting the first derivative equal to zero and solving, we can see that X = 0 is a stationary point, while second derivative evaluated at Σ 2 X = 0 is nonnegative. The minimum value of Var(β^0 ) is σ2/ X i.

21

22

Solutions to Chapter 4 Σ

Σ

(b) Var(β0^ ) = {σ2/N }{ Xi2/( X2i − NX 2 )} = {σ2/N }Q, say. Clearly, Q is infinite when Xi = a, i = 1, · · · , N , or Xi = b, i = 1, · · · , N . Since Q has the form {Var(X) + (E(X))2}/ Va r ( X Σ ) , which is minimum when EX is at the center of the distribution, it follows that Xi = N (a + b)/2, i.e., N/2 of Xi’s are equal to a and N/2 of Xi’s are equal to b.

4.9

(i) β^1 = (Y2 + Y4 + Y6 − Y1 − Y3 − Y5)/6, with variance σ2/6. (ii) β^1 = {5(Y2 − Y5) + 8(Y4 − Y3) + 11(Y6 − Y1)}/48, with variance 420σ2/(48)2.

The ratio of variances is then 32/35.

^ is a direct consequence of property 3 of 4.10 (a) That the ε^ i’s are uncorrelated with the Yi’s Result 4.2.4. To directly verify it without invoking the result, denote Xi∗ = Xi − X and x∗ = (X1∗, · · · , X N∗ )′. Then

By

Σ

Yi − Y = β0 + β1Xi + εi − (β0 + β1X − ε) = β1Xi∗ + εi − ε. ∗ i Xi = 0 and (4.2.18),

Σ

^β1 =

= β1 +

Xi∗εi . x∗ 2

So δ1 := β^1 − β1 = ε′x∗/ x∗ .

2

Since β^0 = Y − β^1 X = β0 + β1X + ε − β^1 X , then δ0 := β^0 − β0 = −δ1X + ε.

For each i,^Yi = (β0 + β1Xi) +(δ0 + δ1Xi) = (β0 + β1Xi) + δ1Xi∗ + ε, with β0 + β1Xi begin a constant. Then ^ i, ^ Cov(Y εj) = Cov(ε + δ1Xi∗, εj − ε − δ1Xj∗) Now Var(δ1) = Var(ε′x∗)/ x∗ Cov(ε, εj − ε) = 0. Then

4

= σ2/ x∗

2

, Cov(δ1, ε) = 0 as 1′x∗ = 0, and

^i, ε Cov(Y ^j) = Cov(δ1Xi∗, εj − δ1Xj∗) = Xi∗ Cov(ε′x∗, εj) − Xi∗Xj∗ Var(δ1) = Xi∗Xj∗/ x∗

2

− Xi∗Xj∗/ x

2

= 0.

Then Y^i and ^εj are uncorrelated. Finally, ∗ ∗ / x 2 + 1/n, Cov(Yi, Y^j ) = Cov(εi, δ1j X + ε) = X i X j

∗

∗ Cov(Yi, ε^j) = Cov(εi, εi − δ1X j− ε) = 1 − 1/n − Xi∗ X ∗j/ x 2,

which are nonzero. So the ^ Yi’s and ε^i’s are correlated with the Yi’s. (b) From the solution to (a), Y^i = (β0 + β1Xi) + δ1Xi∗ + ε and ε^i = εi − δ1Xi∗ − ε. Note ΣN ^−Y^ 1 2 = that ε^ = 0. Then the slope of the regression line of ε^i on Y^i is i=1 Y^i ε^i / y 2 y^ ′ ^ε / y ^ − Y^ 1 = 0. On the other hand, let Yi∗ = Yi − Y . Then the slope of 2 ΣN ΣN 2 the 2regression line of ε^i on Yi is i=1 Yiε^i/ y∗ = i=1 (Y^i + ε^i)ε^i/ y∗ =2 2 ε^ / y∗ . Now ^ε 2 = Σ (Yi − ^ β0 −^β1Xi)2 = Σ (Yi∗ − ^ β1Xi∗)2 = y∗ − β^1x∗ 2 2 2 2 ∗′ ∗ ∗ ∗ ∗′ ∗ ∗ and by ^β1 = x y / x , ε = y − (x y ) / x 2. Then the slope is 2 2 2 ∗′ ∗ ∗ ∗ 1 − (x y ) / x y .

Solutions to Chapter 4

23

4.11 Since the regression model is Yi = β1Xi + εi, we need to minimize S(β1) =

N Σ

(Yi − β1Xi)2.

i=1 ΣN 2

ΣN Since S′(β1) = −2[ XiYi − β1 X ], setting S ′ (β^1 ) = 0 yields β^1 = i i=1 i=1 Σ N XiYi/ ΣN X2. Since εi are uncorrelated and Var(εi) = σ2, then Var(β ^1) = i=1 i i=1 ΣN ΣN ΣN 2 2 2 2 Var(Xiεi)/( i=1 X i ) = σ / i=1 X i. i=1 Σ Σ 0 0 0 0 2 4.12 Minimizing i (Y −γ )ij with respect to µ, τi, βj, and γij, i = 1, · · · , a, i j j ij −µ −τ −β j = 1, · · · , b, we obtain the normal equations Σ Σ ΣΣ 0 Y·· = abµ0 + b τi 0 + a βj0 + γij , i

Σ

j

Σ

i

j

Yi· = bµ0 + bτi0 + β0j + γ0 ij, i = 1, · · · , a, j j Σ Σ Y·j = aµ0 + τ 0i + bβ0j + γ0 ij, j = 1, · · · , b, i

i

0 Yij = µ0 + τ 0i+ β0 + j γ , ij i = 1, · · · , a, j = 1, · · · , b.

The only LIN set of ab equations are the last normal equations, since we can obtain the first three by summing the last over both i and j, over j alone, and over i alone. The rank of the estimation space is ab, which is equal to the number of observations, leaving no degrees of freedom for the error space. Hence, although we may obtain b.l.u.e.’s of estimable functions, we cannot carry out further inference such as confidence interval estimation or hypothesis testing.

4.13 We see that (Xβ)′H = β′X′GX′X = (Xβ)′ using (3.1.7). Again, by (3.1.7), (X′Xβ)′H = β′X′XGX′X = (X′Xβ)′.

4.14 Let X = (X1, X2) and β = (β′1, β′2)′. Then y = Xβ + ε and X1β1 = (X1, O)β. If X1β1 is estimable, then there exists T such that (X1, O) = T′X = (T′X1, T′X2), i.e., X1 = T′X1 and T′X2 = O. Let x ∈ C(X1) ∩ C(X2). Then x can be written as x = X1a = X2b. Then x = X1a = T′X1a = T′X2b = 0. ThusC (X1)∩ C(X2) = {0 } .

Conversely, supposeC (X1)∩ C(X2) = 0{ .} Let β0 be any LS solution to the model, and partition it into β10 and β02. From property 5 of Result 4.3.1, to show that X1β1 is 0 0 0 estimable, it suffices to show that X1β1 is invariant to the choice of β . First, ^y = Xβ is the projection of y onto C(X1, X2), so it is unique. Next, y ^ is the sum of two unique 0 0 0 vectors respectively in C (Xi), i = 1, 2. Then, since ^ y = X1β1 +X2β2 and Xiβi ∈ C(Xi), i = 1, 2, Xiβ0 iare unique vectors, and so are invariant to the choice of β0.

4.15 Suppose c′β is estimable for all c. Then, c′H = c′, where H = GX′X. In particular, H = Ik if and only if columns of X are LIN. Σa Σa 4.16 We have seen that i=1 ciτi is estimable if and only if i=1 ci = 0. Hence, τi is not estimable for any i. Σa Σb 4.17 A basis set of a + b − 1 estimable functions is µ + 1 τi + 1 βj, τ1 − τi, i = Σa Σa i=1 j=1 a b 2,a· · · , a, and β1 − βj, j =Σ2, · · · , b.

If c τ is estimable, we must have ciτi = i i Σ Σ Σ a a a d (τ — τ ). Since i=1 Σa d− d τ =( d (τ − d i=1 − i 1 i i=1 i 1 Στi) = τ1 i=1 i i=1 i i i=1 i − d1)τ1 Σi=1 a a i=2 diτi, which implies that c1 = i=1 di −d1 = d2 +· · ·+da, c2 = −d2, · · · , ca = −da, Σa i.e., i=1 ci = 0.

24

Solutions to Chapter 4

4.18 (a) Let y = Xβ + ε, where y = (Y11, · · · , YN,1, Y12, · · · , YN,2)′, X =

1N 0N

ε = (ε11, · · · , εN,1, ε12, · · · , εN,2)′ with Var(ε) = σ2V, where V =

0N , and 1N IN ρIN . ρIN IN

Then, ^ GLS = (X′V−1X)−1X′V−1y β ! ΣN Y i1 1 τ^ 1 = = Σi=1 .

N N τ^2 i=1 Yi2 (b) Since Va r( β^ GLS

) = (X′V−1X)−1σ2 =

√ s.e.(τ ^1 ) = σ/ N . Substituting

σ 2

N

1 ρ ρ 1

√ , we have s.e.(τ^ ) = σ/ N , and 1

^GLS )′V−1(y − Xβ^GLS )/(2N − 2)}1/2 σ ^ GLS = {(y − Xβ for σ, we obtain the estimated s.e.’s of τ^1 and τ^2.

4.19 (a) Equating (4.5.5) with (4.2.3) and setting y = Xz, we must have X′V−1Xz = X′V−1X(X′X)−1X′Xz, which is true. (b) Since V−1 = (1 − ρ)−1I − ρ(1 − ρ)−1[1 + (N − 1)ρ]−1J, and since 1′y = 0, the result follows.

4.20 First assume that c′β is estimable in the model y = Xβ + ε; then there exists t such that c′β = E(t′y) = E(t′KK−1y) = E(l′z), which implies that c′β is estimable in the model z = Bβ + η.

Next, assume that c′β is estimable in the model z = Bβ + η. This implies that there exists a vector t such that c′β = E(t′z) = E(t′K−1y) = E(l′y), and therefore, c′β is estimable in the model y = Xβ + ε.

4.21 (a) Since V = KK′, X′VX = (K−1X)′(K−1X) = B′B where B is the regressor matrix in the transformed linear model (4.5.3). Hence, (X′VX)β = B′Bβ is estimable. (b) Since V = KK′, B = K−1X, z = K−1y, and η = K−1ε in the transformed OLS model, c′β is estimable if and only if c′ = c′H, where H = (B′B)−B′B, i.e., if and only if c′ = c′(B′B)−B′B = c′(X′K′−1K−1X)−X′K′−1K−1X = c′(X′V−1X)−X′V−1X. 4.22 First, we show that for any unbiased estimator d′y of c′β, Var(d′y) ≥ Var(c′β0 GLS ) with equality holding if and only if d′y = c′β0GLS . Let c′ = t′X. Then c′β0GLS = t′Xβ0GLS = 0 ′ ′ ′ ′ ′ ′ ′ ty ^ G L S . From d Xβ = E(d y) = c β = t Xβ for all β, d y^ G L S = d XβGLS = 0 ′ ′ 0 ′ ′ 0 ′ ′ ′ t XβGLS = c βGLS . Then d y − c βGLS = d (y^ G L S + ^ε G L S ) − t y^ G L S = d ^ε G L S . From ′ ′ 0 ′ ′ property 3 of Result 4.5.1, Cov(d ^ε G L S , c βGLS ) = 0, so Var(d y) = Var(d ^ε G L S + ′ 0 ′ ′ 0 ′ 0 c βGLS ) = Var(d ^ε G L S ) + Var(c βGLS ) ≥ Var(c βGLS ), with equality holding if and ′ ′ ′ only if d ^ε G L S is constant. Since E(d ^ε GL S ) = 0, the latter is equivalent to d ^ε G L S = 0, 0 or d′y ≡ c′β GLS .

Solutions to Chapter 4 ′

′

25 ′

′

0

Second, let ci = t X. Then ciβGLS = t y^ G L S , so by property 2 of Result 4.5.1, Cov(c′1β0GLS , c2′ β0GLS ) = t1′ Var(y)t2 = σ2t′1X(X′V−1X)−X′t2 = σ2c′1(X′V−1X)−c2.

4.23 From (4.5.7), WX = KPC(LX)LX = KLX = X as L = K−1, showing the second identity in (4.5.8). It follows that C(X) = C(WX) ⊂ C(W). On the other hand, again from (4.5.7), C(W) = C(X(X′V−1X)−X′V−1) ⊂ C(X). Then C(X) = C(W), showing the first identity in (4.5.8). Finally, W2 = KPC(LX)LKPC(LX) = KPC(LX)PC(LX) = W, showing the last identity in (4.5.8).

4.24 The least squares estimates sub jectΣ to the constraint θ1 + θ2 + θ3 = 180◦ are θ^i = 3 Yi − Y + 60◦, i = 1, 2, 3, where Y = i=1 Yi/3.

4.25 Recall that Var(β^0 ) = {σ2

Σ

Xi 2}/{N

Σ

(Xi − X)2}, and

Σ Var(β^1 ) = σ2/ (Xi − X)2.

Consider X ∈ [a, b]. Assume that a proportion p of the X values assume the value a, so Σn 2 that a proportion 1−p assume the value b. Then, X = p(a−b)+b, and X 2i −NX = i=1 n(a — b)2p(p + 1). Minimizing this as a function of p yields p = 1/2. Therefore, 50% of the X values assume the value a and 50% assume the value b. ^ = y, and ^βr = (IN − VC(C′VC)−1C′)y.

4.26 β 4.27 r(X) = 2. We can verify that β1 + β3 is estimable, with b.l.u.e. (2Y1 + 3Y2— Y3 + Y4)/5, whose variance is 3σ2/5. It can be verified that β2 is estimable, with b.l.u.e. (3Y1 +2Y2 + Y3 − Y4)/5 whose variance is 3σ2/5.

4.28 ε ^r = y − y^r = (y − y)^− (yr^− y)^= ε − ^ (yr^− y),^ where y^and ε^are the unconstrained vectors of fitted values and residuals, respectively. Note that ^ ε⊥ C(X) while both y^ r 2 ε 2 + y^ r − y ^ 2. and ^ y are in C(X). As a result,^ε ⊥ ^ (yr −^y). Then SSEr = ε^r = ^ 4.29 In the proof of Result 4.6.4, we saw SSEr = SSE + (A′β0 − b)′(A′GA)−1(Aβ0 − b).

Then E(SSEr) = E(SSE) + E[(A′β0 − b)′(A′GA)−1(Aβ0 − b)] = σ2(N − r) + E[(A′β0 − b)′(A′GA)−1(Aβ0 − b)] = σ2(N − r) + tr[(A′GA)−1 Cov(A′β0)] + [E(A′β0 − b)]′(A′GA)−1 E(Aβ0 − b) where the second equality is due to property 4 of Result 4.2.4, and the third equality is due to Result 4.2.3. From Result 4.2.4 and A′H = A′ (because A′β is estimable), Cov(A′β0) = σ2A′GX′XG′A = σ2A′GA while E(A′β0 − b) = A′β −b. Note that A has q columns. Then E(SSEr) = σ2(N − r) + σ2 tr(Iq) + +(A′β − b)′(A′GA)−1(Aβ − b) = σ2(N − r + q) + (A′β − b)′(A′GA)−1(Aβ − b).

26

Solutions to Chapter 4

4.30 (a) Let β = (µ, τ1, · · · , τa)′. µ = a′1β and τ1 + τ2 = a′2β, where a1 = (1, 0, · · · , 0)′ and a2 = (0, 1, 1, 0, · · · , 0)′. Directly check that ai /∈ R(X) while b := 2a1 +a2 ∈ R(X). (b) The constraints µ = s and τ1 + τ2 = t can be written as A′β = d, where A = (a1, a2) and d = (s, t)′. We have seen that C (A) /⊂ R (X) and dim( C (A)) = r(A) = 2, so the dimension ofC (A)∩ R(X) is at most 1. On the other hand, b ∈ C(A) ∩ R (X). As a result, C(A) ∩ R(X) is spanned by b. Under the constraint A′β = d, it is clear that b′β = 2µ + τ1 + τ2 = 2s + t. Then by Result 4.6.5, the vector of fitted values under the constraint A′β = d is identical to that under the constraint 2µ + τ1 + τ2 = 2s + t.

Solutions to Chapter 5

5.1 First, each e′ix = Xi has a normal distribution. In particular, E(Xi 2) < ∞. As a result, x has a well-defined mean µ and a variance-covariance matrix Σ. Now for each c, E(c′x) = c′µ and Var(c′x) = c′Σc, so by assumption, c′x ~ N (c′µ, c′Σc). The proof is complete by using property 2 of Result 5.2.4.

5.2 (a) Using Result 5.1.1 with B = 1 Σ−1, b = Σ−1µ, b0 = 1 µ′Σ−1µ, A = O, a = 0, 2

2

and a0 = 0, 1 1 I = πk/22k/2|Σ|1/2 exp{ µ′Σ−1ΣΣ−1µ) − µ′Σ−1µ 2 2 = (2π)k/2|Σ|1/2, from which the required result follows. (b) Using Result 5.1.1 with A = O, a = 0, a0 = 1, b = 0, b0 = 0, and B = we evaluate the integral to be √π .

1 1 , 1 4

3

5.3 By equating terms, it is easy to verify that µ=

3 , 1

Σ−1 =

−1/2 , 4

2 −1/2

√ and k = 2π det(Σ)1/2 = 4π/ 31. Using property 2 of Result 5.2.4 with c = (3, −1)′, we see that 3X√ 1 − X2 ∼ N (8, 140/31), so that P(3X1 − X2 < a) = 0√. 0 1 if and only if P(Z < (a−8)/ 140/31) = 0.01, where Z ∼ N (0, 1). Equating (a−8)/ 140/31 = −2.26 and solving, we get a = 3.056.

5.4 Let Y1 = X1 + X2 and Y2 = X1 − X2. We are given that y = (Y1, Y2)′ ∼ N2(0, I2).

1/2 1/2 y.

Then, X1 = (Y1 + Y2)/2 and X2 = (Y1 − Y2)/2, i.e., x = (X1, X2)′ = 1/2 −1/2 Hence, x ∼ N2(0, 12 I2).

5.5 The m.g.f. of x has the form of the m.g.f. of a trivariate normal distribution; equating terms, we see that , , 4 5 −1 2 µ = −2 1

and

Σ = −1 2

3 1 .

1 6

5.6 (a) Directly follows from Result 5.2.4. (b) By Result 5.2.5, Var(Y1 − Y2) = 1/σ1

−1/σ2

σ2

ρσ1σ2

1/σ1

ρσ1σ2

σ22

−1/σ2

1

= 2(1 − ρ). ρ measures the linear dependence between X1 and X2.

27

28

Solutions to Chapter 5

5.7 Use the hint to get that the answer is 1/3.

5.8 (a) log f (x) = − k2log(2π) − 12 log |Σ| − 21 x′Σ−1x, so that k 1 1 h(x) = 2 log(2π) + 2 log |Σ| + 2

∫

f (x)x′Σ−1xdx

k 1 1 log(2π) + log |Σ| + E(x′Σ−1x) 2 2 2 k 1 1 = log(2π) + log |Σ| + tr(Σ−1Σ) 2 2 2 1 1 = [log(2πe)k + log |Σ|] = log[(2πe)k|Σ|].

2 2 =

(b) First show that for i = 2, · · · , k, Σ k

h(X1, · · · , Xk) = h(X1) +

h(Xi|Xi−1, · · · , X1), and

i=2

h(Xi) − h(Xi | Xi−1, · · · , X1) ≥ 0, with equality implying that f (X1· ,· · , Xi) = f (Xi)f (Xi−1,· · ·, X1) almost everywhere (since the logarithmic function is strictly concave); this implies that X1, · · · , Xk are independently distributed. Hence, Σ k

h(X1, · · · , Xk) ≤ 1

log{(2πe)k|Σ|} ≤

2

i=1 Σk

i.e.,

1

i=1 k

|Σ| ≤

h(Xi),

Y

log{2πe|Σ |}, ii 2

i.e.,

|Σii|,

i=1

with equality if and only if X1, · · · , Xk are independent, i.e., if and only if Σij = O, i/ = j.

5.9 From Result 5.2.5, y ~ Nk(0, σ2PP′), and the result follows from the orthogonality of P. 5.10 By the singular value decomposition, there exists an orthogonal matrix P such that PΣP′ = D, where D = diag(λ1, · · · , λr, 0). Let B = (EP, O), where E = diag(λ1−1/2, · · · , λr−1/2). Let b = −Bµ. It is easy to verify that z = Bx + b ∼ Nr(0, I). ,Σ Σ Σ 11

12

13

5.11 Let µ = (µ′1, µ′2, µ′3)′, and Σ =

Σ21 Σ22 Σ23 . Apply Result 5.2.12 to get the Σ31 Σ32 Σ33 conditional distribution as Nk3 (µ3.12, Σ33.12) where µ3.12 = µ3 + B′13(c1 − µ1) + B′23(c2 − µ2), Σ11

Σ12

Σ21

Σ22

Σ33.12 = Σ33 − Σ31

Σ32

with B′13 B′23 = Σ31

Σ32

−1

and Σ11 Σ21

Σ12 Σ22

−1

Σ 13 Σ23

.

Solutions to Chapter 5

29

5.12 Since Σ=

Σ11 Σ21

Σ12 Σ22

is symmetric and p.d., Σ22 is symmetric and p.d., so is nonsingular. Then by Result 2.1.4, Σ−1 =

∆11 ∆21

∆12 ∆22

where ∆11 = (Σ11 − Σ12Σ−221Σ21)−1 = Σ−111.2 . Then Σ11.2 = ∆−1.11In order for the components of x1 to be mutually independent conditionally on x2, it is necessary and sufficient that Σ11.2 is a diagonal matrix, which is equivalent to ∆11 being a diagonal matrix.

5.13 If x1 = µ1.2(x2), then clearly x1 is a linear function of x2. Conversely, if x1 = Ax2 +a = f (x2), then f (x2) is an unbiased linear predictor with MSE equal to 0, so it must be the b.l.u.p.

5.14 (a) X2 ∼ N (µ2, σ2) and X3 ∼ N (µ3, σ2). (b) (X1 | X2, X3) is normal with conditional mean µ1 + ρ(x2 − µ2)/(1 − ρ2) − ρ2(x3 − µ3)/(1−ρ2) and conditional variance σ2(1−2ρ2)/(1−ρ2); it reduces to the marginal distribution of X1 when ρ = 0. (c) Since Cov(X1 + X2 + X3, X1 − X2 − X3) = σ2(−1 − 2ρ), the two variables are independent if ρ = −1/2.

1 0 0 0 0 0 and B = .

0 0 0 0 1 0 Then (see Exercise 5.2 7) C o v (x′Ax, x′Bx) = E[x′Ax−tr(AΣ)][x′Bx− tr(BΣ)] = ρ2 ρ ′ 2 tr(AΣBΣ) = 2 tr = 2ρ2 . Similarly, Var(x Ax) = 2 tr(AΣAΣ) = 0 0 1 ρ 0 0 2 tr = 2 and Var(x′Bx) = 2 tr(BΣBΣ) = 2 tr = 2, from which 0 0 ρ 1 the result follows.

5.15 (a) Write Cov(X12, X22) = Cov(x′Ax, x′Bx), where A =

(b) follows from Definition 5.2.7. (c) follows from Definition 5.2.8.

5.16 By Result 5.2.14, X̃ i = µi + σij(Xj − µj)/σjj with σjj = σ2.jThe proof follows by direct verification.

5.17 From Result 5.2.21, |ρj5|(q+1,··· ,k)| = 1 if and only if |ρ(εj, ε5)| = 1, where εj = Xj − X̃ j with X˜ j = µj + Cov(Xj, x2)Σ2−2 Cov(x2, Xj)(x2 − c2) and likewise for ε5. From Result 5.2.17, |ρ(εj, ε5)| = 1 if and only if εj = d1ε5 + d2 for some constants d1 /= 0 and d2.

If the latter holds, then Xj = X˜ j + d1(X5 − X̃ 5 ) + d2, implying Xj is linear function of X5 and x2. Conversely, if Xj = a′x2 + bX5 + c, then replacing Xj with εj + X˜ j and X5 with ε5 + X˜ 5 , εj − bε5 = f (x2), where f (x2) is a linear function of x2. Since both εj and ε5 are uncorrelated with x2, then Var(f (x2)) = Cov(εj − bε1, f (x2)) = 0, so f (x2) = εj − bε5 = 0 as the mean of every εi is 0. Then εj = bε5. Note that b /= 0 because otherwise εj = 0 so that ρj5|(q+1,··· ,k) is not well-defined. But then |ρj5|(q+1,··· ,k)| = 1.

30

Solutions to Chapter 5

5.18 From E(X) = E[E(X | Y )] for any jointly distributed X and Y , Cov(X1, X2) = E(X1X2) − [E(X1)][E(X2)] = E[E(X1X2 | X1)] − E(X1) E[E(X2 | X1)] = E[X1(α + βX1)] + E(X1) E(α + βX1) = β[E(X2) − {E(X1)}2] = βσ2.

1

1

Hence, Corr(X1, X2) = Cov(X1, X2)/(σ1σ2) = βσ1/σ2.

5.19 From Result 5.2.22, ρ0(1,··· ,k) = 1 if and only if Var(ε) = 0, where ε = X0 − W , where W is the b.l.u.p. of X0 based on x(1). Then ρ0(1,··· ,k) = 1 if and only if X0 = W and the proof follows from Exercise 5.13.

5.20 The product of the m.g.f.’s of the two random variables on the right side gives the m.g.f. of the random variable on the left side. ∫u Σ∞ sum and 5.21 P(U ≤ u) = 0 f (s)ds, where f (u) is given in (5.3.3). Interchange the infinite the integral, and successively integrate by parts to obtain P(U ≤ u) = j=0 P(X2 = j) P(X1 ≥ k/2 + j), and the right side is an expression for P(X1 − X2 ≥ k/2).

5.22 T =

1 (1 − ρ2)

(Z2 − 2ρZ Z + Z2) 1

1

2

2

where Zi = (Xi — µi)/σi. Clearly, (Z1, Z2)′ has a N2(0, 0, 1, 1, ρ) distribution.

We can write the quadratic form T as z′Az, where z = (Z1, Z2), and A = 1/(1 − ρ2) −ρ/(1 − ρ2) . Since AΣ = I, which is idempotent, T has a chi-square 2 ρ/(1 ρ ) 1/(1 − ρ2) − − distribution with 2 d.f. Σk 5.23 (a) By Result 5.4.5, x′D−1x = X2/σ2 has a χ2(k, λ) distribution with λ = i i i=1 Σ k µ′D−1µ = µ2/2σi2. By Result 5.3.4, E(U ) = k + 2λ and Var(U ) = 2(k + 4λ). i=1 i (b) Let Σ = Γ′Γ, with r(Γ) = k, and let z = Γ′−1x. Then, z ∼ Nk(Γ′−1µ, I), and U = z′z. By Result 5.4.2, U ∼ χ2(k, λ) with λ = µ′Σ−1µ. (c) We can show that x′Ax ∼ χ2(k − 1, µ′aµ/2).

5.24 From Result 5.4.4, it suffices to show that property 1 = ⇒ 2 =⇒ 3 =⇒ 1. Since Σ is nonsingular, all these implications can be checked directly.

5.25 The results follow from the required integrations.

5.26 (a) Use Result 5.4.3 with r = 1, so E(x′Ax) = κ1(x′Ax). (b) Var(x′Ax) = κ2(x′Ax). (c) Since odd moments of x − µ are zero, Cov(x, x′Ax) = E[(x − µ)(x′Ax − µ′Aµ − tr(AΣ))] = E[(x − µ)(x − µ)′A(x − µ)] + 2 E[(x − µ)(x − µ)′Aµ] — E[(x − µ) tr(AΣ)] = 2ΣAµ 5.27 Use Result 5.4.4 with Σ = I.

Solutions to Chapter 5

31

5.28 From the hint, we can write x′Ax = z′(Γ′AΓ)z. Then, from Result 5.4.4, x′Ax has a chi-square distribution if and only if Γ′AΓ is idempotent, in which case the d.f. of the chi-square distribution is p = r(Γ′AΓ). Now from ΓΓ′ = Σ and Result 1.3.13, (Γ′AΓ)2 = Γ′AΣAΓ = Γ′AΓ if and only if ΣAΣAΣ = ΣAΣ. Clearly p ≥ r(ΣAΣ).

On the other hand, from (Γ′AΓ)3 = Γ′AΣAΣAΓ = Γ′AΓ, r(ΣAΣ)≥ p, so p = r(ΣAΣ). ′ 5.29 (a) Since Cov(x, Bx) = B′ and′ Cov(Bx, ′ ′ Bx) = BB , from Result 5.2.14, the b.l.u.p. of x based on Bx is^x = B BB B x = Px. Then y = (I − P)x ∼ Nk(0, I − P). (b) Direct expansion.

(c) Since y is independent of^x = Px and the latter is a function of Bx, from (b), for any fixed x0, the conditional distribution of x′Ax + a′x given Bx = Bx0 is the same as U (x0) = y′Ay + (2APx0 + a)′y + (x′0PAPx0 + a′Px0). If x′Ax + a′x and Bx are independent, then the distribution of U (x0) is always the same as the unconditional distribution of x′Ax + a′x. (d) If E[U (x0)] = E(y′Ay) + x′0PAPx0 + a′Px0 is invariant to x0, then PAP = O and a′P = 0. The latter yields a′PB′ = a′B′ = 0, i.e., Ba = 0. (e) Since Cov(y) = I − P, from Result 5.4.2, Var[U (x0)] = Var[y′Ay + (2APx0 + a)′y] = 2[tr{[A(I − P)]2} + 2(2APx0 + a)′(I − P)(2APx0 + a)].

The invariance then implies PA(I — P)(AP) = O. Then from (d), PAAP = (AP)′AP = O, so AP = O. (f) This part is identical to the reduction argument at the end of the proof of Result 5.4.7.

5.30 From Result 5.4.4, U ∼ χ2(m, λ = µ′A−1µ/2). , 1 −2 1 1 2 5.31 Writing Q = x′Ax, with A = −2 4 −2 , we see that Q ∼ χ .1 6 1 −2 1 5.32 (a) Let B = Σ/σ2 − λIk. By the elementary row transformations R1 − Rk, · · · , Rk−1 − Rk, followed by the column transformations C1+Ck, · · · , Ck−1+Ck, B is equivalent to L, which is lower triangular. Setting |B| = |L| = (1−λ−ρ)k−1[1−λ+(k−1)ρ] = 0, and solving for λ, we see that the eigenvalues of σ2Σ are λ1 = σ2(1 − ρ) with multiplicity k − 1, and λ2 = σ2[1 + (k − 1)ρ] with multiplicity 1.

2 (b) A22 = J2/k = kJk/k2 = Jk/k = A2. Since A1 = Ik − A2, A1 is idempotent too. k A1A2 = (Ik − Jk/k)(Jk/k) = O. It is easy to verify that λ1A1 + λ2A2 = Σ. (c) First, Q1 = x′A1x/λ1 and Q2 = x′A2x/λ2 are independent since, from (b), A1ΣA2 = A1[λ1A1 + λ2A2]A2 = 0. Since A1Σ/λ1 = A1, which is idempotent of rank r(A1) = tr(A1) = tr(Ik) − [tr(Jk)]/k = k − 1, it follows from Result 5.4.5 that Q 1 ∼ χ2 , the noncentrality parameter being 0 since µ = 0. Similarly, k−1 2 since A2 is idempotent of rank 1, Q2 ∼ χ1.

5.33 Verify that B is symmetric, and that BΣ is idempotent. Now r(BΣ) = tr(BΣ) = tr[I − (Σ−1J)/(1′Σ−11)] = k − 1. Also, BΣB = B, so that r(B) = r(BΣB) ≤ r(BΣ) = k − 1.

But, r(B) ≥ r(BΣ) = k − 1, so that r(B) = k − 1. (a) By Result 5.2.5, y ∼ N (BΣµ, σ2B).

32

Solutions to Chapter 5 (b) y′Σy/σ2 = 2x′B′ΣBx/σ2 = x′Bx/σ2, which has a χ2(k − 1, λ) with λ = µ′ΣBΣµ)/2σ . When µ = 0, y′Σy ∼ σ2χ2 , while when µ /= 0, y′Σy ∼ k−1

σ2χ2(k − 1, λ).

5.34 (a) The proof is identical to the following proof for (b). (b) From (5.4.18), the necessary and sufficient conditions for the independence of x′Ax + a′x and x′Bx + b′x are (Γ′AΓ)(Γ′BΓ) = Γ′AΣBΓ = O, (Γ′AΓ)((2Γ′Aµ + Γ′a) = Γ′AΣ(2Aµ + a) = 0, (Γ′BΓ)((2Γ′Bµ + Γ′b) = Γ′BΣ(2Bµ + b) = 0, (2Γ′Aµ + Γ′a)′(2Γ′Bµ + Γ′b) = 4(Aµ + a/2)′(Bµ + b/2)′ = 0 From Γ′M = O if and only if ΓΓ′M = ΣM = O for any matrix M, the above equalities are equivalent to those in (5.4.19).

5.35 We can verify that Q1 = x′A1x and Q2 = x′A2x, with A1 =

1 2(1 − ρ)

1 −1 , −1 1

A2 =

1 1 .

1 1

Since A1Σ is idempotent of rank 1 and noncentrality parameter 0, and since A1ΣA2 = 0, the results follow.

5.36 The result follows directly from Result 5.4.7 and the positive definiteness of Σ.

5.37 The proof of the necessity part of Result 5.4.9 only uses the independence of x′Ajx. From Result 5.4.7, for i /= j, A′iAj = O, so C(Ai) ⊥ C(Aj). Since A1 + · · · + AL = I, Σ L C(A ), so L j=1 r(A ) = k. then Rk = ⊕j=1 j j Σ ΣL 5.38 (a) k ≤ r(I − Aj) + r(Aj) = r( i/=j Ai) + r(Aj) ≤ i=1 r(Ai) = k. (b) This follows from the spectral decomposition of a symmetric matrix. (c) From (a) and (b), (k− m1) + (k − m0) = k, so m0 + m1 = k. Therefore, Aj is idempotent. ΣL Σ (d) Since Aj is idempotent, Aj = A2 = Aj( Ai)Aj = A3 + AjAiAj = j i=1 i/=j j Σ Σ Aj + i/=j AjAiAj. Thus i/=j Aj AiAj = O. (e) Since Ai are n.n.d., from (d), AjAiAj = O for i /= j. Then AiAj = O. (f) Use Results 5.4.4 and 5.4.7.

5.39 Without loss of generality, let µ = 0. (a) The p.d.f. of x2 is −1 f (x2) = ck−q |V22|−1/2hk−q (x′2V22 x2),

and the conditional covariance is ∫ −1x2)′dF 1|x 2 (x1). (x1 − V12V22−1x2)(x1 − V12V22 x −1x2). The Suppose V11.2 = D′D, and let y = (Y1, · · · , Yq)′ = D−1(x1 − V12V22 covariance matrix is ∫ yy′dFy,x2 (y, x2)/f (x2) = C = {cij}, say.

Solutions to Chapter 5

33

We are given that C does not depend on x2. Now, ∫ ∞ c = Y 2dF (y, x )/f (x ), 11

−∞

1

y,x2

2

2

and we can write c11c k−q |V22|−1/2 hk−q (x′2V−1x2) = ck−q+1 |V22 |−1/2

∫ ∞ −∞

22

2

′

−1

Y 12h k−q+1 (Y1 + x2 V 22 x 2)dy 1.

Set Z = x′2V−221x2, and b = 2ck−q+1/ck−q. (b) Let V = diag(v11, · · · , vkk). Independence of the components of x implies that the Qk √ m.g.f. of x can be written as ψ(t′Vt) = ψ(t2vii). Let ui = ti vii. Then, i i=1 Σk Qk ψ( i=1 ui2) = i=1 ψ(ui2). This equation (called Hamel’s equation) has solution ψ(z) = exp(cz) for some constant c. Therefore, ψ(t′Vt) is the m.g.f. of a normal random vector.

5.40 The m.g.f. of By is MBy(t) = My(B′t) = ψ(t′BVB′t) exp{t′Bµ}.

The result follows.

Solutions to Chapter 6

6.1 Let D = X − 1N x′. Then D is an N × k matrix and SN = D′D. Then r(SN ) = r(D), so SN has rank less than k if and only if the k columns of D are not LIN. Σm 6.2 W ∼ xjx′j , where x1, · · · , xm are i.i.d. ∼ Nk(0, Σ). Then B′WB ∼ j=1 Σm ′ ′ ′ ′ ′ ′ ′ j=1 B xj(B xj) . Since B x1, · · · , B xm are i.i.d. ∼ Nq(0, B ΣB), then B WB ∼ Wq(B′ΣB, m). ΣN 6.3 (a) E(W) = E(xix′i) = N (Σ + µµ′). i=1Σm (b) Suppose V = yiy′, where y1, · · · , ym are i.i.d. ∼ N (ν, Σ). If ν = −µ, then i i=1 Σm y y ′ ∼ Σm ′ (y1, · · · , ym) ∼ −(x1, · · · , xm), so i=1 i i i=1 xixi, yielding V ∼ W. On the other hand, suppose V ~ W. Then E(V) = E(W). From (a), νν′ = µµ′.

If either ν or µ is null, then the other one is null, and so ν = —µ. Otherwise, ν(ν′µ) = µ(µ′µ), giving µ = cν for some scalar c. Then νν′ = µµ′ = c2νν′, giving c2 = 1, i.e., c = ±1. Then ν = ±µ. ^ ML = N−1 SN , it follows that | Σ^ML |N/2 = N −1 kN/2 |S N |N/2; substituting 6.4 Since Σ N N this into the expression (6.1.9), we see that L(µ ^ M L,^ΣML) can be written as ^ ML) = L(µ ^ ML, Σ

exp(− Nk ) (2π)Nk/2

N —1

2 kN/2

N

|SN |N/2

.

6.5 The proof that xm+1 = (m + 1)−1(mxm + xm+1) is immediate from the definition of the sample mean, and some algebra. To show the second identity, we write Σ

m+1

Sm+1 =

(xi − xm+1)(xi − xm+1)′

i=1 m Σ

=

(xi − xm + xm − xm+1)(xi − xm + xm − xm+1)′

i=1

+ (xm+1 − xm+1)(xm+1 − xm+1)′ = Sm + m(xm − xm+1)(xm − xm+1)′ + (xm+1 − xm+1)(xm+1 − xm+1)′.

Using the first identity for xm+1 and simplifying, we get Sm+1 = Sm + m(m + 1)−1(xm+1 − xm+1)(xm+1 − xm+1)′, which is the required identity.

6.6 Let b = Σ1/2a, and B = Σ−1/2SN Σ−1/2; then max a

a′SN a a′Σa

= max b

b′Bb b′b

= max b

b′ (b′b)1/2

B

b (b′b)1/2

.

35

36

Solutions to Chapter 6 Let c = b/(b′b)1/2, B = PΛP′ be the spectral decomposition of the symmetric matrix B, and d = (d1, · · ·, dk)′ = P′c. Let λ1, · ,· ·λk be the eigenvalues of B, i.e., of Λ. Note that c = 1, P′c = 1 and max{c′Bc} = max{c′PΛP′c} = max{d′Dd} = max c

c

d

d

k Σ

d2i λi = max λi.

i=1

1≤i≤k

Now, |SN − cΣ| = 0 implies that |Σ−1/2SN Σ−1/2 − cIk| = 0, i.e., that |B − cIk| = 0.

Hence, the solution to |SN − cΣ| = 0 gives the eigenvalues λi of B. We have shown that '

Na maxa aaS'Σa is the largest root of |S N − cΣ| = 0. Replacing max by min, the other result follows.

6.7 We have SNi ∼ Wk(Σ, Ni − 1), i = 1, 2. In Exercise 6.2, setting B = a, we see that a′SN ia ∼ a′Σaχ2 Ni−1 . Also, a′SN 1a and a′SN a2 are independent since SN 1 is independent of SN2 . Hence a′SN1 a/a′SN2 a ∼ FN1−1,N2−1.

6.8 The joint bivariate normal pdf of (Xij, Xi5), i = 1, · · · , N can be written as (see Example 5.2.6) Y N √ i=1

1

1 2 2 exp − (xij − µj) /σj 2 2πσj

1 ×√ 2πσ 5.j

1 2 2 exp − i5[x −5µ − β(x ij − jµ )] /σ 5.j 2

ΣN 2 ∗ where σ5.j = σ52(1 − ρ2) and β = ρσ5/σj. Let S∗jj = i=1 (Xij − Xj) , S 55 = Σ ΣN N 2 ∗ i=1 (Xi5 − X5) , and Sj 5 = √ i=1 (Xij − Xj)(Xi5 − X5). Consider the transforma∗ ∗ tions U1 = {Sj 5 − βSj j }/σ5.j Sj∗j , U2 = {S5∗5 − Sj52 /Sj∗j }/σ5.j2 , and U3 = Sj∗j /σ2j, with −3(S ∗ )−1/2. The joint p.d.f. of U1, U2 and U3 is Jacobian σj−2σ5.j j j

1 (N −2)/2−1 (N −1)/2−1 u3 . const. × exp − (u12 + u 2 + u3) u2 2 Since U 2 + U2 + U3 = 1

1

2

{S∗ /σ2 − 2ρS∗ /σjσ5 + S∗/σ2}, we can write the joint

(1−ρ ) density of Sj∗j , S5∗5, and Sj∗5 as

jj

j

j5

55

5

−1) σj−(N −1)σ−(N (1 − ρ2)−(N −1)/2 ∗ ∗ 5 ∗2)(N −4)/2 √ N −1 (sjj s55 − sj5 π2 Γ(N − 2)/2Γ(N − 1)/2 1 × exp − {s∗ /σ2 − 2ρs∗ /σ σ + s∗ /σ2 . jj j j5 j j 55 5 2(1 − ρ2) √ Let Sjj = Sj∗j /(N − 1), S55 = S5∗5/(N − 1), and Sj5 = (N − 1)r SjjS55, with Jacobian 4(N − 1)3SjjS55. The joint pdf of Sjj, S55 and r is then

(N − 1)N−1 f (sjj, s55, r) =

(N −2)/2 (N −2)/2

s (σ j σ5 )N −1(1 − ρ2)(N −1)/2πΓ(N − 2) jj

s55

× (1 − r2)(N−4)/2 exp{−(N − 1)/2(1 − ρ2)} 2

× [sjj/σj − 2ρr{sjjs55}

1/2

2

/σjσ5 + s55/σ5 ] √ 2p−1 which follows by using Γ(2p) = 2 Γ(p)Γ(p + 1/2)/ π, and simplifying.

Solutions to Chapter 6

37

6.9 The proof of Result 6.2.3 shows that conditional on almost every value of Z, the distribution is S11.2 = U′(A −PC(AZ))U is the same Wq(Σ11.2, N −1 −k + q). Therefore, S11.2 is independent of Z, hence independent of S22 = Z′AZ. Meanwhile, S11.2 is a the transpose of (A− PC(AZ))U multiplied by itself, and S21 = Z′A(ZB′ +U) only depends on Z′AU. Let the columns of U be u(1), , u(q). Then for any two i, j, Cov((A − PC(AZ))u(i), Z′Au(j)) = (A − PC(AZ)) Cov(u(i), u(j))AZ = σ̃ ij (A − PC(AZ))AZ = O. This implies the conditional independence of S11.2 and S21 given Z. But the conditional distribution of S11.2 is the same as its unconditional one. Then the (unconditional) independence follows.

6.10 (a) ^ ρ0(1,2,3) = 0.6378. (b) ρ^0,3|1,2 = 0.6356.

6.11 ρ^20(1,··· ,k) = 1 − |S|/{S00|S

(1)

|}, which follows from (6.2.9) and (6.2.10).

6.12 Beta(k/2, (N − k − 1)/2) distribution, which follows from (6.2.11) and (B.13).

6.13 If xj ∼ 2Nk(µ, Σ), j = 1, · · · , N , then D2 jhas an approximate χ2 k distribution, so that E(d ) ≈ k. The exact result is seen by using the fact that Nd2/(N − 1)2 has j

a Beta(k/2, (N − k − 1)/2) distribution so that E(d2) = (N−1) j

2

N

k (N −1)

j

= (N − 1)k/N .

6.14 (a) Since Z(1) + · · · + Z(N) = Z1 + · · · + ZN = NZ N , the first equality easily follows.

Next, from Cov(Zi − ZN , ZN ) = Cov(Zi, ZN ) − Var(ZN ) = 0, z − ZN 1N = (Z1 − ZN , · · · ZN − ZN )′ is independent from ZN , so that the order statistics of z − ZN 1N are also independent of ZN . However, the ith order statistic of z − ZN 1N is Z(i) − ZN .

As a result, Cov(Z(i) — ZN , NZ N ) = 0, yielding Cov(Z(i), ZN ) = Cov(ZN , NZ N ) = 1. (b) follows directly from the definition of SW .

6.15 Since yi = Pxi + q, xi = P−1(yi — q). Verify that yN = PxN + q, and that SN,y = PSN P′. Using the form of rij defined below (6.3.2) on the yi’s, we see that −1 (yj − y ) rij,y = (N − 1)(yi − yN )′SN,y N

= (N − 1)(xi − xN )′P′(P′)−1SN−1P−1P(xj − xN ) = rij; i.e., rij is invariant under the nonsingular linear transformation. So are b1,k and b2,k defined by (6.3.2) and (6.3.3). The coefficients do not depend on µ and Σ.

Solutions to Chapter 7

7.1 Property 1 of Corollary 7.1.1 is a direct consequence of Result 5.2.5, while property 2 follows from Result 5.2.4. Property 3 follows from the orthogonality of X and I − P. ^ = σ2(X′X)−1.

7.2 For β = (β0, β1, β2)′, Var(β) (a) The statistic t = β^ e.(β^ 0/s.ˆ 0) has a Student-t distribution with N − 3 d.f. under H0, where s.ˆ e.(β^0) denotes an estimate of the standard error of β^0. (b) The statistic t = (β^ e.(β^ 1 − β^ 2)/s.ˆ 1 − β^ 2) has a Student-t distribution with N − 3 d.f. under H0. The sˆ . e. term on the denominator is obtained from (4.4.2), substituting 2 2 σ ^ for σ . ^1,H = 0. (c) H 0: X∗1= 0 implies H :0 β 1= 0. Clearly, β ^0,H = β^0, β^2,H = β^2, while β Σ ^ ^ The test statistic F 0 = (N−3)Q/SSE ∼ F 1,9 under H0 where Q = β 1X i(2β 0+ Σ β^1 X i + 2β^2 X i2 ), and SSE = (Yi − β^0 − β^1 X i − β^2 X 2i ). , 0 0 0 1 0 0 7.3 We can write H as C′β = d with C′ = 0 0 0 0 1 0 and d = 0. Then, 0 0 0 0 0 1 F (H) = (N − 6)(SSEH − SSE)/3SSE ∼ F3,N−6 under H, where SSE = y′(IN − P)y, SSEH = y′(IN − PH )y, and PH = P − P 1 = X(X′X)−1X′ − X(X′X)−1C[CX′X)−1C]−1C′(X′X)−1X′. If we replace y by y−Y 1N , then, SSEnew = (y − Y 1N )′(IN − P)(y − Y 1N ) = SSE, since (IN − P)1N = 0. A similar reasoning gives SSEnew,H = SSEH , so that F (H) is unaltered.

7.4 Let µ + τ1 = µ + τ2 = µ + τ3 = β, say. Under H, the model becomes Yij = β + εij, Σ3 Σ Σn i 2 and β0 = ( 3i=1 niY i·)/N , where N = n1 + n2 + n3, and SSEH = j=1 Y ij − H i=1 Σ3 Σ3 Σni Σ3 2 i=1(niY i·)2/N . For this model, SSE = i=1 j=1 Y ij − i=1 Yi·2/ni, and F (H) = (N − 3)(SSEH − SSE)/(2SSE) ∼ F2,N−3 under H. 7.5 Least squares estimates are ^ θi = Y−i· Y0·, i = 1, · ·, ·m. The required distribution follows from Result 4.4.1 and Result 7.1.2.

7.6 The two lines intersect at (x0, y0) if and only if they intersect at a point (0, y0∗). This is seen by writing a shifted model Y5,i = β5∗,0 + β5∗,1X5∗,i + ε5,i, where β5∗,0 = β5,0 + β5,1x0, and X5∗,i = (X5,i − x0). To test β1∗,0 = β2∗,0 = β, the F -statistic is F (H) = (n1 + n2 − 4)(SSEH − SSE)/SSE ∼ F1,n1+n2−4

39

40

Solutions to Chapter 7 under H. Here, SSE is the full model sum of squares given in (7.2.10), and 2 nl Σ Σ ^H − βl, ^ 1, HX5∗,i)2, and SSEH = (Y5,i − β Σ Σ 5=1 i=1 Σ2 Σ Σ 2 Y5,i − {X5· Y5,iX5,i/ X } 5 i 5=1 i i 5,i , β^H = Σ2 Σ 2 n1 +Σ n2 − 5=1 {X5·2/ i X5,i } i(Yl,i−βH )Xl,i

Σ

where X5· =

i

^ X5,i , and β 5,1,H =

Σ

^ 2 i Xl,i

, l = 1, 2.

7.7 The null distribution of R2 is Beta(k/2, (N − k − 1)/2), so E(R2) = k/(N − 1).

7.8 The design matrix is X′ =

0 · · · 0 1 −c 0 ··· 0 0

0 0··· 0

1 −c 0 · · · 0

, and β′ = (λ , λ ).

1

2

(a) Then, ^λ1 = [(1 + c2)Yk− c3Yk+1 − c2Yk+2]/(1 + c2 + c4), and ^ λ2 = [cYk + Yk+1 − c(1 + c2)Yk+2]/(1 + c2 + c4), with ^ = Var(β)

σ2 (1 + c2 + c4)

1 + c2 c

c .

1 + c2

(b) Under the restriction λ1 = −λ2 = λ, say, using results from section 4.6.1, we get ^ = [Yk − (1 + c)Yk+1 + cYk+2]/2(1 + c + c2), with Var(λ) ^ = σ2/2(1 + c + c2). λ (c) Q/σ2 ∼ χ2, where ^ 1 Q=

(1 − c + c2) 2

(1 + c + c2)Y k + (1 − c3)Y k+1 − c(1 + c + c2)Y k+2 (1 + c2 + c4)

2

.

7.9 The restricted least squares estimate of β subject to the model constraint 1′β = 0 is Σ4 2 βr^ = y − Y 14 , where Y = i=1 Yi/4. Also, SSEr = 4Y . The reduced and restricted 1 111 model consists of imposing H: β1 = β2, i.e., β = 0. We can show that (see 1 −1 0 0 section 8.1) β^r,H = y − z, where z′ = (Y +21 (Y1 − Y2), Y − 21 (Y1 − Y2), Y , Y ). Also, SSEr,H = 4(Y )2 + (Y1 − Y2)2/2. Then, 2(Y1 − Y2)2 .

F0 = Σ 4 ( i=1 Yi)2 7.10 Under the full model (7.3.6), E(SSE) = σ2(N −r). On the other hand, under H: β1 = 0, SSEH = y′(I − P2)y. Let µ = X1β1 + X2β2. Then (I − P2)µe = (I − P2)X1β1, so from Result 4.2.3, E(SSEH ) = tr((I − P2)σ2Ip) + µ′(I − P2)µ = σ2(N − r2) + β′1X′1(I − P2)X1β1.

Then E(SSEH ) − E(SSE) = σ2(r − r2) + β′1X′1(I − P2)X1β1. ′

0

0

7.11 MSE = {y (I1 − 2P)y}/(N − r).1 Let2 ^ε = y − ^y = y − X1β1 − X2β2 = (I − P)y.

E(MSE) = σ tr(I − P) = σ (N − r), which proves property 2 of Result 7.3.7.

N −r N −r

Solutions to Chapter 7

41

7.12 (a) Let P be the projection matrix onto C(X1, X2). Then Cov(y^ ) = σ P and 2 2 Cov(y ^ H ) = σ P1, so Cov(y^− y^H ) = σ (P − P1). From property 3 of Result 2.6.7, C(X1, X2) = C(X1) ⊕ (I − P1)C(X2) = C(X1) ⊕ C((I − P1)X2).

Since C(X1) ⊥ C((I − P1)X2), then P = P1 + PC((I−P1)X2), i.e., P − P1 = (I − P1)X2)[X′2(I − P1)X2]−X′2(I − P1).

2

2 (b) We have (N − r 1 )σ^H /σ2 ∼ χ2N −r1 and (N − r)σ ^ 2 /σ 2 ∼ χ2N −r , where r1 = r(X1) and r = r(X1, X2). Then Var(σ^2H) = σ4/(N − r1) and Var(σ^ 2 ) = σ4/(N − r), so Var(σ ^ 2)/ Var(σ ^H2 ) = (N − r)/(N − r1) ≥ 1.

7.13 We have y^ H − µ = P1y − µ = P1ε − (I − P1)µ and y ^ − µ = Py − µ = Pε, where 2 P1 is the projection onto C(X1), and P the one onto C(X1, X2). Then E( y^ H − µ ) = E( P1ε 2) + (I − P1)µ 2 2 = E(ε2′P1ε) + (I − P1)X2β2 2. By Result 4.2.3,2 E(ε′P1ε) =2 σ2r1. Then E( y^ H − µ ) = r1σ +2 (I − P )X2β2 2. Likewise, E( y ^ − µ )(r1 + r2)σ .

2 2 2 Then E( y^2H − µ 2 ) − E( y ^ − µ ) = (I − P − r ^ 1)X2β2 2σ , so E( y H − µ ) < 2 2 E( y ^ − µ ) if and only if (I − P1)X2β2 1 < r2σ . ^0, ^ ^2, while^β1,H = 0. The test 7.14 H 0: X∗ = 0 implies H0 : β1 = 0. Clearly,^β0,H = β β2,H = β Σ statistic F0 = (N −3)Q/SSE ∼ F1,9 under H0 where Q = β^1 X i (2β^0 +β^1 X i +2β^2 X 2i), Σ and SSE = (Yi − β^0 − ^ β1Xi − β^2X2i).

7.15 We describe the analysis for the balanced incomplete block design (BIBD). There are a levels of treatment, each replicated r times. There are b blocks, and the number of treatments per block is k. The total number of obsevations is N = ar = bk. The design is symmetric if a = b. We let λ = r(k − 1)/(a − 1) denote the number of times each pair of treatments appears in the same block. To test H0: τ1 ·=· · = τa, the ANOVA decomposition is SSTc = SSTradj. + SSb5ock + SSE, where a

SSTc =

SSTradj. =

b

ΣΣ

a

(Yij − Y ··) =

i=1 j=1

2

b

ΣΣ

Y 2 ij− Y 2··/N, with N − 1d.f.

i=1 j=1 2

a b 1Σ k Σ δij Y·j Y i· − λa i=1 k j=1

with a − 1 d.f.

b

SS

b5ock

1 Σ ·j 1 ·· = k j=1 Y 2 − N Y 2 with b − 1 d.f.

SSE = SSTm − SSTradj. − SSb5ock with N − a − b + 1d.f. where δij = 1 is treatment i appears in block j and 0 otherwise. The test statistic is F = MSTradj./MSE ∼ Fa−1,N−a−b−1 under H0.

7.16 The estimates of β0 and β1 are not changed when β2 is set to 0. This is a consequence , 1 —1 1 of the orthogonality of the columns of the matrix X = 1 0 −2 , so that X′X = 1 1 1 diag(3, 2, 6). The least squares estimates of β0 and β1 are ^ β0 = (Y1 + Y2 + Y3)/3, and β^1 = (Y3 − Y1)/2.

42

Solutions to Chapter 7

7.17 In general, suppose H tests equality of (µ+τi)/di, i = 1, · · · , a, with common value equal to γ, say, and di’s are known constants. Under H, Yij =Σ diβ + εij. The Σ design matrix is X = (d11n , · · · , da1n )′. We can verify that γ0 = ( nidiYi·)/( nid2). Hence, 1 a i H Σ Σi i Σ Σ F (H) = (N — a)[ i Yi·2/ni − (γ 0)2 i nidi 2]/[(a − 1) i j (Yij − Y i·)2] ∼ Fa−1,N −a under H. 7.18 In the model Yij = µi + εij, r(X) = a. The hypothesis H can be writen as C′β = d, where C′ is an (a − 1) × a matrix , 1 −2 0 0 ··· 0 0 0 2 −3 0 ··· 0 0 . .

0 0 3 −4 · · · 0 0. , . . . .

0 0 0 0 · · · a − 1 −a β = (µ1, · · · , µa)′, and d is an (a − 1)-dimensional vector of zeroes. The test statistic has the form (7.4.5) and is distributed under H as an Fa−1,a(n−1) distribution. Alternately, we can write H as µ + τ1 = 2(µ + τ2) = · · · = a(µ + τa) = δ, say. That is, i(µ + τi) = δ, i=Σ 1, . . . , a =⇒ (µ + τi) = δ/i, i = 1, . . . , a. Under H2, the one-way ANOVA model a s.t. i=1 τi = 0 reduces to 1 Yij = δ + ϵij, i = 1, . . . , a, j = 1, . . . , n. i We can show that the LS estimate is

Σ ai=1 Yi·/i δ = Σa 2 i=1 (1/i ) 0

Then we can find ΣΣ a

SSEH =

n

(Yij − δ0/i)2

i=1 j=1

We know SSE for the one-way ANOVA model is ΣΣ a

SSE =

n

(Yij − Y i·)2

i=1 j=1

The F -statistic is obtained from (7.2.9).

7.19 The hypothesis can be written as C′β = d, with C′ = (0, −1/2, 1, −1/2), β = (µ, τ1, τ2, τ3)′, and d = 0. From (7.4.5), F (H) = (3n − 3)Q/SSE ∼ F1,3n−3 under Σ3 Σn 1 2 H, where Q = 2n i=1 j=1(Yij − Y i·)2.

3 [Y 2· − 2 (Y 1· + Y 1·)] , and SSE = 7.20 Consider the nested sequence of subspaces SH0 ⊃ SH1 ⊃ SH2 ⊃ SH3 ⊃ SH4 ⊃ SH5 ⊃ SH 6 = {0}. The least squares estimates are: µ0 = Y ···, τi 0 = Y i·· − Y ···, θj0 = Y ·j· − Y ···, γk0 = Y ··k − Y ···, (τθ)0ij = Y ij· − Y ···, and (θη)0jk = Y ·jk − Y ···. Hence, Q(H0) = Σ Σ Σ Σ Σ Σ 2 i j k (Yij· − Y ·jk − Y ·j· ) with b(a + c − 1), SSE = i j k (Y ijk − Y ij· − Σ Σ Y ·jk + Y ·j· )2 with b(a− 1)(c −1) d.f., Q(H 0 | H 1 ) = a j k (Y ·jk − Y ·j· − Y ··k − Y ··· )2 Σ wΣ ith Σ (b − 1)(c − 1) d.f., Q(H1 | H2) = ab k(Y ··k − Y ···)2 with c − 1 d . fΣ . , Q(H2 | H3) = 2 c i j(Y ij·−Y i··−Y ·j·−Y ···) with (a−1)(b−1) d.f., Q(H3 | H4) = ac j (Y ·j· −Y ···)2 2 Σ with b − 1 d.f., Q(H 4 | H5 ) = bc i(Y i·· — Y··· )2 with a − 1 d.f., Q(H 5 | H 6 ) = abcY ··· with 1 d.f. The ANOVA table can be completed based on this information.

Solutions to Chapter 7

43

7.21 If all three factors have a levels each, and n = 1, the design corresponds to an a × a Latin Square Design (LSD): Yijk = µ + τi + θj + ηk + εijk, for i, j, k = 1, · · · , a. We assume that εijk5 ∼ N (0, σ2). This corresponds to a square with a rows (Factor A) and a columns (Factor B); each of the resulting a2 cells contains one of the a levels of Factor C occurring once and only once in each row and each column.

Note that in an LSD, N = a2. The ANOVA decomposition is SSTc = SSA + SSB + SSC + SSE, where ΣΣΣ a

SSTc =

a

a

(Yijk —

i=1 j=1 k=1

SS =

Σa Y 2 i··

A

a Σ Y·j·2

B

j=1 a

SS C =

Y···2

a

i=1

SS =

−

ΣY

1 N

2 2 Y··· ) with (a − 1) d.f.

with (a − 1) d.f.

N −

Y···2

a

N

2

2

with (a − 1) d.f. and

Y··· a − N with (a − 1) d.f.

··k

k=1

To test the null hypothesis of no difference in effects due to Factor C, the test statistic is F = MSC/MSE which has an F(a−1),(a−1)(a−2) distribution under the null hypothesis. ΣN ΣN 4 7.22 θ^OLS = 2 i=1Yit2/i t , and follows a normal distribution with mean θ and varii i=1 Σ N ance 4σ2/ i=1 t4i. The 100(1 − α)% confidence interval for θ is 2σ θ^OLS ± q ^ ΣN

4 i=1 ti

tN — 1,α/2.

7.23 By Corollary 7.1.1, the least squares estimates 2of these parametric functions of β have normal distributions. Also, SSE/(N − p) ∼ χ . The resulting t-intervals have the N −p √ √ form in (7.4.3). The 95% C.I. for β 1 is β^1 ± 4.73 11/4. For β2 , it is β^2 ± 4.73 3. For β 3, √ √ ^1−β^2±4.73 59/4. For β1+β3, it is β^1+β3^±4.73 27/4. it is β^3±4.73. For β1−β2, it is β 7.24 Verify that β^ = H

Σ Σ 5

i(X5,i − X5·)(Y5,i − Y 5·)

Σ Σ 5

,

i (X 5,i − X5· )2

^5,0,H = Y 5· − ^βHX5·, l = 1, 2. β

^ 1,0,H )/β ^ H . Let U = (β ^2,0,H −β ^ 1,0,H )−δβ ^^ H = A point estimate of δ is then^ δ = ^(β2,0,H −β (Y 2· − Y 1·) + β^H (X1· − X2· − δ^). Then, U ∼ N (0,uσ2), with " # (X 1· − X 2· − ^ δ)2 1 1 2 2 . σu = σ + +Σ Σ 2 n1 n2 (X − X ) 5· 5 i 5,i

44

Solutions to Chapter 7 Then, SSEH =

ΣΣ 5

(Yi5,i − Y 5·)2 − β^H2 (X5,i − X5·)2, and T =

U/σu [SSEH/σ2 u(n1 + n2 − 3)]1/2

∼ t n1+n2−3

under H: δ = 0. We can obtain the limits of the C.I. by solving the quadratic equation obtained by setting T 2 ≤ tn1+n 2−3,α/2.

7.25 We use Scheffe’s projection approach to construct a simultaneous confidence set for c′β for all c ∈ L in the GLS model. Let A = Σ−1/2, so that AΣA′ = IN , and r(AX) = r(X) = r. Let z = Ay. Then, E(z) = AXβ, and Cov(z) = σ2IN , so that the linear model z = AXβ + ε1, ε1 = Aε ∼ N (0, σ2IN ) corresponds to (4.1.1). The GLS solution β0GLS and error variance estimate σ2GLS are respectively given by (4.5.4) and property 3 of Result 4.5.1. The Scheff´e simultaneous confidence set in this case has the form 0 {β : |c′βGLS − c′β| ≤ σ^GLS (dFd,N −r,α )1/2 [c(X′ Σ−1 X)− c]1/2 ∀ c ∈ L}.

Solutions to Chapter 8 ΣN 8.1 L(θ, y) = (2π)−N/2 exp{− 12 t=1 (Yt − θYt−1)2}. The MLE of the θ is the minimizer of ΣN 2 the function t=1 (Yt − θYt−1) , and is ^= θ

ΣN

t=1 YtYt−1 .

2 t=1 Y t

ΣN

8.2 (a) We can verify that the least squares solutions are µ0 = Y ···, τ 0 = Y i·· − Y ···, i Σa Σb Σn 0 j β = Y ·j· − Y ···, so that SSE = i=1 Σj=1 k=1 (Yijk − Y i·· − Y ·j· + Y ···)2 = a Σ a Σb Σb Σ n 2 n i=1 j=1 (Y ij· − Y i·· − Y ·j· + Y ···) + i=1 j=1 k=1 (Yijk − Y ij·)2 with d.f. abn − (a + b − 1) = n∗, say. We have σ^ 2 = SSE/n∗. Let µij = µ + τi + βj, with ^ij = µ0 + τi0+ βj0. The joint p.d.f. of Yijk is µ , , 1 ΣΣΣ exp − 1 2 abn/2 (Y ijk — µij )2 , ,2σ2 , (2πσ ) , i j k Σ 1 Σ 1 exp − = (Y ijk — µ^ ij )2 + n (µ^ij − µij)2 , 2 abn/2 2 , (2πσ ) , 2σ i,j,k i,j because the product term vanishes by independence.

From the properties of exponential families, it follows that ^ σ2 and µ ^ij are jointly complete, sufficient statistics 2

2

2

2

for σ and µij. Since E ( σ^ ) = σ , σ ^ is unbiased and a function of the complete, sufficient statistics, so it must be the MVUE of σ2. (b) Suppose X = (X1, X2, X3), where X1 corresponds to µ, X2 corresponds to τ1, · · · , τa, and X3 corresponds to β1, · · · , βb. We know that C(Xi) ⊥ C(Xj), i /= j. Σa Σb Now, Z1 = i=1 ciτi0 ∈ C(X2) and Z2 = j=1 ciβj0 ∈ C(X3), so that Z1 and Z2 are independent. Let c denote an N = 1 + a + b dimensional vector with c1, · · · , ca in positions 2, · · · , a + 1, and zeroes elsewhere. Since Z1 = c′(X′X)−X′y, and c′(X′X)−X′[I − X(X′X)−X′] = 0, it follows that Z1 and SSE are independent, i.e., Z1 and U are independent. A similar calculation shows that Z1 and U are independent.

8.3 Since N Y

1 √ QN 2πσ2{ i=1 X i1 X i2 }1/2 i=1 1 2 × exp − 2σ2X X (Y i — β1Xi1 — β2Xi2 ) i1 i2 ( ) 2 ΣN Σ Yiβ1 N N Y iβ 2 Σ Yi = k1 exp − , + + 2σ2Xi1Xi2 σ2Xi2 σ2Xi1

f (y; β1 , β2 , σ2) =

i=1

i=1

i=1

45

46

ΣN it follows that {

2

Yi , i=1 Xi1Xi2

ΣN

Yi

,

ΣN

i=1 Xi1

Solutions to Chapter 8 Yi

} are complete-sufficient statistics for

i=1 Xi2

(β1, β2, σ2). Let β˜1 = Q1/Q2, where

ΣN −N Yi/X i1 , and Yi /X i2 + Σ N i=1 i=1 i=1 X i2 /Xi1 N Σ N2 Q2 = Xi1/Xi2 + ΣN —X /X .

N Σ

Q1 =

We can verify that E(Q1) =

i2

i=1

i=1

(N Σ

i1

) — X i2 /Xi1 i=1 N2

Xi1 /Xi2 + ΣN

i=1

β1,

so that E(β˜1 ) = β1. Similarly, let β˜2 = Q3/Q4, where ΣN N Σ −N i=1 Yi /Xi2 , and Q3 = Yi /X i1 + Σ N i=1 X i1 /Xi2 i=1 N Σ N2 Q4 = Xi2 /Xi1 + ΣN —X /X . i=1

i=1

We can verify that E(Q3) =

i1

i2

)

(N Σ

—N Xi2 /Xi1 + ΣN X i1 /Xi2 i=1 i=1 2

β2,

so that E(β˜2) = β2. β1˜and β2˜are respectively MVUE’s of β1 and β2. It can be verified that )−1 (N Σ N2 2 Var(β̃ 1 ) = σ Xi1/Xi2 − Σ N , and X /Xi1 i=1 i=1 i2 )−1 (N Σ N2 2 Var(β̃ 2 ) = σ Xi2/Xi1 − Σ N .

X /Xi2 i=1 i=1 i1 ^1) and Var(β^2) respectively.

These may be compared with Var(β 8.4 We can show that a

Λ=

Σ

2

2 ni/2

(σ^ i/σ^ )

,

i=1

2

where σ denotes the MSE from the fitted regression in the ith group, and σ2 is the ^i ^ MSE from fitting the same model to all N observations.

The statistic −2 log Λ has an approximate χ2 a−1 distribution.

8.5 (a) From (8.2.6),

ΣN

t t=1ε^

2+

ΣN

^t−1 t=1 ε ε2t 2

ΣN

t=2 ε^tε^t−1 ΣN ε2 t=1 ^ t=1 ^t ΣN Σ 2 2 N +1 1 + ε^N ) t−1 — (ε^ t=1ε^2 ε^2t+ = t=1 — 2r^ε, ΣN ^ 2 t=1 ε t

DW =

ΣN

−

Solutions to Chapter 8 47 Assuming that the effects of ε2 and ε2 are small compared to the overall sum, we ^1 ^N get the result. ΣN −p ΣN −p (b) We require moments of DW = U/V , where U = i=1 νiξi2 and V = i=1 ξ2i (see the proof of Result 8.2.6). Since DW and V are independently distributed, it s follows2 that E(DW ) = E(Us)/ E(V s), i.e., moments of DW are ratios of moments.

Since ξ has a χ2 distribution, the sth cumulant of U is i

1

N −p s 1

κs(U ) = 2 − (s − 1)!

Σ

s

νi.

i−1

ΣN −p ΣN −p 2 In particular, κ1(U ) = i−1 νi and κ2(U ) = 2 i−1ν . iThe moments of V are those of a χ2N −p variable, so that E(V ) = N— p, and E(V 2 ) = (N − p)(N − p + 2).

Therefore, E(DW ) = Var(DW ) =

N −p 1 Σ νi = ν, and N − p i=1 # "N −p Σ ν2 − (N − p)ν2 .

2

(N − p)(N − p + 2)

i i=1

Since N −p

Σ

νi = tr{(I − P)A} = tr(A) − tr[X′AX(X′X)−1], and

i=1 N −p

Σ

2

ν i = tr{(I − P)A}

2

i=1

= tr(A2) − 2 tr{X′A2X(X′X)−1} + tr{X′AX(X′X)−1}2, the result follows.

8.6 SSE(i) = SSE − ε^2/(1 − pii) = SSE − r ∗ 2 σ^ 2 (see Result 8.3.5), from which the result i

i

follows after simplification.

(i)

8.7 By Example 2.1.3, ′

PZ = PX +

(I − PX )yy (I − PX ) y′(I − PX )y

′

εε = P X + ^^′ ε^ ^ ε

from which the result follows. ^ ) = β, ^ and T (F ^ (i)) = β^(i)), 8.8 In (8.3.14), let (x′, Y ) = (x′i, Yi), F = F^ (so that T (F ε = −1/(N − 1), and omit the limit. This gives ^ (i) }.

SICi = −(N − 1) {β^ − β Since ^ (i) = [X′(i)X(i)]−1X′(i)Y(i), and β ^ = (X′X)−1X′(i)Y(i) + (X′X)−1xiYi, β

48

Solutions to Chapter 8 we see that ^ − (X′X)−1xiYi. (X′X)−1X′(i)Y(i) = β Substituting this into the form of β^ ( i ) and simplifying, we see that ^−β ^ (i) = (X′X)−1x i β

ε ^i , 1 − pii

from which the form of SICi in (8.3.18) follows.

8.9 Since Cov(X˜1β) = σ2X1(X′1X1)−X′1 = σ2P1 and Cov(X1β0) = σ2X1(X′X)−X′1 := σ2V, we only need to show P1 ≥ V, i.e., P1 − V is n.n.d., and equality holds if R(X1) ∩ R(X2) = {0}. For any vector v, v′P1v = P1v 2, while v′Vv = v′X1(X′X)−X′1v = (P1v)′X1(X′X)−X′1(P1v) = (P 1v′, 0)X(X′X)−X′

P1v 0

= Pz

2

2

2

where P is the projection onto C(X) and z = ((P1v)′, 0)′. Clearly, Pz ≤ z = P1v 2. Then v′P1v ≥ v′Vv, so P1 ≥ V. Furthermore, if R(X1) ∩ R(X2) = {0}, then there is T, such that X1T = X1 and X2T = O; see the proof of Exercise 4.14. Then since P1v ∈ C(X1), it can be written as X1Tt, and so z = XTt, giving Pz = z. Then 2 2 it follows that v′Vv = z = P1v , so v′P1v = v′Vv, yielding P1 = V. 8.10 For the simple linear regression model, ∗

Wi = Σ N

β^2 (Xi − X)2

1

j=1 (Yj — Y )2/(N − 1) (Xi − X)2 2 . = (N − 1)ρX,Y ΣN — X)2 j=1 (Xj ΣN Since pii = 1/N + (Xi − X)2/ j=1 (Xj − X)2 in a simple regression model, the result follows.

8.11 (a) From Exercise 8.7, we see that ′

PZ = PX +

^^ ε ε . ′ ε^ ^ ε 2

ε^ Let pZii be the ith diagonal element of PZ ; pZii ≤ 1 for all i, implying that pii + ^ε'i^ε, and the result follows. (b) These follow directly from the relations in properties 2 and 3 of Result 8.3.4.

8.12 1 Ci =

pσ ^2

^ −^ ^ −β ^ (i)). (β β (i) ) ′ X ′ X(β

Since ^−β ^ = (X′X)−1x i β (i)

ε ^i , 1 − pii

it follows that ^ (i) )′ X′ X(β ^ −β ^ (i) ) = x′i(X′X)−1xi ( β^ − β

ε2 ^i .

(1 − pii)2

Solutions to Chapter 8

49

8.13 Use the matrix result that if A is a k × k nonsingular matrix, while B and C are k × l matrices, then, |A − BC′| = |A||I − C′A−1B| (see Exercise 2.8). Set A = X′X, and B = C = xi, so that |X′(i)X(i)| = |X′X − xix′i| = |X′X||1 − x′i(X′X)−1xi| = |X′X|(1 − pii).

Since σ2(i) = σ2(N − p − r2)/(N − p − 1), the result follows. i 8.14 We know that for i = 1, · · · , N , APi = 1 −

|Z′(i)Z(i)| SSE(i) |X′(i)X(i)| =1− SSE |X′X| |Z′Z|

where Z = (X, y), and Z(i) = (X(i), y(i)). Since |Z′(i)Z(i)| = |Z′Z − ziz′i| = |Z′Z|(1 − z′i(Z′Z)−1zi) = |Z′Z|(1 − pZii), where pZii is the ith diagonal element of PZ (see Exercise 8.7), we see that APi = pZii, and the first result follows directly from Exercise 8.7. The second result follows from (8.3.8).

8.15 By definition of influence curve, ICT,F (x) = ∫

ψ(x − T (F )) ψ′(x − T (F )) dF

.

For the median, ψ(z) = sign(z), and d dz

sign(z) = 2δ(z = 0),

where δ(z) = 1 if z = 0 and is 0 otherwise. Therefore ∫ ∫ ∫ ψ′(x − T (F )) dF = 2δ(z = 0) dF = 2δ(z = 0)f (z) dz = 2f (T (F )).

It follows that ICT,F (x) =

sign[x − T (F )] .

2f [T (F )]

Solutions to Chapter 9

9.1 Verify that X′X is equal to , N 0 ΣN 2 0 i=1(Xi1 − X1) ΣN 0 i=1 (Xi1 − X1)(Xi2 − X2) ΣN since i=1 (Xij − X j ) = 0, j = 1, 2. Also, 2

r12 =

0

ΣN

i=1(Xi1 − X1)(Xi2 − X2) ΣN 2 i=1 (Xi1 − X1)

Cov(Xi1, Xi2)}2 Var(X ) Var(Xi2) i1

ΣN

{ = ΣN

i=1(Xi1 − X1)(Xi2 − X2)}2

i=1 (Xi1 − X1)2

ΣN

2 i=1 (Xi2 − X2)

Simplifying the (2, 2)th element of σ2(X′X)−1, we see that Var(β^1 ) = σ2

,Σ N i=1

(Xi1 − X1)2(1 − r212 ).

The width of the 95% confidence interval for β1 is 2tN−3,.025 est. s. e.(β^1).

9.2 Let y = (Y1, · · · , YN )′, X = (1N , X1), where X1 is an N × k matrix, β = (β0, β1, · · · , βk)′, and ε = (ε1, · · ·, εN )′ with ε ∼ N (0, σ2IN ). Now, H: βj = d is equivalent to H: C′β = d where C′ is a (k + 1)-dimensional vector with 1 in the jth position and zero elsewhere. Following the method in section 7.3.2, we obtain ^ = Y − x′B−1(X′1y − NY x) β B−1(X′1y − NY x)

,

where B = X′1X1 − N xx′, so that C′β^ − d = β^j − d, and Q = (β^j − d)2/ajj, ajj being the jth diagonal element of (X′X)−1. Also, s = 1, and SSE = y′(I − X(X′X)−1X′)y, which has a χ2 N −k−1 distribution. Then, by Result 4.3.5, Q F (H) =

SSE/(N − k − 1)

∼ F1,N −k−1

under H. 9.3 Since σ^2 = y′(I − P)y, by Exercise 5.27, E(σ^ 2 ) = σ2, and Var(σ^ 2 ) = 2σ4/(N − p) =

M S E ( σ^ 2 ). Similar calculations show that E(y′A1y) = (N − p)σ2/(N − p + 2), the bias being 2σ2/(N − p + 2). Also, Var(y′A1y) = 2(N − p)σ4/(N − p + 2)2. Then, MSE(y′A1y) = Var(y′A1y) + (Bias)2 = 2σ4/(N − p + 2) < M SE(σ ^ 2 ).

9.4 Y0 = x′0β + ε0, where ε0 ∼ N (0, σ2). By the Gauss–Markov Theorem, ^ Y0 = x′0β^ ∼ 2 ′ 1 ′ ′ − N (x 0β, σ x 0(X X) x0).

51

52

Solutions to Chapter 9 (a) E(Y0) = E(x′0β + ε0) = x′0β, and since ε0 is independent of ε, Var(Y0 − Y^0) = Var(Y0) + Var(Y^0 ) = σ2 + σ2x′0(X′X)−1x0. Replacing σ2 by its least squares estimate gives the required result. We know that (N − p)σ^ 2 /σ 2 ∼ χ2N −p . The distribution of the given statistic is therefore t N −p . (b) The 95% symmetric two-sided prediction interval for Y0 is ^ ± tN −p,0.025σ[1 {x′0β ^ + x′0(X′X)−1x0]1/2}. (c) The 100(1 − α)% lower bound for γ(η) is ˆ − zα^σ[1 + x′0(X′X)−1x0]1/2, γ(η)

where ˆ γ(η) = x′0β^ + zησ, assuming σ is known.

9.5 The unrestricted least squares estimates of the parameters are obtained by minimizing Σ Σ2 Σnl (Y5,i−β5,0 −β5,1X5,i)2 as β^5,0 = Y 5· −β^5,1 X 5· and β^5,1 = [ nl (Y5,i−Y 5·)(X5,i− 5=1 Σi=1 i=1 nl X5·)]/ i=1 (X5,i − X5·)2, l = 1, 2. Also, the unrestricted error sum of squares is SSE = Σ 2 Σ nl Σ nl 2 2 2 5,1 5=1 [ i=1 (Y5,i − Y 5·) − β^ i=1 (X5,i − X5·) ]. (a) For a test of parallelism, the restricted least squares estimates and SSEH under Σ2 Σ nl (Y − β − H: β = β = β , say, are obtained by minimizing 1,1

2,1

1

5=1

β1X5,i)2 as

i=1

5,i

5,0

β^5,0,H = Y 5· − β^1,H X5·, l = 1, 2, 2 nl 2 nl ΣΣ ΣΣ ^ β 1,H = (Y5,i − Y 5·)(X5,i − X5·)/ (X5,i − X5·)2.

5=1 i=1

5=1 i=1

Also, 2

SSEH =

nl

ΣΣ

(Y5,i − Y 5·)

5=1 i=1 2

SSEH − SSE =

Σ

2

2 nl 2 ΣΣ (X5,i − X5·)2, − β^1,H 5=1 i=1

nl

β^1,H 2

Σ

2

i=1

5=1

l ΣΣ

2

(X5,i − X5·) − β^1,H 2

and

n

(X5,i − X5·)2.

5=1 i=1

The statistic F =

SSEH − SSE F SSE/(N − 4) ~ 1,N −4

distribution under H. (b) We test the hypothesis that the two lines meet at a point on the y-axis, i.e., H : β1,0 = β2,0 = β0, say. The restricted least squares estimates and SSEH under H are obtained by minimizing l ΣΣ

2

n

(Y5,i − β0 − β5,1X5,i)2

5=1 i=1

as

(

Σ nl

),( 2 2 Σ i=1 Y5,i X5,i X β^0,H = Y·· − 5· Σnl Σ n 2 l 2 N− i=1 X 5,i i=1 X 5,i 5=1 5=1 ,Σ nl nl Σ β^5,1,H = (Y5,i − β^0,H )X5,i X25,i, l = 1, 2. i=1

2

Σ X1·

i=1

) , and

Solutions to Chapter 9

53

Also, the restricted error sum of squares is SSE H =

2 nl Σ Σ

(Y5,i − ^ β0,H − ^ β5,1,H X5,i)2,

5=1 i=1

and the test statistic F = (SSEH —SSE)/ SSE/(N { − 4)} has an F1,N−4 distribution under H. (c) The restricted least squares estimates and SSEH under H: β1,0 = β2,0 = β0; β1,1 = β2,1 = β1, say, are obtained by minimizing l ΣΣ

2

n

(Y5,i − β0 − β1X5,i)2

5=1 i=1

as

β^0,H = Y ·· − β^1,H X··, and ( 2 nl ) , 2 nl ΣΣ ΣΣ ^ β 1,H = (Y5,i − Y ··)(X5,i − X··) (X5,i − X··)2.

5=1 i=1

5=1 i=1

The restricted error sum of squares is 2

SSEH =

nl

ΣΣ

2

(Y5,i − Y ··)2 − (β^1,H )2

5=1 i=1

nl

ΣΣ

(X5,i − X··)2.

5=1 i=1

Then, F (H) = (N − 4)(SSEH − SSE)/(2SSE) ∼ F2,N−4 under H. Σ nk Σ nk Σ nk 9.6 (a) Let Y k = Yk,i/nk, Xk = Xk,i/nk, S(k) = (Xk,i − Xk)2, and i=1 i=1 XX i=1 Σ k (k) SXY = ni=1 (Yk,i − Y k)(Xk,i − X(1) k), k = 1, 2. The least squares estimates are (2) (1) (2) ^ ^ i = Y i, i = 1, 2, and β = [SXY + SXY ]/[SXX + SXX ]. The vertical distance α ^ 1 − X2). between the lines is D = (α1 − α2) + β(X1 − X2), with D^ = Y 1 − Y 2 − β(X ^ = β, we see that E(D) ^ = D. Since E(Y k) = αk, for k = 1, 2, and E(β) (b) We can verify that^αk ∼ N (αk, σ2/nk), k = 1, 2, and ! σ2 . ^ ∼ N β, (1) β (2) SXX + SXX 2

2

(1)

(2)

Hence, d = ^D − D ∼ N (0, σ {1/n1 + 1/n2 + (X1 − X2) /[SXX + SXX ]}). An unbiased estimate of σ2 is SSE/(N — 3), which is independent of D ^ − D. A 95% symmetric C.I. for D is ^ ± [F1,n +n −3,.05 ] 1/2 sˆ ^ ).

D . e.(D 1 2 ^ ∗′β ^ ∗− β ^ ∗′( θ ) β^∗ (θ) > 0 for θ > 0. Since 9.7 We must show that β ∗′ ∗

∗′

∗

^β ^β −^ β (θ)β ^ (θ) = y∗′X∗{(X∗′X∗)−2 − (X∗′X∗ + θIp)−2}X∗′y∗, it is sufficient to show that the matrix (X∗′X∗)−2 − (X∗′X∗ + θI)−2 is p.d. Suppose X∗′X∗ = PDP′, where P is orthogonal and D = diag(d1, · · · , dp). Then (X∗′X∗)−2 − (X∗′X∗ + θI)−2 = PD−2P′ − (PDP′ + θIp)−2 = P[D−2 − (D + θIp)−2]P′, which is p.d., since D−2 − (D + θIp)−2 = diag(d1−2 − (d1 + θ)−2, · · · , dp−2 − (dp + θ)−2), and clearly, di −2 − (di + θ)−2 > 0 for all i.

54

Solutions to Chapter 9

9.8 (a) Let P1 be the projection matrix in the smaller model y = X1β1 + ε. Consider (I − P1)y = (I − P1)X1β1 + (I − P1)X2β2 + (I − P1)ε = (I − P1)X2β2 + (I − P1)ε, and let r((I − P1)X2) = p2. Then, ^ 2 = [((I − P1)X2)′(I − P1)X2]−1((I − P1)X2)′(I − P1)y β = (X′2(I − P1)X2)−1X′2(I − P1)y. (b) From Result 5.4.8, ^β2 is independent of SSE1 = y′((I − P1)y if and only if (X′2(I − P1)X2)−1X′2(I − P1)(σ2IN )(I − P1) = O which can happen only if X′2(I—P1) = O, which contradicts the model assumption.

Hence, they are not independent.

9.9 (a) From Exercise 4.14, X1β1 is estimable under the expanded model. Since X1 has full column rank, β1 = MX1β1 with M = (X′1X1)−1X′1. Then β1 is estimable. (b) From (4.2.36), ˜β1 = [X′1(I − P2)X1]−1X′1(I − P2)y; note that X′1(I − P2)X1 is invertible because (I− P2)X1 has full column rank.

Then ˜ 1) = σ2[X′1(I − P2)X1]−1.

Cov(β ^ 1) = σ2(X′1X1)−1. Let A = X′1(I − P2)X1). From On the other hand, Cov(β Sherman–Morrison–Woodbury formula in Result 1.3.8, with B = X′1P2, C = I, and D = P′2X1 therein, (X′1X1)−1 = (A + BCD)−1 = A−1 − A−1X′1P2(I + P2X1A−1X′1P2)−1P2X1A−1, thus (X′1X1)−1 ≤ A−1 with “=” if and only if A−1X′1P2(I + P2X1A−1X′1P2)−1P2X1A−1 = O. Since A is nonsingular and (I + P′2X1A−1X′1P2)−1 is p.d., the latter is equivalent to P2X1 = O, i.e., all the columns of X1 are orthogonal to those of X2. Σk 9.10 Let M1 be the model Yi = β0 + j=1 βjXij + εi, i = 1, · · · , N , i.e., y = Xβ + ε. We can Σ · · · , N , or y = Wδ + ε, write the bigger model M asi Y = 0β + k+1 2 j=1 β j X ij+ ε ,i i = 1, ′ ′ where W = X xk+1 , and δ = (β , βk+1) . Then, ^δ = (W′ W) −1 W ′ y.

Let (x′0, x0,k+1)′ denote a vector of new predictors. Then, the predicted response under Model M2 is Y^0 = X xk+1 δ^ .

Solutions to Chapter 9

55

Let c = x′k+1(I − P)xk+1)−1 and d = (X′X)−1X′xk+1. Then, (X′X) −1 + cdd′ −cd −cd′ c 2 ′ 1 2 ′ − ′ = σ [x 0(X X) x0 + c(d x0 − x0,k+1) ]

^0) = σ 2(x ′ , x0,k+1) Var(Y 0

x0 x0,k+1

≥ σ2x′0(X′X)−1x0, the last term on the right being Var(Y^0) under M1.

9.11 If t2j is the square of the t-statistic for tetsting H0: βj = 0 in the model Yi = β0 + Σk j=1 βjXij + εi, i = 1, · · · , N , it is the extra-SS F -statistic given by Fj = tj2 =

SSEk−1 − SSEk SSEk/(N − k)

,

so that t2SSEk SSEk−1 =

j

N −k

+ SSEk.

Since Mallow’s Ck−1 is defined by Ck−1 =

SSEk−1 − (N − 2(k − 1)). ^σ2

Substitute for SSEk−1 and simplify t o give tj2SSEk + SSEk − (N − 2k) − 2 σ σ ^22(N − k) ^2 t SSEk = j + Ck − 2. σ^2 (N − k)

Ck−1 =

Solutions to Chapter 10

10.1 Write the two-part hypothesis in the form C′β = d, where C′ =

c1,1 c2,1

· · · c1,a , β′ = (µ , 1 · · · , µa), · · · c2,a

c1,2 c2,2

and d = (d1, d2)′. The test statistic follows directly from Result 7.2.1. Σb Σa Σa Σa 10.2 U1a = j=1 Uij = Uii + j/=i Uij = Ni· − 5=1 j=1 nijn5j/N·j = Ni· − Ni· = 0, i = 1, · · · , a. This implies that r(U) ≤ a − 1.

10.3 (a) From (10.1.9), we see that for i = 1, · · · , a, E(Wi) = Ui1τ1 + · · · + Uiaτa = φi, say. Hence, for i = 1, · · · , a, Wi is the b.l.u.e. of φi, from which the required result follows, irrespective of the rank of U. (b) In (10.1.3), in order that , a Σ E

b

i=1 j=1

Σ

cijYij

a ΣbΣ

, b a Σ Σ cij + cij

i=1 j=1

i=1

=µ

j=1

τi +

a Σb Σ j=1

! cij θj

i=1

Σa Σ Σ b cij = 0, and b cij = 0, is to be function only of θ’s, we must have j=1 i=1 j=1 Σb j=1 (Σ a Σ Σb j=1 dj = b i=1 c )θ = j=1 d θ , where iΣ= 1, Σ · · · , a. Then, RHS = ij j j j b a j=1 i=1 cij = 0.

10.4 (a) Formulas for Var(ya), Var(yb), and Cov(ya, yb) follow directly from the assumptions of the model (10.1.3). Σb Σa (b) Since Wi = Yi·· − j=1 nijY·j·/n·j, and Zj = Y·j· − i=1 nijYi··/ni·, the required results follow directly from (a). (c) Cov(w, yb) = Cov(ya − ND−1yb, yb) = N − ND−1Db = O. The other result is b

b

similar.

10.5 For any filled cell, the proof of property 1 of Result 10.1.6 is obvious since then µij = E(Yijk). Property 2, property 3 and property 4 are immediate consequences of property 1 and property 2 of Result 4.3.1.

10.6 To show (a), for i = 1, · · · , n, we can write the function in (a) as ( ) Σb Σa nij kj n (µ + τk + θj + (τθ)kj) , nij(µ + τi + θj + (τθ)ij) − n·j j=1

k=1

which shows it is estimable. The proof of estimability of the function in (b) is similar.

57

58

Solutions to Chapter 10

10.7 (a) Yes (b) No 10.8 We see from Example 4.1.3 X′X =

N

n′

n

D

where N = n1 + · · · + na, n = (n1, · · · , na)′ and D = diag(n1, · · · , na). Then G = diag(0, n1−1, · · · Σ , na−1) is a (X′X)−. Let β = (µ, τ1, · · · , τa)′ and cj = (0, cj1, · · · , cja)′, a Σathe contrasts are orthogonal if j = 1, 2. Then τi =the c′j β.left From Definition 4.3.3, and only if c′ Gc i=1 = 0.cj,iBut hand side exactly c c /n , so the first part of 2

1i 2i

i

Σa ′ Result 10.1.2 follows. From Gauss-Markov theorem, the b.l.u.e. of cj β = i=1 cjiτi is 0 0 ′ c β and one choice of β , according to Example 4.2.5, is (0, Y 1·, · · · , Y a·)′, so c′ β0 = j j Σa c Y .

From Result 4.2.4, the covariance of the b.l.u.e.’s of the two contrasts is ji i· i=1 σ2c′1Gc2 and since the contrasts are orthogonal, we see from above derivation that covariance is 0.

1

i=1

10.9 Set cij = 1/b for i = 1, · · · , a and use property 3 of Result 10.1.6.

10.10 Define ui = ni·/N , i = 1, · · · , a and vj = n·j/N , j = 1, · · · , b. The least squares solutions are µ0 = Y ···, τi0 = Y i·· − Y ···, θj0 = Y ·j· − Y ···, and (τθ)ij0 = Y ij· − Y i·· − Y ·j· + Y ···, Σ Σ Σ for i = 1, · · · , a, and j = 1, · · · , b. Also, SSE = i j k(Yijk − Y ij·)2 ∼ σ2χ2 N −ab .

Under H: (τθ)ij = 0 for all i, j, the restricted solutions are 0

µ H = Y ···, 0

τ i,H = Y i·· − Y ···, and 0

θ j,H = Y ·j· − Y ···, while SSEH =

ΣΣΣ i

j

(Yijk − Y i·· − Y ·j· + Y ···)2 ∼ σ2χ2N −(a+b−1) . k

Then, F (H) =

(SSEH − SSE)/(a − 1)(b − 1) SSE/(N − ab)

~ F (a−1)(b−1),N −ab

under H. 10.11 If we impose the restrictions τ 0i = 0, i = 1, · · · , a and µ0 = 0, we can solve the normal equations in the two-way nested model for θ0 = Yij· /nij = Y ij·, j = 1, · · · , bi; j(i)

i = 1, · · · , a. The g-inverse corresponding to this solution is verified to be G = where D = diag{1/nij}.

0 0 , 0 D

Solutions to Chapter 10

59

10.12 Since the expected value of any observation is estimable, it follows that µ + τi + θj(i) is estimable, as is any linear combination. The form of the b.l.u.e. follows from Result 4.4.1 Σ as j wij Y ij·.

10.13 Minimizing ΣΣ a

S(µi, γ1, γ2) =

b

(Yij − µi − γ1Zij − γ2Wij)2

i=1 j=1

with respect to µi, i = 1, · · · , a, γ1 and γ1, we obtain µ^ i = Y i· − γ^1 Z i· − γ ^2 W i· , Σa Σa Σ Σa — i=1 Swzi ai=1 Syw i S i=1 wwi i=1 Syz i Σa Σa , and γ^1 = Σa Szzi Swwi — ( i=1 Σ Swz )2 Σa i=1 Σa Si=1 Σa Szw a i S yw — yz S i i i i=1 zzi i=1 i=1 i=1 Σa Σa a γ^2 = Σ 2 i=1 Swwi i=1 Szzi — ( i=1 Szwi ) where, b

Swwi = Syzi =

Σ

2 2 (W ij − Wi·), i=1 b

Σ

Σ

Σ

2

2

(Z ij − Zi·),

i=1

(YijZij − Y i·Zi·),

i=1 b

Swyi =

b

Szzi =

Swzi =

b

Σ

(WijZij − Wi·Zi·), and

i=1

(Yij Wij − Wi·Zi·).

i=1

(a) It may be verified that E(Syzi ) = γ1Szzi + γ2Swzi , E(Sywi ) = γ2Swwi + γ1Swzi , so that E(γ^1) = γ1, i.e., γ1^is an unbiased estimator of γ1. (b) Using the notation in the text, we see that Eww Var(γ ) = σ2 , ^1 Ezz Eww − (Ewz )2 Ezz Var(γ , and ^2) = σ2 2 Ezz Eww − Ewz(Ewz ) 2 Cov(γ , γ ) = −σ . ^1 ^2 E E − (E )2 zz

ww

wz

(c) Clearly, γ^ 1 and γ^ 2 are independent if and only if Cov(γ^1 , γ^ 2 ) =Σ0 (under the normala Σb Z W = ity assumption). That is, they are independent if and only if ij i=1 j=1 ij 0. Σ2 10.14 Σ b (Wij − µi)2 with respect to µi to obtain Set Wij = Yij − γiXij, and minimize µi0 = W i· ; then, SSEW =

Σ2i=1 Σbj=1

i=1

j=1

(Wij − Wi·)2. Substitute for Wij = Yij − γiXij,

60

Solutions to Chapter 10 Σ2

Σb and Wi· = Y i·−γiXi· into SSEW . Minimize S∗ = i=1 j=1 (Yij −γiXij −Y i· +γiXi·)2 with respect to γi to obtain Σb (Yij − Y i·)(Xij − Xi·) j=1 , i = 1, 2. γ^i = Σb 2 j=1 (Xij — Xi· ) Then, 2

SSE =

ΣΣ

b

(Yij − µ0 i− γ^i X ij ) 2 ∼ χ22b−4 .

i=1 j=1

Under H, start with the model Yij = µi + γXij + εij, and obtain the solutions µ0i = W i· , i = 1, 2, and Σ2

Σb

i=1

(Yij − Y i·)(Xij − Xi·)

j=1 Σ2 Σ b 2 i=1 j=1 (Xij — Xi·)

γ^ =

.

Then, 2

SSEH =

ΣΣ

b i γ (Yij − µ0 − ^ X ij ) 2 ∼ χ22b−3 ,

i=1 j=1

and F (H) =

(SSEH − SSE)/1

~ F 1,2b−4

SSE/(2b − 4)

under H. It can be verified that this is the same statistic we obtain when we test for parallelism between two lines.

10.15 (a) The normal equations are ΣΣ i

where N =

^0 Y·· = N β Σ Yij(Zi − Z) = β^1 ni(Zi − Z)2,

j

i

Σa

i=1 ni. Then,

SS(β0, β1) = [Y 2··/N ] + β^12 (b) SS(µ, τ1, · · · , τa) =

Σa i=1

Σ

ni(Zi − Z)2.

i

(Y /ni) − Y /N . Hence, 2

2

i·

··

SS(β0, β1) − SS(µ, τ1, · · · , τa) =

Σa d2 − ( 2 Σa cidi)2 c i=1 i=1 i i=1 i , Σa 2 i=1 ni(Zi − Z)

Σa

√ √ where ci = ni(Y i· − Y ··) and di = ni(Zi − Z). The difference in SS is always nonnegative (by Cauchy–Schwarz inequality), with equality only when ci ∝ di, i = 1, · · · , a, i.e., only when Y i· − Y ·· = w(Zi − Z), where w is a constant.

Solutions to Chapter 10

61

0 10.16 Consider the model Wijk = µij + εijk, where Wijk = Yijk − γijZijk. Here, µij = Wij·, Σ Σ Σ 2 ∗ and SSEW = i j k(Wijk − Wij·) . Let S denote the expression obtained by

replacing Wijk and Wij· respectively by Yijk− γijZijk and Y ij· − γijZij· in SSEW .

Minimizing S∗ with respect to γij gives ,Σ Σ γ^ i j = (Yijk − Y ij·)(Zijk − Zij·) (Zijk − Zij·)2, and k k ΣΣΣ SSE = [Yijk − Y ij· − γ^ij (Z ij k − Zij·)]2. i

j

k

Repeating the same steps with Wijk = Yijk − γZijk under H: γij = γ gives Σ Σ Σ

j k(Yijk − Y ij·)(Zijk − Zij·) , and Σ Σ Σ 2 ijk — Z ij· ) j (Z i k ΣΣΣ SSEH = [Yijk − Y ij· − γ^ (Z ij k − Zij·)]2 ∼ χ2abc−(ab+1)

γ ^H =

i

i

j

k

under H. Hence, F (H) = (abc−2ab)(SSEH −SSE)/(ab−1)SSE ∼ Fab−1,abc−2ab under H. 10.17 Let Ω1 = C⊥(X) and Ω2 = C(Z). Then Ω1⊥ +Ω2 = C(X, Z) and PΩ1 Ω2 = (I − P)C(Z) = C((I − P)Z). Then the proof follows from property 2 of Result 2.6.7.

10.18 Let SH0 = {y = Xτ + Zβ + ε}, with dim(SH ) = r+q < p+q. Let H1: Zβ = 0, let τ 0 = (X′X)−X′y and Y^H = Xτ 0 .

H1

0

Then,

1

H1

Y^H0 = Xτ 0 + Zβ0 = X[(X′X)−X′y − (X′X)−X′Zβ0] + Zβ0 = Xτ 0H1+ (I − P)Zβ0 = Y^H1 + (I − P)Zβ0.

The sums of squares are R(H0) = Y^H′ 0 Y^H0 = R(H1) + β0′Z′(I − P)Zβ0, so that R(H0 | H1) = R(H0) − R(H1) = β0′Z′(I − P)Zβ0.

Note that dim(SH1 ) = r, so that dim(SH0 )−dim(SH1 ) = q. Now, R(H1)/σ2 ∼ χ2(r, λ1), where λ1 = 2σ 112 {(Xτ +Zβ)′[X(X′X)−X′](Xτ +Zβ)}. Also, R(H0 | H21)/σ2 ∼ χ2(q, λ2), where λ2 = 2 (β′Z′(I − P)Zβ). Since SSE = y′y − R(H0) ∼ χ , R(H1) and 2σ N −r−q R(H | 0 H1) are distributed independently of each other, and of SSE, the usual F -tests follow.

10.19 We can write c′5 = √ 1

5(5+1)

(0, 1, · · · , 1, −l, 0, · · · , 0). Since we can verify that c′51 = 0,

c′5β is a contrast in the τ ’s and must be estimable. By Result 4.4.1, " 5 # " 5 # Σ Σ 1 1 0 ˆ c′5β = c′5β0 = √ τ 0i − lτ 5+1 = √ Y i· − lY 5· , l(l + 1) i=1 l(l + 1) i=1

62

Solutions to Chapter 10 since τi0 = Y i·, i = 1, · · · , a. Since Y i· ∼ N (µ + τi, σ2/n), Va r ( c5 ′̂ β) = l(l + 1)σ2/n, and σ2 = SSE/a(n − 1). The 95% marginal C.I. for c′5β is ^ σ ^ c′5β0 ± n t a(n−1),.025 To construct Scheff´e simultaneous confidence intervals to control FBER at α, note that (C′β0)′(C′GC)−1(C′β0)/(a

— 1)

MSE

∼ F (a − 1, N − a, λ),

where λ is the non-centrality parameter and C′ ∈ R(a−1)×a given by ,

√1 2 √1 6 √1 12

.

C′ = . .

√ 1

(a−1)a

−1 √ 2 √1 6 √1 12

−2 √ 6 √1 12

0 0

··· ···

0 0

−3 √ 12

···

0

.

.

.

1 √(a−1)a

1 √(a−1)a

1 √(a−1)a

···

√ −a

0

The 100(1 − α)% Scheff´e intervals are r c′ β0 ± MSE A

(a − 1)

. . . .

(a−1)a

F a−1,N −a,α

n

for l = 1, · · · , a − 1.

10.20 Let X be a univariate test statistic whose null distribution has a continuous c.d.f. F . By definition, its lower-sided p-value is F (X) and upper-sided p-value is—1 F (X). Define the left-continuous inverse of F , F ∗(u) = inf { x : F (x)≥ u} . Then for u ∈(0, 1), under the null distribution, P (F (X) < u) = P (X < F ∗(u)) = P (X ≤ F ∗(u)) = F (F ∗(u)).

By continuity of F , F (F ∗(u)) = u. Then P (F (X) < u) = u, so F (X) is uniformly distributed on (0, 1) and so is 1 − F (X).

10.21 (Hsu, 1984) derived constrained simultaneous C.I.s for multiple comparisons with the best. Assumptions are: 1. independent samples of size n for each of a treatment levels, 2. distributions of the responses only differ in their locations.

In the balanced one-way ANOVA model, Yij ∼ N (µ + τi, σ2), and the assumptions are met. For each i = 1, · · · , a, Hsu constructed simultaneous C.I.s for τi − maxA/=i τA. The idea is this: consider i = 1; if the upper bound for τ1 — maxA/=1 τA is < 0, then level 1 is not competitive and can be discarded. Suppose for i = 3, the lower bound for τ3 — maxA/=3 τA is ≥ δ−ast, where δ∗ is close to zero, then level 3 is acceptable. Thus, this method is useful to see which treatment levels are useful or not useful. Hsu constructed simultaneous C.I.s for τi − maxA/=i τA, for i = 1, · · · , a as τi − max τA ∈ (Y i· − max Y A· ± d)± A/=i

A/=i

Solutions to Chapter 10

63

or, equivalently as [−(Y i· − max Y A· − d)−, (Y i· − max Y A· − d)+], A/=i

A/=i '

where c− = min(c, 0) and c+ = max(c, 0). Obtain d = √dnS , where S =

√

MSE and

∫ ∞∫ ∞ 0

−∞

(Φ(z + d′S))a−1dΦ(z) dQ(S) = α,

Q denoting the distribution of S/σ. For details, see (Hsu, 1984). {− } exp {− y /2 } × exp { y′Xβ} .

10.22 (a) The p.d.f. of y is f (y) = (2π)−N/2 exp Xβ /2 ′ ′ { } { } Noting that exp y Xβ = exp (Py) (Xβ) , the sufficiency of Py follows from Neyman’s factorization criterion (Lehmann and Casella, 1998, Section 1.6). Next, suppose h(Py) is an integrable function and ∫ h(Py)f (y) dy = 0 2

2

for all values of Xβ. Let Q = (q1, · · · , qN ), with q1, · · · , qr consisting of an orthonormal basis of C(X) and the rest consisting of an orthonormal basis of C(X)⊥. Then P = Q Ir O Q′, Q′Xβ = (θ′, 0′)′, where θ = (θ1, · · · , θr)′ with θi = q′iXβ. Mak e change of variable z = (z′1, z′2)′ = Q′y with z1 ∈ Rr. Then h(Py) = h(Q z01 ) := k(z1) and from the factorization of f (y), the above condition is equivalent to ∫ 2 1 0= k(z1 ) exp{− z } exp{z′Q′Xβ} dz (2π)N/2 ∫ 2 1 = k(z1 ) exp{− z1 2 − z2 } exp{z′1θ} dz 1 dz 2 (2π)N/2 ∫ 2 1 = k(z1 ) exp{− z1 } exp{z′1θ} dz 1. r/2 (2π) Note that θ can take any values in Rr. Therefore, the Laplace transform of k(z1) is constant 0, yielding k(z1) ≡ 0. Then h(y) ≡ 0, showing the completeness of Py. (b) If V is p.d., then V] = V−1. By comparing (4.5.4) and (10.5.3), β0 in Result 10.5.1 is the same as β0GLS . Let L = V−1/2. From Section 10.5.1, β0GLS is the LS solution to fit the model z = Mβ + η, where z = Ly, M = LX, and η = Lε ∼ N (0, IN ).

By (a), Mβ0GLS is complete and sufficient for Mβ, which combined with M = LX yields the proof. (c) We have Cov(Lε) = LVL′ = In O so Lε = (η′, 0)′ with η ∼ N (0, In). Then (10.5.5) follows easily. It is easy to check V] is a g-inverse of V. Let Q be the matrix of projection onto C(M′2) = R(M2), so that M′2A′ = (AM2)′ = (−M1Q)′ = −QM′1. Let T = ((I − Q)M′1, M′2)′. Then X′V]X = (M′ ,1 M′2)

In A′

A In + A′A

M1 = T′ T M2

(S.1)

and Tv = 0 ⇐⇒ M1(I − Q)v = M1v + AM2v = 0 and M2v = 0 ⇐⇒ Xv = 0, i.e., R(T)⊥ = R(X)⊥, so r(X′V]X) = r(T′T) = r(T) = r(X). Next, put K = L−1

64

Solutions to Chapter 10 and let S1 consist of the first n rows of K′N, and S2 consist of the rest of the rows of K′N. Then X′V]VN = (M′ 1, M′2)

In A A′ In + A′A

In O K′N = (I − Q)M′1 S1.

O O

On the other hand, M′1S1 + M′2S2 = (M′1, M′2)K′N = X′N = O, so premultiplying both sides by I − Q and combining with the above display yields X′V]VN = O. Thus V] ∈ ΩV. (d) From Section 4.6.1, there is λ0 that together with β0 solves M′1M′1β0 + M′2λ0 = M′1z1,

M2β0 = z2.

(S.2)

From (S.1), X′V]Xβ0 = (I − Q)M′1M1(I − Q)β0 + M′2M2β0.

On the other hand, X′V]y = (M′1, M′2 )

In A′

A In + A′A

z1 z2

= (I − Q)M′1z1 + (I − Q)M′1Az2 + M′2z2 = (I − Q)(M′1M1β0 + M′2λ0) + (I − Q)M′1AM2β0 + M′2M2β0 = (I − Q)M′1M1β0 − (I − Q)M′1M1Qβ0 + M′2M2β0.

Then β0 solves (10.5.3). (e) The solutions to z2 = M2β can be written as β = β∗ + (I— M2−M2)γ, where γ 2 is any vector inR N−n. Plugging this into z1 = M1β + η and z1− M1β yields the proof. (f) From Result 10.5.1, Xβ0 is invariant to the choices of β0 and V]. Hence we can let β0 be as in (d). Then from (e), we have M1β0 = M1β∗ + Bγ0, with γ0 2 minimizing w —Bγ . Since Cov(η) = IN−n, from (a), Bγ0 is a complete and sufficient statistic of Bγ. It follows that M1β0 = Bγ0 + M1β∗ is a complete and sufficient statistic of M1β = Bγ + M1β∗ under the restricted model (10.5.5), so 0 M1β 1β 0 LXβ0 = MM = LXβ under 2β is a complete and sufficient statistic of M2β (10.5.5). Since L is nonsingular, Xβ0 is a complete and sufficient statistic of Xβ. (g) Since c′β is estimable, it is equal to t′Xβ for some t. From Result 10.5.1, c′β0 = t′Xβ0 is an unbiased estimator of t′Xβ with finite variance and is a function that only depends on Xβ0. Since Xβ0 is a complete and sufficient statistic of Xβ, then c′β0 = t′Xβ0 is the unique b.u.e. of c′β (see Lehmann and Casella, 1998, Section 2.1).

Solutions to Chapter 11

11.1 (a) Let V] and V] denote any two members of ΩV. Since r(X′V] X) = r(X′V] X), 1 2 1 2 there exists a matrix H such that H(X′V] X) = X′V] X. By Result 10.5.1, X′V] y 1

2

2

is essentially b.l.u.e. for X′V2] Xβ. Let β^1 be such that X′V1] X β^ = X′V1] y. Again

by Result 10.5.1, ^ ′ ] H(X′V]1y) = H(X′V] X 1 β ) = b.l.u.e. (HX V1 Xβ) = b.l.u.e. (X′V2] Xβ) = X′V]2y.

Hence H(X′V] X = X′V] y) is equivalent to (X′V] X = X′V] y). Since H is 1 1 2 2 nonsingular, (X′V] X = X′V] y) is equivalent to (X′V] X = X′V] y), and the 1

1

2

2

result follows. (b) Since C(N) = N (X′), we have X′N = O and N′X = O. Let u be an N -dimensional vector inC (X) ∩(VN); then u = Xr for some r and u = VNs for some s. Also, C N′VNs = N′u = N′Xr = 0, which implies that (Ns)′VNs = s′N′VNs = 0, implying that VNs = 0, i.e., u = 0. This implies that C (X) andC (VN) are essentially disjoint, with only 0 in common. This, together with (VN) C ⊂ C (V) implies that C(X, VN) ⊂ C(X, V).

(S.3)

Since C(X) and C(VN) are essentially disjoint, r(X, VN) = r(X, V).

The result follows from S.3 and S.4.

11.2 The form of Var(µ ) follows from (11.3.1) using Var(Y ^GLS fying.

(S.4) ) = σ2 + σ2/n , and simplii·

τ

ε

i

11.3 Since y = (y1′ , · · · , ya′ )′, where yi = (Yi1, · · · , Yin)′. Then y′((Ia ⊗ Jn)/n − JN /N )y ,Jn/n ′ ′ .. = (y1 , · · · , ya ) .

, y1 .

Jn/n

=

1 n

a Σ i=1

yi′ Jnyi/n −

— y′(1N 1′N /N )y ya

a

Σ (1′y)2 = n(Y i·)2 − N (Y ··)2 = SSA. N i=1

11.4 From Exercise 11.3, SSA = y′My, where M is a symmetric idempotent matrix of rank a − 1 given by M = (Ia ⊗ Jn)/n − JN /N = (Ia − Ja/a) ⊗ Jn/n. Now y = µ1N + η, where η = γ ⊗ 1n + ε ∼ N (0, V) with γ = (γ1, · · · , γa)′ and V = Ia ⊗ (σ2Jn + σ2In).

Since M1N = 0, SSA = η′Mη. Then from

γ

ε

MV = [(Ia − Ja/a) ⊗ Jn/n](Ia ⊗ (σγ 2Jn + σε2In)) = (σ2ε + nσ2γ)M 65

66

Solutions to Chapter 11 2

2

2

and Result 5.4.5, SSA/ E(MSA) ∼ χa−1 and E(MSA) = σε + nσγ . Next, y − y ^ = (IN −Ia⊗Jn/n)y = Bε, where B 2= Ia⊗(In−Jn/n) is idempotent with rank N −a. Then 2 2 ′ SSE/ E(MSE) = ε Bε/σ ∼ χ and E(MSE) = σ . By Bη = B(γ ⊗ 1n + ε) = Bε, N −a

ε

ε

we also have SSE = ′η Bη. Since MB = [(Ia−Ja/a)⊗Jn/n][Ia⊗(In−Jn/n)] = O, from the above display, MVB = O, so by Result 5.4.7, SSA/ E(MSA) and SSE/ E(MSE) are independent.

11.5 Here, y ∼ N (Xβ, V), and L(β, V; y, X) is given in (11.2.8). Let σ2 = (σ2, σ2, · · · , σ2 )′. ε

1

m

Then, l(β, V; y, X) = −

N

1 1 log(2π) − log |V| − (y − Xβ)′V−1(y − Xβ).

2 2 2

Then, ∂l = X′V−1y ′ −1 — X V Xβ, ∂β ∂2l ′ −1 X. ′ = −X V ∂β∂β From (11.2.7), ∂V/∂σ2 = ZjZ′ . From (2.7.13), ∂ log |V|/∂V = V−1. Then by the chain j

j

rule of differentiation ∂V−1 ∂V = −V−1 2 V−1 = −V−1 ZjZ′j V−1, ∂σj ∂σ2j Then

1 ∂l = − tr(V−1

∂σ2j ∂2l

2 1

= tr(V ∂σj2∂σh2 2

∂ log |V| = tr[Z′j V−1 Zj]. ∂σj2

1 — Xβ)′V− Z Z′ V− (y − Xβ), ZjZj ) + (y j j 2 ′

−1

′

1

−1

ZjZj V

′

1

′

ZhZh) − (y − Xβ) V

−1

′

−1

ZjZj V

′

ZhZhV

−1

(y − Xβ),

and ∂l

= −X′V−1Z Z′ V−1(y − Xβ).

2

j

∂β∂σj

j

At the true parameter values, E(y− Xβ) = 0, and E(y so that E − E −

∂2l ∂β∂β′

∂2l ∂σ2∂σ2 j

h

∂l E − ∂β∂σ2 j

− Xβ)′A(y − Xβ) = tr(AV),

= E(X′V−1 X) = X′V−1X; !

= 1 tr(V−1 Zj Z′j V−1 ZhZ′h) + 0; 2

! = −X′V−1Z jZ′jV−1(0) = 0.

Let Sjh = tr(V−1ZjZ′j V−1ZhZ′h) for j, h = 1, · · · , m + 1, and let S ∈ R(m+1)×(m+1) = {Sjh}. Then, the Fisher information matrix is I=

X′V−1X O O S/2

Solutions to Chapter 11

67

From the standard result on MLE, as N β^ M L 2 σ^ M L

∼N

1, β , I−1 σ2

approx.

Consequently, ^ ML ) ≈ (X′V−1X)−1, Var(β 2 Var(σ ^ ML ) ≈ 2S −1 , and 2 ^ ,σ Cov(β ML ^ ML ) ≈ O.

11.6 The ANOVA estimators are 2 σ ^γ = {MSA − MSB(A)}/bn, 2 σ ^θ = {MSB(A) − MSE}/n, and 2 σ ^ε = MSE.

The ML solutions are σγ2 = {(1 − 1/a)MSA − MSB(A)}/bn, σθ2 = {MSB(A) − MSE}/n, and σε2 = MSE. Based on the range of values assumed by σ2 and σ2, the ML estimators are as follows: γ

θ

, ,{MSA − MSB(A)}/bn if σ2 ≥γ 0, σ2 ≥ θ 0, 2 2 if σγ ≥ 0, σθ < 0, ^γ = {MSA − σ ˜ε}/bn , , 0 otherwise; , , {MSB(A) − MSE}/n if σ2γ ≥ 0, σ2θ ≥ 0, σ^2θ = SSA/(nab) + SSB(A)/(nab) − MSE/n if σ 2< 0, σ 2≥ 0, γ θ , 0 otherwise; , , M SE if σ2θ ≥ 0, 2 2 σ2 ^ε = {SSE + SSB(A)}/[a(bn − 1)] if σγ2≥ 0, σθ2 < 0, , if σ < 0, σ < 0. , SSTm/abn

σ2

γ

θ

11.7 For the two-way cross-classified model without interaction, the ML equations are SSA SSB SSAB + SSE 1 a − 1 b − 1 abn − a − b + 1 + 2 + = + + + 2 δ4 δ11 δ12 δ0 δ 11 δ 12 δ02 SSA 1 a−1 = + 2 δ11 δ11 δ4 SSB 1 b−1 = + , 2 δ12 δ12 δ4 where δ0 = δ1 = σε2, δ11 = σε2 + bnσ2γ, δ12 = σ2ε+ anσ2,θ and δ4 = δ11 + δ12 − δ0. Again, the MLE of σ2 εis MSE, while estimates of the remaining variance components must be obtained by numerical solution of these nonlinear equations.

68

Solutions to Chapter 11

11.8 The ith nonzero diagonal element of I − P is qii = (N − r)/N , i = 1, · · · , r. Then N N Σ 1Σ 2 Σ rii + (µ4 − 3σ ) r2ij 2 i=1 i=1 j=1 N

Var(y′Ay) = Var(y′(I − P)y) + 2σ4 = Var(y′(I − P)y) + 2σ4

4

1 2

(µ4 − 3σ4 + 1)

N Σ Σ rii2 + r2ij . i=1

i/=j

Now, µ4 − 3σ4 > −2 =⇒ Var(y′Ay) is minimum when rij = 0, for all i, j, i.e., if A = I − P. Σ Σ 11.9 We see from the proof of Result 11.2.4 that y′Ay = ε′Aε = i aiiε2i + 2 i<j aijεiεj.

Then Σ E(y′Ay) = E(ε′Aε) = σε2 aii.

(S.5) i

Next E[(y′Ay)2] = E(ε′Aεε′Aε) = E[tr(εε′Aεε′A)] =E

Σ

εiεjajkεkε5a5i

=

i,j,k,5

Σ

E[εiεjajkεkε5a5i].

i,j,k,5

Since the εi are i.i.d. with mean 0, E[εiεjajkεkε5a5i] = 0 if one of i, j, k, l is not equal to any of the others. Hence the expectation is nonzero only in the following four cases, i) 2 i = j /= k = l, with value σε4aik , ii) i = k /= j = l, with valueσε4aij2 , iii) i = l /= j = k, with value σ4aiiajj, and iv) i = j = k = l, with value µ4a2 . Therefore, ε

Σ

ii

Σ 2 2 2 σ4ε aik + σ4εaij + σ4ε aiiajj + µ4aii i Σ i/=k i/=j i/=j Σ Σ Σ = 2 σ4a2 + σ4aiiajj + µ4a2 − 3 σ2a2

E[(y′Ay)2] =

Σ

Σ

ε ik

4

ε

ii

i,j

i,j 2

4

= 2σε tr(A ) + σε

Σ

!2

i

ii i

+ (µ4 − 3σε2)a′a.

aii

i

This combined with (S.5) then yields (11.2.15).

11.10 From Result 11.2.3, the logarithm of the REML is, up to an additive constant, 1 1 1 1 l(σ2ε, σ2γ) = − log |V| − log[1′N V−11 N | − (y − µ ^GLS 1N ). ^GLS 1N )′V− (y − µ 2 2 2 Let λ = σ2 ε+ nσ2. γFrom the proof of Result 11.3.6 and (11.3.3), |V| = σ2(n−1)aλa, ε

V−1 = Ia ⊗ σ−2[In − (σ2/λ)Jn]. ε

γ

Then 1′N V−11N = aσε−21′n[In − (σγ2/λ)Jn]1n = an/λ.

From the proof of Result 11.3.6 and (11.3.15), we see that (y − µ1N )′V−1(y − µ1N ) =

Solutions to Chapter 11

69

SSE/(2σ ) + SSA/(2λ) + an(Y ·· − µ) /(2λ). Given σ and σ , this is maximized by 2

2

2

ε

2

ε

γ

Y ··, which by definition is µ^ G L S . Thus ′

(y − µ ^GLS 1N ) V

−1

(y − µ ^ GLS 1N ) =

Combining the displays then yields the proof.

SSE SSA + .

2σ2ε 2λ

Solutions to Chapter 12

12.1 Let Y |p ∼ Binom(m, p) and p∼ Beta(α, β), 0 ≤ p ≤ 1, α > 0, and β > 0. The joint distribution of Y and p is m k 1 p (1 − p)m−k pα−1(1 − p)β−1 k B(α, β) m 1 = pα+k−1(1 − p)m−k+β−1 k B(α, β)

P(Y = k, p) =

∫1 where B(α, β) = 0 pα−1(1 − p)β−1dp = Γ(α)Γ(β)/Γ(α + β) is the Beta function. The marginal distribution of Y is then ∫ 1 m 1 P(Y = k) = α+k−1(1 − p) m−k+β−1 dp k B(α, β) p 0 m B(α + k, m − k + β − 1) = , k B(α, β) which is the Beta-Binomial distribution and does not belong to the exponential family.

It is straightforward to verify that E(Y ) = Ep{E(Y | p)} = m E(p) = mα/(α + β), and Var(Y ) = Ep{Var(Y | p)} + Varp{E(Y | p)} = E{mp(1 − p)} + Var(mp) mαβ m2αβ = + (α + β)(α + β + 1) (α + β)2(α + β + 1) mαβ(α + β + m) = . (α + β)2(α + β + 1) 12.2 (a) For the GLM with Inverse link Yi | x i, β ∼ exp

−x′iβ · Yi + log(x′iβ) 1 + log(Y /φ i i) − log Y i − log Γ(1/φi ) . φi φi

(b) For the GLM with Log link Yi | xi, β ∼ exp

− exp{−x′iβ} · Yi − x′iβ 1 + log(Y /φ )i − log Y i − log Γ(1/φi ) . i φi φi

(c) For the GLM with Identity link Yi | xi, β ∼ exp

−Yi/(x′iβ) − log(x′iβ) 1 + log(Y /φ )i − log Yi − log Γ(1/φi ) , i φi φi 71

72

Solutions to Chapter 12

12.3 Under the suggested model, Yt ∼ Poisson(β0 + β1t). Suppose we are given N years of count data (here N = 20), Yt1 = y1, · · · , Yt20 = yN , say, so that we can directly write down the likelihood function ( N ) N 1 L(β , β ) = exp Σ (β + t β ) Y(β + t β )yi .

0 1 0 i 1 0 i 1 QN i=1 i=1 i=1 yi!

The log-likelihood is N

l(β0, β1) = c − (Nβ0 + β1

Σ

N

ti) +

i=1

Σ

ki log(β0 + β1ti),

i=1

where c denotes a constant term with respect to β0 and β1. The first partial derivatives of l(β0, β1) with respect to β0 and β1 are N

Σ ∂l(β0, β1) ki = −N + , ∂β0 (β 0 + β1ti) i=1 N

N

i=1

i=1

Σ ∂l(β0, β1) kiti Σ = — ti + . ∂β1 (β0 + β 1ti ) Estimates of β0 and β1 are obtained by setting these equal to zero, and solving the resulting equations. The likelihood can be viewed as Y N

L =

exp{θiyi − b(θi) − ci(yi)}, so that

i=1 N

log L =

Σ

N

{θiyi − b(θi)} −

i=1

Σ

ci(yi).

i=1

12.4 (a) Under Model 1, Yi are independent Poisson(λi), i = 1, 2. Under Model 2, Y1/(Y1 + Y2) is Binom(Y1 + Y2, p), where Y1 + Y2 is Poisson(λ1 + λ2). The likelihood under Model 1 for y1, y2 = 0, 1, · · · , λ1 > 0, and λ2 > 0 is L1 = exp{−(λ1 + λ2)}λy1 λy2 /y1!y2!

1

2

The likelihood under Model 2 can be written as L2 =

(y1+y2) y1 + y2 y y exp{−(λ1 + λ2)}(λ1 + λ2) 1 2 p (1 − p) y1 (y1 + y2)!

= exp{−(λ1 + λ2)}

{p(λ1 + λ2)}y1 {(1 − p)(λ1 + λ2)}y2 , y !y !

1

2

which reduces to L1 since p = λ1/(λ1 + λ2). (b) L1 can be written as ) (N Σ [yi log λi − λi − log yi!] .

L1 = exp i=1

This has the exponential family form with w(λi) = log λi, b(λi) = λi and c(yi) = — log yi!. The canonical link is thus the log link. Likewise, we may rewrite L2 as y

y

exp(−λ̃ 1 )λ̃ 1 exp(−λ̃ 2 )λ̃ 2 1 × 2 , L2 = y1! y2!

Solutions to Chapter 12

73

with ˜ λ1 = p(λ1 + λ2) > 0 and λ˜ 2 = (1 − p)(λ1 + λ2) > 0, so that the canonical link is once again the log link. (c) In order to incorporate a covariate X, set g(λi) = log λi = α + βXi, say.

12.5 Let Y ∼ Bernoulli(π). The probit link and its inverse are defined by g(π) = Φ−1(π) = η and g−1(η) = Φ(η) = π.

The complementary log-log (or Cloglog) link and its inverse are g(π) = log[− log(1 − π)] = η, and g−1(η) = 1 − exp(− exp(η)) = π.

All other expressions in the IRLS algorithm are the same except that the dπ in the two dη cases are φ(η) and exp(η) × exp(− exp(η)).

12.6 Consider the Poisson loglinear model defined in Example 12.1.2 and discussed in section 12.5. Write down in detail the steps in the IRLS algorithm for estimating β and show that the weight matrix W and the vector of adjusted dependent responses z are given by (12.5.1) and (12.5.2), respectively.

Iteration 0 consists of the following steps.

1. Specify an initial vector for β, say β(0).

2. Evaluate η(0) = Xβ(0), through the linear predictor function.

3. Evaluate λ(0) = g−1(η(0)) = exp(η(0)). dηi 4. Compute dλ = 1/λi, i = 1, · · · , N and evaluate at it λ(0). i

5. Compute the adjusted dependent variate for i = 1, · · · , N : iZ(0) = ηi (0) + (Yi − dηi (0) (0) (0) λ(0) − i ) dλi = η i + (Yi/λ i 1) where the derivative is evaluated at λ .

6. Compute weights W (0) = wi ( dηi )−2 = λ , all terms evaluated at the estimates i

i

b''(θi) dλi

at the 0th iteration, and recalling φ = 1.

(0) ′ 7. Let z(0) = (Z(0)1 , · · · , Z(0) = diag(λ(0)1, · · · , λ(0) N ) and W N).

(0) (0) 8. Fit the WLS regression z = Xβ + ε .

9. Obtain β(1) = (X′W(0)X)−1X′W(0)z(0).

Iterate Steps 1-9 starting from β(1), β(2), · · · until convergence. The converged estimate is β^M L. The above derivation also proves (12.5.1) and (12.5.2).

12.7 (a) The Anscombe residual is defined to be the transformation that bring y closest to normality and then standardized to have mean 0 and variance 1. If Y~ Poisson, the function y2/3 brings it closest to normality, which leads to the form of the Anscombe residual for a Poisson GLIM. The deviance residual in (b) follows from the definition and the form of di for the Poisson GLIM. 12.8 (a) For integer y, it can be seen that Γ(y + c) = Y (j + c). Γ(c) y−1

j=0

Apply this in the p.m.f. f (y; µ, α) and use the log-link which gives µi = exp(x′iβ). This gives the required likelihood. Part(b) follows the IRLS steps shown in the text.

74

Solutions to Chapter 12

12.9 For each i, given Yi+, suppose that the random vector yi = (Yi,1, · · · , Yi,J )′ has a multinomial distribution, Multinom(Yi+, πi,1, · · · , πi,J ), with p.m.f.

J

Yi+!

p(yi ; πi ) = QJ

Y

j=1 y i,j ! j=1

Γ(Yi+ + 1) πyi,j , y πi,ji,j = Q J i,j j=1 Γ(y i,j + 1)

ΣJ where πi = (πi,1, · · · , πi,J )′, with 0 ≤ πi,j ≤ 1 and j=1 πi,j = 1. For each i, assume that we have a k-dimensional vector of predictors zi,j, which may also vary by level j. Suppose that the proportions πi,j are proportional to the regression function gi,j(β, zi,j), j = 1, . . . , J. For simplicity of notation, denote these by gi,j(β) and let ΣJ Gi(β) = j=1 gi,j(β). Then, πi,j =

gi,j , Gi

and the multinomial likelihood is proportional to LM (β; y1 , · · · , yN ) ∝

Y N Y J i=1 j=1

gi,j (β) Gt(β)

yi,j

,

where the subscript M stands for multinomial.

We can use the well-known Multinomial-Poisson (M-P) transform to rewrite the multinomial likelihood as an equivalent Poisson likelihood, the Poisson-trick. Define an additional N -dimensional vector of parameters φ = (φ1·,· · , φN )′. The idea behind the Multinomial-Poisson (M-P) transformation is to assume that independently for all i, Yi+ ∼ Poisson(φiGi(β)).

For each i = 1, . . . , N , we can write down the M-P mixture p.m.f. as P(yi and Yi+ = yi+) = P(Yi+ = yi+) × p(yi), The marginal p.m.f. of yi can be obtained by summing the joint p.m.f. over all possible values of Yi+ to yield the Poisson likelihood YY N

LP (φ, β; y1, · · · , yN ) ∝

J

(gi,j(β)eφi )yi,j exp(−gi,j(β)eφi ).

i=1 j=1

Let ^i(β) = arg max L P(φ, β). φ φi

Substituting φ^i (β ) from this into the previous expression, we can write that as a function of β only and see that the resulting LP (β; y1, · · · , yN ) is equivalent to LM(β; y1, · · · , yN ). It follows that the maximum likelihood estimates of β and their asymptotic variances are identical for LP (φ, β) and LM(β; y1, · · · , yN ).

Solutions to Chapter 13

13.1 Since B0 = GX′Y, E(B0) = GX′ E(Y) = GX′XB = HB, Cov(β0(i) , β 0(j)) = Cov(GX′y(i), GX′y(j)) = GX′ Cov(ε(i), ε(j))XG′ = σijGX′XG′ by (13.1.5). This proves property 1 of Result 13.1.2. From (13.1.14),^ E(Y) = P E(Y) = PXB = XB, ^ = (I − P) E(E) = O, Cov(y E(E) ^ (i),^y(j)) = Cov(Py(i), Py(j)) = P Cov(y(i), y(j))P = σijP, Cov(ε ^ (i), ε ^(j)) = Cov((I − P)ε(i), (I − P)ε(j)) = (I − P) Cov(ε(i), ε(j))(I − P) = σij(I − P), proving property 2. Likewise, 0 Cov(β(i) , ε(j)) = Cov(GX′y(i), (I − P)ε(j)) = σijGX′(I − P) = O,

proving property 3.

13.2 For any i, j, Cov(M′1ε(i), M′2ε(j)) = M′1 Cov(ε(i), ε(j))M2 = σijM′1M2 = O, so M′1ε(i) and m′2ε(j) are uncorrelated, and hence by their joint normality, are independent.

13.3 There are p responses for each of the n replications at each crossed level of Factor A and Factor B. (a) The MANOVA decomposition is yAkr = y + (yA· − y) + (y·k − y) + (yAk − yA· − y·k + y) + (yAkr − yAk ).

Summing over l = 1,· · · , g; j = 1, · · · , b; r = 1,· · · , n gives the breakup of the Corrected Total SSCP as Corrected Total SSCP = SSCP(Factor A) + SSCP(Factor B) + SSCP(Interaction) + SSCP(Error) from which the MANOVA table follows. Σb Σ n Σg ′ • r=1 (yAkr − y)(yAkr − y) with gbn − 1 Corrected Total SSCP = A=1 k=1 d.f. Σg • SSCP(Factor A) = bn(yA· — y)(yA· − y)′ with g − 1 d.f. Σb A=1 • SSCP(Factor B) = gn(y·k − y)(y·k − y)′ with b − 1 d.f. k=1 Σg Σb ′ • k=1 n(yAk −yA· −y·k +y)(yAk −yA· −y·k +y) SSCP(Interaction SSCP) = A=1 with (g − 1)(b − 1) d.f. Σg Σ b Σn (yAkr − y • Ak )(yAkr − yAk )′ with gb(n − 1) d.f. r=1 Error SSCP = A=1 k=1 This matrix will be p.d. provided p ≤ gb(n − 1). (b) Note that if n = 1 (no replication, we cannot make inference on the interaction effect). The hypotheses are H0: γ11 = γ12 = · · · = γgb = 0 versus H1: at least one γAk /= 0. The Wilk’s Lambda statistic is Λ∗ =

|Error SSCP| |Interaction SSCP + Error SSCP| 75

76

Solutions to Chapter 13 and we reject H0 at level α for small values of the statistic, i.e., (using Bartlett’s multiplier) if — gb(n − 1) −

p + 1 − (g − 1)(b − 1) log Λ∗ > χ2(g−1)(b−1)p (α) 2

If we reject H0, interaction effects exist, and the factor effects do not then have a clear interpretation. (c) To test for Factor A main effects via H0: τ 1 = · · · = τ g = 0 versus H1: at least one τ A /= 0. Let Λ∗ =

|Error SSCP| |SSCP(Factor A) + Error SSCP|

Using Bartlett’s multiplier the decision rule is to reject H0 at level α if — gb(n − 1) −

p + 1 − (g − 1) log Λ∗ > χ2(g−1)p (α) 2

To test for Factor B main effects via H0: β1 = · · · = βg = 0 versus H1: at least one βk /= 0, Let Λ∗ =

|Error SSCP| | SSCP(Factor B) + Error SSCP|

Using Bartlett’s multiplier the decision rule is to reject H0 at level α if −Cbrgb(n − 1) −

p + 1 − (b − 1) log Λ∗ > χ2(b−1)p (α) 2

^0 ∼ Wn(Σ0, m − 1).

13.4 From Result 6.1.2, (a)^µ0 ∼ Nn(µ0, Σ0/m), (b) mΣ 13.5 To prove property 1 of Result 13.2.2, write y = (y1′ , · · · , y′m)′, X = (X′1, · · · , X′m)′, Z = diag(Z1, · · · , Zm), γ = (γ′1, · · · , γ′m)′, V = diag(V1, · · · , Vm), W = V−1, and ε = (ε′1, · · · , ε′m)′. Then y = Xτ + Zγ + ε, V = Cov(y) = diag(V1, · · · , Vm), W = diag(W1, · · · , Wm), and Σγ := Cov(γ) = diag(Dγ, · · ,· Dγ). By the assumption, V is nonsingular. Then 1 (y − Xτ )′V−1(y − Xτ ) 2 where C is independent of τ . Therefore, the MLE of τ is the same as the GLS. Since by Σm the assumption, X′V−1X = i=1 X′iWiXi is nonsingular, then the MLE (and GLS) is l(τ ; y, X) = −C −

′

τ^ = (X V

−1

−1

X)

′

X V

−1

y,

(S.6) Σm which is (13.2.9). From Result 11.2.1, the essentially-unique b.l.u.e. of0 c′ τ + i=1 ci′γ i = c0′ τ + c γ′ is c 0 τ^′ + c γ^′ , where c = (c1,′ · · · , c′m) ′, and ′ ′ ′ −1 ^ 1 ,′ · · · , γ^ m ) γ ^ = ΣγZ V (y − Xτ^ ) = (γ ′

with γ^ i = DγZiWi(y − X i τ^ ), giving (13.2.10). From (S.6), (13.2.11) follows. Put K = ′ X(X ′ WX) −1 X ′W. Then γ ^ = ΣγZ W(I − K)y. By K being idempotent and WKV = ′ K ′

′

^ = ΣγZ W(I − K)V(I − K )WZΣγ Cov(γ) = ΣγZ′(I − K′)2WZΣγ = ΣγZ′(I − K′)WZΣγ = ΣγZ′[W − WX(X′WX)−1X′W]ZΣγ.

Solutions to Chapter 13 ′

77 ′

′

Since γ^ = (γ^ γm) , the major submatrices on the right hand side corresponding to 1, · · · , ^ Cov(γ^i) is exactly the right hand side of (13.2.12). This completes the proof of property 1. As a by-product, for i /= j, ′

′

Cov(γ ^ i,^γj ) = −DγZiWiXi(X WX)

−1

′

Xj WjZjDγ.

When θ is unknown, the log-likelihood now is l(τ , θ; y, X) = −

1 1 log[(2π)N/2|V(θ)|] − (y − Xτ )′[V(θ)]−1(y − Xτ ) 2 2

^ −1(y − Xτ ). Then by It follows that the MLE of τ must minimize (y − Xτ )′[V(θ)] the same argument for (13.2.9), (13.2.13) is proved. The MLE of Cov(τ^) follows from (13.2.11) and the invariance principle. Then property 2 is proved.

13.6 In the formulas in section 13.2.2, set Vi = σ2I + ZiDγZ′i. ∫ 13.7 For any w > 0, P( y ≤ w) = t ≤w ckh(t′t) dt. By change to spherical coordinates, t = rω, where r > 0 and ω = 1, dt = rk−1 dr dω. Then P( y ≤ w)

∫ w

∫ w ckh(r2)rk−1 dr dω = Sk —1

ckh(r2)rk−1 dr

0

0

where Sk−1 is the surface area of the unit sphere in Rk, so is equal 2πk/2/Γ(k/2). Then differentiating both sides of the above equation yields the desired result.

13.8 From (13.3.4) and (13.3.5), σ^∗ 2 =

^ LS )′(y − Xβ ^ LS ) (y − Xβ N −p

=

^ε ′L S ^ε L S N −p

.

Then property 3 follows from property 4 of Result 4.2.4, which holds for all ε with E(ε) = 0 and Cov(ε) = cI, two properties that a spherical distribution has (see section 5.5.2.2).

13.9 The GLS estimator of β has the form in (4.5.5), i.e., ^β = (X′V−1X)−1X′V−1y, while the GLS estimator if σ2 is ^2 = σ

1 SSEGLS, N −p

where SSEGLS was shown in (4.5.9). It is easily verified that the maximum likelihood estimator of β coincides with its GLS estimator for any density generator h(.), and the fitted vector and residual vector are also the same as we showed in section 4.5. In fact, the distribution of the MLE of β is the same for all h(.) within the elliptical family.

However, the distribution of the MLE of σ2 is affected by departures from normality within the family of elliptical distributions. If V = IN , the LS estimates of β and SSE have the forms in (4.2.3) and (4.2.16) respectively. Under the model (13.3.3), the GLS estimates of β and σ2 have these properties: 1. 1 ^β ∼ Ep(β, σ2(X′V−1X)−1, h) ^ )′ V−1 (y − Xβ^ ) is an unbiased estimator of σ2.

2. 2 σ^ 2 = 1 (y − Xβ N −p

78

Solutions to Chapter 13 To see this, let y ~ EN (Xβ, σ2V, h), and suppose My(t) exists and has the form given in Definition 5.5.11. The m.g.f. of (X′V−1X)−1X′V−1y is My(V−1X(X′V−1X)−1t) = ψ(t′(X′V−1X)−1X′V−1VV−1X(X′V−1X)−1t) ×

exp{t′(X′V−1X)−1X′V−1Xβ},

from which property 1 follows. The proof of property 2 is left to the reader.

13.10 Cook’s distance has the same form as the quantity under the normal linear model, and is hence invariant to h(.).

13.11 Under the GIG prior, β = σz + u, where σ and z ∼ Np(0, Ω) are independent. Since σ2 ∼ IG(a, b), 1/σ2 ∼ Gamma(a, 1/b) (see item 16 of Appendix B). Then ξ := 2b/σ2 ∼ 2 Gamma(a, 2) = χ2a . From z (b/a)z +u= √ +u β= √ ξ/(2b) ξ/(2a)

√ and (b/a)z ∼ Np(0, (b/a)Ω), β follows a p-variate t-distribution with d.f. 2a, location u, and scale matrix (b/a)Ω (see Example 5.5.3).

13.12 Under the GIG prior in Result 13.4.3, y = Xβ + σz, where z ~N (0, I) is independent of (σ2, β). Then from β| σ2∼ N (µ, σ2Ω), y| σ2 ∼ N (Xµ, σ2(XΩX′ + I)). Applying Exercise 13.11, with y playing the role of β, y follows an N -variate t distribution with d.f. ν = 2a, location Xµ, and scale matrix (b/a)(XΩX′ + I).

13.13 For ease of notation, write v = σ2 and v = σ2. By Bayes rule, ^ ^ f (y; β, v) × π(β, v) π(β, v y) = , f (y) | ∫ where f (y) = f (y; β, v) × π(β, v) dv dβ.

Plug in the conditional p.d.f. of y given (β, v) and π(β, v) = 1/v. Then 1 1 1 2 1 π(β, v | y) = exp − y − Xβ × N/2 2v v f (y) (2πv) −N/2−1 1 v 2 = exp − y − Xβ . f (y)(2π)N/2 2v By y − Xβ

2

2 ^ 2 + ^ε 2, = X(β^ − β) + ε^ = X(β − β)

π(β, v | y) =

v−N/2−1

exp f (y)(2π)N/2

−

^ 1 X(β − β) 2v

1 ′ −1 ′ ^ (v ^ = exp − (β − β) X X)(β − β) 2

2

2 1 ^ ε 2v v−N/2−1 (N − p)v^ × exp — . f (y)(2π)N/2 2v

−

Then by |v−1X′X| = v−p|X′X|, √ 1 |v−1X′X| exp ′ −1 ′ ^ − (β − ^ β) (v X X)(β − β) π(β, v | y) = p/2 (2π) 2 (N − p)v^ v−(N −p)/2−1 . √ × exp — 2v (N −p)/2 ′ f (y)(2π) |X X|

(S.7)

Solutions to Chapter 13

79

Given v, the second factor on the right hand side does not involve β. Then property 1 of Result 13.4.4 follows. Next, write m = (N − p)/2. Integrating out β on both sides, 1 −m−1 −mv^/v e .

(S.8) √ π(v | y) = ×v m ′ f (y)(2π) |X X| Since the first factor on the right hand side only depends on y, property 2 of Result 13.4.4 follows. Thus, ^ v(X′X)−1), β | v, y ∼ N (β,

v | y ∼ IG(m, mv). ^

By applying Exercise 13.11 to the posterior distribution of (β, v), property 3 follows, immediately yielding property 4. If X∗ is a new N × p matrix, then y∗ = X∗β +√ vε with ε ∼ N (0, I) independent of all the other random variables. From the above display, ' y∗ |v, y ∼ N (X∗β, ^ v[I + X∗(X′X)−1X∗ ]). Then applying Exercise 13.11, property 5 follows.

13.14 This follows directly from Lindley and Smith (1972). Note that the marginal distribution of Y is normal, i.e., Y ~ N (A1A2θ2, C1 + A1C2A′1) and θ1 θ| 2 ∼ N (A2θ2, C2). Then, θ1 |y ∼N (Dd, D), where D−1 = A′1C1−1A1 + C2−1, and d = A′1C1−1y + C2−1A2θ2. We can repeat this step for the next hierarchy.

13.15 Let a = y — Xµ and recall Ω∗−1 = Ω−1 + X′X. Then µ∗ = Ω∗(Ω−1µ + X′y) = Ω∗(Ω−1µ + X′Xµ + X′a) = Ω∗(Ω∗−1µ + X′a) = µ + Ω∗X′a, giving y′y + µ′Ω−1µ − µ∗′Ω∗−1µ∗′ = (Xµ + a)′(Xµ + a) + µ′Ω−1µ − (µ′ + a′XΩ∗)Ω∗−1(µ + Ω∗X′a) = (µ′X′Xµ + 2µ′X′a + a′a) + µ′Ω−1µ − (µ′Ω∗−1µ + 2µ′X′a + a′XΩ∗X′a) = a′(I − XΩ∗X′)a.

From Sherman–Morrison–Woodbury formula, I − XΩ∗X′ = I − X(Ω−1 + X′X)−1X′ = I − X(I + ΩX′X)−1ΩX′ = (I + XΩX′)−1, completing the proof. ΣNj 2 2 2 2 13.16 Suppose νj = Nj − 1, and S j = i=1 (Yij − Y j) /νj, for j = 1, 2. Then, σj /{νjS j } ∼ −2 χν j , i.e., it has an inverse chi-square distribution with νj d.f. It follows that the posterior distribution of σ 2 /S 2 σ2/σ2 S 2 /S 2 F = σ22/S22 = S22/S21 = σ12/σ22 1

1

2

1

1

2

has an Fν1,ν2 distribution.

13.17 (a) π(µ, σ2 | Y ) ∝ π(Y | µ, σ2)π(S2 | σ2) 1 ∝ σ−N exp − 2 [νS2 + N (µ − Y )2] 2σ where

,

√ N N 2 ; π(Y | µ, σ ) = √ exp − 2 (Y − µ) 2σ 2πσ also, (N —1)S2/σ2 ∼ χ2 . The joint posterior distribution is then 2

N −1

π(µ, σ2 | Y ) ∝ π(µ, σ2)π(Y | µ, σ2)π(S2 | σ2), which has the normal-χ2 form.

80

Solutions to Chapter 13 (b) π(µ, σ2 | Y ) ∝ σ−(N+1) exp −

1 {νS2 + N (µ − Y )2 2σ2

The marginal poterior has a tN−1 distribution.

13.18 Yes, it is a conjugate class.

13.19 First consider an arbitrary y = Xβ + σε, ε ~ Np(0, I), with (β, σ2) having a prior of the form in Result 13.4.4. For ease of notation, write v = σ2 and^v = ^σ2. Also, put m = (N — p)/2. From (S.8) and v y| ∼ IG(m, mv) ^(see Exercise 13.13), a comparison of the normalizing constants yields 1

m (mv) ^ . = Γ(m) f (y)(2π)m |X′X|

√

2

Then with v^ = y − y ^ /(N − p), where y ^ = PC(X)y, Γ((N − p)/2) Γ(m) √ √ f (y) = = .

(2πmv^) m |X′ X| (π y − y ^ 2)(N −p)/2 |X′X| For two different models, since the parameters have the same structure of priors, they can only be distinguished by Xi and pi. Thus √ 2 (N −p1)/2

B01 = Γ((N − p0)/2) × (π y − y^ 1 ) Γ((N — p1)/2)

2 (N −p )/2

(π y − y ^0 )

′ 1

√ |X X1| .

0

′

|X 0 X 0|

13.20 Under the model, y = Xβ + ε and β are jointly normally distributed with y β

∼N

Xµ IN + XΩX′ , σ2 ΩX′ µ

XΩ Ω

.

Then from Result 5.2.12, β | y ∼ N (µ∗, σ2Ω∗), where µ∗ = µ − ΩX′(IN + XΩX′)−1(Xµ − y), Ω∗ = Ω − ΩX′(IN + XΩX′)−1XΩ = Ω[IN − X′(IN + XΩX′)−1XΩ].

By Sherman–Morrison–Woodbury formula, Ω∗ = Ω(IN + X′XΩ)−1 = (Ω−1 + X′X)−1.

Meanwhile, µ∗ = [Ip − ΩX′(IN + XΩX′)−1X]µ + ΩX′(IN + XΩX′)−1y ^ = (IN + ΩX′X)−1µ + ΩX′[IN − XΩ(Ip + X′XΩ)−1X′](Xβ^ + ε) ^ + ^ε ) . = Λµ + Ω[IN − X′XΩ(Ip + X′ XΩ) −1 ]X′ (Xβ Note that X′ ^ ε = 0 and Λ = Ip − ΩX′ (XΩX′ + IN ) −1 X. Then ^ µ∗ = Λµ + Ω[IN − X′XΩ(Ip + X ′ XΩ) − 1 ]X′ Xβ ^ = Λµ + ΩX′[IN − XΩ(Ip + X ′ XΩ) − 1 X′ ]Xβ ^ = Λµ + ΩX′(IN + XΩX′)−1Xβ^= Λµ + (Ip − Λ)β, 2 giving the first formula in (13.4.26). Next, given y, y∗ = Xβ +ε, where β ~ N (µ∗, σ Ω) 2 2 ∗ ∗ ∗ and ε ∼N (0, σ IN ) is independent of β. Then y ∼ N (Xµ , σ (Ω + IN )), giving the second formula in (13.4.26).

Solutions to Chapter 13

81

13.21 The state equation is xt+1 = ρxt + wt, and the observation equation is Yt = xt.

13.22 The autoregressive signal-plus-noise model is defined by Zt = Yt + vt Yt = ρYt−1 + wt, where vt and wt are independent error processes, vt ∼ N (0, σv 2) and wt ∼ N (0, σw2 ). Express this in the form of (13.5.1) and (13.5.2); the Kalman filter recursions are obtained from (13.5.7)–(13.5.9).

Bibliography

Harville, D. A. (1997), Matrix Algebra from a Statistician’s Perspective, Springer-Verlag: New York.

Hsu, J. C. (1984), “Constrained simultaneous confidence intervals for multiple comparisons with the best,” The Annals of Statistics, 12, 1136–1144.

Lehmann, E. L. and Casella, G. (1998), Theory of Point Estimation, Springer-Verlag, New York, 2nd ed.

Lindley, D. V. and Smith, A. F. M. (1972), “Bayes estimates for the linear model,” Journal of the Royal Statistical Society Series B, 34, 1–18.

83