Solutions to Chapter 1
1.1 |a • b| = | − 9| = 9, while kak kbk =
√ √ 6 22 ∼ = 11.489 > 9.
1.2 To verify the Cauchy–Schwarz inequality, first see that the inequality holds trivially if a and b are zero vectors. We therefore assume that both a and b are nonzero. Let c be the vector c = xa − yb, where x = b0 b, and y = a0 b. Clearly, c0 c ≥ 0. We express c0 c in terms of x and y: c0 c = (xa − yb)0 (xa − yb) = x2 a0 a − 2xya0 b + y 2 b0 b. Since c0 c ≥ 0, and using the definitions of x and y, we see that (b0 b)2 (a0 a) − 2(a0 b)2 (b0 b) + (a0 b)2 (b0 b) ≥ 0 and dividing by b0 b in the last inequality, we see that (b0 b)(a0 a) − (a0 b)2 ≥ 0, which verifies the Cauchy–Schwarz inequality. We use the Cauchy–Schwarz inequality to deduce the triangle inequality, which can be written in an equivalent form ka + bk2 ≤ (kak + kbk)2 . The expression on the left is ka + bk2 = (a + b) • (a + b) = a • a + 2a • b + b • b = kak2 + 2a • b + kbk2 while the expression on the right is (kak + kbk)2 = kak2 + 2kakkbk + kbk2 . Comparing these two formulas, we see that the triangle inequality holds if and only if a • b ≤ kak kbk. By Cauchy–Schwarz inequality, |a • b| ≤ kakkbk, so the triangle inequality follows as a consequence of the Cauchy–Schwarz inequality. The converse is also true; i.e., if the triangle inequality holds, then a • b ≤ kakkbk holds for a and for −a, from which Cauchy–Schwarz inequality follows. If equality holds, i.e., if a • b = kakkbk, then b = ca, for some scalar c. Hence, a • b = ckak2 , and kakkbk = |c|kak2 . For nonnull a, this implies that c = |c|, so that c ≥ 0. If b 6= 0, then b = ca, with c > 0. 1.3 Since x0 y = 0 = x0 z = y0 z, it follows that we must a2 + b2 = 1, and √ solve the equations √ 2 2 a − b = 0, for which the solutions are a = ±1/ 2 and b = ±1/ 2. 1.4 Let
1 1 V= 0 −1
2 0 0 −2 . 1 1 −1 1 1
2
Solutions to Chapter 1 Using elementary column transformations C2 − 2C1 , and then, C3 − C2 , the matrix V becomes 1 0 0 1 −2 0 , V= 0 1 0 −1 −1 0 so that the column rank of V (or the dimension of its column space) is 2 < 3. Therefore, v1 , v2 and v3 are linearly dependent. It is easily seen that v1 and v2 are LIN, and that v3 = v2 − 2v1 .
1.5 It is easy to see that c1 v1 + c2 v2 + c3 v3 = 0 results in the following three equations, and the only solution is c1 = 0, c2 = 0, and c3 = 0: 2c1 + 8c2 − 4c3 = 0 3c1 − 6c2 + 3c3 = 0 2c1 + 5c2 + c3 = 0. 1.6 Using elementary transformations, we can show that A is equivalent to the matrix −3 3 3 0 4 4 , 0 0 −1 so that the columns of A are LIN. 2 1 3 1.7 We can solve the system = c1 + c2 to get c1 = −1 and c2 = 1. Hence, u 3 2 5 is in Span{v1 , v2 }. 1.8 If v1 ,P · · · , vm are linearly dependent, then there are scalars c1 , · · · , cm not Pall zero, such m that c v = 0. For any k with c = 6 0, we then have v = − i i k k i=1 i6=k (ci /ck )vi , showing property 1 in Result 1.2.2. Suppose without loss of generality that Pvs 1 , · · · , vs are linearly dependent and c1 , · · · , cs are constants, not all zero, such that P i=1 ci vi = 0. Let cj = 0 for all j = s + 1, · · · , n. n Then c1 , · · · , cn are not all zero and i=1 ci vi = 0. Hence v1 , · · · , vn are linearly dependent, showing property 2 in Result 1.2.2. 1.9 1. Let S = {v1 , · · · , vn } denote a set of nonzero orthogonal vectors, and let u belong to the span of S with u = c1 v1 + · · · + cn vn . For a fixed i = 1, · · · , n, take the inner product of each side with vi . Since vi • vj = 0, i 6= j, u • vi = c1 (v1 • vi ) + · · · + cn (vn • vi ) = ci (vi • vi ). Hence, ci = (u • vi ) (vi • vi ). To verify linear independence, set u = c1 v1 +· · ·+cn vn = 0. This implies that ci = 0, i = 1, · · · , n, which in turn implies LIN of {v1 , · · · , vn }.
Solutions to Chapter 1
3
2. By definition, every v ∈ V1 + · · · + Vm can be written as v1 + · · · + vm , where vi ∈ Vi , i = 1, . . . , m. Let wi ∈ Vi , i = 1, . . . , m, such that we also have v = w1 + · · · + wm . Then (w1 − v1 ) + · · · + (wm − vm ) = 0. For each i, pre-multiply both sides by (wi − vi )0 . For j 6= i, since Vi ⊥ Vj , then (wi − vi )0 (wj − vj ) = 0. As a result, (wi − vi )0 (wi − vi ) = kwi − vi k2 = 0, giving wi = vi . Hence by definition, the sum of the Vi ’s is a direct sum. 1.10 Since {v1 , · · · , vm } is a basis of V, the vectors are LIN. So from y1 = v1 ,
yk = vk −
k−1 X
yi0 vk yi , k = 2, · · · , m, kyi k2 i=1
yk 6= 0, implying that zk = kyykk k are well-defined and each has length 1. On the other hand, for 1 ≤ j < k ≤ m, yj0 yk = yj0 vk −
j−1 k−1 X X y 0 vk yi0 vk 0 yi0 vk 0 i y y = − y y − y0 y . i j 2 j 2 i 2 j i ky k ky k ky k i i i i=1 i=1 i=j+1
k−1 X
If k = 2, then j = 1, and it is straightforward to see that y10 y2 = 0. Suppose we have shown that for all i < j < k, yi0 yj = 0. Then the above identity shows that for all j < k, yj0 yk = 0. By induction, the yi ’s are orthogonal to each other. Then z1 , · · · , zm are orthonormal. Since they are LIN from Exercise 1.9, and there are m of them, they form an orthonormal basis of V. 1.11 We have W∩(W ⊥ ∩V) ⊂ W∩W ⊥ = {0}. From W ⊂ V and W ⊥ ∩V ⊂ V, W⊕(W ⊥ ∩V) ⊂ V. On the other hand, for any v ∈ V, there are unique w ∈ W and u ∈ W ⊥ , such that v = w + u. Since w ∈ V, then u = v − w ∈ V, and so u ∈ W ⊥ ∩ V. As a result V ⊂ W ⊕ (W ⊥ ∩ V). Then V = W ⊕ (W ⊥ ∩ V). From the paragraph below Definition 1.2.8, dim V = dim W + dim(W ⊥ ∩ V), completing the proof of property 1. Next, by W ⊥ ∩V ⊂ W ⊥ , (W ⊥ ∩V)⊥ ∩V ⊃ (W ⊥ )⊥ ∩V = W ∩V = W. On the other hand, if v ∈ V and v ⊥ (W ⊥ ∩ V), then by property 1, there are unique w ∈ W and u ∈ W ⊥ ∩ V such that v = w+u. From assumption, v ⊥ u. Meanwhile w ⊥ u. Then u0 u = u0 (v−w) = 0, so u = 0. Then v = w ∈ W. As a result, (W ⊥ ∩V)⊥ ∩V ⊂ W. Then (W ⊥ ∩V)⊥ ∩V = W, showing property 2. 1.12 In order for a matrix A = {aij } i, j = 1, 2, 3, to commute with the matrix B, we require that AB = BA. Computing the product on both sides, and equating them, we see that the conditions are a11 = a22 = a33 , a12 = a23 , and a21 = a31 = a32 = 0. 1.13 By repeated multiplication, we see that k a A = 0 k
β , 1
where, β = b(1 + a + · · · + ak−1 ). 1.14 It is easily verified that the product (A − B)C is equal to Ak − Bk , where, C = Ak−1 + Ak−2 B + · · · + ABk−2 + Bk−1 . 1.15 Clearly, (A0 A)0 = A0 (A0 )0 = A0 A, and (AA0 )0 = (A0 )0 A0 = AA0 . 1.16 If A = O, then clearly A0 A = O. To show the converse, let the column vectors of A be a1 , · · · , an . Since A0 A = {a0i aj }, if A0 A = O, then for all i = 1, · · · , n, a0i ai = 0, giving ai = 0, and so A = O.
4
Solutions to Chapter 1
1.17 (a) From Result 1.3.5, tr(A
k X
xi x0i ) = tr(
i=1
k X
Axi x0i ) =
i=1
=
k X
k X
tr(Axi x0i )
i=1
k X tr(x0i Axi ) = (xi Axi ).
i=1
i=1
(b) From property 4 of Result 1.3.5, it follows that tr(B−1 AB) = tr(BB−1 A) = tr(A). 1.18 Suppose A = {aij } and B = {bij } are both lower triangular matrices and P aii = ai and n bii = bi . Then aij = bij = 0 for all i < j. Now C = AB = {cij } with cij = k=1 aik bkj . If i > j, then for every k = 1, · · · , n, either i > k or k > j, so aik bkj = 0, giving cij = 0. Then C is lower triangular. If i = j, then aik bkj 6= 0 if and only if k = i, so cii = aii bii = an bn . This completes the proof for the lower triangular case. The proof for the upper triangular case is similar. Pm Pn 1.19 Properties 1, 3, 5, and 6, of Result 1.3.5 are obvious. From tr(AB) = i=1 j=1 aij bji = Pn Pm Pn Pm j=1 i=1 aij bji = j=1 i=1 bji aij = tr(BA), Property 2 follows. Property 4 is a direct consequence, since we can regard ABC as the product of AB and C, or of A and BC. Both tr(AA0 ) and tr(A0 A) equal the sum of squares of the elements of A, which is nonnegative, and shows property 7, of which property 8 is an immediate consequence. 1.20 If A = {akl } is lower triangular, then akl = 0 for all k < l. Now Mij = {mkl }, where akl if k < i, l < j a if k ≥ i, l < j k+1,l mkl = ak,l+1 if k < i, l ≥ j ak+1,l+1 if k ≥ i, l ≥ j. If i > j, then for k < l, the second case in the display is not possible, and all the other cases are possible, each one giving value 0. Therefore, mkl = 0, so that Mij is lower triangular. Furthermore, for any k = j, · · · , i − 1, mkk = ak,k+1 = 0, so Mij has at least one diagonal element equal to zero. This verifies the first fact. From (1.3.1), the ith diagonal element of A−1 is |Mii |/|A|. If the diagonal elements of A are d1 , · · · , dn , then the diagonal elements of Mii are d1 , · · · , di−1 , di+1 , · · · , dn . Therefore |Mii | = d1 · · · di−1 di+1 · · · dn and |A| = d1 · · · dn . Then the ith diagonal element of A−1 is 1/di . 1.21 Using Definition 1.3.7, |A| = −8. 1.22 It is easy to verify the given resursive expression, from which we obtain ∆n = (1 + a2 + a4 + · · · + a2n ) = [1 − a2(n+1) ]/[1 − a4 ]. 1.23 This follows directly from property 7 of Result 1.3.6. 1.24 Using Definition 1.2.7, we can show that the value of this determinant is 1 +
Pn
i=1 ai .
1.25 The determinant to be calculated is |(1 − ρ)In + ρJn |. If ρ 6= 1, then from 12 of Result 1.3.6, the determinant is (1 − ρ)n |In + ρ(1 − ρ)−1 1n 10n | = (1 − ρ)n [1 + ρ(1 − ρ)−1 10n 1n ] = (1 − ρ)n [1 + nρ(1 − ρ)−1 ] = (1 − ρ)n−1 [1 + (n − 1)ρ].
Solutions to Chapter 1
5
1.26 We find the RREF(A : I) = (B : C), say. We see that the matrix 1 2 −1 0 B = 0 −4 0 0 0 has a row of zeroes, so that A is singular. 1.27 If A is n × n and invertible, then there is an n × n matrix B such that AB = I. Then |A||B| = |AB| = |I| = 1, so |A| = 6 0. 1.28 We verify Result 1.3.8. The first four properties are straightforward. See Theorem 18.2.8 in Harville (1997) for a proof of the Sherman–Morrison–Woodbury formula. To prove property 6, (A−1 a)(b0 A−1 ) 0 −1 (A ± ab ) A ∓ 1 ± b0 A−1 a = I ± ab0 A−1 ∓
ab0 A−1 ab0 A−1 ab0 A−1 − . 1 ± b0 A−1 a 1 ± b0 A−1 a
Let us denote the scalar b0 A−1 a by c. Hence, ab0 A−1 ab0 A−1 = acb0 A−1 = cab0 A−1 = (b0 A−1 a)ab0 A−1 so that (A−1 a)(b0 A−1 ) (A ± ab0 ) A−1 ∓ 1 ± b0 A−1 a = I ± ab0 A−1 ∓
ab0 A−1 ab0 A−1 − (b0 A−1 a) = I. 0 −1 1±bA a 1 ± b0 A−1 a
1.29 (a) Since (I − A(I + BA)−1 B)(I + AB) = (I + AB) − A(I + BA)−1 (I + BA)B = I + AB − AB = I. Similarly, we can show that (I + AB)(I − A(I + BA)−1 B) = I, and it follows that (I + AB)−1 = I − A(I + BA)−1 B. (b) Note that we can write aIk + bJk = a(Ik + ab Jk ) = a(Ik + ab 1k 10k ). Set ab 1k = A and 10k = B in (a) and use that result to get −1 1 b −1 0 (aIk + bJk ) = Ik + 1k 1k a a " −1 # 1 b kb = Ik − 1 k 1 + 10k a a a ab 1 b 1 Ik − Jk = I k − Jk . = a a(a + kb) a a(a + kb) 1.30 (a) Use Result 1.3.6. Since A is orthogonal, |A|2 = |AA0 | = I| = 1, which implies |A| = ±1. (b) This follows from Definition 1.2.9 and Definition 1.3.11. (c) AB(AB)0 = ABB0 A0 = AA0 = In . 1.31 Since r(A) = 1, A can be written as ab0 for some a 6= 0 and b 6= 0. Then by property 12 of Result 1.3.6, |In +A| = 1+b0 a = 1+tr(b0 a), which by Result 1.3.5 is 1+tr(ab0 ) = 1 + tr(A).