Signals systems and inference 1st edition oppenheim solutions manual

Signals Systems and Inference 1st Edition Oppenheim Solutions Manual Visit to download the full and correct content document: https://testbankdeal.com/dow nload/signals-systems-and-inference-1st-edition-oppenheim-solutions-manual/

Signals, Systems & Inference Alan V. Oppenheim & George C. Verghese c 2016 Chapter 7 Solutions

Note from the authors These solutions represent a preliminary version of the Instructors’ Solutions Manual (ISM).

The book has a total of 350 problems, so it is possible and even likely that at this preliminary stage of preparing the ISM there are some omissions and errors in the draft solutions. It is also possible that an occasional problem in the book is now slightly different from an earlier version for which the solution here was generated. It is therefore important for an instructor to carefully review the solutions to problems of interest, and to modify them as needed. We will, from time to time, update these solutions with clarifications, elaborations, or corrections.

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

Many of these solutions have been prepared by the teaching assistants for the course in which this material has been taught at MIT, and their assistance is individually acknowledged in the book. For preparing solutions to the remaining problems in recent months, we are particularly grateful to Abubakar Abid (who also constructed the solution template), Leighton Barnes, Fisher Jepsen, Tarek Lahlou, Catherine Medlock, Lucas Nissenbaum, Ehimwenma Nosakhare, Juan Miguel Rodriguez, Andrew Song, and Guolong Su. We would also like to thank Laura von Bosau for her assistance in compiling the solutions.

1 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.1 y6 1

y6 1

y6 1

y6 1

1 2

1 2

1 2

A

D

1 2

C

B 1 2

(a)

-

1 x

1 2

D -

1 x

-

1 2

-

1 2

1 x

1 x

1.

1 1 1 P (A ∩ D) = P y > , x > = 2 2 4 1 2

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

P (A) = P (D) =

Since P (A ∩ D) = P (A)P (D), events A and D are independent.

2.

1 1 1 P (C ∩ D) = P y < , x < = 2 2 4 P (C) = P (D) =

1 2

Since P (C ∩ D) = P (C)P (D), events C and D are independent.

3.

1 1 P (A ∩ B) = P y > , y < =0 2 2 P (A) = P (B) =

1 2

Since P (A ∩ B) 6= P (A)P (B), events A and B are not independent.

4. Another similar case that was not asked in the problem in the book:

P (A ∩ C) = P

1 1 y > ,x < 2 2

P (A) = P (C) =

=

1 4

1 2

Since P (A ∩ C) = P (A)P (C), events A and C are independent.

2 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

(b) Note that P (A ∩ C ∩ D) = 0 since there is no region where all three sets overlap. However, P (A) = P (C) = P (D) = 12 , so P (A ∩ C ∩ D) 6= P (A)P (C)P (D) and the events A, C, and D are not mutually independent, even though they are pairwise independent as seen in (a). For a collection of events to be independent, we require the probability of the intersection of any of the events to equal the product of the probabilities of each individual event. So for the 3–event case, pairwise independence is a necessary but not sufficient condition for independence.

3 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.2 We wish to determine the conditional probability P (S = k | R = k), where S denotes what was sent, and R denotes what is received. Recall the conditional probability of an event A, given that the event B with P (B) > 0 has happened, is defined by P (A | B) =

P (A ∩ B) P (B)

Therefore, P (S = k | R = k) =

P (S = k and R = k) P (R = k)

5 0.05 = ≈ 0.21 0.05 + 0.10 + 0.09 24 0.08 4 P (S = 2 | R = 2) = = ≈ 0.19 0.13 + 0.08 + 0.21 21 0.15 15 P (S = 3 | R = 3) = = ≈ 0.44 0.12 + 0.07 + 0.15 34 T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

P (S = 1 | R = 1) =

The probability of transmission error incurred in using this system is P (transmission error) = 1 − P (no transmission error) =1−

3 X

P (S = k and R = k)

k=1

= 1 − (0.05 + 0.08 + 0.15) = 0.72

4 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.3 (a) a+b 2 Z b b2 + ab + a2 1 v 2 dv = E[V 2 ] = b−a a 3 2 (b − a) σV2 = E[V 2 ] − µ2V = 12 µV =

(b) Independence implies V and W are uncorrelated. Hence

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

µY = µV + µ W = a + b 2 σY2 = σV2 + σW =

(b − a)2 6

If V and W were not independent, the expression for σY2 above would have contained the additional term 2σV,W , where σV,W is the covariance of the two random variables.

5 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.4 (a) Recall the tilde notation Z̃ = Z − µZ and note that we have the relations Z̃ = 2X̃ − Ỹ and

1 W̃ = X̃ + Ỹ .

2

Then, by definition, σZW

= E[Z̃ W̃ ]

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

1 = E[(2X̃ − Ỹ )(X̃ + Ỹ )] 2 1 = 2E[X̃ 2 ] − E[Ỹ 2 ] 2 1 2 = 2(E[X ] − E[X]2 ) − (E[Y 2 ] − E[Y ]2 ) 2 1 = 2(8) − (3) = 29/2 .

2

(b) Z and W are affine functions of X and Y so they will also be bivariate Gaussian random variables. The form for the joint distribution of bivariate Guassians is given in Example 7.5 of the text.

z − µZ w − µW fZ,W (z, w) = c exp −q , σZ σW where

c=

and q(v, w) =

1 p

2πσZ σW

1 − ρ2

1 (v 2 − 2ρvw + w2 ) .

2(1 − ρ2 )

So we will need to know the means and variances of Z and W as well as the correlation coefficient ρ between Z and W . µZ = 2E[X] − E[Y ] + 5 = 1 1 µW = E[X] + E[Y ] − 1 = −1 2 σZ2

= E[Z̃ 2 ] = E[(2X̃ − Ỹ )2 ] = 4E[X̃ 2 ] − 4E[X̃ Ỹ ] + E[Ỹ 2 ] = 4(E[X 2 ] − E[X]2 ) − 4(E[XY ] − E[X]E[Y ]) + E[Y 2 ] − E[Y ]2 = 4(8) − 4(−4 + 2) + 3 = 43 .

6

© 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

2 σW

1 = E[W̃ 2 ] = E[(X̃ + Ỹ )2 ] 2 1 = E[X̃ 2 ] + E[X̃ Ỹ ] + E[Ỹ 2 ] 4 1 = 8 − 2 + (3) = 27/4 .

4

29/2 29 σZW =p =p σZ σW (43)(27)/4 (43)(27)

ρ=

fZ,W (z, w) = = = =

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

So finally,

z − µ Z w − µW exp −q , σZ σW 1 − ρ2

1 p

2πσZ σW

!!! (z − 1)2 29 z−1 w+1 (w + 1)2 √ p − 2p + 43 27/4 (43)(27) 43 27/4 !!!

(43)(27) (z − 1)2 29 z−1 w+1 (w + 1)2 1 √ exp − √ p − 2p + 640 43 27/4 8π 5 (43)(27) 43 27/4

1 1 √ exp − 27(z − 1)2 − 4(29)(z − 1)(w + 1) + 4(43)(w + 1)2 .

640 8π 5 1 √ exp − 8π 5

1 2(1 − ρ2 )

7 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.5 Since σV2 = E[V 2 ] − {E[V ]}2 and E[V ] = 0 is given, we get E[V 2 ] = 4. Furthermore, E[X] = E[Y ] = 2.

Correlation rX,Y = E[XY ] = E[(2 + V )(2 − V )] = E[4 − V 2 ] = 4 − E[V 2 ] = 0 Covariance σX,Y = E[(X − E[X])(Y − E[Y ])] = E[(X − 2)(Y − 2)] = E[−V 2 ] = −4 Correlation coefficient

σX,Y = −1 σX σY

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

ρX,Y =

Since rX,Y = E[XY ] = 0, two random variables X and Y are orthogonal. Since σX,Y 6= 0, the two random variables are correlated.

8 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.6 (a) We begin by defining the centered random variables e = X − µX , X

Ye = Y − µY

and similarly Ze = Z − µZ ,

Ve = V − µV ,

f = W − µW .

W

Note that because µX

= µZ + µV

µY

= βµZ + µW

and

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

we can subtract these equations from the defining equations for X and Y to obtain e = Ze + Ve X f.

Ye = β Ze + W

(1) (2)

2 = E[X e 2 ], σ 2 = E[Ye 2 ], and so on, and noting that the specified uncorrelatNow since σX Y edness properties imply E[ZeVe ] = σZV = 0 and similarly σZW = 0, we can simply square each of the equations displayed above and take expectations, obtaining 2 σX

= σZ2 + σV2

σY2

2 = β 2 σZ2 + σW .

Similarly, for the covariance we take the product of the equations (1) and (2), then compute expectations and use the fact that σV W = 0, to obtain e Ye ] = σXY = βσ 2 .

E[X Z Finally, we have the correlation coefficient ρXY

= =

σXY σX σY βσZ2 q

.

2 ) (σZ2 + σV2 )(β 2 σZ2 + σW

(b) Setting α = σZ2 /σ 2 , we have that βα

ρXY = p

(1 + α)(1 + β 2 α)

.

Some exploration of limiting cases: 9 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

p If β 2 1/α, then ρXY ≈ ± α/(1 + α), with the sign being that of β. If α max{1, 1/β 2 } then ρXY ≈ ±1, with the sign being that of β. This corresponds to the fluctuations in both Z and βZ being much larger than those in V and W , so Y ≈ βX.

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

On the other hand, if α min{1, 1/β 2 }, then ρXY ≈ βα. In particular, for α 1/|β|, ρXY ≈ 0, because now fluctuations in X and Y are dominated by those in V and W respectively.

10 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.7 y6 1

y6 1

y6 1

y6 1

1 2

1 2

1 2

A

D

1 2

C

B 1 2

(a)

-

1 x

1 2

D -

1 x

-

1 2

1 x

1 2

-

1 x

1. As an example, choose E = A, F = C, and G = D. We saw in part (a) that A and C are independent. Let’s check if they are independent when conditioned on D: 1 P (A ∩ D) 1 = 41 = P (D) 2 2

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

P (A | D) = P (C | D) =

1 P (C ∩ D) 1 = 41 = P (D) 2 2

P (A ∩ C | D) =

P (A ∩ C ∩ D) =0 P (D)

Since P (A ∩ C | D) 6= P (A | D)P (C | D), events A | D and C | D are not independent even though A and C are independent when not conditioned on event D. 2. As an example, choose J = A∪C, K = B, and L = C. J and B are not independent: P (J) = P (A) + P (C) − P (A ∩ C) =

3 4

1 2

1 1 1 P (J ∩ B) = P x < , y < = 6= P (J)P (B) 2 2 4 P (B) =

However, we can show that J and B are independent when conditioned on L = C: P (J | C) =

1 P (J ∩ C) = 21 = 1 P (C) 2

1 P (B ∩ C) 1 P (B | C) = = 41 = P (C) 2 2

P (J ∩ B | C) =

1 P (J ∩ B ∩ C) 1 = 41 = = P (J | C)P (B | C) P (C) 2 2

11 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

(b) The statement is TRUE. To see this, let us define a new event E = W ∩ Q, and use it as follows: P (V ∩ W ∩ Q) P (Q) P (V ∩ E) = P (Q) P (V |E)P (E) = P (Q) P (W ∩ Q) = P (V |E) P (Q) = P (V |W ∩ Q)P (W |Q).

P (V ∩ W |Q) =

(3)

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

As an example, we may use the previously defined events B, C, and D. The probability P (B ∩ C|D) may be found from the definition of conditional independence by: P (B ∩ C|D) =

1/4 1 P (B ∩ C ∩ D) = = .

P (D) 1/2 2

If we use the result from Equation 3 we have that P (B ∩ C|D) = P (B|C ∩ D)P (C|D) =

P (B ∩ C ∩ D) P (C ∩ D) 1/4 1/4 1 = = .

P (C ∩ D) P (D) 1/4 1/2 2

12 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.8 This is a lot easier after Chapter 9, but only uses elementary concepts. We solve (a) and (b) together.

There are two possible decision rules, D1 or D2 , D1 ⇒ {m b = 0 : A, m b = 1 : B}, D2 ⇒ {m b = 0 : B, m b = 1 : A} The probability of error, Pe , is given by Pe = P (m b 6= m) = P (m = 1, m b = 0) + P (m = 0, m b = 1) = P (m = 1)P (m b = 0 | m = 1) + P (m = 0)P (m b = 1 | m = 0) T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

Substituting in the given values, we get, Pe,D1 = (0.4 × 0.5) + (0.6 × 0.3) = 0.38 Pe,D2 = (0.4 × 0.5) + (0.6 × 0.7) = 0.62

Since Pe,D1 is the lower of the two, the minimum-probability-of-error decoder is D1 .

The minimum-probability-of-error decoder can be chosen intuitively. One can observe that, since m = 1 has equal probability of resulting in either outcome, it will not affect the final probability of error. In the case m = 0, it is clear that m b = 0 is the more probable outcome, hence one can conclude that assigning A to m b = 0 results in less error.

13 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.9 (a) There are three possible values for R: −2, 0, or 2. The probability for each value, P (R = r), is given as, P (R = r) = P (X = x, N = r − x) = P (X = x)P (N = r − x) The independence of X and N allows the last equality.

P (R = 2) = P (X = 1)P (N = 1) = p(1 − g) P (R = 0) = P (X = 1)P (N = −1) + P (X = −1)P (N = 1) = 1 − g − p + 2gp

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

P (R = −2) = P (X = −1)P (N = −1) = g(1 − p)

(b) The choice of our estimate will be based on the conditional probabilities P (X = x | R = r): R=2 or -2 For each case, only a unique combination of X and N exists. For R = 2, X = N = 1 and for R = −2, X = N = −1. Therefore, only one reasonable estimate is possible for both b = 1 and R = −2 → X b = −1 cases: R = 2 → X R=0 We can further simplify the probability, P (X = x | R = 0) =

P (X = x, R = 0) P (N = −x) = P (R = 0) P (R = 0)

We compare two possible cases, x = 1 and x = −1.

P (N = −1) 1−p = P (R = 0) 1 − g − p + 2gp P (N = 1) p P (X = −1 | R = 0) = = P (R = 0) 1 − g − p + 2gp

P (X = 1 | R = 0) =

b will be −1.

Since 21 < p < 1, P (X = −1 | R = 0) has the larger value. Our estimate, X,

14 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

(c) We expand the given equation for P (correct) as following, b = −1 | X = −1) + P (X = 1)P (X b = 1 | X = 1) P (correct) = P (X = −1)P (X

b = −1, N = 1 | X = −1) + P (X b = −1, N = −1 | X = −1) = P (X = −1) P (X

b = 1, N = 1 | X = 1) + P (X b = 1, N = −1 | X = 1) + P (X = 1) P (X   X b = −1 | X = −1, N = n) P (N = n)P (X = P (X = −1)  n∈{−1,1}

 + P (X = 1) 

 X

b = 1 | X = 1, N = n) P (N = n)P (X

n∈{−1,1}

b = −1 | R = 0) + P (N = −1)P (X b = −1 | R = −2) = P (X = −1) P (N = 1)P (X

b = 1 | R = 2) + P (N = −1)P (X b = 1 | R = 0) + P (X = 1) P (N = 1)P (X T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

Using the values for conditional probabilities from (b), we get

p2 (1 − p)2 P (correct) = g + (1 − p) + (1 − g) p + 1 − g − p + 2gp 1 − g − p + 2gp 2 2 1 − p + 2pg(2p − 1) − g (2p − 1) = 1 − g − p + 2gp

This decision rule that chooses the X of maximum conditional probability, given R, also maximizes the probability of being correct.

15 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.10 (a) Given each hypothesis (H0 , H1 ), the number of photon arrivals is from 0 to ∞. Since the sum of probabilities should equal 1, we get the following: ∞ X k=0 ∞ X

P (k | H0 ) = P (k | H1 ) =

k=0

∞ X k=0 ∞ X

A0 v0k = 1 ⇒

A0 = 1 ⇒ A0 = 1 − v 0 1 − v0

A1 v1k = 1 ⇒

A1 = 1 ⇒ A1 = 1 − v 1 1 − v1

k=0

Therefore,

(b) We use Bayes’ rule.

P (k | H1 ) = (1 − v1 )v1k

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

P (k | H0 ) = (1 − v0 )v0k

P (m | H1 )P (H1 ) P (m) 0.5 × (1 − v1 )v1m = 0.5 × (1 − v0 )v0m + 0.5 × (1 − v1 )v1m (1 − v1 )v1m = (1 − v0 )v0m + (1 − v1 )v1m

P (H1 | m) =

(c) The probability of error, given as Pe , can be defined as follows (for more information, refer to Section 9.3.1): Pe = P (H0 )P (0 H10 | H0 ) + P (H1 )P (0 H00 | H1 ) X X P (k | H1 ) = P (H0 ) P (k | H0 ) + P (H1 ) D1

= 0.5 ×

X

D0

(1 − v0 )v0k + 0.5 ×

k≥n0

X

(1 − v1 )v1k

0≤k<n0

v n0 + 1 − v1n0 = 0 2 One might be tempted to take the derivative with respect to n0 to find the n∗0 that minimizes Pe . However, n0 is an integer, so this approach will be inaccurate.

An alternative approach is to use the hypothesis testing framework from Section 9.2.1, where we learn that Pe is minimized if, for each possible number of photons, we chose the hypothesis H with bigger P (H | k). We only have to compare the numerators, 16 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

0.5 × (1 − v1 )v1k and 0.5 × (1 − v0 )v0k , since the denominators are equal. Note that these two functions intersect at one point, k = k ∗ . For k > k ∗ , P (H1 | k) > P (H0 | k) and k ≤ k ∗ , P (H1 | k) < P (H0 | k). If we let k ∗ = n0 , we notice that the receiver in the problem description already follows the desired decision rule that minimizes Pe , with k ∗ equal to n∗ .

P (H1 | n∗0 ) = P (H0 | n∗0 ) n∗

n∗

⇒ (1 − v1 )v1 0 = (1 − v0 )v0 0

⇒ ln(1 − v0 ) + n∗0 ln(v0 ) = ln(1 − v1 ) + n∗0 ln(v1 )

1−v0 ln 1−v 1

⇒ n∗0 = v1 ln v0 T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

As described above, this is the best decision rule.

(d) Since P (H0 ) 6= P (H1 ), we need to generalize n∗0 from above.

0 )(1−v0 ) ln PP (H (H1 )(1−v1 )

n∗0 = ln vv10 A simple calculation yields the following result

P (H1 ) = 0.7 → n∗0 = 0 → Pe = 0.5 P (H1 ) = 1 → n∗0 = −∞ → Pe ∼ 0

P (H1 ) = 1 makes sense, since if the light signal is always sent, the probability of error would be 0.

17 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.11 (a) In terms of conditional probability, we have to calculate P (Y = y | c used), where y = {−1, 0, 1} and c = {A, B}. The conditional probability could further be simplified as P (Y = y | c used) = P (Y = y, X = 1 | c used) + P (Y = y, X = −1 | c used) = P (X = 1)P (Y = y | X = 1, c used) + P (X = −1)P (Y = y | X = −1, c used)

The conditional probabilities in the last equation can be read straightly from the given figure.

For Channel A, 3 3 = 4 8 1 1 1 P (Y = 0 | A used) = 0.5 × + 0.5 × = 4 4 4 3 3 P (Y = 1 | A used) = 0.5 × + 0.5 × 0 = 4 8

For Channel B,

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

P (Y = −1 | A used) = 0.5 × 0 + 0.5 ×

1 5 + 0.5 × 1 = 4 8 3 3 P (Y = 1 | B used) = 0.5 × + 0.5 × 0 = 4 8 P (Y = −1 | B used) = 0.5 ×

(b) We use the following relation, which is Bayes’ rule.

P (A used, Y = −1) P (Y = −1) P (A)P (Y = −1 | A used) = P (Y = −1)

P (A used | Y = −1) =

Note that the denominator could further be simplified as P (Y = −1) = P (Y = −1, A used) + P (Y = −1, B used) = P (A used)P (Y = −1 | A used) + P (B used)P (Y = −1 | B used) 3 5 5 − 2α = α × + (1 − α) × = 8 8 8

18 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Therefore, P (A used | Y = −1) =

α× 3 3α P (A used, Y = −1) = 5−2α8 = P (Y = −1) 5 − 2α 8

(c) We first need to calculate P (B used | Y = −1) as in (b).

P (B used | Y = −1) =

(1 − α) × 58 P (B used, Y = −1) 5(1 − α) = = 5−2α P (Y = −1) 5 − 2α 8

Therefore,

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

P (A used | Y = −1) > P (B used | Y = −1) 5(1 − α) 3α > ⇒ 5 − 2α 5 − 2α 5 ⇒α> 8

5 − 2α is positive as α itself is positive and cannot be bigger than 1. Therefore, if 5 8 < α ≤ 1, we would decide that channel A was used upon receiving Y = −1.

Next, we need to figure out the range of α for which the Channel A will be chosen, regardless of the received Y value. Notice that Y = 0 is only possible through channel A, regardless of the α value (α cannot be 0 though, as Y = 0 cannot happen in the first place.) Thus, we focus on Y = 1. The following needs to hold: P (A used | Y = 1) > P (B used | Y = 1) ⇒

3 3 8 ×α 8 × (1 − α) > 3(1−α) 3(1−α) 3α 3α 8 + 8 8 + 8

⇒α>

1 2

Combining the three cases, we reach the conlcusion that channel A will be decided, regardless of Y value, for 5 <α≤1 8 (d) We use the idea from previous problems. If P (A used | Y = sequence) > P (B used | Y = sequence), channel A will be chosen; otherwise, channel B will be chosen. The problem can be broken down into two parts: 1) b 6= 0 and 2) b = 0.

If b 6= 0, channel A will always be chosen, as channel B does not allow for the received value of 0.

19 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

If b = 0, P (A used | Y = sequence) > P (B used | Y = sequence) P (Y = sequence | A used)P (A used) P (Y = sequence | B used)P (B used) ⇒ > P (Y = sequence) P (Y = sequence) a c ⇒ P (Y = −1 | A) P (Y = 1 | A) × α > P (Y = −1 | B)a P (Y = 1 | B)c × (1 − α) a c a c 3 3 5 3 α> (1 − α) ⇒ 3c α > 5c (1 − α) ⇒ 8 8 8 8 5c ⇒α> c 5 + 3c c

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

Therefore, channel A will be chosen for 5c5+3c < α ≤ 1; otherwise, channel B will be chosen.

20 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.12 (a) False. Suppose (X, Y ) take the values (−1, 0), (0, 1), and (1, 0) with equal probability (= 13 ). Then E[XY ] = 0 since either X or Y is 0 for every outcome, and also E[X] = 0, because it is distributed symmetrically around 0. Hence E[XY ] = E[X]E[Y ], so X and Y are uncorrelated.

Now note that (X 2 , Y 2 ) take the values (1, 0) and (0, 1) with respective probabilities 23 and 31 . Thus E[X 2 Y 2 ] = 0, because either X 2 or Y 2 is 0 for every outcome. However, E[X 2 ] = 32 and E[Y 2 ] = 31 , so E[X 2 Y 2 ] 6= E[X 2 ]E[Y 2 ], i.e., X 2 and Y 2 are uncorrelated.

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

(b) True. By definition of independence, fX,Y (x, y) = fX (x)fY (y). Then we have, Z Z E[g(X)h(Y )] = g(X)h(Y )fX,Y (x, y)dxdy Z Z = g(X)fX (x)dx h(Y )fY (y)dy = E[g(X)]E[h(Y )]

(c) False. The definition of conditional probability is, as stated in Eq. (7.27), fX|Y (x | y) =

fX,Y (x, y) fY (y)

which is same as the given equation in the problem. fX,Y (x, y) = fX (x)fY (y) must hold if X and Y are independent.

21 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.13 2 2 r 2 . Let’s start with what we know: (a) We would like to show that rXY ≤ rX Y

2 2 2 ≤ σX σY σXY

2 2 rX = µ2X + σX

rY2 = µ2Y + σY2 rXY = µX µY + σXY

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

2 2 σ2 , Then substituting back into inequality σXY ≤ σX Y

2 (rXY − µX µY )2 ≤ (rX − µ2X )(rY2 − µ2Y )

2 2 2 2 µ2X µ2Y µ2X µ2Y ≤ rX rY − µ2X rY2 − µ2Y rX + rXY − 2rXY µX µY +

2 2 2 2 rXY − 2rXY µX µY ≤ rX rY − µ2X rY2 − µ2Y rX 2 2 2 2 rXY ≤ rX rY − µ2X rY2 − µ2Y rX + 2rXY µX µY 2 2 ≤ rX rY − E[(µX Y − µY X)2 ]

Since we know that E[(µX Y − µY X)2 ] ≥ 0,

2 2 2 rXY ≤ rX rY .

A more direct route to the proof is to start with the observation that E[(αX − Y )2 ] ≥ 0 2 − 2αr 2 for all α, expand and take expectations to get α2 rX XY + rY ≥ 0, and then note that this is possible if and only if this quadratic in α has at most one real root, which then yields the desired inequality. (b) Using the vector space picture, we can imagine that vector X and vector Y are separated by an angle θ. Then the inner product of X and Y is expressed as

E[XY ] =

p

E[X 2 ]E[Y 2 ] cos(θ)

rXY = rX rY cos(θ) .

22 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Since cos2 (θ) ≤ 1,

2 2 2 rXY ≤ rX rY .

Y

θ

(c) Let’s start with

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

X

−σX σY ≤ σXY ≤ σX σY

−σ 2 ≤ rXY − µ2 ≤ σ 2

After adding µ2 to both sides,

−σ 2 + µ2 ≤ rXY ≤ σ 2 + µ2

−r2 + 2µ2 ≤ rXY ≤ r2

23 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.14 (a) The probability of error is the sum of the numbers in all the off-diagonal boxes, because these are precisely the outcomes that contribute to an error for the given decision rule, so

Perror =

X

P (j sent ∩ k received)

j6=k

= 0.10 + 0.09 + 0.13 + 0.21 + 0.12 + 0.07 = 0.72

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

(b) For minimum probability of error, for each received k, declare in favor of the sent message j that corresponds to the largest element in the column. Thus, j is declared to be sent according to the rule   2 :k=1 1 :k=2 j=  2 :k=3

The probability of deciding correctly is the sum of the joint probabilities that j was sent and k received for each of the pairs above.

Pcorrect = P (j = 2 ∩ k = 1) + P (j = 1 ∩ k = 2) + P (j = 2 ∩ k = 3) = 0.13 + 0.10 + 0.21 = 0.44 The probability of guessing in error and the probability of guessing correctly must sum to 1. Thus,

Perror = 1 − Pcorrect = 0.56 < 0.72

(c) The expected cost is X

c(j, l)P (j sent ∩ l declared)

j,l

24 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

For the decision rule in (a), P (j sent ∩ l declared) is the same as P (j sent ∩ l received), and these terms are all displayed in the table. Multiply each j, k entry in the table by the corresponding c(j, k), and add the resulting numbers to obtain the risk. (d) Rewrite the expected cost as X

c(j, l)P (j sent ∩ l declared |k received)P (k received)

j,k,l

The optimum decision rule minimizes the inner part of this, namely X

c(j, l)P (j sent ∩ l declared |k received)

j,k,l

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

for each received k, because the decision rule specifies what to declare for each received signal, regardless of what’s specified for some other received signal. But given that k is received, the fact that we declare l is independent of the fact that j was sent. So we can write the expected cost as XX l

j

c(j, l)P (j sent |k received)P (l declared|k received)

Any (deterministic) decision rule makes P (l declared |k received) = 1 for some l and 0 for the others. For minimum expected cost, we should make P (l declared |k received) = 1 for the l that has the smallest value of X

c(j, l)P (j sent|l received)

j

In the case of the all-or-none cost, this means we should choose l to be the j for which P (j sent|k received) is largest. This is the same rule we had for minimum probability of error in (b).

25 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.15 (a) The marginal distributions of V and W associated with f + (v, w) and f − (v, w) individually are Gaussian, so the marginals associated with the given fV,W (v, w) are Gaussian as well.

However, the contours of constant density for fV,W (v, w) are evidently not ellipses, so the joint distribution is not bivariate Gaussian. (b) The random variables V and W are not independent. For instance, knowing nothing about the value of V , the distribution of W is Gaussian, but given that V = v0 0, the figure suggests that W will be bimodal. So information about W is obtained by knowing the value of V , which indicates that these two variables are not independent.

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

(c) The four-quadrant symmetry of fV,W (v, w) shows that E[V W ] = 0, and E[V ] = 0 = E[W ]. Therefore, we conclude that V and W are uncorrelated.

26 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.16 (a)

(i) The expected value of Z is µZ = E[cQ + V ] = E[cQ] + E[V ] = cE[Q] + E[V ] = c. e + Ve , where (ii) The variance computations are all simplified by recognizing that Ze = cQ Ze denotes the deviation of Z from its mean value µZ , and similarly for the other variables. σZ2 = E[Ze2 ] e 2 + 2cQ e Ve + Ve 2 ] = E[c2 Q 2 = c2 σQ + σV2 + 2cσQ,V

(iii) The covariance of Z and Q is T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

e σZ,Q = E[ZeQ] e + Ve )Q] e = E[(cQ 2 = cσQ + σV,Q

(iv) The covariance of Z and V is given by

σZ,V = E[ZeVe ]

= cσQ,V + σV2

(b) We want to choose parameters a and b so as to minimize E[(Q − bZ − a)2 ]. If we assume that b is fixed and given, and defining a new random variable X = Q−bZ, it is an easy fact that the value of a that minimizes E[(Q−bZ −a)2 ] = E[(X −a)2 ] is a = E[X] = µQ −bµZ .

Thus, we want to minimize the expression 2 E[(Q − bZ − µQ − bµZ )2 ] = variance(Q − bZ) = σQ + b2 σZ2 − 2bσQ,Z

(4)

with respect to b. This expression is a quadratic function of b, and the minimum is found by taking the derivative with respect to b, setting it equal to 0 and solving for b: d 2 (σ + b2 σZ2 − 2bσQ,Z ) = 2bσZ2 − 2σQ,Z db Q = 0.

Solving for b we find that b=

σQ,Z σQ =ρ , 2 σZ σZ

(5)

σ

where ρ is the correlation coefficient defined as ρ = σQQ,Z σZ . Then a is given by a = µQ − ρ

σQ µZ . σZ

(6)

27 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

With these parameters, the mean squared error is given by 2 b 2 ] = σQ E[(Q − Q) + b2 σZ2 − 2bσQ,Z

from Equation 4

σQ 2 2 = σQ + ρ2 σ Q − 2ρσQ,Z σZ 2 2 σ σ Q,Z Q,Z 2 = σQ + 2 −2 2 σZ σZ 2 = σQ −

2 σQ,Z

σZ2

2 2 2 = σQ − σQ ρ

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

2 = σQ (1 − ρ2 ).

28 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

Solution 7.17 (a) The mean and variance of X can be written in terms of the means and variances of Q and W : X =Q+W µX = µQ 2 2 2 σX = σQ + σW

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

Since Q and W are independent Gaussians, their sum is Gaussian. Hence knowing the mean and variance of X suffices to write down its pdf:

1 (x − µX )2 fX (x) = √ exp − 2 2σX 2πσX (b) The covariance σXQ is most easily computed by first noting that the respective deviations from their mean for the random variables X, Q and W satisfy e =Q e+W f, X

e = X = µX , and similarly for the other variables. Then where X σXQ = E[(X − µX )(Q − µQ )] e Q] e = E[X e 2 ] + E[Q eW f] = E[Q 2 = σQ

since Q and W are independent (and therefore uncorrelated).

The correlation coefficient ρXQ is ρXQ =

2 σQ σXQ =q σX σQ σ2 + σ2 σ Q

W Q

σQ

=q

2 + σ2 σQ W

Note for use in part (d) below that 1 − ρ2XQ =

2 2 σW σW = 2 + σ2 2 . σQ σX W

1. ρXQ is not a function of µQ σ

Q 2. If σW σQ , then noise wins and ρXQ ⇒ σW

29 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

If σQ σW , then signal wins and ρXQ ⇒ 1 (c) The joint pdf of fX,Q (x, q) can be found from fX|Q (x|q) and fQ (q): fX,Q (x, q) = fX|Q (x|q)fQ (q) ! (q − µQ )2 1 fQ (q) = √ exp − 2 2σQ 2πσQ

1 (x − q)2 fX|Q (x|q) = √ exp − 2 2σW 2πσW 1 (x − q)2 (q − µQ )2 fX,Q (x, q) = exp − − 2 2 2πσQ σW 2σW 2σQ

!

fQ|X (q|x) =

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

(d) To find the conditional density fQ|X (q|x), using the fact that we already know fX,Q (x, q) and fX (x), write fX,Q (x, q) fX (x)

(x − q)2 (q − µQ )2 (x − µX )2 1 σX exp − =√ − + 2 2 2 2σW 2σQ 2σX 2π σW σQ 2 ! q − µQ|x 1 p = exp − 2 2σQ (1 − ρ2 ) σQ 2π(1 − ρ2 )

!

where we have introduced the notation

µQ|x =

2 x + σ2 µ σQ W Q 2 + σ2 σQ W

and ρ is streamlined notation for ρXQ . The algebra that gets you to this final result is straightforward, though painful!

We conclude that the conditional distribution of Q, given X = x, is Gaussian, with mean 2 (1 − ρ2 ). µQ|x and variance σQ (e) The choice of constant that minimizes the expected or mean squared deviation of a random variable from that constant is just the expected value of the random variable. In this case, since we are given that X = x, we need the expected value of Q given X = x. Hence b MMSE (x) = E[Q|X = x] = µQ|x = Q

2 x + σ2 µ σQ W Q 2 + σ2 σQ W

.

This estimate is indeed an affine function of x. As a sanity check on the answer, note that b MMSE (x) ≈ x, which makes sense, since X = Q+W . On if the noise intensity is low, then Q 30 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.

b MMSE (x) ≈ µQ , because the measurement the other hand, if the noise dominates, then Q is very unreliable.

The corresponding conditional mean square error is 2 σ2 σQ W 2 2 MMSE = σQ (1 − ρ ) = q 2 + σ2 σW Q

This expression also takes the values that one would expect for the extreme cases of high noise and low noise.

T a his th nd wo o eir is rk w r sa co pro is ill le u vi pr de o rse de ot st f a s d s ec ro n an o te y y p d le d th a a ly by e rt ss fo U in o e r te f t ss th nite gr hi in e ity s w g us d S of or stu e o tat th k ( de f i es e in nt ns co w cl le tr p or ud a uc y r k an ing rnin tors igh d on g. in t la is w D no the iss tea s t p W em ch er or in ing m ld a itt W tio ed id n . e W eb )

(f) If Q and W are no longer independent, then X is not guaranteed to be Gaussian, so we cannot proceed further to compute the requested pdf’s.

31 © 2016 Pearson Education, Inc., Hoboken, NJ. All rights reserved. This material is protected under all copyright laws as they currently exist. No portion of this material may be reproduced, in any form or by any means, without permission in writing from the publisher.