CS 361 Probability & Statistics for Computer Science-University of Illinois - Homework 8

downloaded from

Scholarfriends

https://scholarfriends.com Downloaded by: writerforacademic@gmail.com | https://scholarfriends.com/singlePaper/307204/cs-361-probability-statistics-for-computerscience-university-of-illinois-homework-8 Distribution of this document is illegal.

CS361 Probability & Statistics

University of Illinois at Urbana-Champaigns

Problem 1 Since this is the probability density function, using the log-likehood function logL(θ) = logP (D|θ) = log

Y

P (xi |θ)

xi ∈D

Substituting the give formula in question to have: ⇔

N X

log(θe−θxi )

i=1

⇔

N X

log(θ) − θxi

i=1

Differenting both sides of the equation to have: N

N

i=1

i=1

X1 N X d logL(θ) = 0 = − xi = − xi dθ θ θ Maniputating the equation to have: N

X N = xi θ i=1

N ⇔ θ = PN

i=1 xi

1

CS361 Probability & Statistics

University of Illinois at Urbana-Champaigns

Problem 2 By the formula in the lecture note, the formula of MLE of λ of poisson distribution would PN ki be: θ̂ = i .

N PN ki 3+1+4+2 ˆ = = 2.5 a.) Day one: θ1 = i 4 PN N ki 2+1+2 Day two: θˆ2 = i = = 1.666 N 3 P N ki 3+2+2+1+4 Day three: θˆ3 = i = = 2.4 N 5 PN ki 13 ˆ b.) Day Four: θ3 = i = = 2.166 N 6 PN ki 3 + 1 + 4 + 2 + 2 + 1 + 2 + 3 + 2 + 2 + 1 + 4 + 13 c.) All day combined: θ̂ = i = = N 4+3+5+6 2.222

2

CS361 Probability & Statistics

University of Illinois at Urbana-Champaigns

Problem 3 According to the lecture note, the MLE derivatoin of geometric example is: L(θ) = 1 (1 − θ)k−1 θ; θ̂ = K 1 a.) Substitute K with r + 1 in the question to have: θ̂ = . Assuming there are x zero r+1 x slot in the roulette, the probability of zero slot can be represented by . Manipulating x + 36 the equations to have: 1 x = r+1 x + 36 36 ⇔x= r 36 As a result, the total number of slots in the roulette would be + 36. r b). This estimate is unreliable, since our data base is not large enough to make the MLE reliable. MLE needs more data to have a precise estimate of the number we want. c). We apply the Log-likehood function to do the estimate. logL(θ) = logP (D|θ) = log

Y

P (xi |θ)

xi ∈D

logL(θ) = logP (D|θ) = log

Y

P (xi |θ) =

xi ∈D

⇔

k X

log((1 − θ)ri θ)

i=1

k X (ri )log((1 − θ) + log(θ) i=1

Taking the differential of both sides of equation and both of them would be equal to zero: k

X −ri 1 d logL(θ) = 0 = ( + )= dθ 1−θ θ

Pk

i=1

⇔θ

i=1 −ri

1−θ

+

k θ

k X (−ri ) = (θ − 1)(k) i=1

⇔ θ̂ =

k−

k Pk

i=1 (−ri )

Assuming there are x zero slot in the roulette, the probability of zero slot can be represented x by . Manipulating the equations to have: x + 36 θ̂ = 3

x x + 36

CS361 Probability & Statistics

University of Illinois at Urbana-Champaigns

⇔

x k = Pk x + 36 k − i=1 (−ri ) 36k x = Pk i=1 (ri )

Then the total number of slots would be: x + 36 = 36 + Pk

36k

i=1 (−ri )

4

CS361 Probability & Statistics

University of Illinois at Urbana-Champaigns

Problem 4 a.) Using the Bayesian inference to do the estimate: Using the given prior probability with the corresponding number of zeros to determine the z value of P (n|z) = · P (z) z + 36 P (n|z = 0) =

0 · 0.1 = 0 36

1 · 0.2 = 0.00541 36 + 1 2 · 0.4 = 0.02105 P (n|z = 2) = 36 + 2 3 P (n|z = 3) = · 0.3 = 0.02307 36 + 3 P (n|z = 1) =

b.) By the formula: P (z = 0|observation) =

P (n|z) · P (z) P (n)

To determine under what situation, the value of P (z = 0|observation) would be non zero, it means to determine under what situation, we can’t say certainly that there is not any zero slot. It only happenes when none zero slot is observed in the experience. Since it any number of zero slot is observed, there is no probability to have non zero slot in the wheel.

2 = 0.05556, according to the formula c.) By the given in formation, the value of P (n) = 36 below: P (n|z) · P (z) P (z|n) = P (n) The results would be:

0 · 0.1 =0 0.05556 0.00541 · 0.2 P (z = 1|n) = = 0.01947 0.05556 0.02105 · 0.4 P (z = 2|n) = = 0.15155 0.05556 0.02307 · 0.3 P (z = 3|n) = = 0.12457 0.05556 P (z = 0|n) =

5

CS361 Probability & Statistics

University of Illinois at Urbana-Champaigns

Problem 5 a.)

L(θ) =

50 35 θ (1 − θ)15 15

b.)

99 54 L(θ) = θ (1 − θ)45 54 c.)

L(θ) = ((1 − θ)2 θ) ·

6

10 5 θ (1 − θ)5 5