Time Series Analysis and Its Applications 4th Edition By Robert H. Shumway, David S. Stoffer INSTRUC

INSTRUCTORS SOLUTION MANUAL

Time Series Analysis and Its Applications Q 4th Edition S Instructor’s Manual

© 2017, R.H. Shumway and D.S. Stoffer

We assume that the package for the text, astsa, has been loaded. See the Package Notes at the web page for the text: http://www.stat.pitt.edu/stoffer/tsa4/

! Please Do Not Reproduce !

Chapter 1 Solutions

1.1 The major differences are how quickly the signal dies out in the explosion versus the earthquake and the larger amplitude of the signals in the explosion. plot(EQ5, ylab="Earthquate/Explosion") lines(EXP6, col=2) legend("topright", c("EQ","EXP"), col=1:2, lty=1)

1.2 Figure 1.1 shows contrived data simulated according to this model. The modulating functions are also plotted. The code is given for part (b); part (a) is given in the text. For part (c), basically remove the cos() part. s = c(rep(0,100), 10*exp(-(1:100)/200)*cos(2*pi*1:100/4)) # part (b) x = ts(s + rnorm(200, 0, 1)) plot(x) lines(c(rep(0,100), 10*exp(-(1:100)/200))) # modulator on same plot as the series

The first signal bears a striking resemblance to the two arrival phases in the explosion. The second signal decays more slowly and looks more like the earthquake. The periodic behavior is emulated by the cosine function which will make one cycle every four points. If we assume that the data are sampled at 4 points per second, the data will make 1 cycle in a second, which is about the same rate as the seismic series.

1.3 Below is R code for parts (a)-(c). In all cases the moving average nearly annihilates (completely in the 2nd case) the signal. The signals in part (a) and (c) are similar. w = rnorm(150,0,1) # 50 extra to avoid startup problems x = filter(w, filter=c(0,-.9), method="recursive")[-(1:50)] # AR x2 = 2*cos(2*pi*(1:100)/4) # sinusoid x3 = x2 + rnorm(100,0,1) # sinusoid + noise v = filter(x, rep(1,4)/4) # moving average v2 = filter(x2, rep(1,4)/4) # moving average v3 = filter(x3, rep(1,4)/4) # moving average par(mfrow=c(3,1)) plot.ts(x, main="autoregression") lines(v,lty="dashed") plot.ts(x2, main="sinusoid") lines(v2,lty="dashed") plot.ts(x3, main="sinusoid + noise") lines(v3,lty="dashed")

1.4 Simply expand the binomial product inside the expectation and use the fact that µt is a nonrandom constant, i.e., γ(s, t) = E[(xs xt − µs xt − xs µt + µs µt ] = E(xs xt ) − µs E(xt ) − E(xs )µt + µs µt

2

1 Chapter 1 Solutions

Series (a)

Modulator (a)

10

1 0.8

5

0.6 0 0.4 −5

−10

0.2

0

50

100

150

0

200

0

50

Series (b)

100

150

200

150

200

Modulator (b)

15

1

10

0.9 0.8

5

0.7 0 0.6 −5

0.5

−10 −15

0.4 0

50

100

150

200

0.3

0

50

100

Fig. 1.1. Simulated series with exponential modulations

= E(xs xt ) − µs µt − µs µt + µs µt 1.5 (a) In each case the signals are fixed, so E(xt ) = E(st + wt ) = st + E(wt ) = st . To plot the means, repeat Problem 1.2 and just plot the signals (s) without the noise. (b) The autocovariance function γ(t, u) = E[(xt − st )(xu − su ) = E(wt wu ), which is one (1) when t = u and zero (0) otherwise.

1.6 (a) Since Ext = β1 + β2 t, the mean is not constant, so the process is not stationary. Note that xt − xt−1 = β1 + β2 t + wt − β1 − β2 (t − 1) − wt−1 = β2 + wt − wt−1 , which is clearly stationary. Verify that the mean is β2 and the autocovariance is 2 for s = 2 and −1 for |s − t| = 1 and is zero for |s − t| > 1. (b) First, write q X 1 [(β1 + β2 (t − j)] 2q + 1 j=−q

q X 1 = (2q + 1)(β1 + β2 t) − β2 j 2q + 1 j=−q

E(yt ) =

= β1 + β2 t

because the positive and negative terms in the last sum cancel out. To get the covariance write the process as ∞ X yt = aj wt−j , j=−∞

where aj = 1, j = −q, . . . , 0, . . . , q and is zero otherwise. To get the covariance, note that we need

1 Chapter 1 Solutions

3

γy (h) = E[(yt+h − Eyt+h )(yt − Eyt )] XX = (2q + 1)−2 aj ak Ewt+h−j wt−k j

k

X σ2 = aj ak δh+k−j , 2 (2q + 1) j,k

∞ X

=

aj+h aj ,

j=−∞

where δh+k−j = 1, j = k+h and is zero otherwise. Writing out the terms in γy (h), for h = 0, ±1, ±2, . . ., we obtain σ 2 (2q + 1 − |h|) γy (h) = (2q + 1)2 for h = 0, ±1, ±2, . . . , ±2q and zero for |h| > q.

1.7 By a computation analogous to that appearing in Example 1.17, we may obtain  6σ 2 h = 0    w 2 h=±1 4σw γ(h) = 2 h=±2 σ    w 0 |h|>2.

2 .

The autocorrelation is obtained by dividing the autocovariances by γ(0) = 6σw Ps 1.8 (a) Simply substitute δs + k=1 wk for xs to see that

δt +

t X

wk = δ + δ(t − 1) +

k=1

|

{z xt

t−1 X

wk + wt .

k=1

}

|

{z

}

xt−1

Alternately, the result can be shown by induction. (b) For the mean, ! t t X X Ewk = δt. wk = δt + Ext = E δt + k=1

k=1

For the covariance, without loss of generality, consider the case s ≤ t. γ(s, t) = cov(xs , xt ) = E{(xs − δs)(xt − δt)} X

s t X =E wj wk j=1

k=1

= E (w1 + · · · + ws )(w1 + · · · + ws + ws+1 + . . . + wt ) =

s X

2 E(wj2 ) = s σw .

2 [or min(s, t) σw ]

j=1

(c) The series is nonstationary because both the mean function and the autocovariance function depend on time, t. p p 2 2 2 , which yields the result. The implication is that (d) From (b), ρx (t − 1, t) = (t − 1)σw / (t − 1)σw tσw the series tends to change slowly.

4

1 Chapter 1 Solutions

(e) One possibility is to note that ∇xt = xt − xt−1 = δ + wt , which is stationary because µx,t = δ and 2 γx (t + h, t) = σw δ0 (h) are both independent of time t, where δ0 (h) is the delta measure.

1.9 Because E(U1 ) = E(U2 ) = 0, we have E(xt ) = 0. Then, γ(h) = E(xt+h xt )

= E U1 sin[2πω0 (t + h)] + U2 cos[2πω0 (t + h)] U1 sin[2πω0 t] + U2 cos[2πω0 t]

2 = σw sin[2πω0 (t + h)] sin[2πω0 t] + cos[2πω0 (t + h)] cos[2πω0 t] 2 = σw cos[2πω0 (t + h) − 2πω0 t] 2 = σw cos[2πω0 h]

by the standard trigonometric identity, cos(A − B) = sin A sin B + cos A cos B. 1.10 (a)

2 2 2 M SE(A) = E xt+` − 2AE(xt+` xt ) + A E(xt ) = γ(0) − 2Aγ(`) + A2 γ(0).

Setting the derivative with respect to A to zero yields −2γ(`) + 2Aγ(0) = 0 and solving gives the required value. (b)

ρ(`)γ(`) 2 2 2 2 + ρ (`) = γ(0) 1 − 2ρ (`) + ρ (`) = γ(0) 1 − ρ (`) .

M SE(A) = γ(0) 1 − 2 γ(0) (c) If xt+` = Axt with probability one, then

2 E(xt+` − Axt ) = γ(0) 1 − ρ (`) = 0 2

implying that ρ(`) = ±1. Since A = ρ(`), the conclusion follows.

P∞ 1.11 (a) Since xt = j=−∞ ψj wt−j , γ(h) = E

∞ X

∞ X

j=−∞ k=−∞

2 ψj wt+h−j wt−k ψk = σw

X

2 ψj ψk δh−j+k = σw

j,k

∞ X

ψk+h ψk ,

k=−∞

where δt = 1 for t = 0 and is zero otherwise. (b) The proof is identical to the one given in Appendix A, Example A.2.

1.12 γxy (h) = E[(xt+h − µx )(yt − µy )] = E[(yt − µy )(xt+h − µx )] = γyx (−h) 1.13 (a)

 2 (1 + θ2 ) + σu2  σw 2 γy (h) = −θσw  0

h=0 h = ±1 |h| > 1.

1 Chapter 1 Solutions

(b)

5

2 σw h=0 2 γxy (h) = −θσw h = −1 0 otherwise.

2 γx (h) = σw h = 0 0 otherwise.

(

ρxy (h) = p

γxy (h) . γx (0)γy (0)

(c) The processes are jointly stationary because the autocovariance and cross-covariance functions depend only on lag h.

1.14 (a) For the mean, write

1 E(yt ) = E(exp{xt }) = exp µx + γx (0) , 2 using the given equation at λ = 1. (b) For the autocovariance function, note that

E(yt+h yt ) = E exp{xt+h } exp{xt }

= E exp{xt+h + xt } = exp{2µx + γx (0) + γx (h)}, since xt + xt+h is the sum of two correlated normal random variables and will be normally distributed with mean 2µx and variance

γx (0) + γx (0) + 2γx (h) = 2 γx (0) + γ(h) .

For the autocovariance of yt , γy (h) = E(yt+h yt ) − E(yt+h )E(yt )

2 1 = exp{2µx + γx (0) + γx (h)} − exp µx + γx (0) 2

= exp{2µx + γx (0)} exp{γx (h)} − 1 .

1.15 The process is stationary because µx,t = E(xt ) = E(wt wt−1 ) = E(wt )E(wt−1 ) = 0; 2 2 2 4 γx (0) = E(wt wt−1 wt wt−1 ) = E(wt2 )E(wt−1 ) = σw σw = σw ,

γx (1) = E(wt+1 wt wt wt−1 ) = E(wt+1 )E(wt2 )E(wt−1 ) = 0 = γ(−1), and similar computations establish that γx (h) = 0, for |h| ≥ 1. The series is white noise.

1.16 (a) For t = 1, 2, . . ., Z 1 sin(2πut)du = −

E(xt ) = 0

1

1 1 cos(2πut) = − cos(2πt) − 1 = 0; 2πt 2πt 0

Z 1 γ(h) =

sin[2πu(t + h)] sin[2πut] du.

0

Using the identity 2 sin(α) sin(β) = cos(α − β) − cos(α + β) gives γ(0) = 1/2 and γ(h) = 0, for h 6= 0.

6

1 Chapter 1 Solutions

(b) This part of the problem is harder than it seems at first and it might be a good idea to omit it in more elementary presentations. The easiest way to tackle the problem is to calculate some probabilities (which can be given as a hint), e.g., because U is uniform on [0, 1], Pr{x1 ≤ 0, x3 ≤ 0} = Pr{sin(2πU ) ≤ 0, sin(2π3U ) ≤ 0} n

o 1 = Pr U ∈ [1/2, 1] ∩ [1/6, 1/3] ∪ [1/2, 2/3] ∪ [5/6, 1] = 0 + 1/6 + 1/6 = 3 but, similarly, Pr[x2 ≤ 0, x4 ≤ 0] =

1 .

4

1.17 (a) The essential part of the exponent of the characteristic [or moment generating] function is n X

λ j xj =

j=1

n X

λj (wj − θwj−1 )

j=1

= −λ1 θw0 +

n−1 X

(λj − θλj+1 )wj + λn wn .

j=1

Because the wt are independent and identically distributed, the characteristic function can be written as φ(λ1 , . . . , λn ) = φw (−λ1 θ)

n−1 Y

φw (λj − θλj+1 )φw (λn )

j=1

(b) Because the joint distribution of the wj will not change simply by shifting x1 , . . . , xn to x1+h , . . . , xn+h , the characteristic function [or MGF] remains the same.

1.18 Letting k = j + h, holding j fixed after substituting from (1.29) yields ∞ X

2 |γ(h)| = σw

h=−∞ 2 = σw

∞ X

∞ X

2 ψj+h ψj ≤ σw

h=−∞ j=−∞ ∞ ∞ X X

|ψk |

k=−∞

∞ X

∞ X

|ψj+h ||ψj |

h=−∞ j=−∞

|ψj | < ∞.

j=−∞

1.19 (a) E(xt ) = E(µ + wt + θwt−1 ) = µ.

2 2 (b) γ(h) = cov(wt+h + θwt+h−1 , wt + θwt−1 , so γx (0) = (1 + θ2 )σw , γx (±1) = θσw , and 0 otherwise. (c) From (a) and (b) we see that, for any θ, both the mean function and autocovariance function are independent of time. (d) From the given formula, and because γx (h) = 0 for |h| > 1, we have

2(n − 1) 1 γx (0) + γx (1) . var(x̄) = n n • • •

2 When θ = 0, γx (±1) = 0 and it’s the classical case, var(x̄) = σw /n.

2 σ σ2 2 2 w When θ = 1, γx (0) = 2σw and γx (±1) = σw , so var(x̄) = n [2 + 2(n−1) ] = nw [4 − n2 ]. In this n case, the variance is about 4 times as large as the uncorrelated case. σ2 σ2 2 2 When θ = −1, γx (0) = 2σw and γx (±1) = −σw ] = nw [ n2 ]. In this , so var(x̄) = nw [2 − 2(n−1) n case, the variance is smaller (n > 2) than the uncorrelated case and the variance is nearly zero for moderate n.

1 Chapter 1 Solutions

7

(e) It’s easier to estimate the mean when θ is negative. Negatively correlated data vary around the mean more tightly than positively correlated data. In essence, you need fewer data to identify the mean if the data are negatively correlated.

1.20 Code for parts (a) and (b) is below. Students should have about 1 in 20 ACF values within the bounds, but the values for part (b) will be larger in general than for part (a). wa = rnorm(500,0,1) wb = rnorm(50,0,1) par(mfrow=c(2,1)) (acf(wa, 20)) # plot and print results (acf(wb, 20)) # plot and print results

1.21 This is similar to the previous problem. Generate 2 extra observations due to loss of the end points in making the MA. wa = rnorm(502,0,1) wb = rnorm(52,0,1) va = filter(wa, sides=2, rep(1,3)/3) vb = filter(wb, sides=2, rep(1,3)/3) par(mfrow=c(2,1)) (acf(va, 20, na.action = na.pass)) # plot and print results (acf(vb, 20, na.action = na.pass)) # plot and print results

1.22 Generate the data as in Problem 1.2 and then type acf(x). The sample ACF will exhibit significant correlations at one cycle every four lags, which is the same frequency as the signal. (The process is not stationary because the mean function is the signal, which depends on time t.)

1.23 The sample ACF should look sinusoidal, making one cycle every 50 lags. x = 2*cos(2*pi*(1:500)/50 + .6*pi)+ rnorm(500,0,1) acf(x,100)

1.24 γy (h) = cov(yt+h , yt ) = cov(xt+h − .7xt+h−1 , xt − .7xt−1 ) = 0 if |h| > 1 because the xt s are independent. When h = 0, γy (0) = σx2 (1+.72 ), where σx2 is the variance of xt . When h = 1, γy (1) = −.7σx2 .

Thus, ρy (1) = −.7/(1 + .72 ) = −.47 1.25 (a) The variance is always non-negative, so for xt a stationary series X

X X n n n X X as xs = cov as γ(s − t)at = a0 Γ a ≥ 0, var as xs , at xt = s

s=1

t

s=1 t=1

thus Γ = {γ(s − t), s, t = 1, . . . , n} is a non-negative definite matrix. (b) Let Yt = xt − x̄ for t = 1 . . . n and construct the n × 2n matrix   0 0 · · · 0 Y1 Y2 · · · Yn  0 · · · 0 Y1 Y2 · · · Yn 0  D= .. ..   ... . .  0

Y1

Y2

···

Yn

0

···

0

With Γ̂n = {γ̂(s − t)}ns,t=1 , it is easy to show that Γ̂n =

1 DD0 . n

Then, for any vector a ∈ IRn , a0 Γ̂na =

n X 1 0 1 a DD0a = c0c = c2i ≥ 0 n n i=1

8

1 Chapter 1 Solutions

for c = D0a. Noting that Γ̂n is symmetric, it will be positive definite (p.d.) if its eigenvalues are positive.

Since the diagonal elements of Γ̂n are γ̂(0), the sum of the eigenvalues of Γ̂n is nγ̂(0). Consequently, Γ̂n is p.d. as long as γ̂(0) > 0. In other terms, if the sample variance of the data is not zero, Γ̂n is p.d.

1.26 (a) Ex̄t =

N n 1 X N µt 1 X xjt = µt = = µt N j=1 N j=1 N

(b) E[(x̄t − µt )2 ] =

N N N 1 XX 1 1 X 2 ejt = γe (t, t) E(x − µ )(x − µ ) = jt t kt t 2 2 N j=1 N j=1 N k=1

(c) As long as the separate series are observing the same signal, we may assume that the variance goes down proportionally to the number series as in the iid case. If normality is reasonable, pointwise 100(1 − α) % intervals can be computed as √ x̄t ± zα/2 γe (t, t)/ N 1.27 Vx (h) =

1 1 E[(xs+h − µ) − (xs − µ)]2 = [γ(0) − γ(h) − γ(−h) + γ(0)] = γ(0) − γ(h).

2 2

1.28 The numerator and denominator of ρ̂(h) are γ̂(h) =

n−h n−h n−h X β2 X 1 X (t − t̄) [β1 (t − t̄) + β1 h][β1 (t − t̄)] = 1 (t − t̄)2 + h n t=1 n t=1 t=1

and

n

γ̂(0) =

β12 X (t − t̄)2 . n t=1

Now, write the numerator as

β12 γ̂(h) = γ̂(0) + − n

n X

n X

2

(t − t̄) − h

t=n−h+1

(t − t̄)

t=n−h+1

Hence, we can write ρ̂(h) = 1 + R where

β12 R= − nγ̂(0)

n X

2

(t − t̄) − h

t=n−h+1

n X

(t − t̄)

t=n−h+1

is a remainder term that needs to converge to zero. We can evaluate the terms in the remainder using m X t=1

and

m X t=1

The denominator reduces to

t2 =

t=

m(m + 1) 2

m(2m + 1)(m + 1) 6

1 Chapter 1 Solutions

nγ̂(0) = β12

X n

t2 − nt̄2

9

t=1

n(n + 1)(2n + 1) n(n + 1)2 − 6 4 n(n + 1)(n − 1) = β12 , 12

= β12

whereas the numerator can be simplified by letting s = t − n + h so that X

h h X β12 2 − R= (s + n − h − t̄) − h (s + n − h − t̄) nγ̂(0) s=1 s=1 The terms in the numerator of R are O(n2 ), whereas the denominator is O(n3 ) so that the remainder term converges to zero.

1.29 (a)

√ E[x̄2 ] Pr{ n|x̄| > } ≤ n 2

Note that,

∞ X

nE[x̄2 ] →

γ(h) = 0,

u=−∞

where the last step employs the summability condition. The variance of x̄ is derived in (1.33). (b) An example of such a process is xt = ∇wt = wt − wt−1 , where wt is white noise. This situation arises when a stationary process is over-differenced (i.e., wt is already stationary, so ∇wt would be considered over-differencing).

1.30 Let yt = xt − µx and write the difference as n n−h X X

n1/2 γ̃(h) − γ̂(h) = n−1/2 yt+h yt − n−1/2 (yt+h − ȳ)(yt − ȳ) t=1

=n

−1/2

t=1 n X

yt+h yt + ȳ

n−h X

yt + ȳ

t=1

t=n−h+1

n−h X

yt+h − (n − h)ȳ

2

t=1

For the first term

E n−1/2

n X

yt+h yt

≤ n

−1/2

t=n−h+1

≤ n−1/2

E

n X

|yt+h yt |

t=n−h+1 n X

2 E1/2 [yt+h ]E1/2 [yt2 ]

t=n−h+1

= n

−1/2

hγx (0)

→ 0, as n → ∞. Applying the Markov inequality in the hint then shows that the first term is op (1). In order to handle the other terms, which differ trivially from n−1/2 nȳ 2 , note that, from Theorem A.5, n1/2 ȳ converging in distribution to a standard normal implies that nȳ 2 converges in distribution to a chi-square random variable with 1 degree of freedom and hence nȳ 2 = Op (1). Hence, n−1/2 nȳ 2 = n−1/2 Op (1) = op (1) and the result is proved.

10

1 Chapter 1 Solutions

1.31 To apply Theorem A.7, we need the ACF of xt . Note that X γx (h) = φj φk E[wt+h−j wt−k ] j,k 2 = σw

X

φh+k φk

k 2 h = σw φ

∞ X

φ2k

k=0

σ 2 φh = w 2, 1−φ and we have ρx (h) = φh for the ACF. Now, from (A.55), we have w11 = =

∞ X

2 ρx (u + 1) + ρx (u − 1) − 2ρx (1)ρx (u) u=1 ∞ X

u+1

φ + φu−1 − 2φu+1

u=1 ∞

=

(1 − φ2 )2 X 2u φ φ2 u=1

= 1 − φ2 .

The limiting result implies that

1 − φ2 ρ̂(1) ∼ AN φ, . n

In order to derive a 100(1 − α) % confidence interval, note that n(ρ̂(1) − φ)2 2 ≤ zα/2 1 − φ2 with probability 1 − α. Looking at the roots of Aφ2 + Bφ + C = 0, where A = (1 +

2 zα/2

n B = −2ρ̂(1),

and C = ρ̂2 (1) −

),

2 zα/2

n

gives the interval

√ B B 2 − 4AC − ± 2A 2A Taking ρ̂(1) = .64, n = 100, z.025 = 1.96 gives the approximate 95% confidence interval (.47, .77).

1.32 (a) E(xt xt+h ) = 0 and E(xt xt+h xs xs+k ) = 0 unless all subscripts match. But t 6= s, and h, k ≥ 1, so all subscripts can’t match and hence cov(xt xt+h , xs xs+k ) = 0.

1 Chapter 1 Solutions

(b) Define yt = xt

Ph

and var(yt ) = σ that

4

λj xt+j , for λ1 , . . . , λh ∈ R arbitrary. Then yt is strictly stationary, h-dependent, Pn 2 j=1 λj . Hence, with ȳn = 1 yt /n, the M -dependent CLT of Appendix A implies

j=1 P h

√

∞ h X

X nȳn →d N 0, γy (`) ≡ N 0, γy (0) ≡ N 0, σ 4 λ2j . j=1

`=−∞

Thus σ −2 n−1/2

n X h X

h X

λj xt xt+j →d N 0, λ2j

t=1 j=1

j=1

which, by the Cramer-Wold device, implies σ −2 n−1/2

n X (xt xt+1 , . . . , xt xt+h )0 →d (z1 , . . . , zh )0 . t=1

(c) This part follows from the proof of Problem 1.30, noting that µx = 0. (d) Using part (c), for large n, √ Pn √ n t=1 xt xt+h /n Pn nρ̂(h) ∼ .

2 t=1 xt /n Since the denominator →p σ 2 , using Slutsky’s Theorem,

11

n−1/2

Pn

2 t=1 xt xt+h /n P σ ; j = 1, . . . , h n 2 2 σ t=1 xt /n

0

→d (z1 , . . . , zh )0 .

Chapter 2 Solutions

2.1 (a)–(e) Detailed code is given in Appendix R. We reproduce it here with additional code that will produce all the necessary results. The model is overparameterized if an intercept is included (the terms for each Q are intercepts); most packages will kick out Q1 or Q4 (R kicks out Q1). The estimated annual increase is β̂ = .167. In general, αi − αj is the average increase (decrease) from quarter i to quarter j. The question asks for α̂4 − α̂3 = .88 − 1.15 = −.27, indicating a decrease of about .27. There is substantial correlation left in the residuals, even at the yearly cycle. trend = time(jj) - 1970 # helps to ‘center’ time Q = factor(rep(1:4,21)) # make (Q)uarter factors reg = lm(log(jj)~0 + trend + Q, na.action=NULL) # without intercept Coefficients: Estimate Std. Error t value trend 0.167172 0.002259 74.00 Q1 1.052793 0.027359 38.48 Q2 1.080916 0.027365 39.50 Q3 1.151024 0.027383 42.03 Q4 0.882266 0.027412 32.19 --Residual standard error: 0.1254 on 79 degrees of freedom Multiple R-squared: 0.9935, Adjusted R-squared: 0.9931 F-statistic: 2407 on 5 and 79 DF, p-value: < 2.2e-16 reg2 = lm(log(jj)~ trend + Q, na.action=NULL) # with intercept plot(log(jj), type="o") # plot data and fit lines(fitted(reg), col=2) dev.new() plot.ts(resid(reg)) # plot residuals dev.new() acf(resid(reg), 20) # ACF of the resids

2.2 (a)–(b) The following code will produce the output. Note that Pt−4 is significant in the regression and highly correlated (zero-order correlation is .52) with mortality. Note: Pt and Pt−4 are also highly correlated.

The AIC and BIC are smaller than the values in Table 2.2, so the addition of the lagged particulate is an improvement. temp = tempr-mean(tempr) ded = ts.intersect(cmort, trend=time(cmort), temp, temp2=temp^2, part, partL4=lag(part,-4)) summary(fit <- lm(cmort~trend + temp + temp2 + part + partL4, data=ded)) pairs(ded) # easiest way is to do all of them cor(ded) AIC(fit)/nrow(ded) - log(2*pi) [1] 4.641492

2 Chapter 2 Solutions

13

BIC(fit)/nrow(ded) - log(2*pi) [1] 4.699677

2.3 The code is given below. The trend stationary data (y) are very regular and the true and estimated mean functions are typically close to each other. On the other hand, the random walk data (x) are very irregular, they’re all over the place and the fitted line and true mean functions are rarely close. par(mfrow=c(4,2), mar=c(2.5,2.5,0,0)+.5, mgp=c(1.6,.6,0)) for (i in 1:4){ x = ts(cumsum(rnorm(100,.01,1))) # data y = ts(.01*1:100 + rnorm(100)) regx = lm(x~0+time(x), na.action=NULL) # regressions regy = lm(y~0+time(y), na.action=NULL) plot(x) # plots abline(a=0, b=.01, col=2, lty=2) # true mean abline(regx, col=4) # fitted line plot(y) abline(a=0, b=.01, col=2, lty=2) abline(regy, col=4) }

# set up

2.4 For the normal regression models we have xt ∼ N (β β 0j z t , σj2 ), for j = 1, 2. Then ln

n n f1 (x x; β 1 , σ12 ) 1 X n n 1 X 0 2 2 2 z ) + = − ln σ + ln σ − (x − β (xt − β 02z t )2 t 1 t 1 2 f2 (x x; β 2 , σ22 ) 2 2 2σ12 t=1 2σ22 t=1

Taking expectations, the fourth term in the above becomes by adding and subtracting β 01z t inside the parentheses E1 [(xt − β 02z t )2 ] = nσ12 + (β β 1 − β 2 )0 Z 0 Z(β β1 − β 2) and, dividing through by n and collecting terms, we obtain the quoted result.

2.5 Using the quoted results and the independence of β̂ β and σ̂ 2 , we have

2 1 n χk 2 2 E1 [I(β β , σ 2 ; β̂ β , σ̂ 2 )] = E1 − ln σ + E ln σ̂ + E 1 1 1 2 2 χn−k χ2n−k

1 1 n 2 2 2 = − ln σ + E ln σ̂ + E χ E E1 1 1 1 1 k 2 χ2n−k χ2n−k

1 n k = − ln σ12 + E1 ln σ̂ 2 + + , 2 n−k−2 n−k−2 which simplifies to the desired result.

2.6 (a) Since Ext = β0 + β1 t, the mean depends on t and hence the process in not stationary. (Note that the points will be randomly distributed around a straight line.) (b) Note that ∇xt = β1 + wt − wt−1 so that E(∇xt ) = β1 and  2 h=0  2σw 2 cov(∇xt+h , ∇xt ) = −σw h = ±1  0 |h| > 1. (c) Here ∇xt = β1 + yt − yt−1 , so E(∇xt ) = β1 + µy − µy = β1 . Also, cov(∇xt+h , ∇xt ) = cov(yt+h − yt+h−1 , yt − yt−1 ) = 2γy (h) − γy (h + 1) − γy (h − 1), which is independent of t.

14

2 Chapter 2 Solutions

2.7 This is similar to part (c) of the previous problem except that now we have E(xt − xt−1 ) = δ, with autocovariance function cov(wt+h + yt+h − yt+h−1 , wt + yt − yt−1 ) = γw (h) + 2γy (h) − γy (h + 1) − γy (h − 1).

2.8 (a) The variance in the second half of the varve series is obviously larger than that in the first half. Dividing the data in half gives γ̂x (0) = 133, 594 for the first and second parts respectively and the variance is about 4.5 times as large in the second half. The transformed series yt = ln xt has γ̂y (0) = .27, .45 for the two halves, respectively and the variance of the second half is only about 1.7 times as large.

Histograms, computed for the two series in Figure 2.1 indicate that the transformation improves the normal approximation. varv1 = varve[1:317] varv2 = varve[318:634] var(varv1) # = 133.4574 var(varv2) # = 594.4904 var(log(varv1)) # = 0.2707217 var(log(varv2)) # = 0.451371 par(mfrow=c(1,2)) hist(varve) hist(log(varve)) plot(log(varve)) # for part (b) acf(log(varve)) # for part (c) plot(diff(log(varve))) # for part (d) acf(diff(log(varve))) # for part (d)

Histogram of log(varve)

100

Frequency

150

0

0

50

50

100

Frequency

200

150

250

200

Histogram of varve

0

50

100 varve

150

1

2

3

4

5

log(varve)

Fig. 2.1. Histograms for varve series xt and yt = ln xt .

(b) The data between 300 and 450 show a positive trend that is similar to the global temperature data.

(Presumably, this is due to a difference in the Earth’s rotation.) (c) The ACF of the yt is positive for a large number of lags and decreases in a linear fashion. (d) The plot of ut and its ACF seem to indicate stationarity. The ACF has one significant value at lag 1 (with a value of −.3974).

Because ut can be written in the form

xt xt − xt−1 xt − xt−1 ut = log = log 1 + ≈ , xt−1 xt−1 xt−1

2 Chapter 2 Solutions

15

it can be interpreted as the proportion of annual change. (e)-(f) Note that 2 γu (0) = E[ut − µu )2 ] = E[wt2 ] + θ2 E[wt−1 ]2 = σw (1 + θ2 ) and

2 γu (1) = E[(wt+1 − θwt )(wt − θt−1 )] = −θE[wt2 ] = −θσw ,

with γu (h) = 0 for |h| > 1. The ACF is ρ(1) = or

θ 1 + θ2

ρ(1)θ2 − θ + ρ(1) = 0

and we may solve for θ=

1±

p

1 − 4ρ2 (1) 2ρ(1)

using the quadratic formula. Hence, for ρ̂(1) = −.3974 p 1 ± 1 − 4(−.3974)2 θ̂ = , 2(−.3974) yielding the roots θ̂ = −.4946, −2.0217. We take the root θ = −.4946 (this is the invertible root, see Chapter 3). Then, γ̂u (0) .3317 2 σw = = = .2665 1 + θ2 1 + (−.4946)2 2.9 The R code for this problem is below. summary(fit <- lm(soi~time(soi), na.action=NULL)) # part (a) Estimate Std. Error t value Pr(>t) (Intercept) 13.70367 3.18873 4.298 2.12e-05 time(soi) -0.00692 0.00162 -4.272 2.36e-05 <- significant slope soi.d = resid(fit) # part (b), detrended data in soi.d plot(soi.d) (length(soi.d)) # = 453 per = as.vector(abs(fft(soi.d))^2/453) # as.vector removes the ts attributes ord = 1:227 freq = (ord-1)/453 plot(freq, per[ord], type="l") # graph cbind(freq , per[ord]) # list

The El Niño peak is around .024 or approx 1 cycle/42 months (freq = 0.02428 and local max per = 1.22). The annual peak is at freq = 0.0838 with per = 9.33.

2.10 (a) ts.plot(oil, gas, col=1:2) will plot the data on the same graph. The data resemble the random walks shown in Figure 1.10. We showed in Chapter 1, Example 1.18, that random walks are not stationary [this is also covered in Problem 1.8]. See Figure 2.2. Use the following URL for more details on the data set or more concurrent series: http://tonto.eia.doe.gov/dnav/pet/pet_pri_spt_s1_w.htm. (b) See the answer to 2.8(d). (c) The code is below and follows the hints. The transformed series look stationary and there is very little autocorrelation left after transformation, so a random walk seems plausible for each series. There are some very extreme values [students are pointed to note the outliers in part (e)]. The two series seem to be moving at the same time [this is stressed in parts (d) and (e)].

16

2 Chapter 2 Solutions

50

100

150

200

250

300

oil gas

2000

2002

2004

2006

2008

2010

Time

Fig. 2.2. Oil in dollars per barrel; gas in cents per gallon poil = diff(log(oil)) pgas = diff(log(gas)) ts.plot(poil, pgas, col=1:2) acf(poil) acf(pgas)

(d) There is strong cross-correlation at the zero lag [.66] and poil one week ahead [.18]. There is also feedback for pgas three weeks ahead [.17]. As noted in the hint, ccf(poil, pgas) has oil leading for negative values of Lag; i.e., R computes corr[poil(t+Lag), pgas(t)] for Lag = 0, ±1, ±2, .... (e) Using either astsa or tsa3.rda, issuing lag2.plot(poil, pgas, 3) will give the plots. Aside from the aforementioned extreme outliers (10-30% changes in oil/gas prices in one week) the data seem fairly linear. Also, there may be some asymmetry noted in the lag 1 plot, but this appearance could be caused by outliers. (f) (i) The hint shows how to set up the regression. Note that the interactions are not significant if you include them. The results are Estimate Std. Error t value Pr(>t) (Intercept) -0.006445 0.003464 -1.860 0.06338 . poil 0.683127 0.058369 11.704 < 2e-16 poilL 0.111927 0.038554 2.903 0.00385 indi 0.012368 0.005516 2.242 0.02534 Residual standard error: 0.04169 on 539 degrees of freedom Multiple R-squared: 0.4563, Adjusted R-squared: 0.4532 F-statistic: 150.8 on 3 and 539 DF, p-value: < 2.2e-16

(ii) Note −0.006445 + 0.012368 = .006. Hence, the two models are Ĝt = −.006 + .68Ot + .11Ot−1

if Ot < 0

Ĝt = .006 + .68Ot + .11Ot−1

if Ot ≥ 0.

which suggests that there is no asymmetry for these data. The prices are FOB (Free On Board: A sales transaction in which the seller makes the product available for pick up at a specified port or terminal at a specified price and the buyer pays for the subsequent transportation and insurance.) so the asymmetry might be found in the gas pump prices.

2 Chapter 2 Solutions

17

(iii) plot.ts(resid(fit)) and acf(resid(fit)) seem to suggest the fit is not bad except for the very extreme outliers.

2.11 These are interesting (see Figure 2.3). Students should realize that we’re all going to die.

0.2 0.0 −0.2 −0.4

globtemp

0.4

0.6

0.8

plot(globtemp) lines(lowess(globtemp),col=2) lines(smooth.spline(globtemp, spar=1),col=4)

1880

1900

1920

1940

1960

1980

Time

Fig. 2.3. lowess is red; smoothing spline is blue

2000

2020

Chapter 3 Solutions

2

∂ρx (1) θ 1−θ 3.1 Note ρx (1) = 1+θ = (1+θ 2 and is bounded. Thus 2 )2 = 0 when θ = ±1. We conclude ρx (1) has a ∂θ maximum at θ = 1 wherein ρx (1) = 1/2 and a minimum at θ = −1 wherein ρx (1) = −1/2.

3.2 (a) Use induction or insert the solution into the equation, i.e., t X

φj wt−j = φ

φj wt−1−j + wt .

j=0

j=0

(b) E(xt ) =

t−1 X

Pt

j j=0 φ E(wt−j ) = 0 2 Pt σw 2(t+1) 2j 2 2j ), using the fact that wt is noise. = 1−φ 2 (1 − φ j=0 φ var(wt−j ) = σw j=0 φ

Pt

(c) var(xt ) = (d) Notice that xt+h =

t+h X j=0

φj wt+h−j =

h−1 X

φj wt+h−j +

j=0

t+h X

φj wt+h−j =

h−1 X

φj wt+h−j +φh

j=0

j=h

t X

φk wt−k =

k=0

h−1 X

φj wt+h−j +φh xt .

j=0

Alternately, just iterate xt+h back h time units. For example, below (3.5) we iterated back k units of time; so in that result put t 7→ t + h and k 7→ h to get xt+h = φh xt +

h−1 X

φj wt+h−j ,

j=0

immediately.

Since xt involves the ws for s ≤ t, h−1

X cov(xt+h , xt ) = cov φj wt+h−j + φh xt , xt = φh var(xt ). j=0

(e) xt is not stationary because the (co)variance depends on time t.

2 (f) As t → ∞, var(xt ) → σw /(1 − φ2 ) by (c), and hence by (d), the autocovariance function is independent of t. (g) Generate more than n observations, for example, generate n + n0 observations, where n0 is fairly large (like 50), and discard the first n0 . i h 2t Pt−1 Pt−1 2j φ2t 1−φ2t φ 2 2 2 (h) Write xt = φt w0 + j=0 φj wt−j . Now, var(xt ) = 1−φ = σw = 2 σw + σw j=0 φ 1−φ2 + 1−φ2 2 σw 1−φ2 independent of t.

3 Chapter 3 Solutions

19

3.3 (a) Most of the work is done in the example. For ease, write θ = φ−1 . Then

γx (h) = cov (θwt+1 + · · · + θh+1 wt+h+1 + θh+2 wt+h+2 + · · ·), (θwt+h+1 + θ2 wt+h+2 + · · ·) 2 2 h 2 = σw (θh+2 + θh+4 + · · ·) = σw θ [θ /(1 − θ2 )].

(b) This follows from the discussion of the causal model. Here we have yt = θyt−1 + vt , so yt = P∞ part j 2 2 2 j=0 θ vt−j . Noting (3.6), (3.7) and that σv = σw θ finishes the verification.

3.4 (a) Write this as (1 − .3B)(1 − .5B)xt = (1 − .3B)wt and reduce to (1 − .5B)xt = wt . Hence the process is a causal and invertible AR(1): xt = .5xt−1 + wt . (b) The AR polynomial is 1 − 1z + .5z 2 which has complex roots 1 ± i outside the unit circle (note |1 ± i|2 = 2). The MA polynomial is 1 − z which has root unity. Thus the process is a causal but not invertible ARMA(2, 1).

3.5 Let ξ1 and ξ2 be the roots of φ(z), that is, φ(z) = (1 − ξ1−1 z)(1 − ξ2−1 z). The causal condition is |ξ1 | > 1, |ξ2 | > 1. Let u1 = ξ1−1 and u2 = ξ2−1 so that φ(z) = (1 − u1 z)(1 − u2 z) with causal condition |u1 | < 1, |u2 | < 1. To show |u1 | < 1, |u2 | < 1 if and only if the three given inequalities hold. In terms of u1 and u2 , the inequalities are: (i) φ2 + φ1 − 1 = −(1 − u1 )(1 − u2 ) < 0 (ii) φ2 − φ1 − 1 = −(1 + u1 )(1 + u2 ) < 0 (iii) |φ2 | = |u1 u2 | < 1

(note φ1 = u1 + u2 and φ2 = −u1 u2 )

[if] If (i)–(iii) hold, then (iii), which is |u1 u2 | < 1, implies at least one of u1 , u2 must be less than 1 in absolute value (both if they are complex). Thus, (iii) is enough to imply |u1 | < 1, |u2 | < 1 in the case of complex roots.

Now suppose the roots are real. Suppose wolog, |u1 | < 1. But, if |u1 | < 1, then (1 ± u1 ) > 0 so for (i) and (ii) to hold, we must have (1 ± u2 ) > 0 or |u2 | < 1 as desired. [only if] If |u1 | < 1, |u2 | < 1 and they are real, then (i) and (ii) hold because (1 ± uj ) > 0 for j = 1, 2; (iii) is obvious.

If |u1 | < 1, |u2 | < 1 and they are complex, u2 = u1 and (i) −|1 − u1 |2 < 0, (ii) −|1 + u1 |2 < 0, (iii) |u1 |2 < 1. 2 √ √ 3.6 The roots of φ(z) = 1 + .9z 2 are ± i/ .9. Because the roots are purely imaginary, θ = arg(i/ .9) = π/2 √ h and consequently, ρ(h) = a .9 cos( π2 h + b), or ρ(h) makes one cycle every 4 values of h. Because ρ(0) = 1 √ h and ρ(1) = φ1 /(1 − φ2 ) = 0, it follows that a = 1 and b = 0 in which case ρ(h) = .9 cos( π2 h). Thus √ √ 5 ρ(h) = {1, 0, − .9, 0, .9 , . . .} for h = 0, 1, 2, 3, 4, . . .. u = ARMAacf(ar=c(0,-.9), lag.max=25) plot(0:25, u, type="h", xlab="Lag", ylab="ACF") points(0:25, u) abline(h=0)

3.7 These problems can be tedious, so we suggest not assigning all three parts. Also, students can check their answers in R using ARMAacf(ar=c(_,_), lag.max=10); fill in the blanks. For the ACFs we have ρ(0) = 1, ρ(1) = φ1 /(1 − φ2 ) and distinct roots:

ρ(h) = c1 z1−h + c2 z2−h

equal roots:

ρ(h) = z0−h (c1 + c2 h)

(a) φ(z) = 1 + 1.6z + .64z 2 = (1 + .8z)2 . This is equal roots case with z0 = −1/.8. Thus ρ(h) = −.8h (a + bh). To solve for a and b, note for h = 0 we have ρ(0) = 1 = a and for h = 1 we have ρ(1) = −1.6/(1 + .64) = −.8(1 + b) or b = .22. Finally, ρ(h) = −.8h (1 + .22h) for h = 0, 1, 2, . . . .

3 Chapter 3 Solutions

1.0

20

●

●

●

0.5

● ● ●

ACF

0.0

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

−0.5

● ● ● ●

●

0

5

10

15

20

25

Lag

Fig. 3.1. ACF for Problem 3.6

(b) φ(z) = 1 − .4z − .45z 2 = (1 − .9z)(1 + .5z). This is the unequal roots case with z1 = 1/.9 and z2 = −1/.5. For the ACF, ρ(h) = a.9h − b.5h where a and b are found by solving 1 = a + b and .4/(1 − .45) = .9a − .5b or a = .88 and b = .12. (c) φ(z) = 1 − 1.2z + .85z 2 . This is the complex roots case, with inverse roots .6 ± .7i. Arg(.6+.7i) and Mod(.6+.7i) give θ = arg(.6+.7i) = .86 radians and |.6+.7i| = .92. Thus, ρ(h) = a .92h cos(.86h+b) where a and b are found by solving 1 = a cos(b) [h = 0] and 1.2/(1 + .85) = a .92 cos(.86 + b) [h = 1].

Solving, b = π and a = −1.

3.8 There’s not much to do in verifying the calculations... almost everything is done in the example. The ACF distinguishes the MA(1) case but not the ARMA(1,1) or AR(1) cases, which look similar to each other (see Figure 3.2). u1 = ARMAacf(ar=.6, ma=.9, lag.max=10) u2 = ARMAacf(ar=.6, ma=0, lag.max=10) u3 = ARMAacf(ar=0,ma=.9, lag.max=10) plot(u1[-1], type="o", col=1, ylab="ACF", xlab="Lag") lines(u2[-1], type="o", col=2) lines(u3[-1], type="o", col=4) legend("topright", c("ARMA","AR","MA"), col=c(1,2,4), lty=1)

3.9 R code: ar = arima.sim(list(order=c(1,0,0), ar=.6), n=100) ma = arima.sim(list(order=c(0,0,1), ma=.9), n=100) arma = arima.sim(list(order=c(1,0,1), ar=.6, ma=.9), n=100) acf2(ar) acf2(ma) acf2(arma)

The results should be close to Table 3.1 with randomness taken into consideration.

3.10 R code and output: (reg = ar.ols(cmort, order=2, demean=FALSE, intercept=TRUE)) Coefficients 1 2

0.8

●

0.6

3 Chapter 3 Solutions

●

21

ARMA AR MA

●

0.4

ACF

●

●

●

0.2

● ● ● ●

0.0

●

●

●

2

●

● ●

●

●

4

● ● ●

6

● ● ●

8

● ●

● ●

10

Lag

Fig. 3.2. ACFs for Problem 3.8 0.4286 0.4418 Intercept: 11.45 (2.394) Order selected 2 sigma^2 estimated as 32.32 predict(reg, n.ahead=4) $pred Time Series: Start = c(1979, 41) End = c(1979, 44) Frequency = 52 [1] 87.59986 86.76349 87.33714 87.21350 $se Time Series: Start = c(1979, 41) End = c(1979, 44) Frequency = 52 [1] 5.684848 6.184973 7.134227 7.593357

P∞ j 3.11 (a) The model can be written as xn+1 = j=1 (−θ) xn+1−j + wn+1 . From this we conclude that P∞ 2 j 2 = σw . x̃n+1 = j=1 (−θ) xn+1−j and MSE = E(xn+1 − x̃n+1 )2 = Ewn+1 P n n j (b) Truncating, we have x en+1 = j=1 (−θ) xn+1−j . Thus  2 ∞ X MSE = E(xn+1 − x enn+1 )2 = E  (−θ)j xn+1−j + wn+1  j=n+1

 = E (−θ)(n+1)

∞ X

2 (−θ)j−(n+1) xn+1−j + wn+1 

j=n+1

h

= E (−θ)(n+1) w0 + wn+1

i2

2 = σw 1 + θ2(n+1) .

There can be a substantial difference between the two MSEs for small values of n, but for large n the difference is negligible.

22

3 Chapter 3 Solutions

3.12 The proof is by contradiction. Assume there is a Γn that is singular. Because γ(0) > 0, Γ1 = {γ(0)} is non-singular. Thus, there is an r ≥ 1 such that Γr is non-singular. Consider the ordered sequence Γ1 , Γ2 , . . . and suppose Γr+1 is the first singular Γn in the sequence. Then xr+1 is a linear combination of x = (x1 , . . . , xr )0 , say, xr+1 = b0 x where b = (b1 , ..., br )0 . Because of stationarity, it must also be true that xr+h+1 = b0 xh , where xh = (xh , . . . , xr+h−1 )0 for all h ≥ 1. This means that for any n ≥ r + 1, xn is a linear combination of x1 , . . . , xr , i.e., xn = b0n x where bn = (bn1 , ..., bnr )0 . Thus, γ(0) = var(xn ) = b0n Γr bn = b0n QΛQ0 bn where QQ0 is the identity matrix and Λ = diag{λ1 , . . . , λr } is the diagonal matrix of the positive eigenvalues (0 < λ1 ≤ · · · ≤ λr ) of Γr . From this result we conclude γ(0) ≥ λ1 b0n QQ0 bn = λ1

r X

b2nj ;

j=1

this shows that for each j, bnj is bounded in n. In addition, γ(0) = cov(xn , xn ) = cov(xn , b0n x) from which it follows that r X 0 < γ(0) ≤ |bnj | |γ(n − j)|. j=1

From this inequality it is seen that because the bnj are bounded, it is not possible to have γ(0) > 0 and γ(h) → 0 as h → ∞.

3.13 First take the prediction equations (3.63) with n = h and divide both sides by γ(0) to obtain Rh φh = ρh .

Partition the equation as in the hint with φh = (φ0h−1 , φhh )0 [note ρ(0) = 1]:

Rh−1 ρ̃h−1 φh−1 ρh−1 = , ρ̃0h−1 1 φhh ρ(h) and solve. We get Rh−1 φh−1 + ρ̃h−1 φhh = ρh−1

(1)

ρ̃0h−1 φh−1 + φhh = ρ(h).

(2)

Solve equation (1) for φh−1 to obtain −1 φh−1 = Rh−1 (ρh−1 − ρ̃h−1 φhh ) .

Substitute this into equation (2) and solve for φhh : φhh =

−1 ρ(h) − ρ̃0h−1 Rh−1 ρh−1 −1 1 − ρ̃0h−1 Rh−1 ρ̃h−1

Next, we must show that the PACF,

E( t δt−h ) q

.

(3)

,

2 ) E( 2t )E(δt−h

can be written in the form of equation (3). To this end, let x = (xt−1 , ..., xt−h+1 )0 . The regression of xt on x −1 −1 is (Γh−1 γh−1 )0 x. The regression of xt−h on x is (Γh−1 γ̃h−1 )0 x. Thus −1 0 t = xt − γh−1 Γh−1 x −1 0 δt−h = xt−h − γ̃h−1 Γh−1 x.

From this we calculate [the calculations below are all similar to the verification of equation (3.66); also, note for vectors a and b, a0 b = b0 a] −1 0 E( t δt−h ) = cov( t , δt−h ) = γ(h) − γ̃h−1 Γh−1 γh−1 .

3 Chapter 3 Solutions

23

Similar calculations show that −1 2 0 E(δt−h ) = var(δt−h ) = γ(0) − γ̃h−1 Γh−1 γ̃h−1 .

Also note that the error of the regression of xt on x is the same as the error of the regression of xt on x̃, where −1 x̃ = (xt−h+1 , ..., xt−1 )0 ; that is, t = xt − (Γh−1 γ̃h−1 )0 x̃. From this we conclude that −1 −1 0 0 E( 2t ) = var( t ) = γ(0) − γh−1 Γh−1 γh−1 = γ(0) − γ̃h−1 Γh−1 γ̃h−1 .

This proves the result upon factoring out γ(0) in the numerator and denominator.

3.14 (a) We want to find g(x) to minimize E[y − g(x)]2 . Write this as E[E{(y − g(x))2 x}]. Minimize the inner expectation: ∂E{(y − g(x))2 x}/∂g(x) = 2[E(y|x) − g(x)] = 0 from which we conclude g(x) = E(y|x) is the required minimum. (b) g(x) = E(y|x) = E(x2 + z|x) = x2 + E(z) = x2 . MSE = E(y − g(x))2 = E(y − x2 )2 = E(z 2 ) = var(z) = 1. (c) Let g(x) = a + bx. Using the prediction equations, g(x) satisfies (i) E[y − g(x)] = 0

(ii) E[(y − g(x))x] = 0

(i) E[y] = E[a + bx]

(ii) E(xy) = E[(a + bx)x]

or From (i) we have a + bE(x) = E(y), but E(x) = 0 and E(y) = 1 so a = 1. From (ii) we have aE(x) + bE(x2 ) = E(xy), or b = E[x(x2 + z)] = E(x3 ) + E(xz) = 0 + 0. Finally g(x) = a + bx = 1 and MSE = E(y − 1)2 = E(y 2 ) − 1 = E(x4 ) + E(z 2 ) − 1 = 3 + 1 − 1 = 3.

Conclusion: In this case, the best linear predictor has three times the error of the optimal predictor (conditional expectation).

Pm−1 2 2 3.15 For an AR(1), equation (3.86) is exact; that is, E(xt+m − xtt+m )2 = σw j=0 ψj . For an AR(1), Pm−1 2j 2 2m 2 j 2 ψj = φ and thus σw j=0 φ = σw (1 − φ )/(1 − φ ), the desired expression.

P∞ 3.16 From Example 3.7, xt = 1.4 j=1 (−.5)j−1 xt−j + wt , so the truncated one-step-ahead prediction using Pn (3.81) is x̃nn+1 = 1.4 j=1 (−.5)j−1 xn+1−j .

From Property 3.7, n x̃nn+1 = .9xn + .5w̃nn = .9xn + .5(xn − .9xn−1 − .5w̃n−1 ) n n = 1.4xn − .9(.5)xn−1 − .5w̃n−1 = 1.4xn − .9(.5)xn−1 − .52 (xn−1 − .9xn−2 − .5w̃n−2 ) n = 1.4xn − 1.4(.5)xn−1 + .9(.52 )xn−2 + .53 w̃n−2 n = 1.4xn − 1.4(.5)xn−1 + 1.4(.52 )xn−2 − .9(.53 )xn−3 − .54 w̃n−3 .. . n X = 1.4 (−.5)j−1 xn+1−j j=1

3.17 E(xn+m − x̃n+m )(xn+m+k − x̃n+m+k ) = E

m−1 X

ψj wn+m−j

j=0

m+k−1 X

ψ` wn+m+k−`

`=0

m−1 m−1 m+k−1

X X X 2 =E ψj wn+m−j ψ` wn+m+k−` = σw ψj ψj+k j=0

`=k

j=0

3.18 (a)–(b) Below reg1 is least squares and reg2 is Yule-Walker. The standard errors for each case are also evaluated; the Yule-Walker run uses Property 3.8. The two methods produce similar results.

24

3 Chapter 3 Solutions

(reg1 = ar.ols(cmort, order=2)) 32.32 (reg2 = ar.yw(cmort, order=2)) 32.84 (reg1$asy.se.coef) (sqrt(diag(reg2$asy.var.coef)))

#

coefs: [1] 0.4286 [2] 0.4418;

#

coefs: [1] 0.4339 [2] 0.4376 ; sigma^2 estimated as

# #

sigma^2 estimated as

se: [1] 0.0397 [2] 0.0397 se: [1] 0.0400 [2] 0.0400

3.19 (a) For an AR(1) we have, xn1 = x1 , xn0 = φx1 , xn−1 = φxn0 = φ2 x1 , and in general, xnt = φ1−t x1 for t = 1, 0, −1, −2, . . .. def (b) w̃t (φ) = xnt − φxnt−1 = φ1−t x1 − φφ2−t x1 = φ1−t (1 − φ2 )x1 .

P∞ P1 P1 1 2(1−t) 2 2 2 2 = (1 − φ2 )2 x21 t=1 φ2t−2 = (1 − φ2 )2 x21 1−φ (c) 2 = t=−∞ φ t=−∞ w̃t (φ) = (1 − φ ) x1 2 2 (1 − φ )x1 .

Pn P1 Pn (d) From S(φ) = (1 − φ2 )x21 + t=2 (xt − φxt−1 )2 = t=−∞ w̃t2 (φ) + t=2 (xt − φxt−1 )2 = Pn (3.107), 2 t=−∞ w̃t (φ) using (c) and the fact that w̃t (φ) = xt − φxt−1 for 1 ≤ t ≤ n. (e) For t = 2, ..., n, xt−1 = φxt−1 and xt − xtt−1 = xt − φxt−1 . For t = 1, x01 = E(x1 ) = 0 so t 0 2 x1 − x1 = x1 . Also, for t = 2, ..., n, Ptt−1 = E(xt − φxt−1 )2 = E(wt2 ) = σw so rtt−1 = 1. For t = 1, 2 0 2 0 2 2 P1 = E(x1 ) = σw /(1 − φ ) so r1 = 1/(1 − φ ) we may write S(φ) in the desired form.

3.20 Although the results will vary, the data should behave like observations from a white noise process and each run should yield different parameter estimates where one is approximately the negative of the other.

Students may also get warnings such as non-convergence or the SEs being NaN; it may also happen that the ar estimate is negative one. x = arima.sim(list(order=c(1,0,1), ar=.9, ma=-.9), n=500) plot(x) acf2(x) sarima(x, 1, 0 ,1)

3.21 The following R program can be used. The estimates are close to the actual values. rep(NA, 10) -> phi -> theta -> sigma2 for (i in 1:10){ x = arima.sim(n = 200, list(ar = .9, ma = .5)) fit = arima(x, order=c(1,0,1)) phi[i] = fit$coef[1]; theta[i] = fit$coef[2]; sigma2[i] = fit$sigma2 } cbind("phi"=phi, "theta"=theta, "sigma2"=sigma2)

3.22 Below is R code for pthis example using Yule-Walker. The asymptotic distribution is normal with mean .99 and standard error (1 − .992 )/50 ≈ .02. The bootstrap distribution should be very different than the asymptotic distribution. (If students use MLE, there might be problems because φ is very near the boundary.

You might alert students to this fact or let them find out on their own.) x = arima.sim(list(ar=.99), n=50) fit = ar.yw(x, order=1) phi = fit$ar # estimate of phi nboot = 200 # number of bootstrap replicates resids = fit$resid[-1] # the first resid is NA x.star = x # x[1] stays the same throughout phi.star = rep(NA, nboot) for (i in 1:nboot) { resid.star = sample(resids, replace=TRUE) for (t in 1:49) x.star[t+1] = phi*x.star[t] + resid.star[t] phi.star[i] = ar.yw(x.star, order=1)$ar } summary(phi.star); hist(phi.star); stem(phi.star) # and so on ...

3 Chapter 3 Solutions

25

3.23 Write wt (φ) = xt − φxt−1 for t = 1, ..., n conditional on x0 = 0. Then zt (φ) = −∂wt (φ)/∂φ = xt−1 .

Let φ(0) be an initial guess at φ, then Pn φ(1) = φ(0) +

t=1 xt−1 (xt − φ(0) xt−1 ) Pn = 2 t=1 xt−1

Pn xt−1 xt Pt=1 , n 2 t=1 xt−1

and the estimate has converged in one step to the (conditional) SS estimate of φ.

3.24 (a) Taking expectation through the model, E(xt ) = α + φE(xt−1 ) + 0 + θ0, we have µ = α + φµ or α = µ(1 − φ). Let yt = xt − µ, then the model can be written as yt = φyt−1 + wt + θwt−1 . Because |φ| < 1 the process yt is causal (and hence stationary) and consequently xt is stationary. The same technique used in Problem 1.17 P∞can be used here to show that yt , and hence xt , is strictly stationary. (b) Because of causality, yt = j=0 ψj wt−j where ψ0 = 1 and ψj = (φ + θ)φj−1 for j = 1, 2, . . . (see P∞ Example 3.11) and hence xt = µ + j=0 ψj wt−j . Thus, by Theorem A.5, x̄ ∼ AN(µ, n−1 V ) where P 2

P∞ j−1 2 ∞ 2 2 ψ = σ 1 + (φ + θ) . Equivalently, x̄ ∼ AN(α/(1 − φ), n−1 V ).

V = σw j w j=0 j=1 φ   (1 + a2 )σs2 , h = 0 3.25 (a) E(xt ) = 0 and γx (h) = E(st +ast−δ )(st+h +ast+h−δ ) = aσs2 , h = ±δ so the process  0, |h| > 1, is stationary.

Also, xt − ax + a2 xt−2δ − · · · + (−1)k ak xt−kδ = st − (−1)k+1 ak+1 st−kδ , and letting k → ∞ Pt−δ ∞ shows st = j=0 (−a)j xt−δj is the mean square convergent representation of st . Note: If δ is known, the process is an invertible MA(δ) process with θ1 = · · · = θδ−1 = 0 and θδ = a. (b) The Gauss-Newton procedure is similar to the MA(1) case in Example 3.30. Write st (a) = xt − ast−δ (a) for t = 1, ..., n. Then zt (a) = −∂st (a)/∂a = st−δ (a) + a∂st−δ /∂a = st−δ (a) − azt−δ (a). The iterative procedure is Pn zt (a(j) )st (a(j) ) Pn a(j+1) = a(j) + t=1 j = 0, 1, 2, . . .

2 t=1 zt (a(j) ) where zt (·) = 0 and st (·) = 0 for t ≤ 0. (c) If δ is unknown, the ACF of xt can be used to find a preliminary estimate of δ. Then, a Gauss-Newton procedure can be used to minimize the error sum of squares, say Sc (a, δ), over a grid of δ values near the preliminary estimate. The values â and δ̂ that minimize Sc (a, δ) are the required estimates.

3.26 (a) By Property 3.9, φ̂ ∼ AN[φ, n−1 (1 − φ2 )] so that φ̂ = φ + Op (n−1/2 ). (b) xnn+1 = φxn whereas x̂nn+1 = φ̂xn . Thus xnn+1 − x̂nn+1 = (φ − φ̂)xn . Using Tchebcheff’s inequality, it is easy to show xn = Op (1). Thus, by the properties of Op (·), xnn+1 − x̂nn+1 = (φ − φ̂)xn = Op (n−1/2 )Op (1) = Op (n−1/2 ) 3.27 Write ∇k xt = (1 − B)k xt =

Pk

j j=0 cj xt−j where cj is the coefficient of B in the binomial expanP k sion of (1 − B)k . Because xt is stationary, E(∇k xt ) = µx j=0 cj independent of t, and (for h ≥ 0) P P P k k h+k cov(∇k xt+h , ∇k xt ) = cov( j=0 cj xt+h−j , j=0 cj xt−j ) = j=0 dj γx (j), that is, the covariance is a time independent (linear) function of γx (0), . . . , γx (h + k). Thus ∇k xt is stationary for any k.

Write yt = mt + xt where mt is the given q-th order polynomial. BecauseP ∇k xt is stationary for any k, q−1 q q we concentrate on mt . Note that ∇mt = mt − mt−1 = cq [t − (t − 1) ] + j=0 cj [tj − (t − 1)j ]; from this it follows that the coefficient of tq is zero. Now assume the result is true for ∇k mt and show it is true for ∇k+1 mt [that is, for k < q, if ∇k mt is a polynomial of degree q − k then ∇k+1 mt is a polynomial of degree

q − (k + 1)]. The result holds by induction.

26

3 Chapter 3 Solutions

3.28 yt = P xt − xt−1 , then the model is yt = wt − θwt−1 , which is invertible. That is, wt = P∞Write ∞ j j 2 θ y = j=0P t−j j=0 θ (xt−j − xt−1−j ). Rearranging wt = xt − θ(1 − θ)xt−1 − θ (1 − θ)xt−2 − · · ·, or ∞ j xt = j=1 θ (1 − θ)xt−j + wt . n n n 3.29 (a) Use induction: yn+1 = δ + φyn , yn+2 = δ + φyn+1 = δ(1 + φ) + φyn , . . . , and j

) n n j yn+j = δ + φyn+j−1 = · · · = δ(1 + φ + · · · + φj−1 ) + φj yn = δ (1−φ (1−φ) + φ yn . j Pm n Pm P m ) φ(1−φm ) δ (b) Note (1) j=1 yn+j = j=1 (xnn+j −xnn+j−1 ) = xnn+m −xn , (2) j=1 δ (1−φ (1−φ) = (1−φ) [m− (1−φ) ] m Pm ) φ(1−φm ) and (3) j=1 φj yn = yn φ(1−φ (1−φ) = (xn − xn−1 ) (1−φ) do the trick. (c) First note that ψ ∗ (z)(1 − φz)(1 − z) = (1 + ψ1∗ z + ψ2∗ z 2 + · · ·)(1 − [1 + φ]z + z 2 ) = 1 yields the given homogeneous equation and solution, that is, ψ0∗ = 1 and ψj∗ = (1 − φj+1 )/(1 − φ) for j ≥ 1.

Pm−1 1 j+1 2 n 2 ) ]. (The summation part can be expanded, of course, Thus, Pn+m = σw [1 + (1−φ) 2 j=1 (1 − φ but it’s not clear that it gives any further insights. Note that for large m, the end terms in the summation, (1 − φj+1 )2 ≈ 1 for large j.)

3.30 See Figure 3.3. The EWMAs are smoother than the data (note the EWMAs are within the extremes of the data). The EWMAs are not extremely different for the different values of λ, the smoothest EWMA being when λ = .75. x = log(varve[1:100]) x25 = HoltWinters(x, alpha=.75, beta=FALSE, gamma=FALSE) # alpha = 1 - lambda x50 = HoltWinters(x, alpha=.5, beta=FALSE, gamma=FALSE) x75 = HoltWinters(x, alpha=.25, beta=FALSE, gamma=FALSE) plot(x, type="o", ylab="log(varve)") lines(x25$fit[,1], col=2) lines(x50$fit[,1], col=3) lines(x75$fit[,1], col=4) legend("topright", c("25%", "50%", "75%"), col=2:4, lty=1) ● ●

4.0

● ●

● ●

●

3.5

●

log(varve) 2.5 3.0

●●

●

●

● ● ●

●

● ●

●

●

● ●

● ●

●

●

●

●

● ● ●

● ● ● ● ●

●

● ●●

●

● ●

●

●

●

●

● ● ● ●

● ●

●●

● ●●

● ●

●●

25% 50% 75%

●●

● ●

●

● ●

●

●

● ●

●●

● ●● ●

● ● ●

●

● ●

●

●

●

●

●

● ●

● ●

2.0

● ●

●

●

1.5

● ●

0

20

40

60

80

100

Index

Fig. 3.3. EWMAs for Problem 3.30

3.31 Follow the steps of Examples 3.38 and 3.39, sarima(gnpgr, 1, 0, 0) will produce the diagnostics.

The results should be similar to those in Example 3.39.

3.32 The ACF of the returns reveals only small amounts of autocorrelation. The most appropriate models seem to be ARMA(1,1) or ARMA(0,3). BIC prefers the ARMA(1,1) whereas AIC prefers the ARMA(0,3). The diagnostics are ok but there are some major outliers that may be affecting the results. R code below:

3 Chapter 3 Solutions

poil = diff(log(oil)) acf2(poil) sarima(poil, 1, 0, 1) sarima(poil, 0, 0, 3)

27

# BIC favors # AIC favors

3.33 An ARIMA(1,1,1) seems to fit the data. Below is R code for the problem: plot(gtemp) plot(diff(gtemp)) acf2(gtemp) sarima(gtemp, 1, 1, 1) sarima.for(gtemp, 10, 1, 1, 1)

3.34 There is trend so we consider the (first) differenced series, which looks stationary. Investigation of the ACF and PACF of the differenced suggest an ARMA(0,1) or ARMA(1,1) model. Fitting an ARIMA(0,1,1) and ARIMA(1,1,1) to the original data indicates the ARIMA(0,1,1) model; the AR parameter is not significant in the ARIMA(1,1,1) fit. The residuals appear to be (borderline) white, but not normal – there are numerous outliers. Fitting the ARIMA(0,1,1) to log(so2) removes the outliers, but it doesn’t noticeably change anything else. plot(so2); plot(diff(so2)); plot(diff(so2)); acf2(diff(so2)); sarima(so2,0,1,1); sarima.for(so2,4,0,1,1);

plot(log(so2)) plot(diff(log(so2)) plot(diff(log(so2)) acf2(diff(log(so2)) sarima(log(so2),0,1,1) sarima.for(log(so2),4,0,1,1)

3.35 (a) plot(sales) - looks like random walk, so difference d = 1, (can do this with log(sales) but it’s not necessary). acf2(diff(sales)) - ACF and PACF both tailing, so choose p = q = 1 sarima(sales,1,1,1, no.constant=TRUE) fits well. (b) ccf(diff(sales), diff(lead)) and lag2.plot(diff(lead), diff(sales),8) suggest that there is a strong linear relationship between diff(sales) and lag(diff(lead),-3). Also, lag2.plot shows strong linear relationship. (c) After fitting the regression, the ACF and PACF indicate an AR(1) for the residuals, which fits well. u = ts.intersect(diff(sales), lag(diff(lead),-3)) ds = u[,1] dl3 = u[,2] (fit1 = lm(ds~dl3)) # beta1hat is highly significant acf2(resid(fit1)) # => an AR(1) for the residuals (fit2 = sarima(ds, 1,0,0, xreg=dl3)) # reg with ar1 errors plot(resid(fit2$fit)) acf2(resid(fit2$fit))

3.36 (a) The cost is extremely large in the beginning of the series and rapidly decreases over time. plot(cpg)

(b) If ct ≈ αeβt , then log ct should be approximately linear. The model fits well. summary(fit <- lm(log(cpg)~time(cpg))) plot(log(cpg)) abline(fit)

(c) The residuals are autocorrelated and behave like an AR(1). plot(resid(fit)) acf2(resid(fit))

28

3 Chapter 3 Solutions

(d) Fit the regression with AR(1) errors, which fits well. sarima(log(cpg),1,0,0, xreg=time(cpg))

3.37 The regression with AR(2) errors is ok: # from problem 2.2 temp = tempr-mean(tempr) ded = ts.intersect(cmort, trend=time(cmort), temp, temp2=temp^2, part, partL4=lag(part,-4)) summary(fit <- lm(cmort~trend + temp + temp2 + part + partL4, data=ded)) # residuals: acf2(resid(fit)) # looks like ar2 sarima(ded[,1], 2,0,0, xreg=ded[,-1])

3.38 (a) The model is ARIMA(0, 0, 2) × (0, 0, 0)s (s can be anything) or ARIMA(0, 0, 0) × (0, 0, 1)2 . √ (b) The MA polynomial is θ(z) = 1 + Θz 2 with roots z = ±i/ Θ outside P∞ the unit circle (because |Θ| < 1).

To find the invertible representation, note that 1/[1−(−Θz 2 )] = j=0 (−Θz 2 )j from which we conclude that π2j = (−Θ)j and π2j+1 = 0 for j = 0, 1, 2, . . . . Consequently wt =

∞ X

(−Θ)k xt−2k .

k=0

(c) Write xn+m = −

P∞

k k=1 (−Θ) xn+m−2k + wn from which we deduce that

x̃n+m = −

∞ X

(−Θ)k x̃n+m−2k

k=1

where x̃t = xt for t ≤ n. For the prediction error, note that ψ0 = 1, ψ2 = Θ and ψj = 0 otherwise.

2 n 2 n (1 + Θ2 ). = σw for m = 1, 2; when m > 2 we have Pn+m = σw Thus, Pn+m 3.39 Use the code from Example 3.46 with ma=.5 instead of ma=-.5.

3.40 The first obvious thing to do is to difference the data as in the examples in Chapter 2. Looking at acf2(diff(chicken)), the differenced data look like an AR(2) with an annual cycle. Using sarima(chicken, 2, 1, 0), there appears to be some small but significant correlation at the annual lag, so try a seasonal model, sarima(chicken, 2, 1, 0, 1, 0, 0, 12, no.constant=TRUE) or sarima(chicken, 2, 1, 0, 0, 0, 1, 12, no.constant=TRUE). Both fit well, but the second model is preferred by the ICs. 3.41 After plotting the unemployment data, say xt , it is clear that one should fit an ARMA model to yt = ∇12 ∇xt . The ACF and PACF of yt indicate a clear SMA(1) pattern (the seasonal lags in the ACF cut off after lag 12, whereas the seasonal lags in the PACF tail off at lags 12, 24, 36, and so on). Next, fit an SARIMA(0, 1, 0) × (0, 1, 1)12 to xt and look at the ACF and PACF of the residuals. The within season part of the ACF tails off, and the PACF is either cutting off at lag 2 or is tailing off. These facts suggest an AR(2) or and ARMA(1,1) for the within season part of the model. Hence, fit an (i) SARIMA(2, 1, 0) × (0, 1, 1)12 or an (ii) SARIMA(1, 1, 1) × (0, 1, 1)12 to xt . Both models have the same number of parameters, so it should be clear that model (i) is better because the MSE is smaller for model (i) and the residuals appear to white (while there may still be some correlation left in the residuals for model (ii)). Below is the R code for fitting model (i), along with diagnostics and forecasting. acf2(diff(diff(unemp),12), 50) sarima(unemp, 2, 1, 0, 0, 1, 1, 12) sarima.for(unemp, 12, 2, 1, 0, 0, 1, 1, 12)

3.42 This data set is an update to unemp, but is in terms of percentage unemployed. The techniques are the same as the previous problem and the model is nearly the same; one addition MA parameter is needed.

3 Chapter 3 Solutions

29

acf2(diff(diff(unemp),12), 60) sarima(UnempRate, 2, 1, 1, 0, 1, 1, 12) sarima.for(UnempRate, 12, 2, 1, 1, 0, 1, 1, 12)

3.43 The monthly (s = 12) U.S. Live Birth Series can be found in birth.dat After plotting the data, say xt , it is clear that one should fit an ARMA model to yt = ∇12 ∇xt . The ACF and PACF of yt indicate a seasonal MA of order one, that is, fit an ARIMA(0, 0, 0) × (0, 0, 1)12 to yt . Looking at the ACF and PACF of the residuals of that fit suggests fitting a nonseasonal ARMA(1,1) component (both the ACF and PACF appear to be tailing off). After that, the residuals appear to be white. Finally, we settle on fitting an ARIMA(1, 1, 1) × (0, 1, 1)12 model to the original data, xt . The code for this problem is nearly the same as the previous problem. plot(birth) acf2(diff(diff(birth),12), 50) sarima(birth, 1, 1, 1, 0, 1, 1, 12) sarima.for(birth, 12, 1, 1, 0, 0, 1, 1, 12)

3.44 Because of the increasing variability, the data, jjt , should be logged prior to any further analysis. A plot of the logged data, say yt = ln jjt , shows trend, and one should notice the differences in the behavior of the series at the beginning, middle, and end of the data (as if there are 3 different regimes). Due to these inconsistencies (nonstationarities), it is difficult to discover an ARMA model and one should expect students to come up with various models. In fact, assigning this problem may decrease your student evaluations substantially.

Next, apply a first difference and seasonal difference to the logged data: xt = ∇4 ∇yt . The PACF of xt reveals a large correlation at the seasonal lag 4, so an SAR(1) seems appropriate. The ACF and PACF of the residuals reveals an ARMA(1,1) correlation structure for the within the seasons. This seems to be a reasonable fit. Hence, a reasonable model is an SARIMA(1, 1, 0) × (1, 1, 0)4 on the logged data. Below is R code for this problem. plot(diff(diff(log(jj)),4)) acf2(diff(diff(log(jj)),4)) sarima(log(jj),1,1,0,1,1,0,4) sarima.for(log(jj),4,1,1,0,1,1,0,4)

Pp 3.45 Clearly j=1 φj xn+1−j ∈ sp{xk ; k ≤ n}, so it suffices to show that x̂n+1 satisfies the prediction equations E[(xn+1 − x̂n+1 )xk ] = 0 for k ≤ n. But, by the model assumption, xn+1 − x̂n+1 = wn+1 and E(wn+1 xk ) = 0 for all k ≤ n.

3.46 First note that xi − xi−1 and xj − xj−1 for j > i = 1, ..., n are uncorrelated. This is because xi − xi−1 ∈ i j i j−1 sp{xk ; k = 1, ..., i} but xj − xj is orthogonal (uncorrelated) to sp{xk ; k = 1, ..., i} by definition of xjj−1 Thus, by the projection theorem, for t = 1, 2, . . ., xtt+1 =

t X

θtk (xt+1−k − xt−k t+1−k )

(1)

k=1

where the θtk are obtained by the prediction equations. Multiply both sides of (1) by xj+1 − xjj+1 for j = 0, ..., t − 1 and take expectation to obtain h i j E xtt+1 (xj+1 − xjj+1 ) = θt,t−j Pj+1 .

Because of the orthogonality E[(xt+1 − xtt+1 )(xj+1 − xjj+1 )] = 0 when j < t, so equation above can be written as h i j E xt+1 (xj+1 − xjj+1 ) = θt,t−j Pj+1 .

(2) Using (1) with t replaced by j, (2) can be written as " !# j

−1 X j−k j θt,t−j = E xt+1 xj+1 − θjk (xj+1−k − xj+1−k ) Pj+1 . k=1

30

3 Chapter 3 Solutions

Thus

( θt,t−j =

γ(t − j) +

j X

θjk E

h

xt+1 (xj+1−k − xj−k j+1−k )

) i

j Pj+1

−1

.

(3)

k=1 k Using (2) we can write E[xt+1 (xj+1−k − xj−k j+1−k )] = θt,t−k Pk+1 so (3) can be written in the form of t t (2.71). To show (2.70), first note that E(xt+1 xt+1 ) = E[xt+1 E(xt+1 |x1 , ..., xt )] = E[(xtt+1 )2 ]. Then, for t = 1, 2, ..., t−1 X j t 2 Pt+1 = E(xt+1 − xtt+1 )2 = γ(0) − E[(xtt+1 )2 ] = γ(0) − θt,t−j Pj+1 j=0

3.47 (a) From the projection theorem, xnn+1 =

Pn

k=1 αk xk , where α = (α1 , ..., αn )

0

satisfies

 2 −1 0 0 · · · 0  −1 2 −1 0 · · · 0   α1   0     0 −1 2 −1 · · · 0   α2   0     .. .  =  . . ..   .. .. ..   ..   ..   . . . . .    0 0 · · · −1 2 −1  αn −1 0 0 0 · · · −1 2 

Solving recursively we get α2 = 2α1 , α3 = 3α1 , and in general, αk = kα1 for k = 1, ..., n. This fact and 1 the last equation gives α1 = − n+1 and the result follows. (n+2) 2 2 (b) MSE= γ(0) − a1 γ(−n) − a2 γ(−n + 1) − · · · − an γ(−1) = σw [2 − n/(n + 1)] = (n+1) σw .

3.48 Let x = (x1 , ..., xn )0 , then x ∼ N(0, Γn ) with likelihood L(x) = |Γn |−1/2 exp{− 21 x0 Γn−1 x} (ignoring a constant). Note that xt−1 = Psp{x1 ,...,xt−1 } xt = E(xt |x1 , . . . , xt−1 ) because of the normality assumption. t Hence, the innovations, t = xt − xt−1 , are independent Normal random variables with variance Ptt−1 for t t−1 t = 1, ..., n. Because xt is a linear combination of x1 , . . . , xt−1 , the transformation of x 7→ , where = ( 1 , . . . , n )0 , is lower triangular with ones along the diagonal; i.e. x = C , where C is lower triangular.

In fact, Problem 3.38 shows that (with θij defined there)   1 0 0 ··· 0  θ11 1 0 ··· 0    θ22 θ 1 ··· 0 21 C= .  .. .. ..  . .  . . . . θn−1,n−1 θn−1,n−2 · · · 1 Thus, L(x) = L(C ). Noting that C ∼ N(0, CDC 0 ), where D = diag{P10 , P21 , . . . , Pnn−1 } we have 1 L(x) = L(C ) = |CDC 0 |−1/2 exp{− 0 C 0 (CDC 0 )−1 C }.

2 This establishes the result noting that |CDC 0P | = |C 2 | |D| and |C 2 | = 1, |D| = P10 P21 · · · Pnn−1 , and in the n 0 0 0 −1 0 −1 exponential C (CDC ) C = D = t=1 (xt − xt−1 )2 /Ptt−1 . t 3.49 These results are proven in Brockwell and Davis (1991, Proposition 2.3.2).

3.50 The proof of Property 3.2 is virtually identical to the proof of Property 3.1 given in Appendix B.

Chapter 4 Solutions

NOTE: With slight changes, you can replace mvspec() with spectrum() or spec.pgram() in this chapter.

4.1 This part is included mainly to have the information listed somewhere in the text. The hint has enough detail to make it through all the calculations. Also, the proofs can be found in numerous sources, e.g., Fuller (1995), Theorem 3.1.1.

4.2 (a)–(b) The code is basically the same as the example and is given below. The difference is the frequencies in the data (which are .06, .1, .4) are no longer fundamental frequencies (which are of the form k/128).

Consequently, the periodogram will have non-zero entries near .06, .1, .4 (unlike the example where all other frequencies are zero). t = 1:128 x1 = 2*cos(2*pi*t*6/100) + 3*sin(2*pi*t*6/100) x2 = 4*cos(2*pi*t*10/100) + 5*sin(2*pi*t*10/100) x3 = 6*cos(2*pi*t*40/100) + 7*sin(2*pi*t*40/100) x = x1 + x2 + x3 par(mfrow=c(2,2)) plot.ts(x1, ylim=c(-16,16), main="freq=6/100, amp^2=13") plot.ts(x2, ylim=c(-16,16), main="freq=10/100, amp^2=41") plot.ts(x3, ylim=c(-16,16), main="freq=40/100, amp^2=85") plot.ts(x, ylim=c(-16,16), main="sum") P = abs(2*fft(x)/128)^2 f = 0:64/128 plot(f, P[1:65], type="o", xlab="frequency", ylab="periodogram")

(c) Use the same code as in the example, but with x = x1 + x2 + x3 + rnorm(100,0,5). Now the periodogram will have large peaks at .06, .1, .4, but will also be positive at most other fundamental frequencies.

4.3 (a) Rewrite the transformation as x = tan−1 Note that

z2 z1

y = z12 + z22 .

∂ 1 ∂u tan−1 u = . ∂x 1 + u2 ∂x

Write the joint density of x and y as g(x, y) = f (z1 , z2 )J, 1 ,z2 ) where J denotes the Jacobian, i.e., the determinant of the 2 × 2 matrix { ∂(z ∂(x,y) }. It is easier to compute  ∂x   −z2  z1 ∂x 2 +z 2 2 +z 2 ∂z ∂z z z 1 2 1 1 2  =  1 2  = 2, =  J ∂y ∂y 2z 2z 1 2 ∂z1 ∂z2

32

4 Chapter 4 Solutions

implying that J = 12 . For the joint density of x and y, we obtain

1 2 1 1 y 2 −1 1 exp − (z1 + z2 ) = × exp − g(x, y) = (2π) 2 2 2π 2 2 for −π < x < π and 0 < y < ∞. Integrating over x and y separately shows that the density factors into a product of marginal densities as stated in the problem. (b) Going the other way, we use 1 1 1 1 1 2 exp{−y/2} = exp − (z12 + z22 ) , h(z1 , z2 ) = g(x, y) = J 2π 2 2π 2 √ 2 2 2 2 2 since z1 + z2 = ( y) (cos (x) + sin (x)) = y.

4.4 Most of the work is done in the footnote. Write the terms in the sum (4.4) as xt,k and note that xt,k and xt,` are independent for k 6= ` because the U s are independent (the U s being uncorrelated is sufficient here; this could be pointed out to students - we kept them as independent to go along with the normal and chi-square/uniform theme discussed in (4.2), which requires independence). γk (h) = E(xt+h,k xt,k )

= E U1k sin[2πωk (t + h)] + U2k cos[2πωk (t + h)] × U1k sin[2πωk t] + U2k cos[2πωk t]

2 = σk sin[2πωk (t + h)] sin[2πωk t] + cos[2πωk (t + h)] cos[2πωk t] = σk2 cos[2πωk (t + h) − 2πωk t] = σk2 cos[2πωk h] Pq and γ(h) = k=1 γk (h) give (4.5).

4.5 (a) Ewt = Ext = 0 by linearity, γw (0) = 1 and zero otherwise; γx (0) = (1 + θ2 ), γx (±1) = −θ, and is zero otherwise. The series are stationary because they are zero mean and the autocovariance does not depend on time but only on the shift.

2 |θ(e−2πiω )|2 , thus (b) By Property 4.3, fx (ω) = σw 2 2 2 fx (ω) = σw |1 − θe−2πiω |2 = σw [1 − θ(e−2πiω + e2πiω ) + θ2 ] = σw [1 + θ2 − 2θ cos(2πω)].

4.6 (a) This is similar to 4.4(b). By Property 4.3, 2 2 fx (ω) = σw |φ(e−2πiω )|−2 = σw |1 − φe−2πiω |−2 =

2 σw .

2 1 + φ − 2φ cos(2πω)

(b) From (4.12), write X

0 ∞ ∞ ∞ 2 −h −2πiωh 2 h −2πiωh 2 X X X σw φ e σw φ e σw 2πiω h −2πiω h fx (ω) = + = (φe ) + (φe ) 1 − φ2 1 − φ2 1 − φ2 h=−∞ h=1 h=0 h=1

2 2 2 σw 1 φe−2πiω σw 1 − φ2 σw = + = = , 1 − φ2 1 − φe2πiω 1 − φe−2πiω 1 − φ2 |1 − φe2πiω |2 1 + φ2 − 2φ cos(2φω) and the spectrum is as claimed, by the uniqueness of the Fourier transform.

4.7 First, note that the autocovariance function is γx (h) = (1 + A2 )γs (h) + Aγs (h − D) + Aγs (h + D) + γn (h) Using the spectral representation directly, Z 1/2

γx (h) = 1 + A2 + Ae2πiωD + Ae−2πiωD fs (ω) + fn (ω) e2πiωh dω −1/2

Substituting the exponential representation for cos(2πωD) and using the uniqueness gives the required result.

4 Chapter 4 Solutions

33

4.8 The product series will have mean E(xt yt ) = E(xt )E(yt ) = 0 and autocovariance γz (h) = Ext+h yt+h xt yt = E(xt+h xt )E(yt+h yt ) = γx (h)γy (h).

Now, by (4.12), (4.13) and (4.14) fz (ω) =

∞ X

γx (h)γy (h) exp{−2πiωh} =

γx (h)e−2πiωh e−2πiωh fy (λ) dλ

−1/2 h=−∞

h=−∞

=

Z 1/2 X ∞

Z 1/2 X ∞

γx (h)e

−2πi(ω−λ)h

Z 1/2 fx (ω − λ)fy (λ) dλ.

fy (λ) dλ =

−1/2 h=−∞

−1/2

4.9 Below is R code that will plot the periodogram on the actual scale and then on a log scale (this produces a generic confidence interval – see Example 4.9 on how to get precise limits). The two major peaks are marked; they are 3 cycles/240 years or the 80 year cycle, and 22 cycles/240 years or about an 11 year cycle. mvspec(sunspotz, taper=0, log="no") # for CI, remove log="no" abline(v=3/240, lty="dashed") # 80 year cycle abline(v=22/240, lty="dashed") # 11 year cycle

4.10 This is like the previous problem; the main component is 1 cycle/16 rows, although there’s not enough data to get significance. par(mfrow=c(2,1)) # for CIs, remove log="no" below mvspec(saltemp, taper=0, log="no") abline(v=1/16, lty="dashed") mvspec(salt, taper=0, log="no") abline(v=1/16, lty="dashed")

4.11 (a) Write the model in the notation of Chapter 2 as xt = β 0z t +wt , where z t = (cos(2πωk t), sin(2πωk t))0 and β = (β1 , β2 )0 . Then   Pn Pn 2

n X t=1 cos (2πωk t) t=1 cos(2πωk t) sin(2πωk t) n/2 0 0   z tz t = P = Pn 0 n/2 n 2 t=1 t=1 cos(2πωk t) sin(2πωk t) t=1 sin (2πωk t) from the orthogonality properties of the sines and cosines. For example, n X

n

cos2 (2πωk t) =

t=1

1 X 2πiωk t e + e−2πiωk t e2πiωk t + e−2πiωk t 4 t=1 n

= because, for example,

n X

e

n 1 X 4πiωk t e + 1 + 1 + e−4πiωk t = , 4 t=1 2 4πiωk t

t=1

Substituting,

β̂1 β̂2

e4πik/n 1 − e4πik = =0 1 − e4πik/n

 Pn

t=1 xt cos(2πωk t)

=

2 n Pn

  = 2n−1/2

t=1 xt sin(2πωk t)

dc (ωk ) ds (ωk )

.

(b) Now, SSE = x0x − 2n−1/2 ( dc (ωk ),

 Pn



ds (ωk ) )  P



t=1 xt cos(2πωk t) n t=1 xt sin(2πωk t)

= x0x − 2 d2c (ωk ) + d2s (ωk ) = x0x − 2Ix (ωk ).

34

4 Chapter 4 Solutions

(c) The reduced model is given by xt = wt , so that RSS1 = For the F -test we have q = 2, q1 = 0, so that F2,n−2 =

Pn

2 0 t=1 xt = x x and RSS is given in part (b).

2Ix (ωk ) n−2 x0x − 2Ix (ωk ) 2

is monotone in Ix .

4.12 By applying the definition to xt−s , we obtain n X

as xt−s = n−1/2

s=1

n−1 X

dx (ωk )

=

dx (ωk )n

as e2πiωk (t−s)

s=1

k=0 n−1 X

n X

−1/2

k=0

n X

as e

−2πiωk s 2πiωk t

e

=

s=1

n−1 X

dA (ωk )dx (ωk )e2πiωk t .

k=0

4.13 Don’t forget to difference the data first. mvspec(diff(chicken), spans=c(3,3), log="no", taper=.5) abline(v=c(1/5,.65,1), lty=2)

# looks ok

Aside from the annual cycle (1) there is a 5 year cycle and another at 1/.65, which is about a 1.5 year cycle.

We have no particular explanation of these cycles; further details may be obtained by contacting the United Poultry Growers association.

4.14 Continuing from Problem 4.8 mvspec(sunspotz, spans=c(7,7), log="no", taper=.5) # mvspec(sunspotz, spans=c(7,7), taper=.5) # for log scale with CIs abline(v=3/240, lty="dashed") # 80 year cycle abline(v=22/240, lty="dashed") # 11 year cycle

4.15 Continuing from Problem 4.9: par(mfrow=c(2,1)) mvspec(saltemp, spans=c(1,1), log="no", taper=.5) abline(v=1/16, lty="dashed") salt.per = mvspec(salt, spans=c(1,1), log="no", taper=.5) abline(v=1/16, lty="dashed")

4.16 R code and discussion below. Also, see Figure 4.1. sp.per = mvspec(speech, taper=0) # plots the log-periodogram - which is periodic x = log(sp.per$spec) # x is the log-periodogram values par(mfrow=c(2,1)) plot.ts(x) # plot x=log-periodogram as a time series x.sp = mvspec(x, span=5) # cepstral analysis, x is detrended by default abline(v=.1035, lty="dashed") cbind(x.sp$freq, x.sp$spec) # this lists the quefrencies and cepstra [52,] 0.101562500 32.7549412 [53,] 0.103515625 34.8354468 <- peak is here [54,] 0.105468750 30.3669195

From Problem 4.6, detrended x in the code above is xj ∼ cos(2π nj D) where n here is sp.per$n.used = 1024. Hence, the cepstrum will have a peak around D/n. Thus D̂ = 1024 × .1035 = 105.984 points, or D̂ = .0105984 seconds (recall, 10,000 observations per second). This agrees with Example 1.24.

Pn 4.17 For yt = ht xt , the DFT is dy (ωk ) = n−1/2 t=1 ht xt e−2πiωk t . Then, the expectation of the modulus squared DFT is

35

0

5

x

10

15

4 Chapter 4 Solutions

0

100

200

300

400

500

Time

0.5 2.0 10.0 50.0

spectrum

Series: x Smoothed Periodogram

0.0

0.1

0.2

0.3

0.4

0.5

frequency bandwidth = 0.00246

Fig. 4.1. Figure for Problem 4.14

2

E|dy ((ωk )| = n

−1

n n X X

−1

Z 1/2 X n

hs ht γx (s − t)e−2πiωk (s−t)

s=1 t=1

=n

hs e

−2πi(ωk −ω)s

−1/2 s=1

n X

ht e

2πi(ωk −ω)t

Z 1/2 fx (ω) dω =

|Hn (ωk − ω)|2 fx (ω) dω.

−1/2

t=1

It follows that

Z 1/2 X X 1 |Hn (ωk + `/n − ω)|2 fx (ω) dω E L−1 |Y (ωk + `/n)|2 = L −1/2 ` ` Z 1/2 = Wn (ωk − ω)fx (ω) dω. −1/2

4.18 (a) Since the means are both zero and the ACF’s and CCF’s  2 γx (h) = −1  0

h=0 h = ±1 |h| ≥ 2

  1/2 γy (h) = 1/4  0

h=0 h = ±1 |h| ≥ 2

 0   −1/2 γxy (h) =   1/2 0

h=0 h=1 h = −1 |h| ≥ 2

do not depend on the time index, the series are jointly stationary. (b) fx (ω) = |1 − e−2πiω |2 = 2(1 − cos(2πω))

and fy (ω) =

1 1 |1 + e−2πiω |2 = (1 + cos(2πω)) 4 2

As ω goes from 0 → 12 , fx (ω) increases, whereas fy (ω) decreases. This means xt has more high frequency behavior and yt has more low frequency behavior. (c)

P

2La 2Lf¯y (.10) 2Lb ≤ ≤ fy (.10) fy (.10) fy (.10)

We can make the probability equal to .90 by setting

=P

2La 2Lb ≤ χ22L ≤ fy (.10) fy (.10)

36

4 Chapter 4 Solutions

2La = χ22L (.95) fy (.10)

and

2Lb = χ22L (.05) fy (.10)

Setting L = 3, χ26 (.95) = 1.635, χ26 (.05) = 12.592, fy (.10) = .9045 and solving for a and b yields a = .25, b = 1.90.

4.19 R code for fitting an AR spectrum using AIC is spec.ar(sunspotz). The analysis results in fitting an AR(16) spectrum, which is similar to the nonparametric spectral estimate.

4.21 R code for fitting an AR spectrum using AIC is spec.ar(rec). The analysis results in fitting an AR(13) spectrum, which is similar to the nonparametric spectral estimate.

4.22 We have 2Lf¯x (1/8)/fx (1/8) ∼ χ22L where fx (1/8) = [1 + .52 − 2(.5) cos(2π/8)]−1 = 1.842 from Problem 4.24(a). For L = 3, we have 2(3)2.25/1.842 = 7.26 and does not exceed χ26 (.05) = 12.59. For L = 11, we have 2(11)2.25/1.842 = 26.87 and does not exceed χ222 (.05) = 33.92. Neither sample has evidence for rejecting the hypothesis that the spectrum is as claimed at the α = .05 level.

4.23 The conditions imply that under H0 : d(ωk + `/n) ∼ CN {0, fn (ω)} and under H1 : d(ωk + `/n) ∼ CN {0, fs (ω) + fn (ω)}. For simplicity in notation, denote d` = d(ωk + `/n) and fs = fs (ω), fn = fn (ω). (a) The ratio of likelihoods, under the two hypotheses would be Q π −L (fs + fn )−L ` exp{−|d` |2 /(fs + fn )} p1 Q = p0 π −L (fn )−` ` exp{−|d` |2 /fn } and the log likelihood involving the data d` is proportional to

X p1 1 1 2 , T = ln ∝ |d` | + p0 fn (fs + fn ) `

(b) Write T =

X fs |d` |2 fn (fs + fn ) `

and note that

2

2 ` |d` |

P

fn under H0 and

∼ χ22L

P 2 ` |d` |2 ∼ χ22L (fs + fn )

under H1 . Hence T ∼

1 fs χ2 2 (fs + fn ) 2L

under H0 and T ∼

1 fs 2 χ 2 fn 2L

under H1 . (c) Here, we note that

fs + fn (SN R + 1) = P χ22L > 2K , PF = P {T > K|H0 } = P χ22L > 2K fs SN R and

4 Chapter 4 Solutions

Pd = P {T > K|H1 } = P

χ22L > 2K

fn fs

=P

χ22L >

37

2K , SN R

where SN R denotes the signal-to-noise ratio. Note that, as SN R → ∞, PF → P {T > 2K} and Pd → 1, and the signal detection probability approaches unity for a fixed false alarm rate, as guaranteed by the Neyman-Pearson lemma.

4.24 The analysis is similar to that of Example 4.18. The squared coherency is very large at periods ranging from 16-32 points, or 272-544 feet (1 point = 17 feet). R code below: x = ts(cbind(saltemp,salt)) s = mvspec(x, kernel("daniell",2), taper=0) s$df # = 10 f = qf(.999, 2, s$df-2) # = 18.49365 C = f/(18+f) # = 0.5067635 plot(s, plot.type = "coh", ci.lty = 2) abline(h = C) cbind(s$freq, s$coh) [,1] [,2] [1,] 0.015625 0.598399213 [2,] 0.031250 0.859492914 <- period 1/.03 = 32 [3,] 0.046875 0.891469033 [4,] 0.062500 0.911331648 <- period 1/.06 = 16 [5,] 0.078125 0.749974642

4.25 (a) The CCF is

γxy (h) = cov(xt+h , yt ) = cov(wt+h , φwt + vt ) = φσ 2 δh−D ,

where δh−D = 1 when h = −D and zero otherwise. Thus, X fxy (ω) = γxy (h) exp(−2πiωh) = φσ 2 exp(2πiωD). h

Also, using Proposition 4.3 fx (ω) = σ 2

and fy (ω) = σ 2 (1 + φ2 ).

Finally,

|φσ 2 exp(2πiωD)|2 φ2 = , σ 2 × σ 2 (1 + φ2 ) (1 + φ2 ) which is constant and does not depend on the value of D. (b) In this case, ρ2xy (ω) = .81/1.81 = .45. The R code to simulate the data and estimate the coherence is given below. Note that using L = 1 gives a value of 1 no matter what the processes are, and increasing L (span) gives better estimates. ρ2xy (ω) =

x = rnorm(1024) y = .9*x + rnorm(1024) u = ts(cbind(x,y)) s1 = mvspec(u, taper=0, plot=F)$coh # use this for L = 1 s3 = mvspec(u, span=3, taper=0, plot=F)$coh # L = 3 and so on s41 = mvspec(u, span=41, taper=0, plot=F)$coh s101 = mvspec(u, span=101, taper=0, plot=F)$coh plot(1:512/1024, s1, col=4, type="l", ylim=c(0,1), ylab="SqCoh", xlab="Frequency") lines(1:512/1024, s3, col=3) lines(1:512/1024, s41, col=2, lwd=2) lines(1:512/1024, s101, col=1, lwd=2) abline(h = .81/1.81, lty="dashed") # for a single sq-coherency plot with confidence bands: plot(mvspec(u, span=101, taper=0, plot=F), plot.type="coh")

38

4 Chapter 4 Solutions

4.26 (a) It follows from the solution to the previous problem that φxy (ω) = 2πωD. Hence the slope of the phase divided by 2π is the delay D. (b) Bigger values of L give better estimates. The code is similar to the previous problem. x = ts(rnorm(1025)) y = .9*lag(x,-1)+ rnorm(1025) u = ts.intersect(x,y) s1 = mvspec(u, taper=0, plot=F)$phase s3 = mvspec(u, span=3, taper=0, plot=F)$phase s41 = mvspec(u, span=41, taper=0, plot=F)$phase s101 = mvspec(u, span=101, taper=0, plot=F)$phase plot(1:512/1024, s1, col=4, type="l", ylim=c(-pi,pi), ylab="Phase", xlab="Frequency") lines(1:512/1024, s3, col=3) lines(1:512/1024, s41, col=2, lwd=2) lines(1:512/1024, s101, col=1, lwd=2) abline(a=0,b=2*pi, lty="dashed") # for a single phase plot with confidence bands: plot(mvspec(u, span=101, taper=0, plot=F), plot.type="phase")

4.27 (a) The R code for the cross-spectral analysis of the two series is below: par(mfrow=c(3,1)) mvspec(prodn, span=7, taper=.5) mvspec(unemp, span=7, taper=.5) plot(mvspec(cbind(prodn,unemp), taper=.5, span=7, plot=FALSE), plot.type="coh")

See Figure 4.2 – Figure 4.5. The log spectra with L = 7, show substantial peaks at periods of 2.5 months, 3 months, 4 months and 6 months for the production spectrum and significant peaks at those periods plus a 12 month or one year periodicity in the unemployment spectrum. It is natural that the series tend to repeat yearly and quarterly so that 12 month and 3 month periods would be expected. The 6 month period could be winter-summer fluctuations or possibly a harmonic of the yearly cycle. The 4 month period could be a three cycle per year variation due to something less than quarterly variation or possibly a harmonic of the yearly cycle (recall harmonics of 1/12 are of the form k/12, for k = 2, 3, 4, ...). The squared coherence is large at the seasonal frequencies, as well as a low frequency of about 33 months, or three years, possibly due to a common low frequency business cycle. High coherence at a particular frequency indicates parallel movement between two series at the frequency, but not necessarily causality. (b) The following code will plot the frequency response functions; see Figure 4.3. w = seq(0,.5, length=1000) par(mfrow=c(2,1)) FR12 = abs(1-exp(2i*12*pi*w))^2 plot(w, FR12, type="l", main="12th difference") FR112 = abs(1-exp(2i*pi*w)-exp(2i*12*pi*w)+exp(2i*13*pi*w))^2 plot(w, FR112, type="l", main="1st diff and 12th diff")

The frequency response resulting from the application of the standard difference, followed by a seasonal difference shows that the low frequencies and seasonal frequencies should be attenuated, the low frequencies by the difference and the seasonal frequencies by the seasonal difference. The filtered series are plotted in Figure 4.3; the first difference obviously eliminates the trend but there are still regular seasonal peaks, gradually increasing over the length of the series. The final filtered series tends to eliminate the regular seasonal patterns and retain the intervening frequencies. (c) As mentioned before, the filtered outputs are shown in Figure 4.4. Figure 4.5 shows the spectral analysis of the three series. The first shows the spectrum of the original production series with the low and seasonal frequency components. The second shows the spectrum of the differenced series and we see that the low frequency components have be attenuated while the seasonal component remain. The third shows the spectrum of the seasonally differenced differences, and the power at the seasonal components is essentially notched out. Economists would prefer a flatter response for the seasonally adjusted series and the design of

4 Chapter 4 Solutions

Production Log Spectrum 10

Unemployment Log Spectrum

15

12mo 4mo

2.5mo 3mo

10

4mo 3mo

0

−5

2.5mo

6mo

6mo

5

5

0

0.1

0.2

0.3

0.4

0.5

0

0

0.1

0.2

0.3

0.4

0.5

Coherence 1 33 mo

(coherence)2

2.5mo

4mo

6mo 9mo

0.8

3mo

F2,8(.01)

0.6 0.4 0.2 0

0

0.1

0.2

0.3

0.4

0.5

Fig. 4.2. Analysis of Production and Unemployment.

Frequency Response for 3.7

Frequency Response for Difference

300

20

250 15 Low pass

200

High pass

150

10

100 5 50 0

0

0.1

0.2

0.3

0.4

0.5

Frequency Response for Seasonal Difference 20

0

0

0.1

0.2

0.3

0.4

0.5

Frequency Response for Product 250

Seasonal cut Trend & Seasonal Cut

200

15

150 10 100 5

0

50

0

0.1

0.2 0.3 frequency

0.4

0.5

0

0

0.1

0.2 0.3 frequency

Fig. 4.3. Squared frequency response of various filters.

0.4

0.5

39

40

4 Chapter 4 Solutions

Production Index 200 150 100 50 0

0

50

100

0

50

100

0

50

100

150

200 First Difference

250

300

350

150 200 250 Seasonal Difference of First Difference

300

350

300

350

10 5 0 −5 −10 10 5 0 −5 −10

150

200 Month

250

Fig. 4.4. Production and Filtered Series.

Production log spectrum

First Difference log spectrum 10

5

5

0

0

Power

10

−5

0

0.1

0.2

0.3

0.4

−5

0.5

0

0.1

0.2 0.3 Frequency

0.4

0.5

Seasonally differenced first difference 10

Frequency

First difference attennuates low frequencies 5 Seasonal attenuates seasonal frequencies 0

−5

0

0.1

0.2 0.3 Power

0.4

0.5

Fig. 4.5. Log spectra of production, differenced production and seasonally differenced production.

seasonal adjustment filters that maintain a flatter response is a continuing saga. Shumway (1988, Section 4.4.3) shows an example.

4.28 Write the filter in the general form (4.99), with a2 = a−2 = 1, a1 = a−1 = 4, a0 = 6. Then A(ω) =

2 X

at e−2πiωt = (e−4πiω + e−4πiω ) + 4(e−2πiω + e−2πiω ) + 6

t=−2

= (6 + 2 cos(4πω) + 8 cos(2πω))

By Property 4.7, the spectrum of the output series is fy (ω) = (6 + 2 cos(4πω) + 8 cos(2πω))2 fx (ω). The spectrum of the output series will depend on the spectrum of the input series but we can see how frequencies of the input series are modified by plotting the squared frequency response function |A(ω)|2 . After plotting the frequency response function, it will be obvious that the high frequencies are attenuated and the lower frequencies are not. The filter is referred to as a low pass filter, because it keeps or passes the low frequencies.

4.29

4 Chapter 4 Solutions ∞ X

41

1 2πiωt e A(ω) + e−2πiωt A∗ (ω) = Re A(ω)e2πiωt 2 k=−∞

= Re (AR (ω) − iAI (ω))(cos(2πωt) + i sin(2πωt)) = AR (ω) cos(2πωt) + AI sin(2πωt)

= |A(ω)| cos(φ(ω)) cos(2πωt) − sin(φ(ω)) sin(2πωt) = |A(ω)| cos(2πωt + φ(ω)).

yt =

ak cos[2πω(t − k)] =

The periodic cosine input is shifted by φ(ω) and multiplied by |A(ω)| =

p A2R (ω) + A2I (ω).

4.30 (a) From Property 4.7, fy (ω) = |A(ω)|2 fx (ω), and fz (ω) = |B(ω)|2 fy (ω) = |B(ω)|2 |A(ω)|2 fx (ω). (b) The frequency response functions of the first difference filter |A(ω)|2 , the seasonal difference filter |B(ω)|2 and the product of the two are shown in Figure 3. Note that the first difference tends to keep the high frequencies and attenuate the low frequency and is an example of a high-pass filter. The seasonal difference tends to attenuate frequencies at the multiples 1/12, 2/12, . . . 6/12 which correspond to periods of 12, 6, 4, 3, 2 months respectively. Frequencies in between are retained and the filter is an example of a notch filter, since it attenuates or notches out the seasonal frequencies. (c) The product of the two filters tends to reject low frequency trends (the high-pass part) and seasonal frequencies (the notch part). Retaining the other frequencies is of interest to economists who seek seasonal adjustment filters. Better ones can be designed by specifying a frequency response for the high-pass part that rises more sharply.

4.31 (a) Using Property 4.7, fy (ω) = [1 + a2 − 2a cos(2πω)]−1 fx (ω). (b) Figure 4.6 plots the frequency response functions of both filters and we note that they are both low-pass filters with different rates of decrease. Recursive filters change the phase information and sometimes this can be important. Running the filters twice, once forward and once backward can fix this problem.

Frequency Responses for α=.8

Frequency Responses for alpha=.1

25

1.5

20

1 15

10 0.5

5

0

0

0.1

0.2 0.3 frequency

0.4

0

0.5

0

0.1

0.2 0.3 frequency

0.4

0.5

Fig. 4.6. Squared frequency response of recursive filters.

4.32 Note first that −1 aM k = M

M −1 X

A(ωj )e2πiωj k = M −1

=

∞ X

at e−2πiωj t e2πiωj k

j=0 t=−∞

j=0 ∞ X

M −1 X

at M −1

t=−∞

M −1 X

e−2πiωj (t−k) =

j=0

∞ X

ak+`M = ak +

`=−∞

X

ak+`M .

`6=0

Thus yt − ytM =

X |k|≥M/2

ak xt−k −

X

X

`6=0 |k|<M/2

ak+`M xt−k ≤

X |k|≥M/2

|ak xt−k | +

X

X

`6=0 |k|<M/2

|ak+`M xt−k |

42

4 Chapter 4 Solutions

≤

X

|ak xt−k | +

|k|≥M/2

X

|ak xt−k | ≤ 2

|k|>M/2

X

|ak xt−k |,

|k|≥M/2

where the last steps follow by writing the separate sums for ` = ±1, ±2, . . . and simplifying. Then X X E[(yt − ytM )2 ] ≤ 4 |aj ||ak |E[|xt−j ||xt−k |] |j|≥M/2 |k|≥M/2

≤4

X

X |aj ||ak |E1/2 [|xt−j |2 ]E1/2 [|xt−k |2 ] ≤ 4γx (0)

X

|j|≥M/2 |k|≥M/2

2 |ak |

,

|k|≥M/2

which goes to zero as M increases as long as the absolute summability condition holds.

4.33 Multiply both sides of the equation by xt+h an use the Fourier representation of the spectra and cross spectra to show that fyx (ω) = A(ω)fx (ω). Also, fy (ω) = |A(ω)|2 fx (ω). Then, by the definition of squared coherence, |A(ω)fx (ω)|2 |fyx (ω)|2 = =1 ρ2y·x (ω) = fx (ω)fy (ω) fx (ω)|A(ω)|2 fx (ω) 4.34 (a) Using the following R code: attach(climhyd) inflow = log(Inflow) # log inflow precip = sqrt(Precip) # sqrt precipitation x = replace(climhyd, 5:6, c(precip,inflow)) u = mvspec(x, span=c(7,7), plot=FALSE) plot(u, ci=-1, plot.type="coh") dev.new() LagReg(precip, inflow, threshold=.009) # part (b) detach(climhyd)

the squared coherencies of inflow with the other series will appear in the last row. It is clear that the precipitation-inflow coherence is uniformly larger than the others so that precipitation should be considered as the major contributor over the entire frequency range. (b) Use a two-step procedure as specified in the example of the section, that is, first run LagReg() with no threshold, then run it again using a value near .009. Based on the .009 threshold, the predictions are extremely accurate. The estimated model is The positive lags, at which the coefficients are large in absolute value, and the coefficients themselves, are: lag s beta(s) [1,] 0 0.045197830 [2,] 1 0.022098634 [3,] 2 0.019057057 [4,] 3 0.009269450 [5,] 13 -0.009137882

4.35 For the model yt =

P∞

r=−∞ βr xt−r + vt , first show that

γy (h) =

∞ X

∞ X

βr γx (h − r + s)βs + γv (h).

r=−∞ s=−∞

Substituting the spectral representations for γx (h) and γv (h) and identifying the representations for fx (ω), fv (ω) as well as the Fourier series for βt , leads to the first required result. Then, note that γxy (h) =

∞ X r=−∞

βr γx (h + r) + γv (h)

4 Chapter 4 Solutions

43

and substitute again. That is, since xt and vt are uncorrelated, γxy (h) =

Z 1/2 X ∞

βr e

2πiωr

fx (ω)e

2πiωh

Z 1/2 dω =

−1/2 r=−∞

B(ω)fx (ω)e2πiωh dω.

−1/2

4.36 Rewrite the first equation as yt − φyt−1 = xt − φxt−1 + vt − φvt−1 = wt + vt − φvt−1 , and identify the righthand side as a first-order MA. The required spectrum is the spectrum of an ARMA(1, 1) process, which also has an first order MA on the righthand side. Assuming that the process is Gaussian, we can equate the ACF’s of the two processes and solve for σ 2 and θ. Letting ut = wt + vt − φvt−1 , we obtain 2 γu (0) = σw + (1 + φ2 )σv2

and γu (1) = −φσv2 . Equating these results to the corresponding results for a first-order MA leads to the equations 2 σ 2 (1 + θ2 ) = σw (1 + φ2 )σv2 and − θσ 2 = −φσv2 relating the two models. Solving the second and substituting back into the first equation leads to the required results.

4.37 (a) Set up the orthogonality condition ∞ X

E[(xt −

as yt−s )yt−u ] = 0, u = 0, ±1, . . . ,

s=−∞

P∞ which leads to the normal equations s=−∞ as γy (u − s) = γxy (u). Noting that γxy (u) = γx (u) and taking Fourier transforms leads to the equation A(ω) =

σ2 1 |1 − φe−2πiω |2 σ2 1 fx (ω) = w2 , = w2 −2πiω 2 −2πiω 2 fy (ω) σ |1 − φe | |1 − θe | σ |1 − θe−2πiω |2

2 /σ 2 . Hence its Fourier which we recognize as the spectrum of a first-order AR process, with variance σw transform will be the autocovariance of a first-order AR process, i.e.

as =

2 σw θ|s| σ2 1 − θ2

for the optimal filter. (b) The mean squared error will be E[(x − x̂t )xt ] = E[(xt −

∞ X

as yt−s )xt ] = γx (0) −

s=−∞ ∞ X

∞ X

as γx (s)

s=−∞

4 2 2 2 2 σw σw σv2 σw σw 1 1 σw 1 1 + φθ |s| − 1 − = . = (θφ) = 1 − φ2 σ 2 1 − θ2 1 − φ2 s=−∞ 1 − φ2 ) σ 2 1 − θ2 1 − φθ σ 2 (1 − θ2 ) (c) To get the optimal finite estimator, use the orthogonality principle to get the equation

γy (0) γy (1) a1 γx (1) = γy (1) γy (0) a2 γx (2)

44

4 Chapter 4 Solutions

We obtain γy (0), γy (1) from Example 3.11 in Chapter 3, which derives the autocovariance of an ARM A(1, 1) process and γx (1), γx (2) from Problem 3.5(b). This leads to the equation

.9583 .8147 a1 .8147 = .8147 .9584 a2 .7333 and the solution a = (.7204, .1527)0 . The mean squared error can be computed from MSE = E[(xt − a1 yt−1 − a2 yt−2 )xt ] = γx (0) − a1 γx (1) − a2 γx (2) = .2064.

The optimal mean squared error from the equation in part (b) is .0364.

4.38 (a) Write

γy (h1 , h2 ) = E

X

au1 ,u2 av1 ,v2 xs1 +h1 −u1 ,s2 +h2 −u2 xs1 −v1 ,s2 −v2

u1 ,u2 ,v1 ,v2

=

X

au1 ,u2 γx (h1 − u1 + v1 , h2 − u2 + v2 )av1 ,v2

u1 ,u2 ,v1 ,v2

(b) Substitute in the autocovariance in (a) the spectral representation, so that Z 1/2 X X γy (h1 , h2 ) = au1 ,u2 e−2πi(u1 ω1 +u2 ω2 ) av1 ,v2 e2πi(v1 ω1 +v2 ω2 ) −1/2 u1 ,u2

v1 ,v2

×fx (ω1 ω2 )e2πi(ω1 h1 +ω2 h2 ) dω1 dω2 Z 1/2 =

|A(ω1 , ω2 )|2 fx (ω1 , ω2 )e2πi(ω1 h1 +ω2 h2 ) dω1 dω2 .

−1/2 2 4.39 We can use the definition (C.13) for Sn (ωk , ω` ) in the white noise case [γ(s − t) = σw , for s = t and 0 otherwise] to write n X 2 Sn (ωk , ω` ) = n−1 σw e−2πi(ωk −ω` )t , t=1

which we know to be 1 when ωk − ω` = 0, ±n, ±2n . . . and 0 otherwise. Then, for example, E[dc (ωk )dc (ω` )] =

1 Sn (−ωk , ω` ) + Sn (ωk , ω` ) 4

+Sn (ω` , ωk ) + Sn (ωk , −ω` )

2 σw [0 + 1 + 1 + 0] 4 2 σ = w, 2

=

for ωk = ω` and is zero otherwise. The other terms are treated similarly.

4.43 Write w = 2Re[a ac − ia as )∗ (x xc − ixs )] = 2(a a0cxc + a0sxs )

xc 0 0 = 2 ( ac as ) . xs

4 Chapter 4 Solutions

Hence,

1 C −Q ac as 2 Q C = 2(a a0c + ia a0s )(C − iQ)(a ac − ia as )

cov z = 4 ( a0c

= a∗ Σa a, where the last step follows because Σ = Σ ∗ .

a0s )

45

Chapter 5 Solutions

−5

0

arf

5

10

15

5.1 (a) A time plot is shown below. Note the apparent trend in the data.

0

200

400

600

800

1000

Time

Fig. 5.1. Plot of data for Problem 5.1(a)

(b) Shown below. ACF dampens slowly indicating long memory, but the PACF lag-1 indicates possible random walk. (c) R code and partial output below (must load fracdiff package): plot(arf) # part (a) acf2(arf) # part (b) library(fracdiff) # part (c) summary(fracdiff(arf,nar=1)) Coefficients: Estimate Std. Error z value d 0.264631 0.009653 27.41 ar 0.863068 0.016921 51.01

p.value <2e-16 <2e-16

(d) The time plot in (a) and the ACF and PACF in (b) suggest nonstationary behavior typically found with a random walk; i.e. slow trend in data, ACF tails slowly, PACF lag-1 near one. (e) A plot of the ACF and PACF [acf2(diff(arf))] suggests stationarity is achieved after differencing.

They indicate an AR(1) or an MA(1) with a small coefficient. (f) Using sarima(arf,1,1,0) to fit an ARI(1,1) model fits the data well, with a significant ar1 coefficient of about .17.

5 Chapter 5 Solutions

arf

−0.2

0.2

ACF

0.6

1.0

Series:

47

10

20 LAG

30

40

0

10

20 LAG

30

40

−0.2

0.2

PACF

0.6

1.0

0

Fig. 5.2. ACF and PACF for Problem 5.1(b)

5.2 R code to fit an ARFIMA(1,1,1) and discussion as comments: acf(abs(nyse), 200) u = fracdiff(abs(nyse)) res = diffseries(abs(nyse), u$d) acf2(res) summary(fracdiff(abs(nyse), nar=1, nma=1)) Estimate Std. Error z value p.value d 0.310079 0.016432 18.870 <2e-16 ar -0.068736 0.006991 -9.831 <2e-16 ma 0.166071 0.007021 23.652 <2e-16

# indicates long memory # initial analysis: d = .18 # get frac diffed series # indicates ARMA(1,1) with small coefs # full estimation

5.3 The ADF gives a completely different result than DF and PP, which both reject the unit root hypothesis. It’s unclear why the difference, although it seems adf.test does different things when k = 0 and when k > 0. plot(globtemp) library("tseries") adf.test(globtemp, k=0) Augmented Dickey-Fuller Test data: globtemp Dickey-Fuller = -4.0442, Lag order = 0, p-value = 0.01 alternative hypothesis: stationary Warning message: In adf.test(globtemp, k = 0) : p-value smaller than printed p-value adf.test(globtemp) Augmented Dickey-Fuller Test data: globtemp Dickey-Fuller = -1.6219, Lag order = 5, p-value = 0.7338 alternative hypothesis: stationary pp.test(globtemp) Phillips-Perron Unit Root Test data: globtemp Dickey-Fuller Z(alpha) = -25.062, Truncation lag parameter = 4, p-value = 0.02003 alternative hypothesis: stationary

5.4 All tests reject unit root hyp in favor of explosive. plot(gnp)

48

5 Chapter 5 Solutions

library("tseries") adf.test(gnp, k=0, alt="e") explosive adf.test(gnp, alt="e") pp.test(gnp, alt="e") ...

# DF

= 0.0696, Lag order = 0, p-value = 0.01

| alt hyp:

# ADF = -0.295, Lag order = 6, p-value = 0.01 | ... # PP = 0.742, Truncation lag parameter = 4, p-value = 0.01 |

5.5 Follows almost directly from the AR(2) example just below (5.33): Rearrange the model as in the AR(2) example and then subtract xt−1 from both sides.

5.6 The time plot of the oil returns indicate GARCH behavior. The [P]ACF indicate an ARMA(1,1) for the mean. The [P]ACF of the squared residuals indicate a GARCH(1,1) with long tailed error distribution. Below is code using a t-distribution for the error. poil=diff(log(oil)) plot(poil) sarima(gnpgr, 1, 0, 1) # fit an ARMA(1,1) acf2(innov^2, 24) # get (p)acf of the squared residuals # innov^2 could be ARMA(1,1) again library(fGarch) summary(fit<-garchFit(~arma(1,1)+garch(1,1), cond.dist= "std", data=poil))

5.9 Following the advice of many authors [see for example Thanoon (1990), J. Time Series Anal., 75-87] we fit a long AR, to the data with threshold xt−3 > 36.6. [This information might be related to the students.

Anything between 36 and 37 will work.] We found an AR(15) worked well, the residuals appeared to be white, but were somewhat heteroscedastic. The following models were fit: xt = α(1) +

15 X

(1)

(1)

xt−3 ≤ 36.6

(2)

(2)

xt−3 > 36.6,

φj xt−j + wt ,

j=1

xt = α(2) +

15 X

φj xt−j + wt ,

j=1

You can use tsDyn but it’s a little quirky. library(tsDyn) vignette("tsDyn") # for package details (u = setar(sunspotz, m=15, thDelay=2)) # thDelay=2 is lag 3 (thDelay=0 is lag 1 - I know, it’s strange) plot(u) # for all the plots

5.10 (a) We transform inflow and take the seasonal difference yt = ln it − ln it−12 which is proportional to the percentage yearly increase in flow. Monthly precipitation has some zero values and we use the square root transformation to stabilize this variable. Fitting to two series separately leads to the two ARIMA models xt = ∇12 Pt = (1 − .812(.029)B 12 )wt and

yt = (1 − .764(.033)B 12 )zt ,

2 with σ̂w = 32.503 and σ̂z2 = .225. (b) Cross correlating the two transformed series xt and (1 − .812B 12 )yt leads to the figure shown below and we note that the inflow series seems to depend on exponentially decreasing lagged values of the precipitation.

5.11 (a) The ACF of the residuals from the ARIMA(0, 0, 0)×(0, 1, 1)12 model is well-behaved with all values (except lag 1) well below the significance levels.

5 Chapter 5 Solutions

49

(b) The CCF has been computed in Problem 5.11 and is shown in Figure 5.3. The exponential decrease, beginning at lag zero, suggests α(B) = δ0 (1 + ω1 B + ω12 B 2 + ω13 B 3 + . . .) for fitting the exponential decrease.

1 CCF of transformed inflow with prewhitened precipitation 0.8

0.6

0.4

0.2

0

−0.2

−0.4

−0.6

−0.8

−1 −30

−20

−10

0 lag

10

20

30

Fig. 5.3. CCF for Problem 5.12

(c) So far, we have yt =

δ xt + ηt 1 − ω1 B

which becomes (1 − ω1 B)yt = δxt + (1 − ω1 B)ηt , or yt = ω1 yt−1 + δ0 xt + nt , where nt = (1 − ω1 B)ηt .

We can run the regression model above as ordinary least squares, even though the residuals are correlated, obtaining yt = .526(.026)yt−1 + .050(.002)xt + nt (d) To model the noise, we take n̂t from the above model and note that n̂t = (1 − .526B)ηt can be solved for ηt by inverting the first order moving average transformation. These residuals, say η̂t can be modeled by an ARIM A(1, 0, 0) × (0, 0, 1)12 model of the form (1 − .384B)ηt = (1 − .796B 12 )zt , where σ̂z2 = .0630. Hence, the final model is of the form yt = where and the noise ηt is as modeled above.

.050 xt + η t , 1 − .526B

xt = (1 − .812B 12 )wt

50

5 Chapter 5 Solutions

(e) A possible general procedure would be to forecast xt and ηt separately and then combine the forecasts using the defining equation above (note that xt and ηt are assumed to be independent). To forecast xt , note that xt = wt − .812wt−12 and xt+m = wt+m − .812wt+m−12 and the forecast would be x̃t+m =

n

−.812wt+m−12 0

m ≤ 12 m > 12,

where the residuals wt come from applying the model to the data xt . The forecast variance for x̃t+m will 2 2 be σw for m ≤ 12 and (1 + .8122 )σw for m > 12. To forecast ηt , note that ηt+m = .384ηt+m−1 + zt+m − .795zt+m−12 , so that

η̃t+m =

.384η̃t+m−1 − .795zt+m−12 .384η̃t+m−1

m ≤ 12 m > 12,

where η̃t = ηt . In this case, solving the defining equation in terms ηt yields ηt =

1 − .796B 12 zt = ψ(B)zt .

1 − .384B

The contribution of this term to the forecast variance will then be E[(ηt+m − η̃t+m )2 ] = σz2

m−1 X

ψj2 .

j=0

5.12 Below is the R code. Note that there is still zero-order correlation among the series, so a model that contains zero-order regressors may have done better (this is entirely reasonable because the data are quarterly, and hence the values evolve over months). x = log(econ5[,1:3]) library(vars) VARselect(x, lag.max=10, type="both") # suggests an order 2 or 3 model summary(fit <- VAR(x, p=2, type="both")) (fit.pr = predict(fit, n.ahead = 24, ci = 0.95)) # 4 weeks ahead fanchart(fit.pr) # plot prediction + error

Chapter 6 Solutions

For the most part, we stopped using bold face for vectors in the text. However, in the solutions, the bold face remains because we’re too lazy to fix it.

6.1 (a)

xt xt−1

=

0 1

−.9 0

xt xt−1

and yt = [1 0]

xt−1 xt−2

+

wt 0

+ vt

(b) For yt to be stationary, xt must be stationary. P Note that for t = 0, 1, 2, ..., we may write x2t−1 = Pt−1 t−1 j t j t (−.9) w + (−.9) x and x = 2t−1−2j −1 2t j=0 j=0 (−.9) w2t−2j + (−.9) x0 . From this we see that x2t−1 Repeating the steps of Problem 3.5, we conclude that setting x0 = √ and x2t are independent. √ w0 / 1 − .92 and x−1 = w−1 / 1 − .92 will make xt stationary. In other words, set σ02 = σ12 = 2 /(1 − .92 ). σw (c) and (d): The plots are not shown here.

6.2 (i) s 6= t: Without loss of generality, let s < t, then cov( s , t ) = E[ s E( t | y1 , ..., ys )] = 0. (ii) s = t: Note that ytt−1 = E(xt + vt | y1 , ..., yt−1 ) = xtt−1 . Thus t = yt − ytt−1 = (xt − xt−1 ) + vt , and t t−1 2 . it follows that var( t ) = var[(xt − xt−1 ) + v ] = P + σ t t t v 6.3 This problem is similar to Example 6.3 and Example 6.6 [minus the estimation part].

6.4 (a) Write x = (x1 , ..., xp )0 , y = (y1 , ..., yq )0 , b = (b1 , ..., bp )0 and B = {Bij }i=1,...,p;j=1,...,q . The projection equations are E[(xi − bi − Bi1 y1 − · · · − Biq yq ) 1] = 0, E[(xi − bi − Bi1 y1 − · · · − Biq yq ) yj ] = 0,

i = 1, ..., p;

i = 1, ..., p,

j = 1, ..., q.

(1) (2)

In matrix notation, the p equations in (1) are E(x x − b − Byy ) = 0 and the p q equations in (2) are E(x xy 0 − by 0 − Byyy 0 ) = 0, as was to be shown. Solving (1) leads to the solution for b; that is b = E(x x) − BE(yy ). Inserting this solution into (2) and then solving (2) leads to the solution for B; that is, −1 E(x xy 0 ) − µxµ0y = B[E(yyy 0 ) − µy µ0y ] or B = Σxy Σyy . −1 (b) Let x b = PMx = µx + Σxy Σyy (yy − µy ) as given in (a). The MSE matrix is E(x x − PMx)(x x − PM x ) 0 = 0 E[(x x − PMx)x x ] because x − PMx ⊥ M and PMx ∈ M. Thus, −1 −1 M SE = E[(x x − µx )x x0 ] − Σxy Σyy E[(yy − µy )x x0 ] = Σxx − Σxy Σyy Σyx ,

noting, for example, that E[(x x − µx )(x x − µx )0 ] = E[(x x − µx )x x0 ].

52

6 Chapter 6 Solutions

(c) Consider writing the equation preceding (6.27) in terms of this question. That is,

x = xt µx = xt−1 Σxx = Ptt−1 Σxy = Ptt−1 A0t t , .

Yt−1 ∼ N y = t µy = 0 Σyx = At Ptt−1 Σyy = Σt −1 Then using (a), x b = µx + Σxy Σyy (yy − µy ) corresponds to xtt = xt−1 + Ptt−1 A0t Σt−1 t , which is t −1 precisely (6.21). Moreover, from (b), the MSE is Σxx − Σxy Σyy Σyx which corresponds to Ptt−1 − t−1 0 −1 t−1 t Pt At Σt At Pt , which is precisely Pt defined in (6.22). Thus the normal theory and the projection theorem results coincide.

6.5 (a) Because y k+1 − y kk+1 ∈ Lk+1 , it suffices to show that y k+1 − y kk+1 ⊥ Lk . But E[yy j (yy k+1 − y kk+1 )0 ] = E{yy j E[(yy k+1 − y kk+1 )0 y 1 , ..., y k ]} = 0 j = 1, 2, . . . , k, as required. (b) From the problem statement, we have

−1 Hk+1 = E[x xk (yy k+1 − y kk+1 )0 ] E[(yy k+1 − y kk+1 )(yy k+1 − y kk+1 )]0 .

Now y k+1 − y kk+1 = Ak+1 (x xk+1 − xkk+1 ) + v k+1 . From this it follows immediately that k E[(yy k+1 − y kk+1 )(yy k+1 − y kk+1 )]0 = Ak+1 Pk+1 Ak+1 + R.

To complete the result write (x xk ⊥ v k+1 ) E[x xk (yy k+1 − y kk+1 )0 ] = E[x xk (x xk+1 − xkk+1 )0 A0k+1 ] = E[x xk (x xk − xk−1 )0 Φ0 A0k+1 ] = Pkk Φ0 A0k+1 . k (c) Using (6.23), In addition, using (6.21),

k k A0k+1 [Ak+1 Pk+1 Ak+1 + R]−1 = [Pk+1 ]−1 Kk+1 . k Kk+1 (yy k+1 − y kk+1 ) = xk+1 k+1 − x k+1 .

From these two facts we find that k k Hk+1 (yy k+1 − y kk+1 ) = Pkk Φ0 [Pk+1 ]−1 Kk+1 (yy k+1 − y kk+1 ) = Jk (x xk+1 k+1 − x k+1 ),

and the result follows. (d) and (e) The remainder of the problem follows in a similar manner.

2 6.6 (a) First write ∇yt = yt − yt−1 = wt + vt − vt−1 . Then, E(∇yt ) = 0, var(∇yt ) = σw + 2σv2 , and 2 cov(∇yt , ∇yt−h ) = −σv for h = ±1 and 0 for |h| > 1. We conclude that ∇yt is an MA(1) with ACF −σv2 2 2 h given by ρ(h) = σ2 +2σ 2 δ1 for h = 1, 2, ... . Note, |ρ(1)| ≤ .5 for all values of σw ≥ 0 and σv > 0. w v (b) The estimates should be about σ bw = .111 and σ bv = .425. With these estimates ρb(1) = −.483, and hence θb = −.77. These values are close to the values found in Ch. 3. R code:

y=log(varve) num = length(y) mu0=y[1] Sigma0=var(y[1:10]) # likelihood Linn=function(para){ cQ = para[1] cR= para[2] kf = Kfilter0(num, y, 1, mu0, Sigma0, 1, cQ, cR)

6 Chapter 6 Solutions

53

return(kf$like) } init.par=c(.1,.1) (est = optim(init.par, Linn, NULL, method=’BFGS’, hessian=TRUE, control=list(trace=1,REPORT=1))) SE = sqrt(diag(solve(est$hessian))) # Summary of estimation estimate = c(sig.w=est$par[1], sig.v=est$par[2]) cbind(estimate, SE)

6.7 Using (6.64), the essential part of the complete log-likelihood (i.e. dropping any constants) is ln |σx2 | +

n n X X x2t (yt − xt )2 2 + ln |σ | + . v 2 rσ σv2 t=1 t x t=1

Following (6.71) and (6.72), the updated estimates will be (using the notation of the problem, ˆ for updates and ˜ for current values) σ̂x2 = n−1

n X [x̃n ]2 + P̃ n t

t=1

t

rt

and σ̂v2 = n−1

n X

[(yt − x̃nt )2 + P̃tn ].

t=1

It remains to determine x̃nt and P̃tn . These can be obtained from (B.9)–(B.10). Write Xn = (x1 , ..., xn )0 and Yn = (y1 , ..., yn )0 and drop the ˜ from the notation. Then

Σxx Σxy Xn ∼ N 0, , Σyx Σyy Yn where Σxx = diag {r1 σx2 , . . . , rn σx2 }, Σxy = Σxx [because E(xt yt ) = E(x2t + xt vt ) = E(x2t )] and Σyy = diag {σv2 , . . . , σv2 }. Using (B.9)–(B.10) it follows that xnt = E(xt |Yn ) =

rt σx2 yt rt σx2 + σv2

and Ptn = var(xt |Yn ) = rt σx2 −

rt2 σx4 rt σx2 σv2 = .

2 2 rt σx + σv rt σx2 + σv2

The stated results now follow. t−2 t−2 −1 6.8 (a) Using Property 6.1, Ptt−1 = φ2 [1 − Kt−1 ]Pt−1 + Q = φ2 ([Pt−1 ] + R−1 )−1 + Q. Note R = σv2 2 and Q = σw −1 (b) To ease the notation, we write Ptt−1 as Pt . Part (a) is then Pt = φ2 (Pt−1 + R−1 )−1 + Q. Using this relationship yields −1 −1 −1 Pt−1 − Pt−1 = φ2 [Pt−1 − Pt−2 ].

(1)

From (1) we see that Pt ≥ [≤] Pt−1 as Pt−1 ≥ [≤] Pt−2 , implying that the sequence {Pt } is monotonic.

2 In addition, the sequence is bounded below by 0 and above by σw /(1 − φ2 ) using the fact that 2 Pt = E(xt − xt−1 )2 ≤ var(xt ) ≤ σw /(1 − φ2 ). t

From these facts we conclude that Pt has a limit, say P , as t → ∞, and from part (a), P must satisfy P = φ2 (P −1 + R−1 )−1 + Q.

(2)

We are given R = Q = 1; solving (2) yields P 2 + (1 − φ2 )P − 1 = 0. (c) Using the notation in (b), Kt = Pt /(Pt + 1) and it follows that Kt → K = P/(P + 1). Also, 0 < (1 − K) = 1/(P + 1) < 1 because P > 0.

54

6 Chapter 6 Solutions

n (d) In this problem, yn+1 = xnn+1 , and in steady state

xnn+1 = φKyn + φ(1 − K)xn−1 n = φKyn + φ2 (1 − K)Kyn−1 + φ2 (1 − K)2 xn−2 n−1 .. . ∞ X = φj K(1 − K)j−1 yn+1−j . j=1

6.11 Using Property 6.2 and for m 6= 0, n: n m xnm = xm m + Jm (xm+1 − xm+1 ).

Because xm is not observed,

m−1 • xm = φxm−1 . m = xm

Moreover,

• xnm+1 = xm+1

and m−1 2 • xm m+1 = xm+1 = φ xm−1 . m m Note, Jm = Pm φ/Pm+1 . Now from (6.22) with Am = 0, m m−1 2 • Pm = Pm = σw .

In addition, m−1 m 2 • Pm+1 = Pm+1 = σw (1 + φ2 ).

Thus, • Jm = φ/(1 + φ2 ).

Inserting the •-ed values in (1) gives the desired result.

Also from Property 6.2 and for m 6= 0, n, n m 2 n m Pm = Pm + Jm (Pm+1 − Pm+1 ). n Noting that Pm+1 = 0, and using the •-ed values above yields the desired result.

2 6.12 The estimates are φb = 0.786 (.065) and σ bw = 1.143 (.135). The missing value estimates are:

t 1 2 3

x_t 1.01 **** 1.05

6 7 8

0.76 **** 1.95

13

-2.81

x_t^n 1.00

1.32

t 53 54 55

x_t -3.12 **** -1.73

61 62 63

-2.28 **** -0.56

66

-1.64

x_t^n -2.36

-1.38

t 93 94 95

x_t -1.78 **** -0.51

x_t^n -1.11

(1)

6 Chapter 6 Solutions

14 15

**** -0.42

40 41 42 43 44 45 46 47 48 49 50

-0.60 **** -1.21 **** 1.29 0.12 **** -0.07 -0.28 **** -2.83

-1.57

-0.88

67 68

**** -2.25

79 80 81

1.45 **** 0.67

85 86 87 88 89 90

1.14 **** 2.00 -0.59 **** -0.60

55

-1.89

1.03

0.04

0.02

-1.51

1.53

-0.58

6.13 Code is nearly the same (just put jj = sqrt(jj)) and everything else is the same. Results are a little better 6.14 This problem is similar to Example 6.11. y = aggregate(unemp, nfrequency=4, FUN = mean) num = length(y) A = cbind(1,1,0,0) # Function to Calculate Likelihood Linn=function(para){ Phi = diag(0,4); Phi[1,1] = para[1] Phi[2,]=c(0,-1,-1,-1); Phi[3,]=c(0,1,0,0); Phi[4,]=c(0,0,1,0) cQ1 = para[2]; cQ2 = para[3] # sqrt q11 and sqrt q22 cQ=diag(0,4); cQ[1,1]=cQ1; cQ[2,2]=cQ2 cR = para[4] # sqrt r11 kf = Kfilter0(num,y,A,mu0,Sigma0,Phi,cQ,cR) return(kf$like) } # Initial Parameters mu0 = c(y[1],0,0,0); Sigma0 = diag(100,4) init.par = c(1.01,.1,.1,.1) # Phi[1,1], the 2 Qs and R # Estimation est = optim(init.par, Linn, NULL, method=’BFGS’, hessian=TRUE, control=list(trace=1,REPORT=1)) SE = sqrt(diag(solve(est$hessian))) u = cbind(estimate=est$par,SE) rownames(u)=c(’Phi11’,’sigw1’,’sigw2’,’sigv’); u # Smooth Phi = diag(0,4); Phi[1,1] = est$par[1] Phi[2,]=c(0,-1,-1,-1); Phi[3,]=c(0,1,0,0); Phi[4,]=c(0,0,1,0) cQ1 = est$par[2]; cQ2 = est$par[3] cQ = diag(0,4); cQ[1,1]=cQ1; cQ[2,2]=cQ2 cR = est$par[4] ks = Ksmooth0(num,y,A,mu0,Sigma0,Phi,cQ,cR) # Plot Tsm = ts(ks$xs[1,,], start=1948, freq=4) Ssm = ts(ks$xs[2,,], start=1948, freq=4) p1 = 3*sqrt(ks$Ps[1,1,]); p2 = 3*sqrt(ks$Ps[2,2,]) windows(height=5)

56

6 Chapter 6 Solutions

par(mfrow=c(2,1), mar=c(3,3,2,1), mgp=c(1.6,.6,0)) plot(Tsm, main=’Trend Component’, ylab=’Trend’) xx=c(time(y), rev(time(y))) yy=c(Tsm-p1, rev(Tsm+p1)) polygon(xx, yy, border=NA, col=gray(.4, alpha = .2)) plot(Ssm, main=’Seasonal Component’, ylim=c(-40,50), ylab=’Season’) xx=c(time(y), rev(time(y))) yy=c(Ssm-p2, rev(Ssm+p2)) polygon(xx, yy, border=NA, col=gray(.4, alpha = .2)) # Forecast n.ahead=12; y = ts(append(y, rep(0,n.ahead)), start=1948, freq=4) rmspe = rep(0,n.ahead); x00 = ks$xf[,,num]; P00 = ks$Pf[,,num] Q=t(cQ)%*%cQ; R=t(cR)%*%(cR) for (m in 1:n.ahead){ xp = Phi%*%x00; Pp = Phi%*%P00%*%t(Phi)+Q sig = A%*%Pp%*%t(A)+R; K = Pp%*%t(A)%*%(1/sig) x00 = xp; P00 = Pp-K%*%A%*%Pp y[num+m] = A%*%xp; rmspe[m] = sqrt(sig) } windows(height=4) par(mar=c(3,3,1,1), mgp=c(1.6,.6,0)) plot(y, type=’o’, main=’’, ylab=’’, ylim=c(200,900), xlim=c(1960,1984)) upp = ts(y[(num+1):(num+n.ahead)]+3*rmspe, start=1979, freq=4) low = ts(y[(num+1):(num+n.ahead)]-3*rmspe, start=1979, freq=4) xx=c(time(low), rev(time(upp))) yy=c(low, rev(upp)) polygon(xx, yy, border=NA, col=gray(.4, alpha = .2)) abline(v=1978.75, lty=3)

6.15 (a) That St−5 should be included is evident from the lag-plot. sarima(rec,2,0,0) rec.r = innov lag.plot2(soi,rec.r,8)

(b) The model is Rt = βSt−5 + φ1 Rt−1 + φ2 Rt−2 + vt where Rt is centered (or detrended) Recruitment and St is centered (or detrended) SOI. This could easily be fit via regression techniques. In state space form, the model is

xt+1 φ1 1 xt β φ1 = + St−5+1 + vt xt φ2 0 xt−1 0 φ2

xt Rt = 1 0 + vt . xt−1 First, we can use regression for initial estimates: R = rec - mean(rec) # center series S = soi - mean(soi) ##-- Regression --## dog = ts.intersect(R, R1=lag(R,-1), R2=lag(R,-2), S5=lag(S,-5), dframe=TRU\E) summary(lm(dog$R ~ 0 + dog$R1 + dog$R2 + dog$S5)) Coefficients: Estimate Std. Error t value dog$R1 1.09076 0.03565 30.594 dog$R2 -0.25683 0.03448 -7.449 dog$S5 -17.73344 1.01053 -17.549

6 Chapter 6 Solutions

57

--Residual standard error: 7.331 on 445 degrees of freedom Multiple R-squared: 0.9324, F-statistic: 2045 on 3 and 445 DF, p-value: < 2.2e-16 ##-- MLE --## fish = ts.intersect(R, S5=lag(S,-5)) y = fish[,1] input = fish[,2] num = length(y); A = array(c(1,0), dim = c(1,2,num)) # Function to Calculate Likelihood Linn=function(para){ phi1=para[1]; phi2=para[2]; cR=para[3]; b1=para[4] mu0 = matrix(c(0,0), 2); Sigma0 = diag(100, 2) Phi = matrix(c(phi1, phi2, 1, 0), 2) Theta = matrix(c(phi1, phi2), 2) Ups = matrix(c(b1, 0), 2) cQ = cR; S = cR^2 kf = Kfilter2(num, y, A, mu0, Sigma0, Phi, Ups, Gam=0, Theta, cQ, cR, S, input) return(kf$like) } # Estimation phi1=1; phi2=-.25; cR=7; b1=-17 init.par = c(phi1, phi2, cR, b1) # initial parameters est = optim(init.par, Linn, NULL, method=’L-BFGS-B’, hessian=TRUE, control=list(trace=1,REPORT=1)) SE = sqrt(diag(solve(est$hessian))) # Results u = cbind(estimate=est$par, SE) rownames(u)=c(’phi1’,’phi2’,’sigv’,’S5’); u estimate SE phi1 1.0896554 0.03552116 phi2 -0.2559608 0.03434807 sigv 7.3063958 0.24426808 S5 -17.7709038 1.00763834

6.16 (a) AR(1): xt+1 = φxt + φvt and yt = xt + vt . (b) MA(1): xt+1 = 0xt + θvt and yt = xt + vt . (c) IMA(1,1): xt+1 = xt + (1 + θ)vt and yt = xt + vt .

6.17 The proof of Property 6.5 is similar to the proof of Property 6.1. The first step is in noting that in this setup, cov(x xt+1 , t |y:t−1 ) = ΦPtt−1 A0t + S cov( t , t ) ≡ Σt = At ΦPtt−1 A0t + R E(x xt+1 |y:t−1 ) ≡ xt−1 xt−1 + Υ ut . t t+1 = Φx Then we may write

xt+1 t

y:t−1 ∼ N

t−1 t−1 Pt+1 xt+1 , t−1 0 0 At P t Φ + S 0

ΦPtt−1 A0t + S Σt

and (6.98), (6.100) follow. To show (6.99), write (x xt+1 − xtt+1 ) = (Φ − Kt At )(x xt − xt−1 ) + [I t Then

wt − Kt ] . vt

,

58

6 Chapter 6 Solutions

Q S − Kt ] S0 R

t Pt+1 = E[(x xt+1 − xtt+1 )(x xt+1 − xtt+1 )0 ] = (Φ − Kt At )Ptt−1 (Φ − Kt At )0 × [I

I . −Kt0

and (6.99) follows.

6.19 Repeat Example 6.14 without “windowing’ the data in lines 3 and 4 of the example code. We also set tol = .001 in line 1 and nboot = 200 in line 2. In contrast to Example 6.14, stochastic regression is appropriate in this case. The estimates using Newton-Raphson with estimated standard errors (“asymptotic’ and“bootstrap’) are below. Compare Figure 6.1 here with Figure 6.10 in the text. Here, a 95% bootstrap confidence interval for φ is: (.76, .97).

BOOT-SE 0.0559607 0.2980171 0.1034184 0.0238318 0.1253356 0.10

0.15

0.20

0.25

φ

0.95

● ● ●● ● ● ● ●● ● ●● ● ●● ●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ●●● ● ● ●● ● ●● ●● ● ●● ● ● ●●● ● ●● ●●● ●● ● ●● ●●● ● ● ● ● ●● ● ● ●●●●●●●● ● ● ● ●● ●● ● ●● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ●●● ●●● ● ● ●● ● ● ● ● ●● ●● ● ●● ●● ● ●● ● ●● ● ● ●●●

0.85

ASYM-SE 0.06023241 0.42057041 0.16874251 0.03496117 0.11386013

0.75

MLE 0.9166138 -0.7384140 1.0300287 0.1111845 1.1984293

●

●

●

σ

0.20

0.25

●

0.65

phi alpha beta sig.w sig.v

●

0.10

0.15

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ●● ● ● ● ●●● ●● ●● ● ● ●● ● ● ●● ●● ●● ● ● ●● ●● ●● ● ● ● ● ● ●●● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ●●● ● ● ● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ●●●●● ● ● ● ●● ● ● ● ● ●● ●● ● ●● ●● ●● ●● ● ● ●●●● ● ●

0.65

0.75

0.85

0.95

Fig. 6.1. Bootstrap distributions, B = 200, of the estimator of φ and σw for Problem 6.20.

6.20 See Figure 6.2. The R code for each part is below: # (a) spline plot(gtemp, type=’o’) lines(smooth.spline(globtemp), col=4) # default GCV lines(smooth.spline(globtemp, spar=.7), col=2) # (b) DLM num = length(globtemp) Phi = matrix(c(2,1,-1,0),2) A = matrix(c(1,0),1) mu0 = matrix(c(-.28,-.28),2) Sigma0= diag(c(100,100)) Linn=function(para){ sigw = para[1]; sigv = para[2]

6 Chapter 6 Solutions

59

cQ = diag(c(sigw,0)) kf = Kfilter0(num,globtemp,A,mu0,Sigma0,Phi,cQ,sigv) return(kf$like) } #estimation init.par=c(.1,.1) (est = optim(init.par, Linn, NULL, method=’BFGS’, hessian=TRUE, control=list(trace=1,REPORT=1))) SE = sqrt(diag(solve(est$hessian))) # Summary of estimation estimate = est$par; u = cbind(estimate, SE) rownames(u)=c(’sigw’,’sigv’); u # smooth sigw=est$par[1] cQ = diag(c(sigw,0)) sigv=est$par[2] ks=Ksmooth0(num,globtemp,A,mu0,Sigma0,Phi,cQ,sigv) xsmoo = ts(ks$xs[1,1,], start=1880) psmoo = ts(ks$Ps[1,1,], start=1880) upp=xsmoo+2*sqrt(psmoo) low=xsmoo-2*sqrt(psmoo) plot(globtemp, type=’o’) lines(xsmoo, ylim=c(-.5,1), ylab=’Temp Deviations’) lines(upp, col=4,lty=2) lines(low, col=4, lty=2) lines(smooth.spline(globtemp), col=2) # this is close- within the bnds anyway using GCV

0.6

# (c) plot(globtemp, type=’p’) lines(xsmoo, col=2) lines(upp, col=2, lty=2) lines(low, col=2, lty=2) lines(smooth.spline(globtemp,spar=.7), col=4, lty=2, lwd=2) lines(smooth.spline(globtemp), col=4) ● ● ●● ● ● ● ● ●

0.4

● ● ● ●

−0.4

−0.2

gtemp 0.0 0.2

● ●● ● ●

●

● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●●● ● ● ● ●●●● ● ● ● ●

1880

1900

1920

1940 Time

1960

1980

●

●

●●

●

● ●●

2000

Fig. 6.2. Blue solid is GCV spline; Blue dashed is spar=.7 spline; Red solid is DLM, (identical to spar=.7 spline); Red dashed for DLM error bnds;

6.25 Below is the code to get the estimates. It looks like a switching model is not needed because α1 is not significant. You might want to give the following suggestions: Below is the code but without observational noise (R = 0) and with α0 held fixed at the drift of all the data.

60

6 Chapter 6 Solutions

y = diff(log(gnp)) alpha0 = mean(y) # this is the drift and it’s held fixed num = length(y); nstate = 4; M1 = as.matrix(cbind(1,-1,1,0)) M2 = as.matrix(cbind(1,-1,1,1)) prob = matrix(0,num,1) # to store pi_2(t|t-1) yp = y # yp to store y(t|t-1) xfilter = array(0, dim=c(nstate,1,num)) # -- Function to Calculate Likelihood --# Linn=function(para){ # model phi1=para[1]; phi2=para[2]; sQ=para[3] sR=para[4]; alpha1=para[5] phi=matrix(0,nstate,nstate) phi[1,1]=phi1; phi[1,2]=phi2; phi[2,1]=1; phi[3,3]=1; phi[4,4]=1 Q = matrix(0,nstate,nstate) Q[1,1]=sQ^2; R=sR^2 # end model xf=matrix(0, nstate, 1) # x filter xp=matrix(0, nstate, 1) # x pred Pf=diag(1, nstate) # filter cov Pp=diag(1, nstate) # prec cov pi11=.75; pi12=.25 # initial switch probs pi22=.75; pi21=.25 pif1=.5; pif2=.5 # pi_j(t|t) - filter #-- begin filtering --# like=0 for(i in 1:num){ xp = phi%*%xf; Pp = phi%*%Pf%*%t(phi) + Q sig1=as.numeric(M1%*%Pp%*%t(M1) + R) sig2=as.numeric(M2%*%Pp%*%t(M2) + R) k1=Pp%*%t(M1)/sig1; k2=Pp%*%t(M2)/sig2 e1=y[i]-M1%*%xp-alpha0; e2=y[i]-M2%*%xp-(alpha0+alpha1) pip1 = pif1*pi11 + pif2*pi21; pip2 = pif1*pi12 + pif2*pi22; den1 = (1/sqrt(sig1))*exp(-.5*e1^2/sig1); den2 = (1/sqrt(sig2))*exp(-.5*e2^2/sig2); denom = pip1*den1 + pip2*den2; pif1 = pip1*den1/denom; pif2 = pip2*den2/denom; pif1=as.numeric(pif1); pif2=as.numeric(pif2) e1=as.numeric(e1); e2=as.numeric(e2) xf=xp + pif1*k1*e1 + pif2*k2*e2 eye = diag(1, nstate) Pf = pif1*(eye-k1%*%M1)%*%Pp + pif2*(eye-k2%*%M2)%*%Pp like=like-log(pip1*den1 + pip2*den2) prob[i]<<-pip2

6 Chapter 6 Solutions

61

xfilter[,,i]<<-xf innov.sig<<-c(sig1,sig2) yp[i]<<-ifelse(pip1 > pip2, M1%*%xp, M2%*%xp) } return(like) } # -- Estimation --# phi1=1; phi2=-.3; sQ=.01; sR=0; alpha1=0 # R does not move off of 0 init.par = c(phi1,phi2,sQ,sR,alpha1) (est = optim(init.par,Linn,NULL,method=’BFGS’,hessian=TRUE,control=list(trace=1,REPORT=1))) SE = sqrt(diag(solve(est$hessian))) u = cbind(estimate=est$par, SE) rownames(u)=c(’phi1’,’phi2’,’sig.w’,’sig.r’,’alpha1’); cat( ’alpha0 =’, alpha0, ’\n’); u alpha0 = 0.008337501 estimate SE phi1 1.23296864 0.1043216858 phi2 -0.29143485 0.1014927439 sig.w 0.00860795 0.0003471278 sig.r 0.00000000 0.0023890671 alpha1 -0.42442477 0.5594017787 *Not significant

6.26 This problem was done in Gauss many years ago. We’re not providing R code, but we give details of the analysis. The number of sunspots rises slowly and decreases rapidly indicating a switch in regime. A threshold model was fit in Problem 5.10. Here we used an AR(2) as the basis of the model for the state, but this could be extended to higher order models.

Consider state equations xt1 = α1 xt−1,1 + α2 xt−2,1 + wt1 xt2 = β0 + β1 xt−1,1 + β2 xt−2,1 + wt2 or in vector form       α1 α2 0 0 xt−1,1 0 wt1 1 0 0 0 xt−2,1   0   0   xt−1,1     =  + +   β xt2 xt−1,2 β0 wt2 1 β2 0 0  0 0 10 xt−1,2 xt−2,2 0 0 

xt1





with

 2 σw1 0 0 0  0 0 0 0  Q= 2  0 0 σw2 0 0 0 0 0 

and observation equation (with yt being the centered sunspots data) 

xt1



x  yt = At  t−1,1  + vt xt2 xt−1,2 where At = [1 0 0 0]

or

[1 0 1 0].

Thus there are two regimes and either (a) yt = µy + α0 + α1 xt−1,1 + α2 xt−2,1 + wt1 + vt

(“yt = xt1 + noise0 )

62

6 Chapter 6 Solutions

(b) yt = µy + β0 + (α1 + β1 )xt−1,1 + (α2 + β2 )xt−2,1 + wt2 + vt

(“yt = xt1 + xt2 + noise0 ).

The final estimates are (we set µ by = ȳ) shown in the table below. The estimated states, x bt1 and x bt1 + x bt2 , are shown in Figure 6.3. Note that the dotted line at 40 indicates the model is selecting x bt1 + x bt2 ; in general, the model selects x bt1 + x bt2 during periods when the data is increasing and then peaks.

Estimates for Problem 6.22 Parameter Estimate SE α1 1.700 0.026 α2 −0.793 0.027 β1 0.355 0.078 β2 −0.272 0.090 σw1 7.495 0.252 σw2 0.001 1.118 σv 0.000 0.371 α0 −0.218 0.334 β0 3.675 1.166

Fig. 6.3. Sunspots analysis, Problem 6.22. x bt1 (|), x bt1 + x bt2 (- - -), and the horizontal dotted line (· · ·)at 40 indicates the model is selecting x bt1 + x bt2 .

6.28 Use the same code as in Example 6.19, but first extract the innovations: ARfit = sarima(diff(log(gnp)),1,0,0) y = log(resid(ARfit$fit)^2) num = length(y) . . .

# or can use the MA(2) fit

Using the notation of Section 6.10, a stochastic volatility model fit to the GNP residuals is given below. phi0

estimates SE 0.001374046 0.1849378

6 Chapter 6 Solutions

phi1 0.909416795 0.1043696 sQ 0.333135052 0.2291385 alpha -9.755616207 1.9498158 sigv0 0.889107526 0.2305288 mu1 -2.106589153 0.5140917 sigv1 2.512724902 0.2958002

63

Chapter 7 Solutions

7.1 Note first that (C − iQ)(vv c − ivv s ) = λ(vv c − ivv s ) implies that

C −Q Q C

vc vs

=λ

vc vs

by equating the real and imaginary parts and stacking. Also,

C −Q vs vs =λ Q C −vv c −vv c is the same equation, showing that there are two eigenvectors for each eigenvalue. Hence, 1 2

C −Q Q C

2p 2p Y p 1 1 = |diag{λ1 , λ1 , . . . , λp , λp }| = λ2j .

2 2 j=1

But, |f |2 =

Y p

2 λj

j=1

Then, it follows that |Σ| =

=

p Y

λ2j .

j=1

2p 1 |f |2 , 2

which verifies the result. To verify the second result, let Y = X − M or Y c − iY Y s = (X X c − M c ) − i(X X s − M s) and note that Y ∗ f −1Y is purely real because f = f ∗ is Hermitian. Then, let W = f −1Y so that Y ∗ f −1Y = Y ∗W = (Y Y 0c + iY Y 0s )(W W c − iW W s) = Y 0cW c + Y 0sW s , because the imaginary part is zero. Now, Y = fW W implies that Y c − iY Y s = (C − iQ)(W W c − iW W s ),

7 Chapter 7 Solutions

or

C −Q Q C

Wc Ws

=

Yc Ys

Then, write the quadratic form 1 ( (X X c − M c )0 2

(X X s − M s )0 )

−1

1 C −Q Xc − Mc Xs − Ms 2 Q C

as ( Y 0c

Y 0s )

C −Q Q C

−1

Yc Ys

= ( Y 0c

Y 0s )

Wc Ws

= (Y Y 0cW c + Y 0sW s ) = Y ∗ f −1Y .

7.2 Substitute Lfˆ from (5.6) into (5.5) to rewrite the negative of the log likelihood as

L X − ln L(X X 1 , . . . , X L ; f ) ∝ L ln |f | + tr f −1 (X X ` − M ` )(X X ` − M ` )∗ `=1

= L ln |f | + L tr{fˆf −1 } = −L ln |fˆf −1 | + L tr{fˆf −1 } + ln |fˆ| = −L ln |fˆP P ∗ | + L tr{fˆP P ∗ } + ln |fˆ| = −L ln |P̂ ∗ fˆP | + L tr{P ∗ fˆP } + ln |fˆ| = −L ln |Λ| + L tr{Λ} + ln |fˆ| p p X X = −L ln λi + L λi − Lp + Lp + ln |fˆ| i=1

=

p X

i=1

(λi − ln λi − 1) + Lp + + ln |fˆ|

i=1

≥ Lp + ln |fˆ| with equality when Λ = I or P ∗ fˆP = I so that fˆ = (P ∗ )−1 P −1 = f 7.3 M SE = γy (0) −

∞ X

β 0r γ xy (−r)

r=−∞

Z 1/2 fy (ω) dω −

= −1/2

Z 1/2 =

Z 1/2 X ∞ −1/2

β 0r e−2πiωr

r=−∞

[fy (ω) − B 0 (ω)ff xy (ω)] dω

−1/2

Z 1/2 =

[fy (ω) − f ∗xy (ω)fx−1 (ω)ff xy (ω)] dω

−1/2

Z 1/2 =

fy·x (ω) dω. −1/2

f xy (ω) dω

65

66

7 Chapter 7 Solutions

7.4 Note first that to find the coherence, we must evaluate ρ2ŷy (ω) =

|fŷy (ω)|2 fŷ (ω)fy (ω)

Note that the Fourier representation of the cross-covariance function, γŷy (h) can be written γŷy = E(ŷt+h yt ) X

∞ 0 =E β r xt+h−r yt r=−∞ ∞ X

=

β 0r γ xy (h − r)

r=−∞

=

Z 1/2 X ∞

β 0r e−2πiωr f xy (ω)e2πiωh dω

−1/2 r=−∞

Z 1/2 =

B 0 (ω)ff xy (ω)e2πiωh dω

−1/2

Z 1/2 =

f ∗xy (ω)fx−1 (ω)ff xy (ω)e2πiωh dω

−1/2

We would also need γŷŷ = E(ŷt+h ŷt ) X

∞ ∞ X 0 0 β r xt+h−r xt−sβ s =E = =

r=−∞ ∞ X

∞ X

s=−∞

βs β 0r Γx (h − r + s)β

r=−∞ s=−∞ Z 1/2 X ∞ −1/2

Z 1/2 =

β 0r e−2πiωr

X

∞ 2πiωs fx (ω) β se e2πiωh dω

r=−∞

s=−∞

B (ω)e2πiωh dω B 0 (ω)ff x (ω)B

−1/2

Z 1/2 =

f ∗xy (ω)fx−1 (ω)fx (ω)fx−1 (ω)ff xy (ω)e2πiωh dω

−1/2

Z 1/2 =

f ∗xy (ω)fx−1 (ω)ff xy (ω)e2πiωh dω

−1/2

Substituting into the definition for squared coherence, we obtain ρ2ŷy (ω) =

|ff ∗xy (ω)fx−1 (ω)ff xy (ω)|2 f ∗xy (ω)fx−1 (ω)ff xy (ω)fy (ω)

=

f ∗xy (ω)fx−1 (ω)ff xy (ω) fy (ω)

which is just ρ2y·x (ω) as given in (5.21).

7.5 Writing the complex version of the regression model as Y c − iY Y s = (Xc − iXs )(B B c − iB B s ) + V c − iV Vs shows that

7 Chapter 7 Solutions

67

Y c = XcB c − XsB s + V c and Y s = XsB c + XcB s + V s which is the matrix equation determining the real regression model. Furthermore, two complex matrices F = Fc − iFs , G = Gc − iGs can be multiplied and the real and imaginary components of the product H = Hc − iHs = F G will appear as components of the real product

Fc −Fs Gc −Gs Hc −Hs = Fs Fc Gs Gc Hs Hc This result, along with isomorphism involving the product of a matrix and a vector justifies the last two equations at the end of the problem. Note also that the least squares solution will be −1 X X L L Xk Xk0 Xk Yk B̂ B = (X ∗ X)−1 X ∗ Y = k=1

k=1

It follows that 0

B̂ B =

L X

Yk Xk

0

X L

k=1

Xk Xk

0

−1

∗ = f̂f xy fˆx−1 ,

k=1

which is the sample version of (5.16). To verify the first part of the last equation, note that Y ∗Y =

L X

|Yk |2 = Lfˆy

k=1

and Y ∗ X(X ∗ X)−1 X ∗Y =

L X

Yk X 0k

X L

X L

Yk X 0k

k=1

=

L X

Yk X k

0

β̂ βt =

∞ X

X k Yk

k=1

X L

X k X 0k

−1 X L

k=1

k=1

X L

−1 X L

X kX k

k=1 k=1 ∗ −1 ˆ = Lf̂f xy fx f̂f xy

and the assertions in the last equations are true.

7.6 Note first that, since

−1 X L

k=1

k=1

=

X k X 0k

0

k=1

hsy t−s

s=−∞

E ψ̂t = E

∞ X

a0r β t−r

r=−∞

=E

X

a0r hsy t−r−s

r,s

=

X

a0r hs Zt−r−s−j β j

r,s,j

Z 1/2 = −1/2

A0 (ω)H(ω)Z(ω)B B (ω)e2πiωt dω.

0 X k Yk

X k Yk

68

7 Chapter 7 Solutions

Now,

∞ X

ψt =

a0r β t−r =

Z 1/2

A0 (ω)B B (ω)e2πiωt dω

−1/2

r=−∞

and the above two expressions are equal if and only if H(ω)Z(ω) = I for all ω. To show that the variance is minimized, subject to the unbiased constraint, note that, for any unbiased estimator X ψ̃t = ψt + a0r gsv (t − r − s), r,s

we would have E[(ψ̃t − ψt )2 ] = E[(ψ̂t − ψt )2 ] + E[(ψ̃t − ψ̂t )2 ] + 2E[(ψ̃t − ψ̂t )(ψ̂t − ψt )].

The first two terms on the righthand side are positive and the result is shown if the cross product term is zero. We have XX E[(ψ̃t − ψ̂t )(ψ̂t − ψt )] = a0r (gs − hs )E[vv t−r−sv 0t−j−k ]h0k aj r,s j,k

Z 1/2 =

A (ω) G(ω) − H(ω) H ∗ (ω) dω 0

−1/2

Z 1/2 =

−1 ∗ A (ω) G(ω) − H(ω) Z(ω) Z (ω)Z(ω) dω 0

−1/2

Z 1/2 =

−1 A0 (ω) G(ω)Z(ω) − H(ω)Z(ω) Z ∗ (ω)Z(ω) dω

−1/2

=0 because G(ω)Z(ω) = H(ω)Z(ω) = I. 7.7 In the model (5.39), make the identifications β t = (µt , αt )0 and δ

t

 δt zt =   .. . δt

δt−τ1  δt−τ2  ..  , . δt−τN

where δt = 0, t = 0 and is zero otherwise. Taking the Fourier transforms gives   1 e−2πiωτ1 −2πiωτ 2  1 e , Z(ω) =  ..  ...  .

Then,

Sz (ω) =

j=1 e

2πiωτj

Now, it follows that Sz−1 (ω) =

e−2πiωτN

PN

−2πiωτj j=1 e

N PN

1

1

1 1 N (1 − |φ(ω)|2 )

=N

φ(ω) 1 φ(ω) 1

1 −φ(ω) −φ(ω) 1

7 Chapter 7 Solutions

69

for ω 6= 0. If we apply Z ∗ (ω) directly to the transform vector Y (ω) = (Y1 , (ω), . . . , Yn (ω))0 , we obtain a 2 × 1 vector containing N Y· (ω) and N X

N Bw (ω) =

e2πiωj Yj (ω),

j=1

which leads to the desired result, on multiplying by the 2 × 2 matrix Sz−1 (ω).

7.8 For computations, it is convenient to let u = t − r in the model (5.66), so that ∞ X

yt =

zuβ t−u + v t .

u=−∞

The estimator is given by (5.68), i.e., β̂ βt =

∞ X

hr y t−r .

r=−∞

The orthogonality principle yields the optimal filter as a solution of E[(β β t − β̂ β t )yy 0t−s ] = 0 or

E[(β β ty 0t−s )] = E[β̂ β ty 0t−s )].

since β t and v t are uncorrelated the lefthand side of the above becomes

∞ ∞ X X E βt β 0t−s−u zu0 = γβ (s + u)zu0 u=−∞

u=−∞

Z 1/2 =

fβ (ω)Z ∗ (ω)e2πiωs dω.

−1/2

The righthand side becomes

X

∞ E β̂ β ty 0t−s = E hr y t−r y 0t−s r=−∞

=

∞ X

hr Γy (s − r)

r=−∞

Z 1/2 =

H(ω)fy (ω)e2πiωs dω.

−1/2

Equating the Fourier transforms of the left and right sides gives Hfy = fβ Z ∗ , or H = fβ Z ∗ (Zfβ Z ∗ + fv I)−1 , where the frequency arguments are suppressed to save notation. To get the final form, note that (4.56) can be written as AC ∗ (CAC ∗ + B)−1 = (A−1 + C ∗ B −1 C)−1 C ∗ B −1 for the complex case, implying the form

H=

1 1 I + Z ∗Z fβ fv

= (Sz + θI)−1 Z ∗

−1

Z∗

1 fv

70

7 Chapter 7 Solutions

for the optimal filters. To derive the mean square error, note that MSE = E[(β β t − β̂ β t )β β 0t ] = E[(β β tβ 0t )] − E[(β̂ β tβ 0t )].

The second term is E[(β̂ β tβ 0t )] = E

X

hs zuβ t−s−uβ 0t

u,s

=

X

hs zu γβ (−s − u)

s,u

Z 1/2 =

H(ω)Z(ω)fβ (ω) dω. −1/2

Combining the two terms gives Z 1/2 [fβ (ω) − H(ω)Z(ω)fβ (ω)] dω.

MSE = −1/2

Suppressing frequency again, we may write the argument as

−1 fβ − fβ Z ∗ Zfβ Z ∗ + fv I Zfβ Then, appeal to the complex version of (4.55), i.e.

−1

∗ ∗ A − AC CAC + B CA = A−1 + C ∗ B −1 C)−1 to write the argument as

1 1 I + Z ∗Z fβ fv

−1

which reduces to (5.72).

7.9 Suppressing the frequency argument, we write E[SSR] = E[Y Y ∗ ZSz−1 Z ∗Y ]

= tr ZSx−1 Z ∗ E[Y Y Y ∗]

= tr ZSz−1 Z ∗ (fβ ZZ ∗ + fv I)

= fβ tr ZSz−1 Z ∗ ZZ ∗ + fv tr ZSz−1 Z ∗

= fβ tr ZZ ∗ + fv tr Sz−1 Sz = fβ tr {Sz } + qfv .

When the spectrum is cumulated over L frequencies, the multiplier appears.

7.10 Again, suppressing the frequency subscripts the model Y = ZB B + V takes the vector form       Y11 V11 1 1 . . . .  .  .  .  ..  .   .  .          Y1N   1 1  B1  V1N  +  =  ,  Y21   1 −1  B2  V21   .  .   .  . ..   ..   ..  ..  1 −1 Y2N V2N

7 Chapter 7 Solutions

71

where B1 is the DFT of µt and B2 is the DFT of α1t , α2t = −α1t . The null hypothesis is that B2 = 0.

Now, in (5.52),

N (Y·1 + Y·2 ) szy = N (Y·1 − Y·2 ) and s2y·z =

2 X N X

2

|Yij | − ( N (Y·1 + Y·2 ) N (Y·1 − Y·2 ) )

i=1 j=1

=

N X

|Y1j |2 +

j=1

=

2 X N X

N X

2N 0

0 2N

−1

N (Y·1 + Y·2 ) N (Y·1 − Y·2 )

|Y2j |2 − N |Y1· |2 − N |Y2· |2

j=1

|Yij − Yi· |2 ,

i=1 j=1

which is (5.85) Under the reduced model, s1y = N (Y1· + Y2· ) and S11 = 2N so that s2y·1 =

N X

|Y1j |2 +

Then, substitute Y·· =

s2y·1 =

|Y2j |2 −

j=1

j=1

to obtain

N X

N X

N |Y1· + Y2· |2 .

2

1 (Y1· + Y2· ) 2

|Y1j |2 +

j=1

N X

|Y2j |2 − 2N |Y·· |2 .

j=1

Then, RSS = s2y·1 − s2y·z = −2N |Y·· |2 + N |Y1· |2 + N |Y2· |2 =N

2 X

|Yi· − Y·· |2

i=1

=

N 2 X X

|Yi· − Y·· |2 ,

i=1 j=1

which is (5.84).

7.11 Use the model yijt = µit + vijt , for i = 1, . . . , I, j = 1, . . . , Ni and write the frequency domain version in terms of the vector Y = (Y11 , . . . , Y1,N1 , Y21 , . . . , Y2N2 , . . . , YI1 , . . . , YINI )0 The matrix Z has N1 ones in elements 1 to N1 of column 1, and zeros elsewhere, N2 ones in elements N1 + 1, . . . , N1 + N2 of column 2, etc. It follows that Sz = Z ∗ Z = diag (N1 , N2 , . . . , NI ) and

72

7 Chapter 7 Solutions

Z ∗Y = (N1 Y1· , N2 Y2· , . . . , NI YI· )0 so that

B̂ B = (Y1· , Y2· , . . . , YI· )0

and A∗B̂ B=

I X

Ai Yi· .

i=1

Finally,

Q(A) = (A1 , A2 , . . . , AI ) diag =

I X |Ai |2 i=1

Ni

1 1 1 , ,..., (A1 , A2 , . . . , AI )0 N1 N2 NI

.

The error variance s2y·z comes from (5.85).

7.12 Each of the spectra N1 X ˆ |Y1j − Y1· |2 f1 =

and fˆ2 =

N2 X

|Y2j − Y2· |2

j=1

j=1

can be regarded as an error power component, computed from the model yijt = µit + vijt for a fixed i = 1, 2. Hence, from Table 5.2, the error power components will have a chi-squared distribution, say, 2(Ni − 1)fˆi ∼ χ22(Ni −1) fi for i = 1, 2, and the two samples are assumed to be independent. It follows that the ratio χ2[2(N −1)] /2(N1 − 1) fˆ1 f2 ∼ F[2(N1 −1),2(N2 −1)] . ∼ 2 1 χ[2(N −1)] /2(N2 − 1) fˆ2 f1 2

It follows that

fˆ1 f1 ∼ F[2(N1 −1),2(N2 −1)] . ˆ f 2 f2

2 7.13 In the notation of (5.113), µ1 = s, µ2 = 0, Σ1 = Σ2 = σw I and

1 s0s π1 1 0 s x − + ln 2 2 σw 2 σw π2 Pn n X 1 1 t=1 s2t π1 = 2 st xt − + ln 2 σw t=1 2 σw π2

n π1 1 X 1 S + ln = 2 st xt − σw t=1 2 N π2

dL (x x) =

When π1 = π2 , the last term disappears and we may use (5.115) for the two error probabilities with

s0s S 2 D = 2 = . σw N

7 Chapter 7 Solutions

73

2 2 7.14 In this case, µ1 = µ2 = 0, Σ1 = (σs2 + σw )I, Σ2 = σw I, so that (5.115) becomes

2

2 1 1 π1 n σs + σw 1 − − 2 x0x + ln dq (x x) = − ln 2 2 2 σw 2 σs2 + σw σw π2

n 2 2 2 X π1 1 σs 1 σ +σ = x2t − ln s 2 w + ln 2 2 2 2 σw (σs + σw ) t=1 2 σw π2

In this case, define the signal-to-noise ratio as

S N

=

σs2 2 σw

so that, for the quadratic criterion with π1 = π2 we accept Π1 or Π2 according to whether the statistic T (x x) =

n X σs2 1 x2t 2 (σ 2 + σ 2 ) 2 σw s w t=1

exceeds or fails to exceed 2

2 1 σs + σw S 1 = ln 1 + .

K = ln 2 2 σw 2 N P P 2 2 2 Now, under Π1 , t x2t ∼ (σs2 + σw )χ2n , whereas, under Π2 , t x2t ∼ σw χn , so that

1 S χ2n T (x x) ∼ 2 N under Π1 and T (x x) ∼

−1 S 1 1+ χ2n 2 N

under Π2 .

7.15 Awake-Heat: Figures 7.1 and 7.2 are the figures corresponding to Figures 7.14 and 7.15 of the text (Example 7.14) except that here, Caudate is included (location number 5). Awake-Shock: The corresponding figures are Figures 7.3 and 7.4 below. The listing below is similar to Table 5.8 but for Awake-Heat and Awake-Shock. Note that χ22 (.95, .975, .99) = 5.99, 7.38, 9.21. loc |e| 1 .467 2 .450 3 .460 4 .363 5 .230

AWAKE-HEAT chi^2 loc |e| 842.01 6 .121 150.16 7 .012 623.62 8 .323 104.45 9 .254 30.32

chi^2 8.09 0.07 64.99 46.76

| | | | | |

loc |e| 1 .410 2 .416 3 .387 4 .370 5 .269

AWAKE-SHOCK chi^2 loc |e| 233.39 6 0.139 138.42 7 0.161 389.46 8 0.309 352.68 9 0.398 107.23

chi^2 13.952 18.848 252.75 539.46

7.16 (a) See Figure 7.5. The P components have broad power at the midrange frequencies whereas the S components have broad power at the lower frequencies. (b) See Figure 7.7. There appears to be little or no coherence between the P and S components. (c) - (d) See Figure 7.6. These figures support the overall conclusion of part (a). (e) See Figure 7.8. The canonical variate series appear to be strongly coherent (in contrast to the individual series).

74

7 Chapter 7 Solutions

Fig. 7.1. Problem 7.15 (Awake-Heat) individual periodograms

Fig. 7.2. Problem 7.15 (Awake-Heat) spectral density of first PC series.

7.17 For p = 3 and q = 1, write (5.158) as 

     x1 b1 1  x 2  =  b2  z +  2  . x3 b3 3

This implies

   2 b1 b3 1 .4 .9 b1 + δ12 b1 b2  .4 1 .7  =  b1 b2 b22 + δ22 b2 b3  . .9 .7 1 b1 b3 b2 b3 b23 + δ32 

Now b1 b2 = .4

and b2 b3 = .7

But b1 b3 = .9

⇒

⇒

b1 =

4 b3 .

7

4 2 b = .9.

7 3

This means that b23 = .9 47 = 1.575. But we also have b23 + δ32 = 1 in which case δ32 < 0 which is not a valid variance.

P P im 0 7.19 Note f (ω) = h Γ P (h) cos(2πhv) − i h Γ (h)P sin(2πhv) = f re (ω) − if im (ω).

P that P Now f (ω) = 0 h Γ (h) sin(2πhv) = h Γ (−h) sin(2πhv) = − h Γ (−h) sin(−2πhv) = − h Γ (h) sin(2πhv) =

7 Chapter 7 Solutions

75

Fig. 7.3. Problem 7.15 (Awake-Shock) individual periodograms

Fig. 7.4. Problem 7.15 (Awake-Shock) spectral density of first PC series.

P −f im (ω); that is, the imaginary part is skew symmetric. Also note that f re (ω)0 = h Γ (−h) cos(−2πhv) = f re (ω); that is, the real part is symmetric.

Next, because β 0 f im (ω)β β is a scalar, β 0 f im (ω)β β = (β β 0 f im (ω)β β )0 = β 0 f im (ω)0β = −β β 0 f im (ω)β β . This 0 im result implies β f (ω)β β = 0 for any real-valued vector β .

7.20 See Figure 7.9. Note the significant peak at ω = 1/3 as opposed to the fourth quarter of EBV (see Figure 7.23).

7.21 (a) The estimated spectral density of rt is shown in Figure 7.10 and supports the white noise hypothesis. (b) The spectral envelope with respect to G = {x, |x|, x2 } is shown in Figure 7.11 where substantial power near the zero frequency is noted. The optimal transformation is shown in Figure 7.12 as a solid line and the usual square transformation is shown as a dotted line; the two transformations are similar.

76

7 Chapter 7 Solutions

Fig. 7.5. Problem 7.16 (a). Estimated spectral densities.

Fig. 7.6. Problem 7.16 (c)-(d). Spectral density of first PC series.

7 Chapter 7 Solutions

Fig. 7.7. Problem 7.16 (b). Estimated coherencies.

Fig. 7.8. Problem 7.16 (c) (d). Squared coherency between canonical variate series.

77

78

7 Chapter 7 Solutions

Fig. 7.9. Problem 7.20. This is the equivalent of Figure 7.23 but for the herpesvirus saimiri.

Fig. 7.10. Problem 7.21 (a). Estimated spectral density of NYSE returns.

Fig. 7.11. Problem 7.21 (b). Spectral envelope of NYSE returns with respect to G = {x, |x|, x2 }.

7 Chapter 7 Solutions

Fig. 7.12. Problem 7.21 (b). The optimal transformation (|) and the usual square transformation (- - -).

79

http://www.springer.com/978-3-319-52451-1