Linear Algebra, Signal Processing, and Wavelets - A Unified Approach: MATLAB Version 1st Edition By

INSTRUCTORS SOLUTION MANUAL

Exercises from Linear algebra, signal processing, and wavelets. A unified approach.

MATLAB version Øyvind Ryan

Sep 26, 2018

Contents

1

Sound and Fourier series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2

Digital sound and Discrete Fourier analysis . . . . . . . . . . . . . . . . . . . . . . . . 29

3

Discrete time filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

4

Motivation for wavelets and some simple examples . . . . . . . . . . . . . . . . 95

5

The filter representation of wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

6

Constructing interesting wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

7

The polyphase representation of filter bank transforms . . . . . . . . . . . . 165

8

Digital images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

9

Using tensor products to apply wavelets to images . . . . . . . . . . . . . . . . . 203

5

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

3

Chapter 1

Sound and Fourier series

Exercise 1.6: The Krakatoa explosion Compute the loudness of the Krakatoa explosion on the decibel scale, assuming that the variation in air pressure peaked at 100 000 Pa.

Solution.

Setting pref =0.00002 Pa and p=100 000 Pa in the decibel expression we get

20 log10

p pref

105 100000 = 20 log10 = 20 log10 0.00002 2 × 10−5 10 10 = 20 log10 = 20 (10 − log10 2) ≈ 194db.

2

Exercise 1.7: The sum of two pure tones a) Consider a sum of two pure tones, f (t) = a sin(2πν1 t) + b sin(2πν2 t). For which values of a, b, ν1 , ν2 is f periodic? What is the period of f when it is periodic?

Solution. sin(2πν1 t) has period 1/ν1 , while sin(2πν2 t) has period 1/ν2 . The period is not unique, however. The first one also has period n/ν1 , and the second also n/ν2 , for any n. The sum is periodic if there exist n1 , n2 so that n1 /ν1 = n2 /ν2 , i.e. so that there exists a common period between the two. This common period will also be a period of f . This amounts to that ν1 /ν2 = n1 /n2 , i.e. that ν1 /ν2 is a rational number. b) Find two constant a and b so that the function f (t) = a sin(2π440t) + b sin(2π4400t) resembles the right plot of Figure 1.1 in the book as closely as possible. Generate the samples of this sound, and listen to it.

5

6

1 Sound and Fourier series

Solution.

The important thing to note here is that there are two oscillations present in the right part of Figure 1.1 in the book: One slow oscillation with a higher amplitude, and one faster oscillation, with a lower amplitude. We see that there are 10 periods of the smaller oscillation within one period of the larger oscillation, so that we should be able to reconstruct the figure by using frequencies where one is 10 times the other, such as 440Hz and 4400Hz. Also, we see from the figure that the amplitude of the larger oscillation is close to 1, and close to 0.3 for the smaller oscillation. A good choice therefore seems to be a = 1, b = 0.3. The code can look this: t = 0:(1/f_s):3; x = sin(2*pi*440*t) + 0.3*sin(2*pi*4400*t); x = x/max(abs(x)); playerobj = audioplayer(x, f_s); playblocking(playerobj);

Exercise 1.8: Playing with different sample rates If we provide another sample rate to the play functions, the sound card will assume a different time distance between neighboring samples. Play and listen to the audio sample file again, but with three different sample rates: 2fs , fs , and fs /2, where fs is the sample rate returned by audioread.

Solution.

The following code can be used playerobj = audioplayer(x, f_s); playblocking(playerobj);

playerobj = audioplayer(x, 2*f_s); playblocking(playerobj);

playerobj = audioplayer(x, f_s/2); playblocking(playerobj);

The sample file castanets.wav played at double sampling rate can be found in the file castanetsdouble.wav, and in castanetshalf.wav for half the sampling rate.

Exercise 1.9: Playing sound with added noise Removing noise from recorded sound can be very challenging, but adding noise is simple.

There are many kinds of noise, but one kind is easily obtained by adding random numbers to the samples of a sound. For this we can use the function rand as follows.

1 Sound and Fourier series

7

z = x + c*(2*rand(size(x))-1);

This adds noise to all channels. The function for returning random numbers returns numbers between 0 and 1, and above we have adjusted these so that they are between −1 and 1 instead, as for other sound which can be played by the computer. c is a constant (usually smaller than 1) that dampens the noise.

Write code which adds noise to the audio sample file, and listen to the result for damping constants c=0.4 and c=0.1. Remember to scale the sound values after you have added noise, since they may be outside [−1, 1].

Solution.

The following code can be used. z = x + c*(2*rand(size(x))-1); z = z/max(abs(z)); playerobj = audioplayer(z, f_s); playblocking(playerobj);

With c = 0.4 the resulting sound can be found in the file castanetsnoisehigh.wav, while with c = 0.1 it can be found in the file castanetsnoiselow.wav.

Exercise 1.10: Playing the triangle wave Repeat what you did in Example 1.4 in the book, but now for the triangle wave of Example 1.5 in the book. Start by generating the samples for one period, then plot five periods, before you generate the sound over a period of three seconds and play it. Verify that you generate the same sound as in Example 1.5 in the book.

Solution.

The triangle wave can be plotted as follows: totransl=1-4*abs(t-T/2)/T; figure() plot([t t+T t+2*T t+3*T t+4*T],repmat(totransl,1,5), ’k-’)

The samples for one period are created as follows. oneperiod=[linspace(-1,1,round(samplesperperiod/2)) ... linspace(1,-1,round(samplesperperiod/2))]; x = zeros(1, antsec*f*length(oneperiod));

Then we repeat one period to obtain a sound with the desired length, and play it as follows. x=repmat(oneperiod,1,antsec*f); playerobj=audioplayer(x, f_s); playblocking(playerobj);

8

1 Sound and Fourier series

Exercise 1.11: Playing the notes in an octave In music theory, an octave is a set of pure tones at frequencies f0 , ..., f11 , f12 so that the ratio of neighboring tones are the same, and so that f12 is double the frequency of f0 , i.e. so that f2 f12 f1 = = ··· = = 21/12 . f0 f1 f11 Make a program which plays all the pure tones in an octave, and listen to it with f0 = 440Hz.

Solution.

The following code can be used. f_s = 44100; num_sec = 1; k = 2^(1/12); f = 440; t = linspace(0, num_sec, f_s*num_sec); for s=0:12 playerobj = audioplayer(sin(2*pi*f*t), f_s); playblocking(playerobj) f = f*k; end

Exercise 1.12: The Karplus-Strong algorithm for making guitar-like sounds Given initial values x0 , ..., xp , the difference equation 1 xn+p+1 − (xn+1 + xn ) = 0 2 of order p + 1 is known to create guitar like sounds. Show that all xn lie in [−1, 1] when the initial values do, and write a function karplus_strong(x_init, f_s)

which takes the initial values (x0 , x1 , ..., xp ) as input, and plays the resulting sound for ten seconds with sample rate fs . Experiment with randomly generated initial values between −1 and 1, as well as different sample rates. What can you say about the frequencies in the resulting sound?

Solution.

The following function can be used

1 Sound and Fourier series

9

function karplus_strong(x_init, f_s) p=size(x_init,1)-1; num_sec = 10; num_samples = f_s*num_sec; z=zeros(num_samples,1); z(1:(p+1)) = x_init; for k=(p+2):num_samples z(k) = 0.5*(z(k-p)+z(k-p-1)); end playerobj = audioplayer(z, f_s); playblocking(playerobj) end

This function can be tested as follows p = 100; f_s = 44100; x_init = 2*rand(p+1,1)-1; karplus_strong(x_init, f_s)

Exercise 1.16: Shifting the Fourier basis vectors Show that sin(2πnt/T + a) ∈ VN,T when |n| ≤ N , regardless of the value of a.

Solution.

Write sin(2πnt/T + a) = cos a sin(2πnt/T ) + sin a cos(2πnt/T ).

Exercise 1.17: Listening to the Fourier series of the triangle wave a) Plot the Fourier series of the triangle wave.

Solution.

The following code can be used. t = linspace(0, T, 100); x = zeros(1,length(t)); for k = 1:2:19 x = x - (8/(k*pi)^2)*cos(2*pi*k*t/T); end figure() plot(t, x, ’k-’)

b) Write code so that you can listen to the Fourier series of the triangle wave. How high must you choose N for the Fourier series to be indistinguishable from the triangle wave itself?

10

1 Sound and Fourier series

Solution.

The following code can be used. x = repmat(x, 1, antsec/T); playerobj=audioplayer(x, f_s); playblocking(playerobj);

Exercise 1.18: Riemann-integrable functions which are not square integrable Find a function f which is Riemann-integrable on [0, T ], and so that infinite.

RT 0

f (t)2 dt is

Solution.

The function f (t) = √1t = t−1/2 can be used since it has the properties Z T

Z T f (t)dt = lim

0

x→0+

t−1/2 dt = lim

x→0+

x 1/2

1/2

h

2t1/2

iT x

1/2

= lim (2T − 2x ) = 2T x→0+ Z T Z T T f (t)2 dt = lim t−1 dt = lim [ln t]x 0

x→0+

x

x→0+

= ln T − lim ln x = ∞. x→0+

Exercise 1.19: When are Fourier spaces included in each other?

Given the two Fourier spaces VN1 ,T1 , VN2 ,T2 . Find necessary and sufficient conditions in order for VN1 ,T1 ⊂ VN2 ,T2 .

Solution.

The space VN1 ,T1 is spanned by pure tones with frequencies 1/T1 , . . . , N1 /T1 , while VN2 ,T2 is spanned by pure tones with frequencies 1/T2 , . . . , N2 /T2 . We must have that the first set of frequencies is contained in the second. This is achieved if and only if 1/T1 = k/T2 for some integer k, and also N1 /T1 ≤ N2 /T2 . In other words, T2 /T1 must be an integer, and T2 /T1 ≤ N2 /N1 .

1 Sound and Fourier series

11

Exercise 1.20: Fourier series of antisymmetric functions are sine series Prove the second part of Theorem 10 in the book, i.e. show that if f is antisymmetric about 0 (i.e. f (−t) = −f (t) for all t), then an = 0, i.e. the Fourier series is actually a sine series.

Solution.

If f is symmetric we obtain

an = =

2 T

Z T /2

2 T

Z 0

f (t) cos(2πnt/T )dt −T /2

f (t) cos(2πnt/T )dt + −T /2

2 T

Z T /2 f (t) cos(2πnt/T )dt 0

Z 2 −T /2 f (t) cos(2πnt/T )dt − f (−t) cos(−2πnt/T )dt T 0 −T /2 Z Z 2 0 2 0 = f (t) cos(2πnt/T )dt − f (t) cos(2πnt/T )dt = 0.

T −T /2 T −T /2 2 = T

Z 0

where we again made the substitution u = −t, and used that cos is symmetric.

Exercise 1.21: More connections between symmetric-/antisymmetric functions and sine-/cosine series Show that a) Any cosine series a0 +

PN

n=1 an cos(2πnt/T ) is a symmetric function.

Solution.

This follows since each cos(2πnt/T ) is symmetric.

PN b) Any sine series n=1 bn sin(2πnt/T ) is an antisymmetric function.

Solution.

This follows since each sin(2πnt/T ) is antisymmetric. c) Any periodic function can be written as a sum of a symmetric - and an antisymmetric (−t) (−t) function by writing f (t) = f (t)+f + f (t)−f .

2 2

12

1 Sound and Fourier series

Solution. (−t) (−t) It suffices to show that g(t) = f (t)+f is symmetric, and h(t) = f (t)−f is antisym2 2 metric. We have that

f (−t) + f (t) f (t) + f (−t) = = g(t) 2 2 f (t) − f (−t) f (−t) − f (t) =− = −h(t). h(−t) = 2 2 PN d) If fN (t) = a0 + n=1 (an cos(2πnt/T ) + bn sin(2πnt/T )), then g(−t) =

N X fN (t) + fN (−t) = a0 + an cos(2πnt/T ) 2 n=1 N fN (t) − fN (−t) X = bn sin(2πnt/T ).

2 n=1

Solution.

Using the symmetry of cos and the antisymmetry of sin we obtain PN fN (t) + fN (−t) a0 + n=1 (an cos(2πnt/T ) + bn sin(2πnt/T )) = 2 2 PN a0 + n=1 (an cos(2πnt/T ) − bn sin(2πnt/T )) + 2 N X =a0 + an cos(2πnt/T ) n=1

PN a0 + n=1 (an cos(2πnt/T ) + bn sin(2πnt/T )) fN (t) − fN (−t) = 2 2 PN a0 + n=1 (an cos(2πnt/T ) − bn sin(2πnt/T )) − 2 N X = bn sin(2πnt/T ). n=1

Exercise 1.22: Fourier series of low-degree polynomials Find the Fourier series coefficients of the periodic functions with period T defined by f (t) = t, f (t) = t2 , and f (t) = t3 , on [0, T ].

1 Sound and Fourier series

13

Solution.

For f (t) = t we get that a0 = T1 2 an = T

Z T

2 = T

2 T

Z T

bn =

RT

tdt = T2 . We also get

0

t cos(2πnt/T )dt 0

2 = T

!

T Z T T T sin(2πnt/T )dt = 0 t sin(2πnt/T ) − 2πn 2πn 0 0 t sin(2πnt/T )dt

0

!

T Z T T T T cos(2πnt/T )dt = − . − t cos(2πnt/T ) + 2πn 2πn 0 πn 0

The Fourier series is thus X T T − sin(2πnt/T ).

2 πn n≥1

Note that this is almost a sine series, since it has a constant term, but no other cosine terms. If we had subtracted T /2 we would have obtained a function which is antisymmetric, and thus a pure sine series.

RT 2 For f (t) = t2 we get that a0 = T1 0 t2 dt = T3 . We also get

an =

2 T

Z T

t2 cos(2πnt/T )dt

0

!

T Z T T 2 T 2 t sin(2πnt/T ) − t sin(2πnt/T )dt = T 2πn πn 0 0

T T T2 = − − = 2 2 πn πn π n Z T 2 t2 sin(2πnt/T )dt bn = T 0 !

T Z T 2 T 2 T = − t cos(2πnt/T ) + t cos(2πnt/T )dt T 2πn πn 0 0 =−

T2 . πn

Here we see that we could use the expressions for the Fourier coefficients of f (t) = t to save some work. The Fourier series is thus 2

T2 X T T2 + cos(2πnt/T ) − sin(2πnt/T ) .

3 π 2 n2 πn n≥1

For f (t) = t3 we get that a0 = T1

RT 3 3 t dt = T4 . We also get 0

14

1 Sound and Fourier series

an =

2 T

Z T

t3 cos(2πnt/T )dt

0 !

T Z T 2 T 3 3T 2 = t sin(2πnt/T )dt t sin(2πnt/T ) − T 2πn 2πn 0 0

3T T2 3T 3 = − − = 2πn πn 2π 2 n2 Z T 2 bn = t3 sin(2πnt/T )dt T 0 !

T Z T 2 T 3 3T 2 = t cos(2πnt/T )dt − t cos(2πnt/T ) + T 2πn 2πn 0 0

=−

T3 T3 3T T 2 3T 3 = − . + + πn 2πn π 2 n2 πn 2π 3 n3

Also here we saved some work, by reusing the expressions for the Fourier coefficients of f (t) = t2 . The Fourier series is thus

T3 X T3 3T 3 3T 3 + cos(2πnt/T ) + − + sin(2πnt/T ) .

4 2π 2 n2 πn 2π 3 n3 n≥1

We see that all three Fourier series converge slowly. This is connected to the fact that none of the functions are continuous at the borders of the periods.

Exercise 1.23: Fourier series of polynomials Write down difference equations for finding the Fourier coefficients of f (t) = tk from those of f (t) = tk−1 , and write a program which uses this recursion to compute the Fourier coefficients of f (t) = tk . Use the program to verify what you computed in Exercise 1.22.

Solution.

Let us define an,k , bn,k as the Fourier coefficients of tk . When k > 0 and n > 0, integration by parts gives us the following difference equations:

1 Sound and Fourier series

2 T

Z T

2 = T

an,k =

15

tk cos(2πnt/T )dt

0

!

T Z T T k kT k−1 t sin(2πnt/T )dt t sin(2πnt/T ) − 2πn 2πn 0 0

kT bn,k−1 2πn Z T 2 bn,k = tk sin(2πnt/T )dt T 0 !

T Z T 2 T k kT k−1 = t cos(2πnt/T )dt − t cos(2πnt/T ) + T 2πn 2πn 0 0 =−

=−

Tk kT + an,k−1 . πn 2πn

When n > 0, these can be used to express an,k , bn,k in terms of an,0 , bn,0 , for which we Tk clearly have an,0 = bn,0 = 0. For n = 0 we have that a0,k = k+1 for all k.

The following program computes an,k , bn,k recursively when n > 0. function [ank,bnk] = find_fourier_coeffs(n, k, T) if n==0 ank = T^k/(k+1); bnk = 0; else ank=0; bnk=0; if k > 0 [ankprev,bnkprev] = find_fourier_coeffs(n, k-1, T); ank = -k*T*bnkprev/(2*pi*n); bnk = -T^k/(pi*n) + k*T*ankprev/(2*pi*n); end end end

Exercise 1.24: Fourier series of a given polynomial 3 Use the previous exercise to find the Fourier series for f (x) = − 13 x3 + 12 x2 − 16 x + 1 on the interval [0, 1]. Plot the 9th order Fourier series for this function. You should obtain the plots from Figure 1.5 in the book.

Solution.

The following code generates the left plot of Figure 1.5 in the book. figure() polycoeffs = [1 -3/16 1/2 -1/3]; antterms=9; ank = zeros(length(polycoeffs),antterms+1); bnk = ank; for k=0:length(polycoeffs)-1 for n=0:antterms [ank(k+1,n+1),bnk(k+1,n+1)] = find_fourier_coeffs(n, k, 1); end end t=0:0.01:1; n=(0:antterms)’;

16

1 Sound and Fourier series

vals = polycoeffs*(ank*cos(2*pi*n*t) + bnk*sin(2*pi*n*t)); polyvals = zeros(length(polycoeffs),length(t)); for k=1:length(polycoeffs) polyvals(k,:) = t.^(k - 1); end plot(t, polycoeffs*polyvals,’r’); hold on axis([0 1 0.975 1]) plot(t,vals,’b’)

The polynomial coefficients are stored in the row vector polycoeffs at the top. The code is written so that it will work for any polynomial: You just need to replace the vector of polynomial coefficients. The Fourier coefficients are stored in the matrices ank, bnk, using the function implemented in the previous exercise. Each row in those matrices correspond to Fourier coefficients of a fixed polynomial. We also multiplied with matrices containing cosine and sine values at all instances in time.

Exercise 1.28: Orthonormality of Complex Fourier basis Show that the complex functions e2πint/T are orthonormal.

Solution.

For n1 6= n2 we have that

he2πin1 t/T , e2πin2 t/T i =

1 T

Z T 0

e2πin1 t/T e−2πin2 t/T dt =

1 T T

Z T

e2πi(n1 −n2 )t/T dt

0

T e2πi(n1 −n2 )t/T 2πi(n1 − n2 ) 0 1 1 − = 0. = 2πi(n1 − n2 ) 2πi(n1 − n2 )

1 = T

When n1 = n2 the integrand computes to 1, so that ke2πint/T k = 1.

Exercise 1.29: Complex Fourier series of f (t) = sin2 (2πt/T ) Compute the complex Fourier series of the function f (t) = sin2 (2πt/T ).

Solution.

We have that

1 Sound and Fourier series

17

2 1 2πit/T −2πit/T (e −e f (t) = sin (2πt/T ) = 2i 1 1 1 1 = − (e2πi2t/T − 2 + e−2πi2t/T ) = − e2πi2t/T + − e−2πi2t/T .

4 4 2 4 2

This gives the Fourier series of the function (with y2 = y−2 = −1/4, y0 = 1/2). This could also have been shown by using the trigonometric identity sin2 x = 12 (1 − cos(2x)) RT first, or by computing the integral T1 0 f (t)e−2πint/T dt (but this is rather cumbersome).

Exercise 1.30: Complex Fourier series of polynomials Repeat Exercise 1.22, computing the complex Fourier series instead of the real Fourier series.

Solution.

We have that Z 1 T −2πint/T 1 h −2πnit/T iT 1 T te te dt = − =− T 0 2πin/T T 2πin 0 Z Z T 1 T 2 −2πint/T 1 h 2 −2πnit/T iT 2 t e t e dt = − + te−2πint/T dt T 0 2πin 2πin 0 0 T2 T2 + 2 2 =− 2πin 2π n Z Z T 1 T 3 −2πint/T 1 h 3 −2πnit/T iT 3 t e dt = − t e t2 e−2πint/T dt + T 0 2πin 2πin 0 0

T3 3 T3 T3 =− + − + 2πin 2πin 2πin 2π 2 n2 T3 3T 3 3T 3 =− + 2 2 + 3 3.

2πin 4π n 4π in

Exercise 1.31: Complex Fourier series and Pascal’s triangle In this exercise we will find a connection with certain Fourier series and the rows in Pascal’s triangle. a) Show that both cosn (t) and sinn (t) are in VN,2π for 1 ≤ n ≤ N .

Solution.

We have that

18

1 Sound and Fourier series

n 1 it (e + e−it ) 2

n 1 it sinn (t) = (e − e−it ) 2i

cosn (t) =

If we multiply out here, we get a sum of terms of the form eikt , where −n ≤ k ≤ n. As long as n ≤ N it is clear that this is in VN,2π . b) Write down the N ’th order complex Fourier series for f1 (t) = cos t, f2 (t) = cos2 t, and f3 (t) = cos3 t.

Solution.

We have that 1 it (e + e−it ) 2 1 1 1 1 cos2 (t) = (eit + e−it )2 = e2it + + e−2it 4 4 2 4 1 it 1 3it 3 it 3 −it 1 −3it 3 −it 3 cos (t) = (e + e ) = e + e + e + e .

8 8 8 8 8 cos(t) =

Therefore, for the first function the nonzero Fourier coefficients are y−1 = 1/2, y1 = 1/2, for the second function y−2 = 1/4, y0 = 1/2, y2 = 1/4, for the third function y−3 = 1/8, y−1 = 3/8, y1 = 3/8, y3 = 1/8. c) In b) you should be able to see a connection between the Fourier coefficients and the three first rows in Pascal’s triangle. Formulate and prove a general relationship between row n in Pascal’s triangle and the Fourier coefficients of fn (t) = cosn t.

Solution.

In order to find the Fourier coefficients of cosn (t) we have to multiply out the expression 1 it −it n ) . The coefficients we get after this can also be obtained from Pascal’s 2n (e + e triangle.

Exercise 1.32: Complex Fourier coefficients of the square wave Compute the complex Fourier coefficients of the square wave using equation (1.15) in the book, i.e. repeat the calculations from Example 1.13 in the book for the complex case. Use Theorem 15 in the book to verify your result.

Solution.

We obtain that

1 Sound and Fourier series

1 yn = T

19

Z T /2 e

−2πint/T

0

1 dt − T T /2

Z T

e−2πint/T dt

T /2

T T −2πint/T e 2πin T /2

1 = −e−πin + 1 + 1 − e−πin + 2πin (

0, if n is even; 1 −πin = .

1−e = πin 2/(πin), if n is odd.

1 =− T

T −2πint/T e 2πin 0

1 + T

Instead using Theorem 15 in the book together with the coefficients bn = 2(1−cos(nπ) we nπ computed in Example 1.13 in the book, we obtain ( ( if n is even; 0, if n is even; 1 0, 1 = yn = (an − ibn ) = − i 2 2 4/(nπ), if n is odd.

2/(πin), if n is odd. when n > 0. The case n < 0 follows similarly.

Exercise 1.33: Complex Fourier coefficients of the triangle wave Repeat Exercise 1.32 for the triangle wave.

Solution.

Using Integration by parts we get 1 T

Z T /2

(4t/T − 1)e−2πint/T dt

0

!

Z T /2 h iT /2 4 − (4t/T − 1)e−2πint/T + e−2πint/T dt T 0 0

1 4 1 −1 − (−1)n − ((−1)n − 1) = 2πin T 2πin/T Z T 1 (3 − 4t/T )e−2πint/T dt T T /2 !

Z T h iT 1 4 − (3 − 4t/T )e−2πint/T − e−2πint/T dt = 2πin T T /2 T /2

1 4 1 = 1 + (−1)n + (1 − (−1)n ) .

2πin T 2πin/T 1 = 2πin

Adding the two several terms cancel, and we obtain − π22n2 (1 − (−1)n ). This means that yn = −4/π 2 n2 for n odd, and 0 otherwise. Using Theorem 15 in the book, this is clearly

20

1 Sound and Fourier series

compatible with our previous computation of the real Fourier coefficients of the triangle wave.

Exercise 1.34: Complex Fourier coefficients of low-degree polynomials Use equation (1.15) in the book to compute the complex Fourier coefficients of the periodic functions with period T defined by, respectively, f (t) = t, f (t) = t2 , and f (t) = t3 , on [0, T ]. Use Theorem 15 in the book to verify your calculations from Exercise 1.22.

Solution.

For f (t) = t we get Z T

1 yn = T =−

te

−2πint/T

0

1 dt = T

!

T Z T T −2πint/T T −2πint/T + − te e dt 2πin 0 2πin 0

T T = i.

2πin 2πn

T T i From Exercise 1.22 we had bn = − πn , for which Theorem 15 in the book gives yn = 2πn for n > 0, which coincides with the expression we obtained. The case n < 0 follows similarly.

For f (t) = t2 we get

1 yn = T =−

Z T 0

1 t2 e−2πint/T dt = T

!

T Z T T 2 −2πint/T T − t e +2 te−2πint/T dt 2πin 0 2πin 0

T2 T2 T2 T2 + 2 2 = + i.

2 2 2πin 2π n 2π n 2πn 2

2

T T From Exercise 1.22 we had an = π2 n2 and bn = − πn , for which Theorem 15 in the 2

2

T book gives yn = 12 πT2 n2 + i πn for n > 0, which also is seen to coincide with what we obtained. The case n < 0 follows similarly.

For f (t) = t3 we get

!

T Z T T 3 −2πint/T T 2 −2πint/T t e − t e +3 t e dt 2πin 0 0 2πin 0 3

T T2 T2 T3 T T3 T3 +3 ( + i) = 3 2 2 + −3 3 3 i= =− 2πin 2πin 2π 2 n2 2πn 4π n 2πn 4π n

1 yn = T

Z T

3 −2πint/T

1 dt = T

3

3

3

T From Exercise 1.22 we had an = 2π3T2 n2 and bn = − πn + 2π3T3 n3 for which Theorem 15 in the book gives

1 Sound and Fourier series

yn =

1 2

3T 3 +i 2π 2 n2

21

T3 3T 3 − 3 3 πn 2π n

=

3T 3 + 4π 2 n2

T3 3T 3 − 3 3 2πn 4π n

i

for n > 0, which also is seen to coincide with what we obtained. The case n < 0 follows similarly.

Exercise 1.35: Complex Fourier coefficients for symmetric and antisymmetric functions In this exercise we will prove a version of Theorem 10 in the book for complex Fourier coefficients. a) If f is symmetric about 0, show that yn is real, and that y−n = yn .

Solution.

If f is symmetric about 0 we have that bn = 0. Theorem 15 in the book then gives that yn = 12 an , which is real. The same theorem gives that y−n = 12 an = yn . b) If f is antisymmetric about 0, show that the yn are purely imaginary, y0 = 0, and that y−n = −yn .

Solution.

If f is antisymmetric about 0 we have that an = 0. Theorem 15 in the book then gives that yn = − 12 bn , which is purely imaginary. The same theorem gives that y−n = 12 bn = −yn .

PN c) Show that n=−N yn e2πint/T is symmetric when y−n = yn for all n, and rewrite it as a cosine-series.

Solution.

When yn = y−n we can write y−n e2πi(−n)t/T + yn e2πint/T = yn (e2πint/T + e−2πint/T ) = 2yn cos(2πnt/T ) PN This is clearly symmetric, but then also n=−N yn e2πint/T is symmetric since it is a sum of symmetric functions.

PN d) Show that n=−N yn e2πint/T is antisymmetric when y0 = 0 and y−n = −yn for all n, and rewrite it as a sine-series.

Solution.

When yn = −y−n we can write

22

1 Sound and Fourier series

y−n e2πi(−n)t/T + yn e2πint/T = yn (−e2πint/T + e2πint/T ) = 2iyn sin(2πnt/T ) PN This is clearly antisymmetric, but then also n=−N yn e2πint/T is antisymmetric since it is a sum of antisymmetric functions, and since y0 = 0.

Exercise 1.37: Fourier series of a delayed square wave Define the function f with period T on [−T /2, T /2) by ( 1, if −T /4 ≤ t < T /4; f (t) = −1, if T /4 ≤ |t| < T /2. f is just the square wave, delayed with d = −T /4. Compute the Fourier coefficients of f directly, and use Property 4 in Theorem 17 in the book to verify your result.

Solution.

We obtain that Z Z 1 −T /4 −2πint/T 1 T /2 −2πint/T e dt − e dt − e dt T −T /2 T T /4 −T /4 T /4

−T /4 T /2

1 −2πint/T 1 −2πint/T 1 −2πint/T e + e + e =− 2πin 2πin 2πin −T /4 −T /2 T /4

1 −πin/2 = −e + eπin/2 + eπin/2 − eπin + e−πin − e−πin/2 2πin 2 1 (2 sin(πn/2) − sin(πn)) = sin(πn/2). = πn πn

1 yn = T

Z T /4

−2πint/T

The square wave defined in this exercise can be obtained by delaying our original square wave with −T /4. Using Property 3 in Theorem 17 in the book with d = −T /4 on the complex Fourier coefficients ( 0, if n is even; yn = 2/(πin), if n is odd, which we obtained for the square wave in Exercise 1.32, we obtain the Fourier coefficients

e

2πin(T /4)/T

( ( 0, if n is even; 0, if n is even; = 2i sin(πn/2) 2/(πin), if n is odd. , if n is odd. πin ( 0, if n is even; = 2 . πn sin(πn/2), if n is odd.

This verifies the result.

1 Sound and Fourier series

23

Exercise 1.38: Find a function from its Fourier series Find a function f which has the complex Fourier series X n odd

4 e2πint/T . π(n + 4)

Hint.

Attempt to use one of the properties in Theorem 17 in the book on the Fourier series of the square wave.

Solution.

Since the real Fourier series of the square wave is X n≥1,n odd

4 sin(2πnt/T ), πn

4 Theorem 15 in the book gives us that the complex Fourier coefficients are yn = − 12 i πn = 2i 1 4 2i 2i − πn , and y−n = 2 i πn = πn for n > 0. This means that yn = − πn for all n, so that the complex Fourier series of the square wave is

−

X 2i e2πint/T . πn

n odd

Using Property 4 in Theorem 17 in the book we get that the e−2πi4t/T (i.e. set d = −4) 2i times the square wave has its n’th Fourier coefficient equal to − π(n+4) . Using linearity, this means that 2ie−2πi4t/T times the square wave has its n’th Fourier coefficient equal 4 to π(n+4) . We thus have that the function ( 2ie−2πi4t/T , 0 ≤ t < T /2 f (t) = −2πi4t/T −2ie , T /2 ≤ t < T has the desired Fourier series.

Exercise 1.39: Relation between the complex Fourier coefficients of f and the cosine-coefficients of f˘ Show that the complex Fourier coefficients yn of f , and the cosine-coefficients an of f˘ are related by a2n = yn + y−n . This result is not enough to obtain the entire Fourier series of f˘, but at least it gives us half of it.

Solution.

The 2nth complex Fourier coefficient of f˘ is

24

1 Sound and Fourier series

1 2T =

Z 2T

f˘(t)e−2πi2nt/(2T ) dt

0

1 2T

Z T

f (t)e−2πint/T dt +

0

1 2T

Z 2T

f (2T − t)e−2πint/T dt.

T

Substituting u = 2T − t in the second integral we see that this is

=

1 2T

Z T

f (t)e−2πint/T dt −

0

1 2T

Z 0

f (u)e2πinu/T du

T

Z T Z T 1 1 −2πint/T = f (t)e dt + f (t)e2πint/T dt 2T 0 2T 0 1 1 = yn + y−n .

2 2 Therefore we have a2n = yn − y−n .

Exercise 1.40: Symmetric filters preserve sine- and cosine-series An analog filter where λs (ν) = λs (−ν) is also called a symmetric filter. a) Prove that, if the input to a symmetric filter is a Fourier series which is a cosine- or sine series, then the output also is a cosine- or sine series.

Solution.

We have that

1 2πint/T −2πint/T s(cos(2πnt/T )) = s (e +e ) 2 1 1 = λs (n/T )e2πint/T + λs (−n/T )e−2πint/T 2 2

1 2πint/T = λs (n/T ) (e + e−2πint/T ) = λs (n/T ) cos(2πnt/T ), 2 so that s preserves cosine-series. A similar computation holds for sine-series.

Ra b) Show that s(f ) = −a g(s)f (t − s)ds is a symmetric filter whenever g is symmetric around 0, and zero outside [−a, a].

Solution.

We have that

1 Sound and Fourier series

25

Z a λs (ν) =

g(s)e−2πiνs ds =

Z a

−a

g(s)e2πiνs ds = λs (−ν),

−a

so that s is symmetric.

Exercise 1.41: Circular convolution and Fourier series Let f, g ∈ VN,T , and let h = f ~ g. Assume that f , g, and h have Fourier coefficients xn , yn , and zn respectively. Show that zn = xn yn .

Solution.

The circular convolution of f and g was defined as h(t) = T1 that

zn =

1 T

Z T 0

h(t)e−2πint/T dt =

1 T2

Z TZ T 0

RT 0

f (u)g(t − u)du. We have

f (u)g(t − u)e−2πint/T dudt

0

Z TZ T 1 f (u)e−2πinu/T g(t − u)e−2πin(t−u))/T dudt = 2 T 0 0 Z Z 1 T 1 T f (u)e−2πinu/T du g(s)e−2πins/T ds = T 0 T 0 = xn yn .

Exercise 1.42: Approximation in norm with continuous functions Assume that f is a function defined on [a, b] with only a finite number of discontinuities, and with all one-sided limits existing. Show that there exists a continuous function g so that kf − gk < .

Solution.

Denote the discontinuities by {ai }N i=1 . We can easily construct a continuous function g which agrees with f outside intervals of the form [ai − , ai + ] (simply connect (ai − , f (ai − )) and (ai + , f (ai + )) with a linear segment). By choosing small enough it is clear that we can make kg − f k as small as we like.

Exercise 1.43: The Dirichlet kernel Recall that the Dirichlet kernel was defined as

26

1 Sound and Fourier series

DN (t) =

N X

e2πint/T

n=−N +1)t/T ) . a) Show that DN (t) = sin(π(2N sin(πt/T )

Solution.

Using the formula for the sum of a geometric series we obtain N X

e2πint/T = e−2πiN t/T

n=−N

2N +1 1 − e2πint/T 1 − e2πint/T

= e2πiN t/T +πint/T −πi(2N +1)t/T =

eπi(2N +1)t/T − e−πi(2N +1)t/T eπint/T − e−πint/T

sin(π(2N + 1)t/T ) . sin(πt/T )

b) Prove that DN (t) satisfies one of the properties of a summability kernel, but fails on the other ones.

Solution.

We have that

1 T

N X

Z T 0

! e

2πint/T

n=−N

dt =

1 T

Z T 1dt = 1, 0

so the first property of a summability kernel is satisfied. The second property is not satisfied, however, since the integral over [0, T /2] goes to infinity as N → ∞. To see why, note that | sin(x)| ≤ |x| for all x. This means that it is enough to prove that R T /2 | sin(π(2N +1)t/T )| dt → ∞ as well. Setting u = π(2N + 1)t/T , we can write this as t 0 Z π(N +1/2) 0

N −1 Z (k+1)π

X | sin(u)| du = u

k=0

kπ

| sin(u)| du = u

Z π | sin(u)|du 0

N −1 X k=0

1 . (k + 1)π

This sum clearly diverges. That third property is not satisfied also follows by using Rb | sin(x)| ≤ |x| (simply note that a | sin(π(2N + 1)t/T )|dt does not converge to zero).

Exercise 1.44: The Fejer summability kernel The Fejer kernel is defined as

1 Sound and Fourier series

27

FN (t) =

N X

1−

n=−N

|n| N +1

e2πint/N .

Clearly FN ∈ VN,T , and of degree N .

2 +1)t/T ) , and conclude from this that 0 ≤ FN (t) ≤ a) Show that FN (t) = N1+1 sin(π(N sin(πt/T ) T2 4(N +1)t2 .

Hint.

Use that π2 |u| ≤ | sin u| when u ∈ [−π/2, π/2].

Solution.

We compute

2

sin (πt/T )

N X n=−N

|n| 1− N +1 N

X 1 = (1 − cos(2πt/T )) 2

n=−N

=

e2πint/N

|n| 1− N +1

1 1 1 − e−2πint/T + − e2πint/T 4 2 4

e2πint/N

X

N n=−N

|n| 1− N +1

e2πint/N .

When one here multiplies out, one sees that most terms cancel (since −1/4 + 1/2 − 1/4 = 0), and we are left with 1 N +1

1 1 1 1 − e−2πi(N +1)t/T + − e2πi(N +1)t/T = sin2 (π(N + 1)t/T ).

4 2 4 N +1

This proves the claim. Finally, using that π2 |u| ≤ | sin u| when u ∈ [−π/2, π/2], we obtain 1 N +1

sin(π(N + 1)t/T ) sin(πt/T )

2 ≤

1 π2 1 T2 = .

N + 1 4 π 2 t2 /T 2 4(N + 1)t2

b) Show that FN (t) satisfies the three properties of a summability kernel.

Solution.

The first property of summability is easily seen to hold. Since from a) FN (t) ≥ 0, the the second property follows from the first. To prove the third, due to a) we only need to RT compare with the integral δ t−2 dt, which exists for any δ.

RT PN c) Show that T1 0 FN (u)f (t − u)du = N1+1 n=0 fn (t).

28

1 Sound and Fourier series

Hint.

Show that FN (t) = N1+1

PN

n=0 Dn (t), and use Exercise 1.43 b).

Solution.

We have that N 1 X 1 Dn (t) = N + 1 n=0 N +1

=1−

N +1+

N X

(N − n + 1)(e

+e

2πint/T

)

n=0

N X N +1−n n=0

! −2πint/T

N +1

(e−2πint/T + e2πint/T )

N X

n (e−2πint/T + e2πint/T ) =1− 1− N +1 n=0

N X |n| = 1− e2πint/T = FN (t).

N +1 n=−N

Using this we obtain that

1 T

Z T FN (u)f (t − u)du = 0

Z N N 1 X 1 T 1 X fn (t).

DN (u)f (t − u)du = N + 1 n=0 T 0 N + 1 n=0

Chapter 2

Digital sound and Discrete Fourier analysis

Exercise 2.8: Computing the DFT by hand Compute F4 x when x = (2, 3, 4, 5).

Solution.

As in Example 2.3 in the book we get      2 1 1 1 1 2 3 1 1 −i −1 i  3     F4  4 = 2 1 −1 1 −1 4 5 1 i −1 −i 5     7 2+3+4+5   1 2 − 3i − 4 + 5i  = −1 + i . =  2  2 − 3 + 4 − 5   −1  −1 − i 2 + 3i − 4 − 5i

Exercise 2.9: Exact form of low-order DFT matrix As in Example 2.3 in the book, state the exact Cartesian form of the Fourier matrix for the cases N = 6, N = 8, and N = 12.

Solution.

For N = 6 the entries are on the form √16 e−2πink/6 = √16 e−πink/3 . This means that the √ entries in the Fourier matrix are the numbers √16 e−πi/3 = √16 (1/2−i 3/2), √16 e−2πi/3 = √ √1 (−1/2 − i 3/2), and so on. The matrix is thus 6

29

30

2 Digital sound and Discrete Fourier analysis

  1 1√ 1 √ 1 1 √ 1√ 1 1/2 − i 3/2 −1/2 − i 3/2 −1 −1/2 + i 3/2 1/2 + i 2/2    √ √ √ √ 1 1 −1/2 − i 3/2 −1/2 + i 3/2 1 −1/2 − i 3/2 +1/2 − i 3/2  F6 = √   −1 √ 1 √ −1 1 √ −1 √ 6 1  1 −1/2 + i 3/2 −1/2 − i 3/2 1 −1/2 + i 3/2 −1/2 − i 3/2 √ √ √ √ 1 1/2 + i 2/2 −1/2 + i 3/2 −1 −1/2 − i 3/2 1/2 − i 3/2 The cases N = 8 and N = 12 follow similarly, but are even more tedious. For N = 8 the entries are √18 eπink/4 , which can be expressed exactly since we can express exactly any sines and cosines of a multiple of π/4. For N = 12 we get the base angle π/6, for which we also have exact values for sines and cosines for all multiples.

Exercise 2.10: DFT of a delayed vector We have a real vector x with length N , and define the vector z by delaying all elements in x with 5 cyclically, i.e. z5 = x0 , z6 = x1 ,. . . ,zN −1 = xN −6 , and z0 = xN −5 ,. . . ,z4 = xN −1 .

For a given n, if |(FN x)n | = 2, what is then |(FN z)n |?

Solution. z is the vector x delayed with d = 5 samples, and then Property 3 of Theorem 7 in the book gives us that (FN z)n = e−2πi5k/N (FN x)n . In particular |(FN z)n | = |(FN x)n | = 2, since |e−2πi5k/N | = 1.

Exercise 2.11: Using the symmetry property Given a real vector x of length 8 where (F8 (x))2 = 2 − i, what is (F8 (x))6 ?

Solution.

By Theorem 7 in the book we know that (FN (x))N −n = (FN (x))n when x is a real vector. If we set N = 8 and n = 2 we get that (F8 (x))6 = (F8 (x))2 = 2 − i = 2 + i.

Exercise 2.12: DFT of cos2 (2πk/N ) Let x be the vector of length N where xk = cos2 (2πk/N ). What is then FN x?

Solution.

The idea is to express x as a linear combination of the Fourier basis vectors φn , and use that FN φn = en . We have that

2 Digital sound and Discrete Fourier analysis

31

2 1 2πik/N −2πikn/N e +e cos (2πk/N ) = 2 1 1 1 1 1 1 = e2πi2k/N + + e−2πi2k/N = e2πi2k/N + + e2πi(N −2)k/N 4 2 4 4 2 4

√ 1 1 1 = N φ2 + φ0 + φN −2 .

4 2 4 2

We here used the periodicity of e2πikn/N , i.e. that e−2πi2k/N = e2πi(N −2)k/N . Since FN is linear and FN (φn ) = en , we have that √ FN (x) =

N

1 1 1 e2 + e0 + eN −2 4 2 4

√

N (1/2, 0, 1/4, 0, . . . , 0, 1/4, 0) .

=

Exercise 2.13: DFT of ck x Let x be the vector with entries xk = ck . Show that the DFT of x is given by the vector with components yn =

1 − cN 1 − ce−2πin/N

for n = 0, . . . , N − 1.

Solution.

We get

yn =

N −1 X

ck e−2πink/N =

k=0

N −1 X k=0

−2πin/N N

=

(ce−2πin/N )k

1 − (ce ) 1 − ce−2πin/N

=

1 − cN .

1 − ce−2πin/N

Exercise 2.14: DFT implementation Extend the code for the function dft_impl in Example 2.4 in the book so that 1. The function also takes a second parameter called forward. If this is true the DFT is applied. If it is false, the IDFT is applied. If this parameter is not present, then the forward transform should be assumed.

2. If the input x is two-dimensional (i.e. a matrix), the DFT/IDFT should be applied to each column of x. This ensures that, in the case of sound, the FFT is applied to each channel in the sound when the entire sound is used as input.

Also, write documentation for the code.

32

2 Digital sound and Discrete Fourier analysis

Solution.

The library version of this function looks as follows. function y = dft_impl(x, forward) sign = -1; if nargin >= 2 & forward == 0 sign = 1; end N = size(x, 1); y = zeros(size(x)); for n = 1:N D = exp(-sign*2*pi*1i*(n-1)*(0:(N-1))/N); for s2 = 1:size(x,2) y(n, s2) = dot(D, x(:, s2)); end end if sign == 1 y = y/N; end end

Exercise 2.15: Symmetry Assume that N is even. a) Prove that, if xk+N/2 = xk for all 0 ≤ k < N/2, then yn = 0 when n is odd.

Solution.

We have that   N/2−1 N −1 X 1  X xk e−2πikn/N + xk e−2πikn/N  yn = √ N k=0 k=N/2   N/2−1 N/2−1 X 1  X =√ xk e−2πikn/N + xk e−2πi(k+N/2)n/N  N k=0 k=0 N/2−1 1 X =√ xk (e−2πikn/N + (−1)n e−2πikn/N ) N k=0 N/2−1 1 X = (1 + (−1)n ) √ xk e−2πikn/N N k=0

If n is odd, we see that yn = 0. b) Prove that, if xk+N/2 = −xk for all 0 ≤ k < N/2, then yn = 0 when n is even.

2 Digital sound and Discrete Fourier analysis

33

Solution.

The proof is the same as in a), except for a sign change. c) Prove the converse statements in a) and b).

Solution.

Clearly the set of vectors which satisfies xk+N/2 = ±xk is a vector space V of dimension N/2. The set of vectors where every second component is zero is also a vector space of dimension N/2, let us denote this by W . We have shown that FN (V ) ⊂ W , but since FN is unitary, FN (V ) also has dimension N/2, so that FN (V ) = W . This shows that when every second yn is 0, we must have that xk+N/2 = ±xk , and the proof is done. d) Prove the following: • xn = 0 for all odd n if and only if yk+N/2 = yk for all 0 ≤ k < N/2.

• xn = 0 for all even n if and only if yk+N/2 = −yk for all 0 ≤ k < N/2.

Solution.

In the proofs above, compute the IDFT instead.

Exercise 2.16: DFT on complex and real data Let x1 , x2 be real vectors, and set x = x1 + ix2 . Use Theorem 7 in the book to show that

1 (FN (x))k + (FN (x))N −k 2

1 (FN (x2 ))k = (FN (x))k − (FN (x))N −k 2i

(FN (x1 ))k =

This shows that we can compute two DFT’s on real data from one DFT on complex data, and 2N extra additions.

Solution.

We have that

(FN (x))k = (FN (x1 + ix2 ))k = (FN (x1 ))k + i(FN (x2 ))k (FN (x))N −k = (FN (x1 ))N −k + i(FN (x2 ))N −k = (FN (x1 ))k + i(FN (x2 ))k , where we have used Property 1 of Theorem 7 in the book. If we take the complex conjugate in the last equation, we are left with the two equations

34

2 Digital sound and Discrete Fourier analysis

(FN (x))k = (FN (x1 ))k + i(FN (x2 ))k (FN (x))N −k = (FN (x1 ))k − i(FN (x2 ))k .

If we add these we get

1 (FN (x))k + (FN (x))N −k , 2 which is the first equation. If we instead subtract the equations we get

1 (FN (x2 ))k = (FN (x))k − (FN (x))N −k , 2i which is the second equation (FN (x1 ))k =

Exercise 2.17: Vandermonde matrices −1 In linear algebra, an N × N -matrix where column j is on the form (x0j , x1j , ..., xN ) for j some number xj is called a Vandermonde matrix. The number xj is called a generator for (the j’th column) of the matrix.

a) Show that a Vandermonde matrix where all the generators are distinct and nonzero is non-singular.

Hint.

Combine elementary row operations with a cofactor expansion along the last column.

Solution.

If we subtract xN −1 times row k from row k + 1 for k ≥ 0, the last column will become equal to e0 . The other entries are now aij = (xj )i − xN −1 xi−1 = xi−1 j j (xj − xN −1 ) (the new matrix has the same determinant). A cofactor expansion is now natural, and leads to the smaller matrix where we drop the first row and the last column. It is enough to prove that this smaller matrix is non-singular. Its entries are xij (xj − xN −1 ) (since i was shifted by one). (xj − xN −1 ) is a common factor for each column which is nonzero and can be moved out of the matrix without affecting if the determinant is nonzero or not. After this we are left with a Vandermonde matrix with generators x1 , ..., xN −2 .

The result now follows by induction. b) Let A be the s × s-matrix obtained by taking s distinct columns of DF TN , and s successive columns of DF TN . Show that A is non-singular.

Solution.

The DFT-matrix has distinct generators for each column. As long as the rows we choose are successive, the non-singularity clearly follows from that of the Vandermonde matrix

2 Digital sound and Discrete Fourier analysis

35

(if we don’t start with the first row, simply move out a constant from each column to relate to a Vandermonde matrix).

Exercise 2.18: Fourier basis vectors with zeros inserted Later we will encounter vectors created by zeroing out certain components in the Fourier basis vectors, and we will have use for the DFT of these. a) Show that 1 2πirk/N (e + e2πi(r+N/2)k/N ) 2 1 (0, e2πir·1/N , 0, e2πir·3/N , . . . , 0, e2πir(N −1)/N ) = (e2πirk/N − e2πi(r+N/2)k/N ).

2

(e2πir·0/N , 0, e2πir·2/N , 0, . . . , e2πir(N −2)/N , 0) =

Do this by computing the DFT of the left hand sides.

Solution.

We prove only the first claim. Component n of the DFT of the left hand side is N/2−1

X

N/2−1

e2πir(2k)/N e−2πin(2k)/N =

k=0

X

e2πi(r−n)k/(N/2) .

k=0

This is clearly zero unless n = r or n = r + N/2. For these two values this computes to N/2. It follows that the DFT of the left hand side is N2 (er + er+N/2 ). Taking the IDFT proves the claim. b) Assume that N (the length of the vector) is divisible by M , and let s be a fixed number. Show that the vector where only components on the form M k + s in (e2πir·0/N , e2πir·1/N , . . . , e2πir(N −1)/N ) are kept (the others zeroed out) can be written as M −1

1 X −2πist/M 2πi(r+tN/M )k/N e e .

M t=0 In other words, such vectors can be written as a sum of M Fourier basis vectors.

Solution.

Again computing component n of the DFT of the vector in question gives N/M −1

X k=0

N/M −1 2πir(M k+s)/N −2πin(M k+s)/N

e

e

=e

2πi(r−n)s/N

X k=0

e2πi(r−n)k/(N/M ) .

36

2 Digital sound and Discrete Fourier analysis

The sum here is zero unless n = r + tN/M for some 0 ≤ t < M , for which the sum computes to N/M . It follows that the DFT of the vector in question is PM −1 −2πits/M N er+tN/M . Again taking the IDFT proves the claim. t=0 e M

Exercise 2.19: Code example Explain what the code below does, line by line: [x, fs] = audioread(’sounds/castanets.wav’); N = size(x, 1); y = fft(x); y((round(N/4)+1):(round(N/4)+N/2), :) = 0; newx = abs(ifft(y)); newx = newx/max(max(newx)); playerobj = audioplayer(newx, fs); playblocking(playerobj)

Comment in particular why we adjust the sound samples by dividing with the maximum value of the sound samples. What changes in the sound do you expect to hear?

Solution.

First a sound file is read. We then restrict to the first 212 sound samples, perform a DFT, zero out the frequencies which correspond to DFT-indices between 210 and 212 − 210 − 1, and perform an IDFT. Finally we scale the sound samples so that these lie between −1 and 1, which is the range we demand for the sound samples, and play the new sound.

Exercise 2.20: Mapping DFT indices to frequencies In the code from the previous exercise it turns out that fs = 44100Hz, and that the number of sound samples is N = 292570. Which frequencies in the sound file will be changed on the line where we zero out some of the DFT coefficients?

Solution.

As we have seen, DFT index n corresponds to frequency ν = nfs /N . Above N = 217 , so that we get the connection ν = nfs /N = n × 44100/217 . We zeroed the DFT indices above n = 215 , so that frequencies above ν = 215 × 44100/217 = 11025Hz are affected.

Exercise 2.21: Plotting the interpolant Implement code where you do the following: • at the top you define the function f (x) = cos6 (x), and M = 3, • compute the unique interpolant from VM,T (i.e. by taking N = 2M + 1 samples over one period), as guaranteed by Proposition 9 in the book,

2 Digital sound and Discrete Fourier analysis

37

• plot the interpolant against f over one period.

Finally run the code also for M = 4, M = 5, and M = 6. Explain why the plots coincide for M = 6, but not for M < 6. Does increasing M above M = 6 have any effect on the plots?

Solution.

The code can look as follows. f = @(t)cos(t).^6; M = 5; T = 2*pi; N = 2*M + 1; t = (linspace(0, T, 100))’; x = f((linspace(0, T - T/N, N))’); y = fft(x)/N; s = y(1)*ones(length(t),1); for k=1:((N-1)/2) s = s + 2*real(y(k+1)*exp(2*pi*1i*k*t/T)); end plot(t, s, ’r’, t, f(t), ’g’); legend(’Interpolant from V_{M,T}’, ’f’)

Exercise 2.22: Proving the general version of the sampling theorem for functions in a Fourier space a) Show that the function ( g(t) =

1 1 sin t − t

0

, if t > 0 , if t = 0

is continuous and strictly increasing on (−π, π).

Solution.

It is clear that g is continuous when t 6= 0. We also have that

lim

t→0

1 1 − sin t t

= lim

t→0

t − sin t 1 − cos t sin t = lim = lim =0 t→0 t→0 t sin t sin t + t cos t 2 cos t − t sin t

where L’hôpitals rule was used twice. This proves that g is continuous at 0 as well.

We will now prove that g 0 (t) > 0 for all t ∈ (0, π). We have that cos t 1 2 + t2 . sin t This is obviously > 0 for t ≥ π/2, since cos t is negative then. In the remainder of the proof we will therefore assume that t < π/2, and cos t > 0.

2 g 0 (t) > 0 is the same as sint2 t > cos t, which can be written as 12 (1 − cos(2t)) > t2 cos t.

This can be written in terms of Taylor series as g 0 (t) = −

38

2 Digital sound and Discrete Fourier analysis ∞ X

∞

(−1)k−1

k=1

22k−1 2k X 1 t > t2k . (−1)k−1 (2k)!

(2(k − 1))! k=1

separating the two first terms on both sides this is the same as t2 − t4 /3 + a1 − a2 + a3 − · · · > t2 − t4 /2 + b1 − b2 + b3 . · · · Where the ai and bi are positive. It is easily checked that the ai and bi are decreasing for t < 2 (so in particular for t < π/2), since a2 is obtained from a1 by multiplying with 4/(7 · 8)t2 < 1, b2 is obtained from b1 by multiplying with 1/(5 · 6)t2 < 1 (and in the next steps we multiply with even smaller values). It is also easily checked that b1 = t6 /24. It follows from a result on the sum of an alternating series that a1 − a2 + a3 − · · · > 0

b1 − b2 + b3 − · · · < t6 /24

Inserting this in the above we see that it is enough to prove that t2 − t4 /3 > t2 − t4 /2 + t6 /24, which can be simplified to t4 /6 > t6 /24, which clearly holds for t < 2.

The case for negative t follows also since g is anti-symmetric.

Any function of period T can also be viewed as a function of period sT , for any integer s. Replacing N with sN , T with sT , and taking the difference between two corresponding terms in the ideal interpolation formula for periodic functions, and the one for non-periodic functions, we get

f (kTs ) sin(π(t − kTs )/Ts )

1 1 − sN sin(π(t − kTs )/(sT )) π(t − kTs )/Ts

.

b) With ak =

1 1 − = g(π(t − kTs )/(sT )), sin(π(t − kTs )/(sT )) π(t − kTs )/(sT )

the difference between the ideal interpolation formula for periodic and non-periodic functions is 1 sN

sM X

f (kTs ) sin(π(t − kTs )/Ts )ak

k=−sM

Show that, for N odd, this can be made arbitrarily small by choosing s big enough.

Hint.

Split the sum into N alternating series, so that there a common f (kTs ) in each series.

The result from a) will be needed in order to bound each of those series.

2 Digital sound and Discrete Fourier analysis

39

Solution.

Since −sM ≤ k ≤ sM , the argument π(t − kTs )/(sT ) to g satisfies πt πM π(t − kTs ) πt πM − ≤ ≤ + . sT N sT sT N Using that N ≥ 2M + 1 this can also be written as πt π π(t − kTs ) πt π − (1 − 1/N ) ≤ ≤ + (1 − 1/N ). sT 2 sT sT 2 By choosing s big enough this clearly lies inside the interval (−π/2, π/2).

If we take N odd, i.e. we take an odd number of samples in each period, the partial sum sl of the above sum for k on the form rN + l, i.e. sl =

1 sN

X

f ((rN + l)Ts ) sin(π(t − (rN + l)Ts )/Ts )arN +l

r|−sM ≤rN +l≤sM

2 will be alternating. Since the ak increase, the sum is bounded by |sl | ≤ sN |g(π/2)| (since any alternating series where the terms decrease has sum less than the biggest term).

Clearly we can get this as small as we want by increasing s. Since there are N partial sums, we can get the total difference as small as we want as well.

Note that this approach avoided t-values close to π. These would be problematic, since g(t) → ∞ when t → π.

Thus, we can prove the more general interpolation formula for a periodic f by considering f as a function with period sT , and letting s → ∞.

Exercise 2.23: Properties of the DFT when N is composite When N = N1 N2 is composite, the following explains a duality between the polyphase components x(k) with components xrN1 +k , and the polyphase components y (k) of y = DFTN x with components yrN2 +k . a) Assume that all x(k) are constant vectors (i.e. x is N1 -periodic). Show that y (k) = 0 for all k 6= 0.

Solution.

We have that x(p) is a constant vector of length N2 for 0 ≤ p < N1 . But then the DFT of all the x(p) has zero outside entry zero. Multiplying with e−2πikn/N does not affect this. The last N2 − 1 rows are thus zero before the final DFT is applied, so that these rows are zero also after this final DFT. After assembling the polyphase components again we have that yrN2 are the only nonzero DFT-coefficients. b) Assume that N = N1 N2 , and that x(p) = 0 for p 6= 0. Show that the y (p) are constant vectors for all p.

40

2 Digital sound and Discrete Fourier analysis

Solution.

Write any number < N as kN1 + l for 0 ≤ k < N2 , 0 ≤ l < N1 . We have that

yrN2 +p =

NX 2 −1 N 1 −1 X k=0

=

NX 2 −1

(l)

xk e−2πi(kN1 +l)(rN2 +p)/N

l=0 (0)

xk e−2πikN1 (rN2 +p)/N =

NX 2 −1

k=0

(0)

xk e−2πikN1 p/N ,

k=0

which is independent of r, so that the y (p) are constant vectors.

Exercise 2.24: FFT implementation which works for both forward and reverse Recall that, in Exercise 2.14, we extended the direct DFT implementation so that it accepted a second parameter telling us if the forward or reverse transform should be applied. Extend fft_impl and fft_kernel_standard in the same way. Again, the forward transform should be used if the forward parameter is not present. Assume also that the kernel accepts only one-dimensional data, and that the general function applies the kernel to each column in the input if the input is two-dimensional (so that the FFT can be applied to all channels in a sound with only one call).

It should be straightforward to make the modifications for the reverse transform by consulting the second part of Theorem 16 in the book. For simplicity, let fft_impl take care of the additional division with N in case of the IDFT. In the following we will assume these signatures for the FFT implementation and the corresponding kernels.

Solution.

The library versions of these functions are as follows: function y = fft_impl(x, f, forward) fwd = 1; if nargin >= 3 fwd = forward; end x = bit_reversal(x); [N, n] = size(x); y = zeros(N, n); for s2 = 1:size(x, 2) y(:, s2) = f(x(:,s2), fwd); end if ~fwd y = y/N; end end

% Compute the DFT of the column vector x using the FFT algorithm function y = fft_kernel_standard(x, forward) N = size(x, 1);

2 Digital sound and Discrete Fourier analysis

41

sign = -1; if ~forward sign = 1; end if N == 1 y = x; else xe = fft_kernel_standard(x(1:(N/2)), forward); xo = fft_kernel_standard(x((N/2+1):N), forward); D = exp(sign*2*pi*1j*(0:(N/2-1))’/N); xo = xo.*D; y = [ xe + xo; xe - xo]; end end

Exercise 2.25: Execution times for the FFT Let us compare execution times for the different methods for computing the DFT. a) Write code which compares the execution times for an N -point DFT for the following three cases: Direct implementation of the DFT (as in Example 2.4 in the book), the FFT implementation used in this chapter, and the built-in fft-function. Your code should use the sample audio file castanets.wav, apply the different DFT implementations to the first N = 2r samples of the file for r = 3 to r = 15, store the execution times in a vector, and plot these. You can use the functions tic and toc to measure the execution time.

Solution.

The code can look as follows. [x0, fs] = audioread(’sounds/castanets.wav’); kvals = 3:15; slowtime=zeros(1,length(kvals)); fasttime = slowtime; fastesttime = slowtime; N = 2.^kvals; for k = kvals x = x0(1:2^k,1); tic; y = dft_impl(x); slowtime(k-2) = toc; tic; y = fft_impl(x, @fft_kernel_standard); fasttime(k-2) = toc; tic; y = fft(x); fastesttime(k-2) = toc; end % a. plot(kvals, slowtime, ’r’, ... kvals, fasttime, ’g’, ... kvals, fastesttime, ’b’) grid on title(’time usage of the DFT methods’) legend(’DFT’, ’Standard FFT’, ’Built-in FFT’)

42

2 Digital sound and Discrete Fourier analysis

xlabel(’log_2 N’) ylabel(’time used [s]’) % b. figure(2) loglog(N, slowtime, ’r’, N, fasttime, ’g’, N, fastesttime, ’b’) axis equal legend(’DFT’, ’Standard FFT’, ’Built-in FFT’)

b) A problem for large N is that there is such a big difference in the execution times between the two implementations. We can address this by using a loglog-plot. Plot N against execution times using the function loglog. How should the fact that the number of arithmetic operations are 8N 2 and 5N log2 N be reflected in the plot?

Solution.

The two different curves you see should have a derivative approximately equal to one and two, respectively. c) It seems that the built-in FFT is much faster than our own FFT implementation, even though they may use similar algorithms. Try to explain what can be the cause of this.

Solution.

There may be several reasons for this. One is that MATLAB code runs slowly when compared to native code, which is much used in the built-in FFT. Also, the built-in fft has been subject to much more optimization than we have covered here. Another reason is that functions in MATLAB copy the parameters each time a function is called. When the vectors are large, this leads to extensive copying, also since the recursion depth is big.

Exercise 2.26: Combining two FFT’s Let x1 = (1, 3, 5, 7) and x2 = (2, 4, 6, 8). Compute DFT4 x1 and DFT4 x2 . Explain how you can compute DFT8 (1, 2, 3, 4, 5, 6, 7, 8) based on these computations (you don’t need to perform the actual computation). What are the benefits of this approach?

Solution.

We get

2 Digital sound and Discrete Fourier analysis

43

    16 1 1 1 1 1 1 −i −1 i  3 −4 + 4i     DFT4 x1 =  1 −1 1 −1 5 =  −4  −4 − 4i 7 1 i −1 −i      20 2 1 1 1 1 1 −i −1 i  4 −4 + 4i     DFT4 x2 =  1 −1 1 −1 6 =  −4  −4 − 4i 8 1 i −1 −i 

In the FFT-algorithm we split the computation of DFT4 (x) into the computation of DFT2 (x(e) ) and DFT2 (x(o) ), where x(e) and x(o) are vectors of length 4 with evenindexed and odd-indexed components, respectively. In this case we have x(e) = (1, 3, 5, 7) and x(o) = (2, 4, 6, 8). In other words, the FFT-algorithm uses the FFT-computations we first made, so that we can save computation. The benefit of using the FFT-algorithm is that we save computations, so that we end up with O(5N log2 N ) real arithmetic operations.

Exercise 2.27: FFT operation count When we wrote down the difference equation for the number of multiplications in the FFT algorithm, you could argue that some multiplications were not counted. Which multiplications in the FFT algorithm were not counted when writing down this difference equation? Do you have a suggestion to why these multiplications were not counted?

Solution.

When we compute e−2πin/N , we do some multiplications/divisions in the exponent. These are not counted because they do not depend on x, and may therefore be pre-computed.

Exercise 2.28: FFT algorithm adapted to real data It is possible to adapt an FFT algorithm to real input. Since yN −n = yn for real input, there is no additional complexity in computing the second half of the DFT coefficients, once the first half has been computed. We will now rewrite equation (2.11) in the book for indices n and N/2 − n as yn = (DFTN/2 x(e) )n + e−2πin/N (DFTN/2 x(o) )n yN/2−n = (DFTN/2 x(e) )N/2−n + e−2πi(N/2−n)/N (DFTN/2 x(o) )N/2−n = (DFTN/2 x(e) )n − e2πin/N (DFTN/2 x(o) )n = (DFTN/2 x(e) )n − e−2πin/N (DFTN/2 x(o) )n .

We see here that, if we already have computed DFTN/2 x(e) and DFTN/2 x(o) , we need one additional complex multiplication for each yn with 0 ≤ n < N/4 (since e−2πin/N

44

2 Digital sound and Discrete Fourier analysis

and (DFTN/2 x(o) )n are complex). No further multiplications are needed in order to compute yN/2−n , since we simply conjugate terms before adding them. Again yN/2 must be handled explicitly with this approach. For this we can use the formula yN/2 = (DFTN/2 x(e) )0 − (DN/2 DFTN/2 x(o) )0 instead. a) Conclude from this that an FFT algorithm adapted to real data at each step requires N/4 complex additions and N/2 complex additions. Conclude from this as before that an algorithm based on real data requires MN = O(N log2 N ) real multiplications and AN = O 32 N log2 N real additions (i.e. half the operation count for complex input).

Solution.

The formulas at the top show that we need only compute the e−2πin/N (DFTN/2 x(o) )n for 0 ≤ n < N/4, i.e. N/4 complex additions. Since N/2 formulas are listed, N/2 complex additions are needed. This translates to N real multiplications, and N/2 + N = 3N/2 real additions. These are exactly half the numbers we obtained for the complex version of the algorithm, for which we obtained MN = O(2N log2 (N )) and AN = O(3N log2 (N )).

It is then not difficult to see that the real version will require MN = O(N log2 N ) real multiplications and AN = O 32 N log2 N real additions. b) Find an IFFT algorithm adapted to vectors with conjugate symmetry, which has the same operation count as this FFT algorithm adapted to real data.

Hint.

Consider the vectors z, w with entries zn = yn + yN/2−n ∈ CN/2 and wn = e2πin/N (yn − yN/2−n ) ∈ CN/2 . From the equations above, how can these be used in an IFFT?

Solution.

Note that

zN/2−n = yN/2−n + yn = yn + yN/2−n = zn wN/2−n = e2πi(N/2−n)/N (yN/2−n − yn ) = −e−2πin/N (yN/2−n − yn ) = e2πin/N (yn − yN/2−n ) = wn , showing that the vectors z and w also have conjugate symmetry. Note also that, if we add and subtract the equations at the top, we obtain 2(DFTN/2 x(e) )n = yn + yN/2−n = zn 2(DFTN/2 x(o) )n = e2πin/N (yn − yN/2−n ) = wn . so that

2 Digital sound and Discrete Fourier analysis

x(e) =

1 IDFTN/2 z 2

45

x(o) =

1 IDFTN/2 w.

2

The point is that the number of arithmetic operations in computing the vectors z and w is the same as the FFT in a), so that the overall complexity is the same.

Exercise 2.29: Non-recursive FFT algorithm Use the factorization in (2.16) in the book to write a kernel function for a non-recursive FFT implementation. In your code, perform the matrix multiplications in equation (2.16) in the book from right to left in an (outer) for-loop. For each matrix loop through the different blocks on the diagonal in an (inner) for-loop. Make sure you have the right number of blocks on the diagonal, each block being on the form

I DN/2k .

I −DN/2k It may be a good idea to start by implementing multiplication with such a simple matrix first as these are the building blocks in the algorithm. Do this so that everything is computed in-place. Also compare the execution times with our original FFT algorithm, as we did in Exercise 2.25, and try to explain what you see in this comparison.

Solution.

The library version of the non-recursive FFT is as follows function y = fft_kernel_nonrec(x, forward) N = size(x, 1); sign = -1; if ~forward sign = 1; end D = exp(sign*2*pi*1i*(0:(N/2 - 1))’/N); nextN = 1; while nextN < N k = 1; while k <= N xe = x(k:(k + nextN - 1)); xo = x((k + nextN):(k + 2*nextN - 1)); xo = xo.*D(1:(N/(2*nextN)):(N/2)); x(k:(k + 2*nextN - 1)) = [xe + xo; xe - xo]; k = k + 2*nextN; end nextN = nextN*2; end y = x; end

If you add the non-recursive algorithm to the code from Exercise 2.25, you will see that the non-recursive algorithm performs much better. There may be several reasons for this. First of all, there are no recursive function calls. Secondly, the values in the matrices DN/2 are constructed once and for all with the non-recursive algorithm. Code which compares execution times for the original FFT algorithm, our non-recursive implementation, and the split-radix algorithm of the next exercise, can look as follows:

46

2 Digital sound and Discrete Fourier analysis

[x0, fs] = audioread(’sounds/castanets.wav’); kvals = 3:15; slowtime = zeros(1,length(kvals)); fasttime = slowtime; fastesttime = slowtime; N = 2.^(kvals); for k = kvals x = x0(1:2^k,1); tic; y = fft_impl(x, @fft_kernel_standard); slowtime(k-2) = toc; tic; y = fft_impl(x, @fft_kernel_nonrec); fasttime(k-2) = toc; tic; y = fft_impl(x, @fft_kernel_splitradix); fastesttime(k-2) = toc; end plot(kvals, slowtime, ’ro-’) hold on plot(kvals,fasttime, ’bo-’) plot(kvals,fastesttime, ’go-’) grid on title(’time usage of the DFT methods’) legend(’Standard FFT algorithm’, ... ’Non-recursive FFT’, ... ’Split radix FFT’) xlabel(’log_2 N’) ylabel(’time used [s]’)

Exercise 2.30: The Split-radix FFT algorithm The split-radix FFT algorithm is a variant of the FFT algorithm. Until recently it held the record for the lowest operation count for any FFT algorithm. It was first derived in 1968 [6], but largely forgotten until a paper in 1984 [2].

We start by splitting the rightmost DFTN/2 in equation (2.15) in the book by using this equation again, to obtain 

  (e)  DFTN/4 DN/4 DFTN/4 x DFT D N/2 N/2   DFTN/4 −DN/4 DFTN/4  x(oe)  .

DFTN x =   DFTN/4 DN/4 DFTN/4  DFTN/2 −DN/2 x(oo) DFTN/4 −DN/4 DFTN/4 The term radix describes how an FFT is split into FFT’s of smaller sizes, i.e. how the sum in an FFT is split into smaller sums. The FFT algorithm we started this section with is called a radix 2 algorithm, since it splits an FFT of length N into FFT’s of length N/2. If an algorithm instead splits into FFT’s of length N/4, it is called a radix 4 FFT algorithm. The algorithm we go through here is called the split radix algorithm, since it uses FFT’s of both length N/2 and N/4. −2πin/N a) Let GN/4 be the on the diagonal.

(N/4) × (N/4) diagonal matrix with e GN/4 0 Show that DN/2 = .

0 −iGN/4

2 Digital sound and Discrete Fourier analysis

47

Solution.

We have that   −2πi·0/N e ··· 0 0 ··· 0 .. .. .. ..   .. ..   . . . . . . 

   . .. −2πi(N/4−1)/N GN/4 0   0 · · · e 0 · · · =  −2πi·0/N 0 −iGN/4   0 · · · 0 −ie · · · 0     . . . . . . .. .. .. .. .. ..   −2πi(N/4−1)/N 0 ··· 0 0 · · · −ie  −2πi·0/N  e ··· 0 0 ··· 0 .. .. .. ..   .. ..   . . . . . .     . .. −2πi(N/4−1)/N   0 · · · e 0 · · ·   =  . . −2πi(N/4)/N   0 .

0 e ··· 0     .. .. .. .. .. ..   . . . . . .

0

···

0

0

· · · e−2πi(N/2−1)/N

= DN/2 . b) Let HN/4 be the (N/4) × (N/4) diagonal matrix GD/4 DN/4 . Verify the following rewriting of the equation above: 

  (e)  GN/4 DFTN/4 HN/4 DFTN/4 DFTN/2 −iGN/4 DFTN/4 iHN/4 DFTN/4  x

 x(oe)  DFTN x =   −GN/4 DFTN/4 −HN/4 DFTN/4  DFTN/2 x(oo) iGN/4 DFTN/4 −iHN/4 DFTN/4      (e)  I 0 GN/4 HN/4 x 0 0 0 I −iGN/4 iHN/4  DFTN/2   x(oe)  0 DFT 0 = N/4 I 0 −GN/4 −HN/4  0 0 DFTN/4 x(oo) 0 I iGN/4 −iHN/4     GN/4 HN/4 (e) I  DFTN/2 x −iG iH N/4 N/4

 DFTN/4 x(oe)  =  GN/4 HN/4  I− DFTN/4 x(oo) −iGN/4 iHN/4

  GN/4 DFTN/4 x(oe) + HN/4 DFTN/4 x(oo) (e) DFTN/2 x + −i GN/4 DFTN/4 x(oe) − HN/4 DFTN/4 x(oo) 

 =  GN/4 DFTN/4 x(oe) + HN/4 DFTN/4 x(oo)  (e) DFTN/2 x − −i GN/4 DFTN/4 x(oe) − HN/4 DFTN/4 x(oo)

Solution.

In the first step we write

48

2 Digital sound and Discrete Fourier analysis

GN/4 0 DFTN/4 DN/4 DFTN/4 DFTN/4 −DN/4 DFTN/4 0 −iGN/4

GN/4 DFTN/4 GN/4 DN/4 DFTN/4 = −iGN/4 DFTN/4 iGN/4 DN/4 DFTN/4

GN/4 DFTN/4 HN/4 DFTN/4 = −iGN/4 DFTN/4 iHN/4 DFTN/4

DN/2 DFTN/2 =

c) Explain from the above expression why, once the three FFT’s above have been computed, the rest can be computed with N/2 complex multiplications, and 2×N/4+N = 3N/2 complex additions. This is equivalent to 2N real multiplications and N + 3N = 4N real additions.

Hint.

It is important that GN/4 DFTN/4 x(oe) and HN/4 DFTN/4 x(oo) are computed first, and the sum and difference of these two afterwards.

Solution.

Computing GN/4 DFTN/4 x(oe) and HN/4 DFTN/4 x(oo) each requires N/4 complex multiplications, i.e. a total of N/2 complex multiplications. Computing GN/4 DFTN/4 x(oe) + HN/4 DFTN/4 x(oo) needs an additional N/4 complex additions, and the same for GN/4 DFTN/4 x(oe) − HN/4 DFTN/4 x(oo) , so a total of N/2 complex additions. Finally we need N more complex additions, since we finally compute DFTN/2 x(e) plus/minus entries from these matrices. d) Due to what we just showed, our new algorithm leads to real multiplication and addition counts which satisfy

MN = MN/2 + 2MN/4 + 2N

AN = AN/2 + 2AN/4 + 4N

Find the general solutions and conclude from these to these difference equations

that MN = O 43 N log2 N , and AN = O 83 N log2 N . The operation count is thus O (4N log2 N ), which is a reduction of N log2 N from the FFT algorithm.

Solution.

With xr = M2r (or A2r ) we obtain the difference equations xr+2 − xr+1 − 2xr = 8 · 2r (h) (or 16 · 2r ). The general solution to the homogeneous equation is xn = A2r + B(−1)r , (p) (p) and a particular solution to the general equation is xn = 43 r2r (or xn = 83 r2r ). The general solutions are thus xr = 43 r2r + A2r + B(−1)r (or xr = 83 r2r + A2r + B(−1)r ).

Since the first terms here are the ones which are dominating, setting 2r = N we obtain

4 8 that MN = O 3 N log2 N , and AN = O 3 N log2 N . e) Write a kernel function for the split-radix FFT algorithm (again this should handle both forward and reverse transforms). Are there more or less recursive function calls in this function than in the original FFT algorithm? Also compare the execution times

2 Digital sound and Discrete Fourier analysis

49

with our original FFT algorithm, as we did in Exercise 2.25. Try to explain what you see in this comparison.

Solution.

The library version of the split-radix algorithm is as follows function y = fft_kernel_splitradix(x, forward) N = size(x, 1); sign = -1; if ~forward sign = 1; end if N == 1 y = x; elseif N == 2 y = [x(1) + x(2); x(1) - x(2)]; else xe = fft_kernel_splitradix(x(1:(N/2)), forward); xo1 = fft_kernel_splitradix(x((N/2 + 1):(3*N/4)), forward); xo2 = fft_kernel_splitradix(x((3*N/4 + 1):N), forward); G = exp(sign*2*pi*1j*(0:(N/4-1))’/N); H = G.*exp(sign*2*pi*1j*(0:(N/4-1))’/(N/2)); xo1 = G.*xo1; xo2 = H.*xo2; xo = [xo1 + xo2; -sign*1i*(xo2 - xo1)]; y = [xe + xo; xe - xo]; end end

If you add the split-radix FFT algorithm also to the code from Exercise 2.25, you will see that it performs better than the FFT algorithm, but worse than the non-recursive algorithm. That it performs better than the FFT algorithm is as expected, since it has a reduced number of arithmetic operations, and also a smaller number of recursive calls.

It is not surprising that the non-recursive function performs better, since only that function omits recursive calls, and computes the values in the diagonal matrices once and for all.

By carefully examining the algorithm we have developed, one can reduce the operation count to 4N log2 N − 6N + 8. This does not reduce the order of the algorithm, but for small N (which often is the case in applications) this reduces the number of operations considerably, since 6N is large compared to 4N log2 N for small N . In addition to having a lower number of operations than the FFT algorithm of Theorem 14 in the book, a bigger percentage of the operations are additions for our new algorithm: there are now twice as many additions than multiplications. Since multiplications may be more time-consuming than additions (depending on how the CPU computes floating-point arithmetic), this can be a big advantage.

Exercise 2.31: Bit-reversal In this exercise we will make some considerations which will help us explain the code for bit-reversal. This is not a mathematically challenging exercise, but nevertheless a good exercise in how to think when developing an efficient algorithm. We will use the notation i for an index, and j for its bit-reverse. If we bit-reverse k bits, we will write N = 2k for the number of possible indices.

50

2 Digital sound and Discrete Fourier analysis

a) Consider the following code j = 0; for i = 0:(N-1) j m = N/2; while (m >= 1 && j >= m) j = j - m; m = m/2; end j = j + m; end

Explain that the code prints all numbers in [0, N − 1] in bit-reversed order (i.e. j). Verify this by running the program, and writing down the bits for all numbers for, say N = 16.

In particular explain the decrements and increments made to the variable j. The code above thus produces pairs of numbers (i, j), where j is the bit-reverse of i. The function bit_reversal in the library applies similar code, and then swaps the values xi and xj in x, as it should.

Solution.

Note that, if the bit representation of i ends with 0 1| .{z . . 1}, then i + 1 has a bit repn

resentation which ends with 1 0| .{z . . 0}, with the remaining first bits unaltered. Clearly n

the bit-reverse of i then starts with 1| .{z . . 1} 0 and the bit-reverse of i + 1 starts with n

10 . . 0}. We see that the bit reverse of i + 1 can be obtained from the bit-reverse of i by | .{z n

replacing the first consecutive set of ones by zeros, and the following zero by one. This is performed by the line above where j is decreased by m: Decreasing j by N/2 when j ≥ N/2 changes the first bit from 1 to 0, and similarly for the next n bits. The line where j is increased with m changes bit number n + 1 from 0 to 1.

Since bit-reverse is its own inverse (i.e. P 2 = I), it can be performed by swapping elements i and j. One way to secure that bit-reverse is done only once, is to perform it only when j > i. bit_reversal makes this check. b) Explain that N − j − 1 is the bit-reverse of N − i − 1. Due to this, when i, j < N/2, we have that N − i − 1, N − j − l ≥ N/2, and that bit_reversal can swap them.

Moreover, all swaps where i, j ≥ N/2 can be performed immediately when pairs where i, j < N/2 are encountered. Explain also that j < N/2 if and only if i is even. In the code you can see that the swaps (i, j) and (N − i − 1, N − j − 1) are performed together when i is even, due to this.

Solution.

Clearly N − i − 1 has a bit representation obtained by changing every bit in i. That N − j − 1 is the bit-reverse of N − i − 1 follows immediately from this. If i is even, the least significant bit is 0. After bit-reversal, this becomes the most significant bit, and the most significant bit of j is 0 which is the case if and only if j < N/2.

2 Digital sound and Discrete Fourier analysis

51

c) Assume that i < N/2 is odd. Explain that j ≥ N/2, so that j > i. This says that when i < N/2 is odd, we can always swap i and j (this is the last swap performed in the code). All swaps where 0 ≤ j < N/2 and N/2 ≤ j < N can be performed in this way.

Solution.

If i < N/2 is odd, then the least significant bit is 1. This means that the most significant bit of j is 1, so that j ≥ N/2, so that j > i.

In bit_reversal, you can see that the bit-reversal of 2r and 2r + 1 are handled together (i.e. i is increased with 2 in the for-loop). The effect of this is that the number of if-tests can be reduced, due to the observations from b) and c).

Exercise 2.33: Writing down lower order DCT’s As in Example 2.32 in the book, state the exact Cartesian form of the DCT matrix for the case N = 3.

Solution.

We first see that d0,3 =

q

1 3 and dk,3 =

q

2 3 for k = 1, 2. We also have that

n 1 n 1 cos 2π k+ = cos π k+ , 2N 2 3 2 so that the DCT matrix can be written as 

q

q

1 3

1 3

q

1 3



q q2 q2   π3 π5  DCT3 =  23 cos π3 21  cos cos 3 2 3 2  q q3 q3

2 2 2 2π 1 2π 3 2π 5 3 cos 3 2 3 cos 3 2 3 cos 3 2 q q q   1

1

1

3 3 3 q  q q  2  2 2 =  3 cos(π/6) 3 cos(π/2) 3 cos(5π/6) q  q q 2 2 2 cos(π/3) cos(π) cos(5π/3) 3 3 3 q q q   1

1

1

3 q 3  q √ 3 √   2 =  23 ( 3/2 + i/2) 0 (− 3/2 + i/2) 3  q q q √ √ 2 2 2 (1/2 + 3i/2) − (1/2 − 3i/2) 3 3 3

52

2 Digital sound and Discrete Fourier analysis

Exercise 2.34: Type-IV DCT N −1

Show that the vectors {cos(2π(n + 1/2)(k + 1/2)/(2N ))}n=0qin RN are orthogonal, p with lengths N/2. This means that the matrix with entries N2 cos(2π(n + 1/2)(k + 1/2)/(2N )) is orthogonal. Since this matrix also is symmetric, it is its own inverse. This (IV) is the matrix DCTN , which can be used for type IV cosine matrices.

Hint.

Compare with the orthogonal vectors dn , used in the DCT.

Solution.

We can write

n + 21 1 2n + 1 1 cos 2π k+ = cos 2π k+ .

2N 2 4N 2 If we consider these as vectors of length 2N , we recognize these p as the unit p vectors d2n+1 in the 2N -dimensional DCT, divided by the factor d = 2/(2N ) = 1/N , so n √ that these vectors have length N . To see that these vectors are orthogonal when we restrict to the first N elements using the property cos(2π(n + 1/2)(2N − k)/(2N )) = − cos(2π(n + 1/2)k/(2N )) as follows: 2N −1 X

n1 + 12 n2 + 21 1 1 cos 2π k+ cos 2π k+ 2N 2 2N 2 k=0

N −1 X n1 + 21 n2 + 21 1 1 cos 2π = k+ cos 2π k+ 2N 2 2N 2 k=0

2N −1 X n1 + 21 n2 + 21 1 1 + cos 2π k+ cos 2π k+ 2N 2 2N 2 k=N

N −1 X n2 + 21 n1 + 12 1 1 k+ cos 2π k+ = cos 2π 2N 2 2N 2 k=0

N −1 X n1 + 21 n2 + 21 1 1 + cos 2π k+ cos 2π k+ 2N 2 2N 2 k=0

N −1 X n2 + 21 n1 + 21 1 1 =2 cos 2π k+ cos 2π k+ .

2N 2 2N 2

N δn1 ,n2 =

k=0

This shows that

2 Digital sound and Discrete Fourier analysis

53

N −1 X

n1 + 12 n2 + 21 1 1 N cos 2π k+ cos 2π k+ = δn1 ,n2 , 2N 2 2N 2 2 k=0 p so that the vectors are orthogonal with lengths N/2.

Exercise 2.35: Modified Discrete Cosine Transform The Modified Discrete Cosine Transform (MDCT) is defined as the N × (2N )-matrix M with elements Mn,k = cos(2π(n + 1/2)(k + 1/2 + N/2)/(2N )). This exercise will take you through the details of multiplication with this matrix. a) Show that r M=

N 0 A (IV) DCTN B 0 2

where A and B are the (N/2) × N -matrices 

 · · · · · · 0 −1 −1 0 · · · · · ·  .. .. .. .. .. .. .. .. 

  A =  . . . . . . . .  = −(IN/2 )↔ −IN/2  0 −1 · · · · · · · · · · · · −1 0  −1 0 · · · · · · · · · · · · 0 −1   1 0 · · · · · · · · · · · · 0 −1  0 1 · · · · · · · · · · · · −1 0 

  B =  . . . . . . . .  = IN/2 −(IN/2 )↔ ,  .. .. .. .. .. .. .. ..  · · · · · · 0 1 −1 0 · · · · · · where where A↔ is the matrix A with the columns reversed. Due to this, any Type-IV DCT algorithm can be used to compute the MDCT.

Solution.

(IV)

Clearly, columns 0, . . . , N/2 − 1 of the MDCT are columns N/2, . . . , N − 1 of DCTN .

For the remaining columns, note first that, for 0 ≤ k < N , the properties cos(2π(n + 1/2)((2N − 1 − k) + 1/2)/(2N )) = − cos(2π(n + 1/2)(k + 1/2)/(2N )) cos(2π(n + 1/2)((k + 2N ) + 1/2)/(2N )) = − cos(2π(n + 1/2)(k + 1/2)/(2N )) are easy to verify. From the first property it follows that columns N/2, . . . , 3N/2−1 of the (IV) MDCT are columns N − 1, N − 2, . . . , 0 of the DCTN , with a sign change (they occur in opposite order). From the second property, it follows that columns 3N/2, . . . , 2N − 1 (IV) of the MDCT are columns 0, . . . , N/2 − 1 of DCTN , with a sign change. This means that, if y is a vector of length 2N , the MDCT of y can be written as

54

2 Digital sound and Discrete Fourier analysis

 −y3N/2 − y3N/2−1 −y3N/2+1 − y3N/2−2      ..   . r    N −y − y (IV)  2N −1 N .  DCTN   y − y 2 0 N −1     y − y 1 N/2+1     ..   . yN/2−1 − yN/2 

The factor

q

N 2

was added since

q

2 N

was added in front of the cosine-matrix in order to

0 A (IV) make DCTN orthogonal. The result now follows by noting that we can write y B 0 for the vector on the right hand side, with A and B as defined in the text of the exercise. b) The MDCT is singular, since it is not a square matrix. We will show here that it still can be used in connection with non-singular transformations. We first define the IMDCT as the matrix M T /N . Transposing what we obtained in a) gives

√

1 2N

0 BT AT 0

(IV)

DCTN

for the IMDCT, which thus also has an efficient implementation. Show that if x0 = (x0 , . . . , xN −1 )

x1 = (xN , . . . , x2N −1 )

x2 = (x2N , . . . , x3N −1 )

and

y0,1 = M

x0 x1

y1,2 = M

x1 x2

(i.e. we compute two MDCT’s where half of the data overlap), then 2N −1 N −1 x1 = {IMDCT(y0,1 )}k=N + {IMDCT(y1,2 )}k=0 .

Thus, even though the MDCT itself is singular, the input can still be recovered from two overlapping IMDCT’s.

Solution.

Applying the MDCT first, and then the IMDCT, gives us the matrix 1 2 Note that

0 BT AT 0

0 A B 0

=

1 2

BT B 0 0 AT A

2 Digital sound and Discrete Fourier analysis

AT A =

IN/2 (IN/2 )↔ (IN/2 )↔ IN/2

55

BT B =

IN/2 −(IN/2 )↔ . −(IN/2 )↔ IN/2

Inserting this in the above gives  IN/2 −(IN/2 )↔ 0 0

↔ 1 IN − (IN )↔ 1 IN/2 0 0  0 −(IN/2 )  = .

0 0 IN/2 (IN/2 )↔  2 0 IN + (IN )↔ 2 0 0 (IN/2 )↔ IN/2 

x0 x1

Assume now that we have computed the MDCT of = (x0 , . . . , x2N −1 ), and of

x1 = (xN , . . . , x3N −1 ). Performing the IMDCT on these two thus gives x2

1 x0 − xrev 1 x1 − xrev 0 1 and 2 x1 + xrev 2 x2 + xrev 1 2 Adding the second component of the first and the first component of the second gives x1 , which proves the result.

Exercise 2.36: Trick for reducing the number of multiplications with the DCT In this exercise we will take a look at a small trick which reduces the number of multiplications we need for DCT algorithm from Theorem 27 in the book. This exercise does not reduce the order of the DCT algorithms, but we will see in Exercise 2.37 how the result can be used to achieve this. a) Assume that x is a real signal. Equation (2.26) in the book said that n n yn = cos π <((DFTN x(1) )n ) + sin π =((DFTN x(1) )n ) 2N 2N n n <((DFTN x(1) )n ) − cos π =((DFTN x(1) )n ) yN −n = sin π 2N 2N for the n’th and N − n’th coefficient of the DCT. This can also be rewritten as

n yn = <((DFTN x(1) )n ) + =((DFTN x(1) )n ) cos π 2N n n (1) − =((DFTN x )n )(cos π − sin π ) 2N 2N

n yN −n = − <((DFTN x(1) )n ) + =((DFTN x(1) )n ) cos π 2N n n (1) + <((DFTN x )n )(sin π + cos π ).

2N 2N Explain that the first two equations require 4 multiplications to compute yn and yN −n , and that the last two equations require 3 multiplications to compute yn and yN −n .

56

2 Digital sound and Discrete Fourier analysis

Solution.

In the last two equations, the first summand is the same, so that one multiplication is needed. Each of the other lines require one more multiplication, so that we end up with a total of three. That the first two equations require 4 multiplications is obvious. b) Explain why the trick in a) reduces the number of additional multiplications in a DCT from 2N to 3N/2.

Solution.

Since there are N/2 pairs on the form (yn , yN −n ), the number of multiplications is reduced from 4 · N/2 = 2N to 3 · N/2 = 3N/2. c) Explain why the trick in a) can be used to reduce the number of multiplications in an IDCT with the same number.

Solution.

In matrix form the two first two equations above can be written

n n yn cos π 2N sin π 2N <((DFTN x(1) )n )

= . n n yN −n sin π 2N − cos π 2N =((DFTN x(1) )n ) This system is easily inverted to

n n yn <((DFTN x(1) )n ) cos π 2N sin π 2N

= . n n yN −n − cos π 2N sin π 2N =((DFTN x(1) )n ) Since this system has the same form, the same trick can be used to reduce the number of multiplications in an IDCT with the same number. After solving this system, an IDFT is applied to recover x(1) . d) Show that the penalty of the trick we here have used to reduce the number of multiplications, is an increase in the number of additions from N to 3N/2. Why can this trick still be useful?

Solution.

The first two equations used 2 additions, while the next two used 3. The trick in this exercise therefore does not change the total number of operations. This trick may still be useful, however, if the CPU spends more time on computing a multiplication than an addition.

2 Digital sound and Discrete Fourier analysis

57

Exercise 2.37: An efficient joint implementation of the DCT and the FFT In this exercise we will explain another joint implementation of the DFT and the DCT, which has the benefit of a lower multiplication count, at the expense of a higher addition count. It also has the benefit that it is specialized to real vectors, with a very structured implementation (this may not be the case for the quickest FFT implementations, as one often sacrifices clarity of code in pursuit of higher computational speed). a) of this exercise can be skipped, as it is difficult and quite technical. For further details of the algorithm the reader is referred to [5]. a) Let y = DFTN x be the N -point DFT of the real vector x. Show that  0 ≤ n ≤ N/4 − 1  <((DFTN/2 x(e) )n ) + (CN/4 z)n <(yn ) = <((DFTN/2 x(e) )n ) n = N/4  <((DFTN/2 x(e) )n ) − (CN/4 z)N/2−n N/4 + 1 ≤ n ≤ N/2 − 1  n=0  =((DFTN/2 x(e) )n ) =(yn ) = =((DFTN/2 x(e) )n ) + (CN/4 w)N/4−n 1 ≤ n ≤ N/4 − 1  =((DFTN/2 x(e) )n ) + (CN/4 w)n−N/4 N/4 ≤ n ≤ N/2 − 1 where x(e) is as defined in Theorem 14 in the book, where z, w ∈ RN/4 defined by zk = x2k+1 + xN −2k−1

0 ≤ k ≤ N/4 − 1,

k

0 ≤ k ≤ N/4 − 1,

wk = (−1) (xN −2k−1 − x2k+1 )

Explain from this how you can make an algorithm which reduces an FFT of length N to an FFT of length N/2 (on x(e) ), and two DCT’s of length N/4 (on z and w). We will call this algorithm the revised FFT algorithm.

Solution.

Taking real and imaginary parts in equation (2.11) in the book for the FFT algorithm we obtain

<(yn ) = < (DFTN/2 x(e) )n + <((DN/2 DFTN/2 x(o) )n

=(yn ) = = (DFTN/2 x(e) )n + =((DN/2 DFTN/2 x(o) )n , These equations explain the first terms <((DFTN/2 x(e) )n ) and =((DFTN/2 x(e) )n ) on the right hand sides in the equations for <(yn ) and =(yn ). It remains to rewrite < (DN/2 DFTN/2 x(o) )n and = (DN/2 DFTN/2 x(o) )n so that the remaining terms on the right hand sides can be seen. Let us first consider the equation for the real part with 0 ≤ n ≤ N/4 − 1. In this case we can write

58

2 Digital sound and Discrete Fourier analysis

<((DN/2 DFTN/2 x(o) )n )     N/2−1 N/2−1 X X 1 = < e−2πin/N (x(o) )k e−2πin(k+ 2 )/(N/2)  (x(o) )k e−2πink/(N/2)  = <  k=0

k=0 N/2−1

X n(k + 12 ) (o) = (x )k cos 2π N/2

k=0

N/4−1

=

X

(x

k=0

(o)

n(k + 21 ) )k cos 2π N/2

N/4−1

n(N/2 − 1 − k + 21 ) + (x )N/2−1−k cos 2π N/2 k=0 !

N/4−1 X n k + 12 (o) (o) = ((x )k + (x )N/2−1−k ) cos 2π N/2 k=0 !

N/4−1 X n k + 21 = zk cos 2π , N/2 X

(o)

k=0

where we have used that cos is periodic with period 2π, that cos is symmetric, and where z is the vector defined in the text of the theorem. When 0 ≤ n ≤ N/4 − 1 this can also be written as N/4−1

X

(CN/4 )n,k zk = (CN/4 z)n ,

k=0

This proves the first formula in the equation for <(yn ).

For N/4 + 1 ≤ n ≤ N/2 − 1, everything above is valid, except for that cos(2πn(k + 1/2)/(N/2)) are not entries in the matrix CN/4 , since n is outside the legal range of the indices. However, N/2 − n is now a legal index in CN/4 , and using that

!

N k + 21 n(k + 21 ) 2 −n cos 2π , = − cos 2π N/2 N/2 we arrive at −(CN/4 z)N/2−n instead, and this proves the third formula in the equation for <(yn ). For the case n = N4 all the cosine entries in the sum are zero, and this completes the proof for the equation for <(yn ).

For the imaginary part, using that sin is periodic with period 2π, and that sin is anti-symmetric, analogous calculations as above give N/4−1

=((DN/2 DFTN/2 x

(o)

)n ) =

X k=0

Using that

((x

(o)

)N/2−1−k − (x

(o)

n k + 21 )k ) sin 2π N/2

! .

2 Digital sound and Discrete Fourier analysis

59

n(k + 12 ) n(k + 21 ) (N/4 − n)(k + 21 ) π sin 2π = cos − 2π = cos 2π − kπ N/2 2 N/2 N/2

1 (N/4 − n)(k + 2 ) = (−1)k cos 2π , N/2 the above equation can be rewritten as N/4−1

X k=0

(N/4 − n)(k + 21 ) ((x(o) )N/2−1−k − (x(o) )k )(−1)k cos 2π N/2

N/4−1

=

X k=0

(N/4 − n)(k + 21 ) wk cos 2π , N/2

where w is the vector defined as in the text of the theorem. When n = 0 this is 0 since all the cosines entries are zero. When 1 ≤ n ≤ N/4 this is (CN/4 w)N/4−n , since cos(2π(N/4 − n)(k + 1/2)/(N/2)) are entries in the matrix CN/4 . This proves the second formula in the equation for =(yn ).

For N/4 ≤ n ≤ N/2 − 1 we can use that cos(2π(N/4 − n)(k + 1/2)/(N/2)) = cos(2π(n − N/4)(k + 1/2)/(N/2)), which is an entry in the matrix DCTN/4 as well, so that we get (CN/4 z)n−N/4 . This also proves the third formula in the equation for =(yn ), and the proof is done. a) says nothing about the coefficients yn for n > N2 . These are obtained in the same way as before through symmetry. a) also says nothing about yN/2 . This can be obtained with the same formula as in Theorem 14 in the book.

Let us now compute the number of arithmetic operations our revised algorithm needs. Denote by the number of real multiplications needed by the revised N -point FFT algorithm b) Explain from the algorithm in a) that

MN = 2(MN/4 + 3N/8) + MN/2

AN = 2(AN/4 + 3N/8) + AN/2 + 3N/2

Hint.

3N/8 should come from the extra additions/multiplications (see Exercise 2.36) you need to compute when you run the algorithm from Theorem 27 in the book for CN/4 . Note also that the equations in a) require no extra multiplications, but that there are six equations involved, each needing N/4 additions, so that we need 6N/4 = 3N/2 extra additions.

Solution.

Two of the six mentioned equations which add to the additions are the ones for computing zk and wk from the x. The other four come from the formulas for <(yn ) and =(yn ), except for the two formulas for n = 0 and n = N/4. MN/4 + 3N/8 and AN/4 + 3N/8 come from the number of operations needed to compute DCTN/4 by reducing it to a DF TN/4 as in the previous exercise. Two such reductions are needed.

60

2 Digital sound and Discrete Fourier analysis

c) Explain why xr = M2r is the solution to the difference equation xr+2 − xr+1 − 2xr = 3 × 2r , and that xr = A2r is the solution to xr+2 − xr+1 − 2xr = 9 × 2r . and show that the general solution to these are xr = 12 r2r + C2r + D(−1)r for multiplications, and xr = 32 r2r + C2r + D(−1)r for additions.

Solution.

Inserting N = 2r and xr = MN in MN = 2(MN/4 + 3N/8) + MN/2 gives xr = 2(xr−2 + 3 · 2r /8) + xr−1 .

Inserting r + 2 for r this is easily reduced to xr+2 − xr+1 − 2xr = 3 × 2r . The case for additions is deduced similarly. We see that the characteristic equations here are the same as those for the split radix-algorithm, so that for a particular solution we (p) should try xr = Ar2r here as well (since also the right hand side is a power with the same base as for split-radix). It is easily shown that, for multiplications the particular (p) (p) solution obtained is xr = 12 r2r , and for multiplications xr = 32 r2r . Inserting the 1 r r solution to the homogeneous equation gives xr = 2 r2 + C2 + D(−1)r for the number of multiplications, and xr = 23 r2r + C2r + D(−1)r for additions. d) Explain why, regardless of initial conditions to the difference equations, MN =

O 12 N log2 N and AN = O 32 N log2 N both for the revised FFT and the revised DCT. The total number of operations is thus O(2N log2 N ), i.e. half the operation count of the split-radix algorithm. The orders of these algorithms are thus the same, since we here have adapted to read data.

Solution.

Regardless of the initial conditions in c), i.e. of C and D, r2r will be the dominating term in the solution. Since r2r = N log2 (N ), the result follows. The revised DCT has the same order as the revised FFT, since they differ only in 3N arithmetic operations. e) Explain that, if you had not employed the trick from Exercise 2.36, we would instead

have obtained MN = O 32 log2 N , and AN = O 43 log2 N , which equal the orders for the number of multiplications/additions for the split-radix algorithm. In particular, the order of the operation count remains the same, but the trick from Exercise 2.36 turned a bigger percentage of the arithmetic operations into additions.

Solution.

Without the trick we would instead have

MN = 2(MN/4 + N/2) + MN/2

AN = 2(AN/4 + N/4) + AN/2 + 3N/2

2 Digital sound and Discrete Fourier analysis

61

rather than the

MN = 2(MN/4 + 3N/8) + MN/2

AN = 2(AN/4 + 3N/8) + AN/2 + 3N/2

from before, and solving these as above gives the stated solution. But note that for both the total operation count RN = MN + AN satisfies RN = 2(RN/4 + 3N/4) + RN/2 + 3N/2 so the total number of operations is of the same order.

The algorithm we here have developed is constructed from the beginning to apply for real data only. Another advantage of the new algorithm is that it can be used to compute both the DCT and the DFT.

Exercise 2.38: Implementation of the IFFT/IDCT In Exercise 2.37 we did not write down corresponding algorithms for the revised IFFT and IDCT algorithms. Use the equations for <(yn ) and =(yn ) in that exercise to show that a) <(yn ) − <(yN/2−n ) = 2(CN/4 z)n =(yn ) + =(yN/2−n ) = 2(CN/4 w)N/4−n for 1 ≤ n ≤ N/4 − 1. Explain how one can compute z and w from this using two IDCT’s of length N/4.

Solution.

For 1 ≤ n ≤ N/4 − 1 the formulas in Exercise 2.37 said that <(yn ) = <((DFTN/2 x(e) )n ) + (CN/4 z)n <(yN/2−n ) = <((DFTN/2 x(e) )N/2−n ) − (CN/4 z)n =(yn ) = =((DFTN/2 x(e) )n ) + (CN/4 w)N/4−n =(yN/2−n ) = =((DFTN/2 x(e) )N/2−n ) + (CN/4 w)N/4−n .

Subtracting the second equation from the first and using conjugate symmetry in DFTN/2 x(e) gives the first equation. Adding the two last equations and using the same conjugate symmetry gives the second equation.

In an IFFT the yn are the known quantities, so that the values of the right hand sides are known. z and w can then be computed with the help of IDCT’s of length N/4. b)

62

2 Digital sound and Discrete Fourier analysis

<(yn ) + <(yN/2−n ) = 2<((DFTN/2 x(e) )n ) =(yn ) − =(yN/2−n ) = 2=((DFTN/2 x(e) )n ), and explain how one can compute x(e) from this using an IFFT of length N/2.

Solution.

Follows in the same way by adding the two first equations, and subtracting the fourth equation from the third. As above DFTN/2 x(e) is known since the yn are known, so that x(e) can be computed with the help of an N/2-point IFFT. Combining a) and b), z, w, and x(e) can be computed with the help of two length N/4 IDCT’s and one N/2-point IFFT. It is straightforward to reconstruct x from z, w, and x(e) .

Chapter 3

Discrete time filters

Exercise 3.2: Some linear operations which are not filters a) In Example 1.2 in the book we looked at time reversal as an operation on digital sound. In RN this can be defined as the linear mapping which sends the vector ek to eN −1−k for all 0 ≤ k ≤ N − 1. Write down the matrix for time reversal, and explain from this why time reversal is not a filter.

Solution.

The matrix for time reversal is the matrix   0 0 ··· 0 1 0 0 · · · 1 0    .. .. .. .. ..  . . . . .   0 1 · · · 0 0 1 0 ··· 0 0 This is not a circulant Toeplitz matrix, since all diagonals assume the values 0 and 1, so that they are not constant on each diagonal. Time reversal is thus not a filter. b) Consider the linear mapping S which keeps every second component in RN , i.e. S(e2k ) = e2k , and S(e2k−1 ) = 0. Write down the matrix for this, and explain why this operation is not a filter.

Solution.

The matrix for the operation which keeps every second component is   1 0 ··· 0 0 0 0 · · · 0 0    .. .. .. .. ..  . . . . . ,   0 0 · · · 1 0 0 0 ··· 0 0

63

64

3 Discrete time filters

where 1 and 0 are repeated in alternating order along the main diagonal. Since the matrix is not constant on the main diagonal, it is not a circulant Toeplitz matrix, and hence not a filter.

Exercise 3.3: Eigenvectors and eigenvalues Consider the matrix   4131 1 4 1 3  S= 3 1 4 1 .

1314 a) Compute the eigenvalues and eigenvectors of S using the results of this section. You should only need to perform one DFT in order to achieve this.

Solution.

The eigenvalues of S are 1, 5, 9, and are found by computing a DFT of the first column.

The eigenvectors are the Fourier basis vectors. 1 has multiplicity 2. b) Verify the result from a) by computing the eigenvectors and eigenvalues the way you taught in your first course in linear algebra. This should be a much more tedious task.

Solution.

The characteristic equation will be a fourth degree polynomial. One usually would guess two of the roots, and then use polynomial division to find a second degree polynomial which can be solved. To find the eigenvectors corresponding to λ = 1, compute       3131 1 3 1 3 1010 1 3 1 3 0 −8 0 −8 0 1 0 1      S − λI =  3 1 3 1 ∼ 0 0 0 0  ∼ 0 0 0 0 .

1313 0 0 0 0 0000 This means that the general eigenvector for λ = 1 takes the form     −1 0 0 −1    x3   1  + x4  0  .

0 1 The two vectors here clearly give an orthonormal basis for the eigenspace for λ = 1.

Note that φ1 = 12 (i, −1, −i, 1) and φ3 = 12 (−i, −1, i, 1), and that the vectors above can be obtained by adding and subtracting these, so that φ1 and φ3 span the eigenspace of λ = 1. The computations for the other eigenvalues are similar. c) Use a computer to find the eigenvectors and eigenvalues of S also. For some reason some of the eigenvectors seem to be different from the Fourier basis vectors, which you would expect from the theory in this section. Try to find an explanation for this.

3 Discrete time filters

65

Solution.

The computer uses some numerical algorithm to find the eigenvectors. However, eigenvectors may not be unique, so you have no control on which eigenvectors it actually selects. In particular, here the eigenspace for λ = 1 has dimension 2, so that any linear combination of the two eigenvectors from this eigenspace also is an eigenvector.

Exercise 3.4: Connection between continuous time- and discrete time filters Let s be a continuous time filter with frequency response λs (ν), and assume that f ∈ VM,T (so that also g = s(f ) ∈ VM,T ). Let x = (f (0 · T /N ), f (1 · T /N ), . . . , f ((N − 1)T /N )) z = (g(0 · T /N ), g(1 · T /N ), . . . , g((N − 1)T /N )) be vectors of N = 2M + 1 uniform samples from f and g. Show that the operation S : x → z (i.e. the operation which sends the samples of the input to the samples of the output) is well-defined on RN , and corresponds to a filter with frequency response λS,n = λs (n/T ).

This explains how we can implement the restriction of a continuous time filter to the Fourier spaces, by means of sampling, a discrete time filter, and interpolation. This also states that any discrete time filter can be expressed in terms of a continuous time filter.

Solution.

With N = 2M + 1 we know that f ∈ VM,T is uniquely determined from x. This means that g = s(f ) also is uniquely determined from x, so that z also is uniquely determined from x. The operation S : x → z is therefore well-defined on RN .

Clearly also s(e2πint/T ) = λs (n/T )e2πint/T . Since the samples of e2πint/T is the vector 2πikn/N e , and the samples of λs (n/T )e2πint/T is λs (n/T )e2πikn/N , the vector e2πikn/N is an eigenvector of S with eigenvalue λs (n/T ). Clearly then S corresponds to a filter with frequency response satisfying λS (2πn/N ) = λs (n/T ).

Exercise 3.5: Replacing a circular convolution with a longer one Let s, x ∈ RN . For M ≥ 2N − 1 define the vectors s̃ = (s0 , s1 , ..., sN −1 , 0, ..., 0, s1 , ..., sN −1 )T x̃ = (x0 , x1 , ..., xN −1 , 0, ..., 0)T

66

3 Discrete time filters

in RM (in s̃, M − (2N − 1) zeros have been inserted in the middle). Show that the first N elements of s̃ ~ x̃ equal s ~ x, so that a circular convolution can be computed as part of a longer circular convolution. A good strategy may now be to choose M as a power of two, since circular convolution can be computed with the help of the DFT, and since there are efficient DFT algorithms when the number of points is a power of two.

Solution.

Circular convolution with s and s̃ correspond to N × N - and M × M circulant Toeplitz matrices. Since x̃ has zero entries after the first N entries, the result would follow if those two circulant Toeplitz matrices have equal upper left N × N corners. That this is the case is straightforward to see.

Exercise 3.6: Replacing a sparse circular convolution with shorter ones Let s, x ∈ RN , and assume that s has at most k nonzero entries, gathered in one segment. a) Show that there exists a s̃ ∈ RM +k−1 and a number a so that, for any r, (s ~ x)r,r+1...,r+M −1 = (s̃ ~ x̃)0,...,M −1 , where x̃ = (xa+r , xa+r+1 , ..., xa+r+M +k−2 ).

In other words, any set of M consecutive entries in s ~ x can be obtained from a circular convolution of size M + k − 1. Thus, a circular convolution of size N can be computed N as M circular convolutions of size M + k − 1.

Solution.

Denote by S the N × N circulant Toeplitz matrix with first column equal to s. Assume that the nonzero entries in the first row of S occur in the interval [a, a + k − 1]. Then the nonzero entries in rows r, ..., r + M − 1 of S are in the columns from a + r to a + r + M − 1 + k − 1 = a + r + M + k − 2, i.e. the interval a + [r, r + M + k − 2] (there are M + k − 1 such columns). With x̃ = (xa+r , xr+1 , ..., xa+r+M +k−2 ), and viewing the submatrix of S from rows r, ..., r + M − 1 and columns from a + [r, r + M + k − 2] as the first M rows of a (M + k − 1) × (M + k − 1)-circulant Toeplitz matrix S̃ with first column s̃, we obtain immediately that (s ~ x)r,r+1...,r+M −1 = (s̃ ~ x̃)0,...,M −1 . b) It is natural to implement the shorter circular convolutions in terms of the FFT. Use this strategy with M + k − 1 = 2r to find the number of operations needed to compute

3 Discrete time filters

67

N the M circular convolutions from a), and write a program which finds the r which gives the lowest number of operations. Create a plot where k is plotted against the number of operations corresponding to the optimal r, and also against the number of operations required by a direct implementation of circular convolution.

Solution.

Recall that an N -point split-radix FFT requires 4N log2 N −6N +8 arithmetic operations.

With M + k − 1 = 2r for N this gives (4r − 6)2r + 8 operations for each of the shorter convolutions. A 2r -point convolution implemented in terms of the FFT (or rather FFT, multiplication in frequency, and IFFT) thus requires 2((4r − 6)2r + 8) + 6 · 2r = (8r − 6)2r + 16 operations. The factor 2 comes from computing both an FFT and an IFFT, and the summand 6 · 2r comes from the multiplications in frequency (each complex multiplication requires 4 real multiplications and 2 real additions, a total of 6 operations for each of the 2r entries). In total forl all the mN/M (or rather dN/M e) shorter convolutions we

N thus need ((8r − 6)2r + 16) 2r −k+1 operations. In a direct implementation, for each of the N entries we need to make k complex multiplications and k − 1 complex additions.

Since each complex addition requires 2 real additions, this totals 6k + 2(k − 1) = 8k − 2 operations for each entry, so that the overall total is (8k − 2)N operations.

The following code finds the r which gives the minimum of ((8r −6)2r +16)/(2r −k +1) for a fixed k, and then stores that minimum value. This is then repeated for all the k, and compared with 8k − 2, as we deduced for a direct implementation.

kvals = 1:50; opt_vals = kvals; for k=kvals rvals = ceil(log2(k)):50; num_ops = ((8*rvals-6).*2.^rvals + 16)./(2.^rvals-k+1); opt_vals(k) = min(num_ops); end plot(kvals,8*kvals-2,kvals,opt_vals)

Although this code does not plot the optimal value for r, this value is very much needed as a guideline for implementations on how to split convolutions into shorter convolutions.

The reason that we iterate r from log2 k in the code is that k ≤ 2r secures that there is room for all the filter coefficients inside the vector s̃ used in the smaller convolution.

From the resulting plot we see that when the number of filter coefficients gets above 5, the FFT-based approach works better. When the number of filter coefficients is below this, a direct implementation of the filter should be chosen.

When the number of points in an FFT gets high, one usually not cares about the terms −6N + 8 from the exact operation count of the Split radix FFT, since they are not dominant terms. In this exercise these terms really mattered, however, since short FFT’s are in focus. If you had dropped the terms −6N + 8, you can verify that ((8r − 6)2r + 16)/(2r − k + 1) should be replaced by (8r + 6)2r /(2r − k + 1), and that a new plot shows that at least 8 filter coefficients are needed in order for the FFT-based approach to outperform the direct implementation.

68

3 Discrete time filters

Exercise 3.7: Dualities between circular convolution and multiplication Show that IDF TN (λ1 ◦ λ2 ) = (IDF TN λ1 ) ~ (IDF TN λ2 ) 1 DF TN (s1 ◦ s2 ) = (DF TN s1 ) ~ (DF TN s2 ) N IDF TN (λ1 ~ λ2 ) = N (IDF TN λ1 ) ◦ (IDF TN λ2 ) Note that there is a factor in front of the last two equations, so that one must be a bit careful here.

Solution.

Setting λ1 = DF TN s1 , λ1 = DF TN s2 , and taking the IDFT on both sides in DF TN (s1 ~ s2 ) = (DF TN s1 ) ◦ (DF TN s2 ), it follows that (IDF TN λ1 ) ~ (IDF TN λ2 ) = IDF TN (λ1 ◦ λ2 ), which is the first equation. Conjugating both sides in this equation, and using that IDF TN = N1 DF TN , we obtain that 1 1 (DF TN λ1 ) ~ (DF TN λ2 ) = DF TN (λ1 ◦ λ2 ).

N2 N Conjugating again gives the second equation. The last equation follows by taking the DFT on both sides of this.

Exercise 3.8: Finding sparse solutions to under-determined linear systems Let x ∈ CN be a vector where we know that at most s components are nonzero. When s is small such vectors are called sparse. Denote the indices of these nonzero components

by S, and let y = DF TN x. In this exercise we will find a procedure for finding x from the values y0 , ..., y2s−1 , i.e. we can recover all components of a sparse vector from an under-determined set of measurements. This procedure is also called a Reed-Solomon code. In some sense this parallels the sampling theorem (which in a similar way fills in the values between the samples), with an assumption on the highest frequency replaced by an assumption on sparsity. Many other results also exist on how to recover sparse vectors, and these results give rise to the field of compressed sensing. See [3] for a review on this field.

Q a) Let z be the vector with components zk = N1 n∈S 1 − e−2πin/N e2πik/N . Show that zk = 0 for k ∈ S, and also that x ◦ z = 0.

3 Discrete time filters

69

Solution.

This follows from that 1−e−2πik/N e2πik/N = 1−1 = 0 is a factor in zk when k ∈ S. Since xk = 0 when k 6∈ S, and zk = 0 when k ∈ S, we have for all k that (x ◦ z)k = xk zk = 0. b) Let w = DF TN z. Show that w0 = 1 and that wn = 0 for n > s. Conclude from this, that x ◦ z = 0, and due the previous exercise that w1 , w2 , ..., ws must fulfill the equations ys + w1 ys−1 + · · · + ws y0 = 0 ys+1 + w1 ys + · · · + ws y1 = 0 .. .. .. .. . . . . =0 y2s−1 + w1 y2s−2 + · · · + ws ys−1 = 0

Solution.

Ps If we multiply out the factors in zk we obtain 1 + r=1 cr e2πirk for some values cr .

PN Since also z = n=1 wn e2πikn/N by the definition of the DFT, we see that w0 = 1 and that wn = 0 for n > s. From the previous exercise, multiplication in time corresponds to convolution in frequency, so that y ~ w = 0. The stated equation system corresponds to (y ~ w)s = · · · = (y ~ w)2s−1 = 0. where we inserted w0 = 1, wn = 0 for n > s.

We now have s equations with s unknowns. By construction this has a solution, but there may not be a unique solution. In the last part of the exercise we will show how to find x, regardless of which solution we choose. c) Assume that v1 , v2 , ..., vs is any solution to the system in b), and extend this to a vector v ∈ RN by setting v0 = 1, vs+1 = · · · = vN −1 = 0. Show that (v ~ y)n = 0 for s ≤ n ≤ 2s − 1, and conclude from the previous exercise that (DF TN ((IDF TN v) ◦ x))n = 0 for s ≤ n ≤ 2s − 1.

Solution.

For s ≤ n ≤ 2s − 1 we have that (v ~ y)n =

N −1 X k=0

vk y(n−k) mod N = yn +

s X

vk y(n−k) mod N = 0.

k=1

Since DF TN ((IDF TN v) ◦ x) = N1 v ~ y from the previous exercise, the result follows.

The vector (IDF TN v) ◦ x has at most s nonzero components (since x has). If we take the columns from DF TN with indices in S, and the rows with indices between s and 2s − 1, the resulting s × s-matrix was shown in Exercise 2.17 to be non-singular. d) Explain that ((IDF TN v) ◦ x)n = 0 for n ∈ S, and conclude that (IDF TN v) ◦ x = 0.

70

3 Discrete time filters

Solution.

The stated non-singular matrix applied to the components of (IDF TN v) ◦ x from S produces zero. These components must thus also be zero, so that (IDF TN v) ◦ x have all components in S equal to 0. Since x is zero outside S, it follows that (IDF TN v) ◦ x = 0. e) In particular we must have that (IDF TN v)k = 0 for k ∈ S, so that N −1 s 1 X 1 X vn e2πikn/N = vn e2πikn/N = 0 N n=0 N n=0

for k ∈ S. This is a polynomial in e2πik/N of degree at most s. Explain how one can find the set S from this polynomial.

Solution.

Simply apply an IDFT, and choose S as the set from the result with smallest coefficients.

Due to roundoff errors the coefficients will not be exactly zero, however. f) Explain how you can recover x from y0 , ..., y2s−1 when the set S is known.

Solution.

The submatrix A from DFTN with columns from S, and rows from 0, ..., s−1 is again nonsingular. The values of x on S, denoted xs can be found by solving AxS = (y0 , ..., ys−1 ). g) Write a function recover_x(y, N)

which uses the procedure obtained by combining b), e), and f) to compute and return x ∈ RN under the assumption that x has at most s nonzero components, and that y contains the first 2s components in y = DF TN x. Test that the code works correctly on a vector x of your own choosing.

Solution.

The function can be implemented as follows. function x=recover_x(y, N) s= size(y,1)/2; A=zeros(s); for k=1:s A(:,s-k+1) = y(k:(k+s-1)); end b=-y((s+1):(2*s)); w=A\b; v = [1; w; zeros(N-(s+1),1)]; inds = find(abs(ifft(v)) <= 0.00001); x_S = exp(-2*pi*i*((0:(s-1))’)*(inds-1)’/N)\y(1:s); x = zeros(N,1); x(inds)=x_S; end

3 Discrete time filters

71

The following code test that the function manages to recover a sparse vector with only four nonzero integer entries, from the 8 first Fourier coefficients.

N=32; x = zeros(N,1); x([3,5,9,12])=[1,4,3,2]; y=fft(x); x=recover_x(y(1:8),N)

Exercise 3.9: Windows We mentioned in Section 2.2 in the book that we obtain some undesirable effects in the frequency representation when we restrict to a block of the signal. Assume that x ∈ RM , and that w = {w0 , . . . , wN −1 } with N < M. We call (w0 x0 , . . . , wN −1 xN −1 , 0, ..., 0) ∈ RM a windowed signal and w ∈ RN a window of length N . a) Use Exercise 3.7 to show that the DFT of the windowed signal is 1 (DF TM (w0 , ..., wN −1 , 0, ..., 0)) ~ (DF TM x) M Solution.

With s1 = (w0 , ..., wN −1 , 0, ..., 0), s2 = x we have that s1 ◦ s2 is the windowed signal.

The result now follows from the second property in Exercise 3.7.

If DF TM (w0 , w1 , ..., wN −1 , 0, ..., 0) is close to (M, 0, ..., 0), this will be close to y = DF TM x. In other words, a good window should satisfy DF TM (w0 , w1 , ..., wN −1 , 0, ..., 0) ≈ (M, 0, ..., 0).

There is a loss when we pass from the signal to the windowed signal, since we can only construct a DFT close to, not equal to, (M, 0, ..., 0). We will not go into techniques for how to find values wi which are close to satisfying this, only evaluate three such in the rest of the exercise. b) The rectangular window is defined by w = {1, 1, . . . , 1}. Show that (DF TM (1, ..., 1, 0, ..., 0))n =

N −1 X k=0

e−2πikn/M =

1 − e−2πinN/M , 1 − e−2πin/M

and use this to check whether DF TM (w0 , w1 , ..., wN −1 , 0, ..., 0) ≈ (M, 0, ..., 0)

72

3 Discrete time filters

Solution.

This has absolute value

sin(πnN/M ) sin(N ω/2) sin(πn/M ) . The yn thus lie on the curve sin(ω/2) , from which

one can see that DF TM (w0 , w1 , ..., wN −1 , 0, ..., 0) ≈ (M, 0, ..., 0) is partially fulfilled.

The frequency response can be plotted as follows N=32; omega = linspace(0, 2*pi, 1000); wd = [ones(1, N) zeros(1, 1000-N)]; plot(omega, abs(fft(wd)), ’k-’);

c) The Hamming window is defined by wn = 2(0.54 − 0.46 cos(2πn/(N − 1))).

Make a plot of DF TM (w0 , w1 , ..., wN −1 , 0, ..., 0), and compare with (M, 0, ..., 0) to see if the Hamming window is a good window.

Solution.

1 Figure 3.1 compares M (DF TN (w0 , w1 , ..., wN −1 , 0, ..., 0)) for the rectangular- and the Hamming window for N = 32. The Hamming window is seen to differ from the rectangular window in two ways:

• It has much smaller values away from 0, • the width of the “main lobe” (i.e. the main structure at the center) is bigger.

As a consequence, windowing with the Hamming window reduces the contribution from higher frequencies, but increases the contribution from the smallest frequencies. The window coefficients for both windows are shown in Figure 3.2.

1.0

1.0

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0.0 3

2

1

0

1

2

3

0.0 3

2

1

0

1

2

3

1 The values M (DF TN (w0 , w1 , ..., wN −1 , 0, ..., 0)) for the rectangular and Hamming win-

Fig. 3.1 dows, which we considered for restricting to a block of the signal.

The frequency response can be plotted as follows n = 0:(N-1); wd = [ 2*(0.54-0.46*cos(2*pi*n/(N-1))) zeros(1, 1000-N)]; plot(omega, abs(fft(wd)), ’k-’);

3 Discrete time filters

73

2.0

2.0

1.5

1.5

1.0

1.0

0.5

0.5

0.0 0

5

10 15 20 25 30

0.0 0

5

10 15 20 25 30

Fig. 3.2 The coefficients of the rectangular and Hamming windows, which we considered for restricting to a block of the signal.

d) The Hanning window is defined by wn = 1 − cos(2πn/(N − 1)).

Repeat the analysis you did above for the rectangular and Hamming windows, for a Hanning window for N = 32.

The Hanning window is used in the MP3 standard, where it is applied to blocks which overlap, contrary to the non-overlapping block pattern we have used. After the Hanning window has been applied, the MP3 standard applies an FFT to the windowed signal in order to make a frequency analysis of that part of the sound.

Solution.

The frequency response can be plotted as follows wd = [1-cos(2*pi*n/(N-1)) zeros(1,1000-N)]; plot(omega, abs(fft(wd)), ’k-’);

A good survey on the windows mentioned here, as well as other ones, can be found in [4]. One lesson to be learned from this exercise is that the windowed signal may not give a good frequency representation. One must therefore be careful when splitting a sound into blocks, as this alters the frequency representation. You can use the function forw_comp_rev_DFT from Example 2.5 in the book to experiment with this. This function accepts a named parameter N , which can be used to split the DFT of a sound into blocks of length N , and eliminate low frequencies before taking an IDFT to reconstruct and listen to the new sound (see also Example 3.31 in the book).

Exercise 3.10: The multiplicative group of integers modulo N In number theory it is known that, when N is a prime number, the numbers 1, ..., N − 1 is a group under multiplication modulo N . In particular this means that for each 1 ≤ a ≤ N − 1 there exists a unique 1 ≤ b ≤ N − 1 so that ab = 1 mod N (the number 1 acts as a unit). b us then called the inverse of a, and we write b = a−1 (this is not the same as 1/a!). Also, it is known that there exists an integer g so that the set

74

3 Discrete time filters

−2 {g q mod N }N q=0 constitutes all numbers 1, ..., N − 1 (the order these numbers appear in may have changed, though). g is called a generator for the group.

a) Find generators for the multiplicative group of integers modulo 11, 23 and 41. Are the generators for each group unique?

Solution.

Even for small primes, one saves tedious computation here with an implementation. The following code returns the smallest generator. function g=find_smallest_generator(N) g = 0; for k=2:(N-1) res = zeros(N-1,1); val = 1; for s=1:(N-1) if res(val) ~= 0 break; end res(val) = s; val = mod(val*k,N); if s == N-1 g = k; return; end end end end

The outer loop starts at 2, because the unit 1 is never a generator. If your run this function for the stated primes, find_smallest_generator(11) find_smallest_generator(23) find_smallest_generator(41)

the generators 2, 5, and 6, respectively, are found.

A generator may not be unique: In the next exercise we state that g −1 is a generator whenever g is. b) Write a function reorder(x, g)

which, for a vector x of length N , and a generator g for the numbers 1, ..., N − 1, returns the vector (xg0 , xg1 , ..., xgN −2 ).

In the next exercise you will see how the multiplicative group of integers modulo N relates to circular convolution.

Solution.

The following code can be used. function xnew=reorder(x, g) N = length(x); xnew = zeros(N-1,1); val = 1;

3 Discrete time filters

75

for s=1:(N-1) xnew(s) = x(val+1); val = mod(val*g, N); end end

Exercise 3.11: The FFT algorithm when N is prime. Rader’s algorithm It is possible to show that, if g is a generator, then g −1 is also a generator. This means −2 −p that {g −p mod N }N is defined as p=0 also constitute the numbers 1, ..., N − 1 (where g −1 p (g ) ).

Replacing n = g −p and k = g q in the FFT algorithm with N prime, we can rephrase it as (we pull k = 0 out of the sum, since the number 0 is not an element in our group)

y0 =

N −1 X

xk

k=0

yg−p = x0 +

N −2 X

xgq e−2πig

−(p−q)

/N

, 0 ≤ p ≤ N − 2.

q=0

Define a = (xg0 , xg1 , ..., xgN −2 ) ∈ RN −1 0

b = (e−2πig /N , e−2πig

−1

/N

, ..., e−2πig

−(N −2)

/N

) ∈ RN −1

c = (yg0 , yg−1 , ..., yg−(N −2) ) ∈ RN −1 Explain that c = x0 + a ~ b, where x0 is added to every component in the vector.

This explains how to compute an N -point DFT (with N prime) using (N − 1)-point circular convolution. Since a circular convolution can be computed using a DFT/IDFT of the same size, so this method effectively reduces an N -point DFT to an (N − 1)-point DFT. Since N − 1 is not prime, we can use the algorithm for the FFT for composite N , to reduce the problem to DFT’s of lengths being smaller prime numbers. Another possibility is to use Exercise 3.5 to replace the circular convolution with a longer one, of length a power of two, effectively reducing the problem to a DFT of length a power of two.

Solution.

Inserting the definitions of a, b, and c in the DFT formula above gives cp = x 0 +

N −2 X q=0

aq bp−q = x0 +

N −2 X q=0

aq b(p−q) mod (N −1) = x0 + (a ~ b)p .

76

3 Discrete time filters

The formula follows. We used that bp−q = b(p−q) mod (N −1) , which follows from that g N −1 equals the unit e: If this was not the case, we would have that g N −1 = g i for some i > 0. This gives that g N −1−i = e, which would imply that g does not generate all numbers from 1 to N − 1. The following code tests that Rader’s algorithm produces the same result as an N -point FFT when N is prime.

N = 41; x=rand(N,1); g=find_smallest_generator(N); a = reorder(x, g); b = reorder(exp(-2*pi*i*(0:(N-1))/N), g); b(2:(N-1)) = flip(b(2:(N-1))); c = cconv(a, b, N-1) + x(1); c(2:(N-1)) = flip(c(2:(N-1))); y = x; y(1) = sum(x); ind = 1; for k = 1:(N-1) y(ind+1) = c(k); ind = mod(ind*g, N); end max(abs(fft(x)-y))

Here we used the built-in function cconv for circular convolution in MATLAB. Later we will consider our own implementation of circular convolution, but this will have a different assumption on the input. In the code above, also note that we reversed the vectors b and c. This is because these used g −1 rather than g to generate their values.

Exercise 3.18: Passing between compact filter notation and matrix form a) Assume that a filter is defined by the formula 1 1 1 1 xn+1 + xn + xn−1 + xn−2 .

4 4 4 4 Write down the corresponding compact filter notation, and the corresponding circulant Toeplitz matrix when N = 8. zn =

Solution.

Here we have that t−1 = 1/4, t0 = 1/4, t1 = 1/4, and t2 = 1/4. We now get that s0 = t0 = 1/4, s1 = t1 = 1/4, and s2 = t2 = 1/4 (first formula), and sN −1 = s7 = t−1 = 1/4 (second formula). This means that S = {= 1/4, 1/4, 1/4, 1/4}, and that the corresponding circulant Toeplitz matrix is   11000011 1 1 1 0 0 0 0 1   1 1 1 1 0 0 0 0   1 0 1 1 1 1 0 0 0  .

S=  4 0 0 1 1 1 1 0 0  0 0 0 1 1 1 1 0   0 0 0 0 1 1 1 1 10000111

3 Discrete time filters

77

b) Given the circulant Toeplitz matrix   1200 0 1 2 0  S= 0 0 1 2 .

2001 Write down the corresponding compact filter notation.

Solution.

We obtain that S = {2, 1}.

Exercise 3.19: Composing two filters Assume that the filters S1 and S2 have the frequency responses λS1 (ω) = 2 + 4 cos(ω), λS2 (ω) = 3 sin(2ω). a) Compute and plot the frequency response of the filter S1 S2 .

Solution.

We have that λS1 S2 (ω) = λS1 (ω)λS2 (ω) = (2 + 4 cos(ω))(3 sin(2ω)).

The frequency response can be plotted as follows. omega=linspace(-pi,pi,100); plot(omega, (2+4*cos(omega)).*(3*sin(2*omega)));

b) Write down the filter coefficients tk for the filter S1 S2 .

Solution.

We have that λS1 S2 (ω) = (2 + 4 cos(ω))(3 sin(2ω)) = (2 + 2eiω + 2e−iω )

3 2iω (e − e−2iω ) 2i

= −3i(1 + eiω + e−iω )(e2iω − e−2iω ) = −3i(e2iω − e−2iω + e3iω + eiω − e−iω − e−3iω ).

From this we see that S1 S2 = 3i{−1, −1, −1, 0, 1, 1, 1}.

78

3 Discrete time filters

Exercise 3.20: Factoring a filter Factor the frequency response of the filter S = {1, 5, 10, 6} as the product of two polynomials in e−iω , one of order 1 and one of order 2. Use this to factor S as a product of two filters, one with two filter coefficients, and one with three filter coefficients.

Solution.

The polynomial 6x3 + 10x2 + 5x + 1 clearly has x = −1 as a root. With polynomial division we obtain 6x3 + 10x2 + 5x + 1 = (x + 1)(6x2 + 4x + 1), where the second factor doesn’t have any real roots. For the frequency response we thus obtain λS (ω) = 1 + 5e−iω + 10e−2iω + 6e−3iω = (e−iω + 1)(6e−2iω + 4e−iω + 1).

The first factor is the frequency response of S1 = {1, 1}, while the second factor is the frequency response of S2 = {1, 4, 6}. It follows that S = S1 S2 .

Exercise 3.21: Plotting a simple frequency response Let S be the filter defined by the equation 1 1 1 1 xn+1 + xn + xn−1 + xn−2 , 4 4 4 4 as in Exercise 3.18. Compute and plot (the magnitude of) λS (ω). zn =

Solution.

The frequency response is

λS (ω) =

eiω (1 − e−4iω ) 1 sin(2ω) 1 iω (e + 1 + e−iω + e−2iω ) = = e−iω/2 .

4 4(1 − e−iω ) 4 sin(ω/2)

Exercise 3.22: Adding two echo filters Consider the two filters S1 = {1, 0, . . . , 0, c} and S2 = {1, 0, . . . , 0, −c}. Both of these can be interpreted as filters which add an echo. Show that 12 (S1 + S2 ) = I. What is the interpretation of this relation in terms of echos?

3 Discrete time filters

79

Solution.

The sum of two filters is again a filter, and the first column in the sum can be obtained by summing the first columns in the two matrices. This means that the filter coefficients in 12 (S1 + S2 ) can be obtained by summing the filter coefficients of S1 and S2 , and we obtain 1 ({1, 0, . . . , 0, c} + {1, 0, . . . , 0, −c}) = {1}.

2 This means that 12 (S1 + S2 ) = I, since I is the unique filter with e0 as first column. The interpretation in terms of echos is that the echo from S2 cancels that from S1 .

Exercise 3.23: Filters with coefficients being powers Assume that S = {1, c, c2 , . . . , ck }. Compute and plot λS (ω) when k = 4 and k = 8, for c = 0.5. How do the choices of k and c influence the frequency response?

Solution.

The frequency response is k X s=0

cs e−isω =

1 − ck+1 e−i(k+1)ω .

1 − ce−iω

It is straightforward to compute the limit as ω → 0 as ck (k + 1). This means that as we increase k or c, this limit also increases. The value of k also dictates oscillations in the frequency response, since the numerator oscillates fastest. When c = 1, k dictates how often the frequency response hits 0.

Exercise 3.24: Convolution and polynomials Compute the convolution of (1, 2, 1) with itself, and interpret the result in terms of a product of polynomials. Compute also the circular convolution of (1, 2, 1) with itself.

Solution.

Set x = (1, 2, 1), and z = x ∗ x. We obtain that (only the terms contributing in the convolution are listed).

80

3 Discrete time filters

z0 = x0 x0 = 1 z1 = x0 x1 + x1 x0 = 2 + 2 = 4 x2 = x0 x2 + x1 x1 + x2 x0 = 1 + 4 + 1 = 6 x3 = x1 x2 + x2 x1 = 2 + 2 = 4 x4 = x2 x2 = 1.

It follows that (1, 2, 1) ∗ (1, 2, 1) = (1, 4, 6, 4, 1). For the circular convolution we have

z0 = x0 x0 + x1 x2 + x2 x1 = 1 + 2 + 2 = 5 z1 = x0 x1 + x1 x0 + x2 x2 = 2 + 2 + 1 = 5 x2 = x0 x2 + x1 x1 + x2 x0 = 1 + 4 + 1 = 6.

It follows that (1, 2, 1) ~ (1, 2, 1) = (5, 5, 6).

Exercise 3.25: Execution times for convolution Implement code which computes t ∗ x in the vectorized- and non-vectorized ways described in Section 3.2.1 in the book (i.e. as a single loop over k or n with the other variable vectorized, or a double loop). As your t, take k randomly generated numbers.

Compare execution times with the conv function, for different values of k. Present the result as a plot where k runs along the x-axis, and execution times run along the y-axis. Your result will depend on how vectorization is performed by the computing environment.

Solution.

The code can look as follows. [x, f_s] = audioread(’sounds/castanets.wav’); x = x(:,1); N= length(x); kmax=100; vals1 = zeros(1, kmax/10); vals2 = zeros(1, kmax/10); vals3 = zeros(1, kmax/10); ind = 1; for k=10:10:kmax t = rand(k, 1); tic; conv(t, x); vals1(ind) = toc; z = zeros(N, 1); tic; for s = 1:k z(k:N) = z(k:N) + t(s)*x((k-s+1):(N-s+1)); end vals2(ind)=toc; z = zeros(N, 1); tic; for n=k:N

3 Discrete time filters

81

for s = 1:k z(n) = z(n) + t(s)*x(n-s+1); end end vals3(ind)=toc; ind = ind+1; end plot(10:10:kmax,log(vals1),10:10:kmax,log(vals2), 10:10:kmax, log(vals3)); legend(’conv’,’simple for’,’double for’)

Exercise 3.26: Implementing convolution in-place Filters are often implemented in hardware, and hardware implementations have strong limitations on the number of local variables. Assume that a filter t has n nonzero coefficients, all known to the hardware, and that the vector x is input. Show that t ∗ x can be implemented in-place (i.e. that t ∗ x can be computed and stored directly in x) by using n − 1 local variables. You can assume that two local variables can be swapped without using an additional local variable. It follows that, since any real convolution can be split into real convolutions with three filter coefficients, only 2 local variables are needed in order to compute any convolution in-place.

Solution.

Assume for simplicity that the nonzero filter coefficients are t0 , ..., tn−1 . We have that x0 is needed in order to compute z0 , ..., zn−1 . Therefore, z0 can’t be stored into the location of x0 until zn−1 has been computed. When we start the computation of zn−1 , z0 , ..., zn−2 must be stored in n − 1 local variables. The final contribution of x0 is with x0 tn−1 in zn−1 . Now, multiply x0 in-place with tn−1 , and compute zn−1 in the same memory location. Finally, swap the result with the local variable holding z0 . As a result, z0 has replaced x0 in the input memory, and the local variables hold z1 , ..., zn−1 instead.

This is continued incrementally.

When the first nonzero filter coefficient is not t0 , it is clear that this procedure will still work, but that the output will be a delayed version.

Exercise 3.27: Circular convolution when there is a different number of coefficients with positive and negative indices Assume that S = {t−E , . . . , t0 , . . . , tF }. Formulate a generalization of Proposition 9 in the book for such filters, i.e. to filters where there may be a different number of filter coefficients with positive and negative indices. You should only need to make some small changes to the proof.

Solution.

We reproduce Figure 3.5 by placing the t−E , t−E+1 and so on in the first column, starting at the top. We then disregard the first and last E + F rows (not

82

3 Discrete time filters

the first and last 2L entries as in step 3). We can form the shorter vector x̃ = (xN −F , · · · , xN −1 , x0 , · · · , xN −1 , x0 , · · · , xE−1 ) in step 1, before we proceed with the convolution.

Exercise 3.28: Implementing circular convolution through convolution Implement a function which takes two parameters t and x, and uses Proposition 9 in the book and convolution to compute the circular convolution of t = (t−L , . . . , t0 , . . . , tL ) and x. Write the function so that, if x has many columns, the circular convolution with each column is computed.

Hint.

The function filter_impl in the library is slightly more general than the one you are asked to implement here, and can serve as a template for your implementation.

Solution.

The code can look like this: function y=filter_impl(t, x) L = (length(t) - 1)/2; N = length(x); y = [x((N - L + 1):N); x; x(1:L)]; y = conv(t, y); y = y((2*L+1):(length(y)-2*L));

Exercise 3.29: Expressing convolution in terms of circular convolution Find a similar statement to that in Proposition 9 in the book, which explains how to express convolution in terms of circular convolution.

Hint.

Add some zeros at the start and end of the vector. How many zeros do you need to add?

Solution.

Let x̃ be the vector where 2L zeros are placed at the end of x. It is then straightforward to check that

3 Discrete time filters

83

(t−L , ..., tL , 0, ..., 0) ~ x̃ = t ∗ x.

The circular convolution here corresponds to the N + 2L cirulant Toeplitz matrix with compact filter notation {t−L , ..., tL }.

Exercise 3.36: Applying low-pass- and high-pass filters deduced from Pascal’s triangle to the audio sample file a) Write code where you apply the low-pass and high-pass filters in examples 3.34 and 3.35 in the book to the audio sample file, and verify that the sounds you get are the same as in these examples. How high must k be in order for you to hear difference from the actual sound? How high can you choose k and still recognize the sound at all?

If you solved Exercise 3.28 you can also use the function filter_impl to perform the filtering, rather than using convolution (which, as mentioned, discards circularity).

Solution.

The code can look like this, when we apply low-pass filters. z = x(:, 1); for kval=1:k z = conv(z,[1/2 1/2]); end playerobj=audioplayer(z, f_s); playblocking(playerobj);

If we apply high-pass filters instead, the code can look like this. z = x(:, 1); for kval=1:k z = conv(z,[1/2 -1/2]); end playerobj=audioplayer(z, f_s); playblocking(playerobj);

b) In your code, it is not necessary to scale the values after applying the low-pass or high-pass filters so that values fit inside [−1, 1]. Explain why this is the case.

Solution.

The low-pass filters compute weighted averages of the input samples. Therefore, as long as the input values are inside the legal range[−1, 1], the output values will as well. Since the filter coefficients sum to one it is easy to see that also the high-pass filters produce values in the legal range.

84

3 Discrete time filters

Exercise 3.37: Constructing a high-pass filter Consider again Example 3.31 in the book. Find an expression for a filter so that only frequencies so that |ω − π| < ωc are kept, i.e. the filter should only keep angular frequencies close to π (i.e. here we construct a high-pass filter).

Solution.

A filter which keeps frequencies so that |ω − π| < ωc is the high-pass filter corresponding to the filter constructed in Example 3.31 in the book. We therefore simply need to add an alternating sign to the filter coefficients from that example.

Exercise 3.38: Combining low-pass and high-pass filters In this exercise we will investigate how we can combine low-pass and high-pass filters to produce other filters a) Assume that S1 and S2 are low-pass filters. Is S1 S2 also a low-pass filter? What if both S1 and S2 are high-pass filters?

Solution.

We have that λS1 S2 (0) = λS1 (0)λS2 (0) which is “large” when the two factors are. Also, λS1 S2 (π) = λS1 (π)λS2 (π) is approximately 0 when the two factors are. Therefore S1 S2 is also low-pass. That S1 S2 is high-pass when the two factors are follows similarly. b) Assume that one of S1 , S2 is a high-pass filter, and that the other is a low-pass filter.

Is S1 S2 low-pass or high-pass?

Solution.

In this case one of λS1 (0) and λS2 (0) is large, and the other approximately zero. There is then no telling if their product is large or approximately zero, so we can’t conclude anything about the filter S1 S2 .

Exercise 3.39: Filters in the MP3 standard Assume that tk are the filter coefficients of a filter S1 , and that S2 is the filter with filter coefficients cos(kωc )tk , where ωc ∈ [0, π). Show that 1 (λS1 (ω − ωc ) + λS1 (ω + ωc )).

2 In other words, when we multiply (modulate) the filter coefficients with a cosine, the new frequency response can be obtained by shifting the old frequency response with ωc λS2 (ω) =

3 Discrete time filters

85

in both directions, and taking the average of the two. We saw that the filters in the MP3 standard were constructed in this way. The collection of these filters are also called a cosine modulated filter bank. It is also possible to construct DFT-modulated filter banks where the shift in frequency is obtained by multiplying with a complex exponential instead, but this produces filters with complex coefficients.

Solution.

We have that

λS2 (ω) =

X k

=

1 2

1 X ikωc (e + e−ikωc )tk e−ikω 2 k !

X X −ik(ω−ωc ) −ik(ω+ωc ) tk e + tk e

cos(kωc )tk e−ikω =

k

k

1 = (λS1 (ω − ωc ) + λS1 (ω + ωc )).

2

Exercise 3.40: Code example a) Explain what the code below does, line by line. [x, f_s] = audioread(’sounds/castanets.wav’); [N, nchannels] = size(x); z = zeros(N, nchannels); for n=2:(N-1) z(n,:) = 2*x(n+1,:) + 4*x(n,:) + 2*x(n-1,:); end z(1,:) = 2*x(2,:) + 4*x(1,:) + 2*x(N,:); z(N,:) = 2*x(1,:) + 4*x(N,:) + 2*x(N-1,:); z = z/max(abs(z)); playerobj=audioplayer(z, f_s); playblocking(playerobj)

Comment in particular on what happens in the three lines directly after the for-loop, and why we do this. What kind of changes in the sound do you expect to hear?

Solution.

In the code a filter is run on the sound samples from the file castanets.wav. Finally the new sound is played. In the first two lines after the for-loop, the first and the last sound samples in the filtered sound are computed, under the assumption that the sound has been extended to a periodic sound with period N. After this, the sound is normalized so that the sound samples lie in the range between −1 and 1. In this case the filter is a low-pass-filter (as we show in b)), so that we can expect that that the treble in the sound is reduced. b) Write down the compact filter notation for the filter which is used in the code, and write down a 5 × 5 circulant Toeplitz matrix which corresponds to this filter. Plot the (continuous) frequency response. Is the filter a low-pass- or a high-pass filter?

86

3 Discrete time filters

Solution.

Compact filter notation for the filter which is run is {2, 4, 2}. A 5 × 5 circulant Toeplitz matrix becomes   42002  2 4 2 0 0    0 2 4 2 0 .    0 0 2 4 2 20024 The frequency response is λS (ω) = 2eiω + 4 + 2e−iω = 4 + 4 cos ω. It is clear that this gives a low-pass filter. c) Another filter is given by the circulant Toeplitz matrix   4 −2 0 0 −2 −2 4 −2 0 0     0 −2 4 −2 0  .    0 0 −2 4 −2 −2 0 0 −2 4 Express a connection between the frequency responses of this filter and the filter from b). Is the new filter a low-pass- or a high-pass filter?

Solution.

The frequency response for the new filter is −2eiω + 4 − 2e−iω = 4 − 4 cos ω = 4 + 4 cos(ω + π) = λS (ω + π), where S is the filter from the first part of the exercise. The new filter therefore becomes a high-pass filter, since to add π to ω corresponds to swapping the frequencies 0 and π.

We could also here referred to Proposition 11 in the book, where we stated that adding an alternating sign in the filter coefficients turns a low-pass filter into a high-pass filter and vice versa.

Exercise 3.44: All-pass filters All-pass filters are filters where |λS (ω)| = 1 for all ω. Such filters are uniquely determined by the phase of the frequency response. Any all-pass FIR filter must clearly be on the form αEd with |α| = 1. It is, however, possible to find other all-pass filters by considering realizations in terms of difference equations. a) If α is any real number, consider the filter S given by the realization zn − αzn−1 = αxn − xn−1 .

Write down the frequency response of this realization, and show that it is an all-pass filter.

3 Discrete time filters

87

Solution. −iω

α−e The frequency response is λS (ω) = 1−αe −iω , and it is straightforward to check that −iω −iω |α − e | = |1 − αe |, so that |λS (ω)| = 1.

b) A filter is said to be linear phase if the phase (argument) of the frequency response is linear in ω, i.e. λS (ω) = Cei(rω+s) for some real r, s, and C. Any delay filter has linear phase since λEd (ω) = e−idω . Show that S and the filter {−α2 , 2α, −1} have frequency responses with the same phase, and explain why it follows from this that S has non-linear phase as long as |α| = 6 1. Since delay filters have linear phase, these filters can not be delays, and thus represent a new kind of all-pass filters.

Hint.

Consult Exercise 3.49 on linear phase filters.

Solution.

We have that λS (ω) =

−α2 eiω + 2α − e−iω (α − e−iω )(1 − αeiω ) = , |1 − αe−iω |2 |1 − αe−iω |2

so that S and {−α2 , 2α, −1} have frequency responses with the same phase. Since {−α2 , 2α, −1} has nonlinear phase, S has so as well. c) Explain that S has an inverse, and that a realization of the inverse system is −αzn + zn−1 = −xn + αxn−1 .

Solution.

Since |λS (ω)| = 1, we simply need to find a realization of the filter with frequency response λS (ω) =

α − eiω , 1 − αeiω

which gives the realization −αzn+1 + zn = −xn+1 + αxn .

Since only coefficients up to n for zn should be listed, a translation with 1 is needed, and this yields the desired expression. d) Explain that combining two all-pass filters as above with different values for α will also give an all-pass filter, different from the ones we combined. This gives some flexibility in how one can construct all-pass filters as well.

88

3 Discrete time filters

Solution.

Note first that, also in the context of IIR filters, composing two filters gives a new IIR filter with frequency response equal to the multiplication of the two frequency responses.

To see why, let S1 and S2 be the two IIR filters X

ak zn−k =

X k

0≤k≤M

X

bk yn−k

ck yn−k =

0≤k≤M

X

dk xn−k .

k

They can be represented in matrix form as Az = By and Cy = Dx, where A, B, C, and D are (infinite) Toeplitz matrices. These matrices also commute (even though they are infinite), and there are only finitely many non-zeros in each row/column of the matrices. we thus obtain CAz = CBy = BCy = BDx. which gives an expression for the combined filter. Since CA and BD again are Toeplitz matrices, S1 S2 is also an IIR filter. It is also clear that it can be written as X X ek zn−k = fk yn−k , 0≤k≤M

k

with e = a ∗ c and f = b ∗ d, and we obtain that −iωk

P P −iωk d e−iωk k bk e P Pk k −iωk = λS1 (ω)λS2 (ω), = −iωk −iωk 0≤k≤M (a ∗ c)k e k ak e k ck e P

λS1 S2 (ω) = P

k (b ∗ d)k e

where we used that convolution in time corresponds to multiplication in frequency. When S1 and S2 are all-pass filters, it follows that |λS1 S2 (ω)| = |λS1 (ω)λS2 (ω)| = |1||1| = 1, so that S1 S2 also is an all-pass filter.

Finally, suppose now that we compose all-pass filters with parameters α1 and α2 .

The numerator in the frequency response of the combined filter has zeros in α1 and α2 , while for the denominator the zeros are 1/α1 and 1/α2 . Unless we choose α1 = 1/α2 (for which all terms in the frequency response will cancel), we thus will get a different filter when they are combined.

Exercise 3.45: The IIR filter for the Karplus-Strong algorithm Let us rewrite the difference equation in Exercise 1.12 as 1 zn − (zn−p + zn−(p+1) ) = 0 2 a) Explain that the frequency response of this realization is 0. Therefore, this realization is only interesting if the initial conditions are chosen so that we don’t obtain a filter.

Solution.

The numerator in λS (ω) is zero, since all bk = 0. We therefore have that λS (ω) = 0 also.

3 Discrete time filters

89

b) Plot the zeros of the characteristic equation for p = 20. Is the filter BIBO stable?

Solution.

We have that a0 = 1, ap = ap+1 = −1/2, so that the characteristing equation is PM −k = 1 − 12 r−p − 12 r−(p+1) . Its zeros can be plotted as follows. k=0 ak r p=20; rts = roots([1 zeros(1, p-1) -1/2 -1/2]); plot(real(rts), imag(rts), ’kx’)

We see from the resulting plot that the zeros are inside the unit circle, so that the filter is BIBO stable. c) What happens with the zeros of the characteristic equation when you increase p?

Attempt to explain why the resulting sound changes more slowly to a static sound when you increase p.

Solution.

It seems that the absolute values of the zeros of the characteristic equation gets closer and closer to 1 as p increases. This also means that their powers decreases slower to zero, so that the resulting sound goes to zero somewhat more slowly. d) A related IIR filter is 1 zn − (zn−p + zn−(p+1) ) = xn .

2 Plot the frequency response of this filter for p = 20. Include only values in the response between 0 and 10.

Solution.

The following code can be used p=20; omega = linspace(0,2*pi,500); plot(omega, abs(1./(1-0.5*exp(-omega*p*i)-0.5*exp(-omega*(p+1)*i))) ); axis([0,2*pi,0,10]);

Exercise 3.46: Direct proof of the orthogonality of the DCT basis The way we proved that the DCT basis was orthonormal was constructive in that it built on the orthogonality of the DFT. In the literature many instead give a direct proof.

Give a direct proof of the orthonormalilty of the DCT basis using only trigonometric identities.

90

3 Discrete time filters

Solution.

Let f be the vector with components fk = cos 2πn1 (k+1/2) , and g the vector 2N

2πn2 (k+1/2) with components gk = cos . If these vectors have length 2N , they are 2N clearly orthogonal when n = 6 n , since both can be written as a sum of two ge2 1

2πn(k+1/2) 1 ometric series (i.e. cos = 2 e2πn(k+1/2)/(2N ) + e−2πn(k+1/2)/(2N ) ), and 2N since (e2πn/(2N ) )2N = 1. It is also straightforward to check that c2N −1−k = ck for k = 0, ..., N − 1, and similarly for d. This implies that 2N −1 X

0=

k=0

ck dk =

N −1 X

ck dk +

k=0

2N −1 X

ck dk =

k=N

N −1 X

ck dk +

k=0

N −1 X

ck d k = 2

k=0

N −1 X

ck dk .

k=0

It follows that c and d are orthogonal also when viewed as vectors of length N . Since the DCT basis vectors are scaled versions of these, they are orthogonal as well. √ If n = 0 the cosine vectors have all components equal to one, so that it has length N . √ This is the reason why a scaling with 1/ N is needed in d0 to achieve orthonormality.

We also have N −1 X

2

cos

k=0

2πn(k + 1/2) 2N

N −1

1 X = 2

cos

k=0

2πn(k + 1/2) N

+1

N −1

=

N 1 X 2πin(k+1/2)/N N + e + e−2πin(k+1/2)/N = , 2 2 2 k=0

where it was easily seen that p the two geometric series summed to zero. This explains the scaling with the factor 2/N in the other DCT basis vectors.

Exercise 3.47: Writing down lower order Sr Consider the averaging filter S = { 14 , 12 , 41 }. Write down the matrix Sr for the case when N = 4.

Solution.

First we obtain the matrix S as 1 1

1 2 4 0 0 0 0 0 4 1 1 1 0 0 0 0 0  4 12 41 1  0   4 21 14 01 0 0 0  0 0  4 2 4 0 0 0  0 0 0 1 1 1 0 0   4 2 4 0 0 0 0 1 1 1 0   4 2 4 0 0 0 0 0 1 1 1 4 2 4 1 1 1 4 0 0 0 0 0 4 2



3 Discrete time filters

91

where we have drawn the boundaries between the blocks S1 , S2 , S3 , S4 . From this we see that 1 1



2 4 0 0  1 1 1 0 4 2 4  S1 =  0 1 1 1  4 2 4 0 0 14 12

 0 0 0 14 0 0 0 0  S2 =  0 0 0 0 1 4 0 0 0

1



4 0 0 0



0 0 0 0  (S2 )↔ =  0 0 0 0 .

0 0 0 41

From this we get 3 1



4 0 0 1 1  2 4 0 .

0 41 12 41  0 0 14 43

4 1 ↔  Sr = S1 + (S2 ) =  4

Exercise 3.48: Computing eigenvalues Consider the matrix   210000 1 1 1 0 0 0   1 0 1 1 1 0 0   S=  3 0 0 1 1 1 0  0 0 0 1 1 1 000012 a) Compute the eigenvalues and eigenvectors of S using the results of this section. You should only need to perform one DFT or one DCT in order to achieve this.

Solution.

In c) we show that this indeed corresponds to a symmetric filter. We then know that the DCT basis vectors are eigenvectors. The eigenvalues can be found on the diagonal in DCTN S(DCTN )T . Note that it is not true that the eigenvalues can be computed by taking a DCT on the first column, as is the case for filters and the DFT. b) Use a computer to find the eigenvectors and eigenvalues of S also. What are the differences from what you found in a)?

Solution.

The following code compares what you found in a) with a direct computation.

S = [2 1 0 0 0 0; ...

1 1 1 0 0 0; ...

0 1 1 1 0 0; ...

0 0 1 1 1 0; ...

0 0 0 1 1 1; ...

0 0 0 0 1 2];

92

3 Discrete time filters

idctmatr = idct(eye(6)); idctmatr % Eigenvectors dct(S*idctmatr) % eigenvalues [V, D]=eig(S); V D

% Eigenvectors % eigenvalues

The two typically will list eigenvectors and eigenvalues in different order. c) Find a filter T so that S = Tr . What kind of filter is T ?

Solution.

Writing down the circulant 12 × 12-matrix for the symmetric filter T = 13 {1, 1, 1} we obtain the two upper blocks   110000 1 1 1 0 0 0   1 0 1 1 1 0 0  T1 =   3 0 0 1 1 1 0 0 0 0 1 1 1 000011

  000001 0 0 0 0 0 0   1 0 0 0 0 0 0 .

T2 =   3 0 0 0 0 0 0 0 0 0 0 0 0 100000

From these expressions it is clear that Tr = T1 + T2↔ = S.

Exercise 3.49: Linear phase filters For a symmetric (2N ) × (2N )-filter the filter coefficients have symmetry around N − 1/2.

Show that, if a filter instead has symmetry around d (where d may or may not be an integer), then its frequency response takes the form λS (ω) = Ce−iωd , with C a real number.

This means that the phase (argument) of the frequency response is −dω or π − dω, depending on the sign of C. In other words, the phase is linear in ω. Such filters are therefore called linear phase filters (see also Exercise 3.44). One can show that linear phase filters also preserve symmetric extensions, but one has to re-define such extensions to have another point of symmetry. An example of linear phase filters which are not symmetric are low-pass filters where the coefficients are taken from odd rows in Pascal’s triangle.

Solution.

If we gather the terms with indices d + k and d − k in the frequency response we obtain td+k (e−iω(d+k) + e−iω(d−k) ) = 2e−iωd td+k cos(kω), which has phase −dω. The sum of all such terms also have phase −dω. The claim follows with C = 2td+k cos(kω).

3 Discrete time filters

93

Exercise 3.50: Component expressions for a symmetric filter Assume that S = t−L , . . . , t0 , . . . , tL is a symmetric filter. Use equation (3.21) in the book to show that zn = (Sx)n in this case can be split into the following different formulas, depending on n: a) 0 ≤ n < L: zn = t0 xn +

n X

tk (xn+k + xn−k ) +

k=1

L X

tk (xn+k + xn−k+N ).

k=n+1

Solution.

We have that L X

tk x(n−k) mod N = t0 xn +

k=−L

L X

tk (x(n−k) mod N + x(n+k) mod N ).

k=1

When 0 ≤ n < L we have that (n + k) mod N = n + k, and ( n−k when k ≤ n (n − k) mod N = . n − k + N when n < k ≤ L The result follows. b) L ≤ n < N − L: zn = t0 xn +

L X

tk (xn+k + xn−k ).

k=1

Solution.

When L ≤ n < N − L we have that (n − k) mod N = n − k and (n + k) mod N = n + k, and the result follows. c) N − L ≤ n < N : zn = t0 xn +

NX −1−n k=1

tk (xn+k + xn−k ) +

L X

tk (xn+k−N + xn−k ).

k=N −1−n+1

Solution.

When N − L ≤ n < N we have that (n − k) mod N = n − k, and ( n+k when k ≤ N − 1 − n (n + k) mod N = . n + k − N when k > N − 1 − n The result follows.

Chapter 4

Motivation for wavelets and some simple examples

Exercise 4.1: The vector of samples is the coordinate vector Show that the coordinate vector for f ∈ V0 in the basis {φ0,0 , φ0,1 , . . . , φ0,N −1 } is (f (0), f (1), . . . .f (N − 1)). This shows that, for f ∈ Vm , there is no loss in working with the samples of f rather than f itself.

Solution.

PN −1 We have that f (t) = n=0 cn φ0,n , where cn are the coordinates of f in the basis {φ0,0 , φ0,1 , . . . , φ0,N −1 }. We now get that f (k) =

N −1 X

cn φ0,n (k) = ck ,

n=0

since φ0,n (k) = 0 when n = 6 k. This shows that (f (0), f (1), . . . .f (N − 1)) are the coordinates of f .

Exercise 4.2: Realizing the DWT in terms of averages and differences Let f (t) ∈ V1 , and let fn,1 be the value f attains on [n, n + 1/2), and fn,2 the value f attains on [n + 1/2, n + 1). Show that • projV0 (f ) is the function in V0 which equals (fn,1 + fn,2 )/2 on the interval [n, n + 1).

• projW0 (f ) is the function in W0 which is (fn,1 − fn,2 )/2 on [n, n + 1/2), and −(fn,1 − fn,2 )/2 on [n + 1/2, n + 1).

Hint.

Apply Lemma 10 in the book.

95

96

4 Motivation for wavelets and some simple examples

Solution.

Since f is constant and equal to f (n) on [n, n+1/2), and constant and equal to f (n+1/2) on [n + 1/2, n + 1), we get that Z N

Z n+1

hf, φ0,n i =

f (t)φ0,n (t)dt =

f (t)dt

n Z n+1

0

Z n+1/2 f (t)dt +

= n

f (t)dt n+1/2

Z n+1/2

Z n+1 f (n)dt +

= n

f (n + 1/2)dt n+1/2

= f (n)/2 + f (n + 1/2)/2 = (f (n) + f (n + 1/2))/2.

The orthogonal decomposition theorem gives that projV0 f =

N −1 X

hf, φ0,n iφ0,n =

n=0

N −1 X

f (n) + f (n + 1/2) φ0,n .

2 n=0

Since φ0,n is 1 on [n, n + 1) and 0 elsewhere, projV0 f is the piecewise constant function which is equal to (f (n) + f (n + 1/2))/2 on [n, n + 1).

Exercise 4.3: Computing projections a) Use Lemma 10 in the book to write down the matrix of projV0 : V1 → V0 relative to φ1 and φ0 , and the matrix of projW0 : V1 → W0 relative to φ1 and ψ0 .

Solution.

From Lemma 10 in the book it follows that √ projV0 (φ1,2n ) = φ0,n / 2 √ projV0 (φ1,2n+1 ) = φ0,n / 2 This means that √ [projV0 (φ1,2n )]φ0 = en / 2 √ [projV0 (φ1,2n+1 )]φ0 = en / 2.

These are the columns in the matrix for projV0 relative to the bases φ1 and φ0 . This matrix is thus

4 Motivation for wavelets and some simple examples

97

  1 1 0 0 0 ··· 0 0 0  1  0 0 1 1 0 · · · 0 0 0 √ . . . . . . . . . .

2  .. .. .. .. .. .. .. .. ..  0 0 0 0 0 ··· 0 1 1 Similarly √ projW0 (φ1,2n ) = ψ0,n / 2 √ projW0 (φ1,2n+1 ) = −ψ0,n / 2 so that √ [projW0 (φ1,2n )]ψ0 = en / 2 √ [projW0 (φ1,2n+1 )]ψ0 = −en / 2.

These are the columns in the matrix for projW0 relative to the bases φ1 and ψ0 . This matrix is thus   1 −1 0 0 0 · · · 0 0 0  1  0 0 1 −1 0 · · · 0 0 0  √ . . . . . . . . .  .

2  .. .. .. .. .. .. .. .. ..  0 0 0 0 0 · · · 0 1 −1 b) Write down the matrix of projV0 : V1 → V0 relative to (φ0 , ψ0 ) and φ0 , and the matrix of projW0 : V1 → W0 relative to (φ0 , ψ0 ) and ψ0 .

Solution.

Note first that, since φ ∈ V0 we must have that projV0 (φ) = φ. Also, since ψ is in the orthogonal complement of V0 in V1 we must have that projV0 (ψ) = 0. It follows that the first columns in the matrix of projV0 relative to (φ0 , ψ0 ) and φ0 are [projV0 (φ0,0 )](φ0 ,ψ0 ) = [φ0,0 ](φ0 ,ψ0 ) = e0 [projV0 (φ0,1 )](φ0 ,ψ0 ) = [φ0,1 ](φ0 ,ψ0 ) = e1 .... ..

The last columns in the matrix of projV0 relative to (φ0 , ψ0 ) are [projV0 (ψ0,0 )](φ0 ,ψ0 ) = [0](φ0 ,ψ0 ) = 0 [projV0 (ψ0,1 )](φ0 ,ψ0 ) = [0](φ0 ,ψ0 ) = 0 .... ..

It follows that the matrix of projV0 relative to (φ0 , ψ0 ) is given by the diagonal matrix where the first half of the entries on the diagonal are 1, the second half 0. In the same

98

4 Motivation for wavelets and some simple examples

way it follows that projW0 : V1 → W0 has a matrix relative to (φ0 , ψ0 ) and ψ0 given by the diagonal matrix where the first half of the entries on the diagonal are 0, the second half 1. c) Generalize the points above to the projections from Vm+1 onto Vm and Wm .

Solution.

Since φm,n ∈ Vm we must have that T (φm,n ) = φm,n . Since ψm,n is in the orthogonal complement of Vm in Vm+1 we must have that T (ψm,n ) = 0. The first half of the columns in the matrix of projVm relative to (φm , ψm ) are [projVm (φm,0 )](φm ,ψm ) = [φm,0 ](φm ,ψm ) = e0 [projVm (φm,1 )](φm ,ψm ) = [φm,1 ](φm ,ψm ) = e1 .... ...

The second half of the columns are

[T (ψm,0 )](φm ,ψm ) = [0](φm ,ψm ) = 0 [T (ψm,1 )](φm ,ψm ) = [0](φm ,ψm ) = 0 .... ...

Thus, as before, the matrix of projVm relative to (φm , ψm ) is given by the diagonal matrix where the first half of the diagonal consists of 1’s, and the second half consists of 0’s. c) follows in the same way.

Exercise 4.4: Finding the least squares error Show that

kprojV0 f − f k2 = hf, f i −

X Z n+1 n

2 f (t)dt .

n

Solution.

We have that kf − projV0 (f )k2 = hf − projV0 (f ), f − projV0 (f )i = hf, f i − 2hf, projV0 (f )i + hprojV0 (f ), projV0 (f )i Now, note that

4 Motivation for wavelets and some simple examples

hprojV0 (f ), projV0 (f )i =

99

N −1 Z n+1 X

2 f (t)dt

n

n=0

From the orthogonal decomposition formula it follows that N −1 Z n+1 X

projV0 (f ) =

n=0

f (t)dt φ0,n (t).

n

This means that the above can be written

= hf, f i − 2

N −1 Z N Z n+1 X n=0

= hf, f i − 2

n=0

n=0

n=0

n

2 NX 2 −1 Z n+1 f (t)dt + f (t)dt

n

N −1 Z n+1 X

n

2 N −1 Z n+1 X f (s)ds f (t)dt + f (t)dt

n

n

N −1 Z n+1 X n=0

= hf, f i −

n

0

N −1 Z n+1 Z n+1 X n=0

= hf, f i − 2

2 N −1 Z n+1 X f (s)ds φ0,n (t)f (t)dt + f (t)dt

n=0

n

2 f (t)dt .

n

Exercise 4.5: Direct sums Let C1 , C2 . . . , Cn be independent vector spaces, and let Ti : Ci → Ci be linear transformations. The direct sum of T1 , T2 ,. . . ,Tn , written as T1 ⊕ T2 ⊕ . . . ⊕ Tn , denotes the linear transformation from C1 ⊕ C2 ⊕ · · · ⊕ Cn to itself defined by T1 ⊕ T2 ⊕ . . . ⊕ Tn (c1 + c2 + · · · + cn ) = T1 (c1 ) + T2 (c2 ) + · · · + Tn (cn ) when c1 ∈ C1 , c2 ∈ C2 , . . . , cn ∈ Cn . Also, when A1 , A2 , . . . , An are square matrices, diag(A1 , A2 , · · · , An ) is defined as the block matrix where the blocks along the diagonal are A1 , A2 , . . . , An , and all other blocks are 0 (see Appendix A in the book). Show that, if Bi is a basis for Ci then [T1 ⊕ T2 ⊕ . . . ⊕ Tn ](B1 ,B2 ,...,Bn ) = diag([T1 ]B1 , [T2 ]B2 , . . . , [Tn ]Bn ).

Solution.

If bj is the j’th basis vectors in Bi , column j in [Ti ]Bi is [Ti (bj )]Bi . Since (T1 ⊕ T2 ⊕ . . . ⊕ Tn )(bj ) = Ti (bj ), the column in [T1 ⊕ T2 ⊕ . . . ⊕ Tn ](B1 ,B2 ,...,Bn ) corresponding to the basis vector bj in Bi is (0, ..., 0, [Ti (bj )]Bi , 0, ..., 0)T , which equals the corresponding column in diag([T1 ]B1 , [T2 ]B2 , . . . , [Tn ]Bn ).

100

4 Motivation for wavelets and some simple examples

Exercise 4.6: Properties of block diagonal matrices Assume that A, B, C, and D are square matrices of the same dimensions. a) Assume that the eigenvalues of A are equal to those of B. What are the eigenvalues of diag(A, B)? Can you express the eigenvectors of diag(A, B) in terms of those of A and B?

Solution.

Assume that λ is an eigenvalue common to both A and B. Then there exists a vector v1 so that Av1 = λv1 , and a vector v2 so that Bv2 = λv2 . We now have that

v diag(A, B) 1 v2

A 0 v1 Av1 λv1 v = = = =λ 1 .

0 B v2 Bv2 λv2 v2

This shows that λ is an eigenvalue for diag(A, B) also, with

v1 v2

an eigenvector.

b) Assume that A and B also are non-singular. Show that diag(A, B) is non-singular, and that (diag(A, B))−1 = diag(A−1 , B −1 ).

Solution.

We have that

diag(A, B)diag(A

−1

,B

−1

−1

A 0 )= 0 B −1

AA−1 0 I0 = = =I 0 BB −1 0I

A 0 0 B

where we have multiplied as block matrices. This proves that diag(A, B) is non-singular, with inverse diag(A−1 , B −1 ). c) Show that diag(A, B)diag(C, D) = diag(AC, BD).

Solution.

We have that

A 0 C 0 AC 0 diag(A, B)diag(C, D) = = = diag(AC, BD) 0 B 0 D 0 BD where we again have multiplied as block matrices.

4 Motivation for wavelets and some simple examples

101

Exercise 4.7: The DWT when N is odd When N is odd, the (first stage in a) DWT is defined as the change of coordinates from (φ1,0 , φ1,1 , . . . , φ1,N −1 ) to (φ0,0 , ψ0,0 , φ0,1 , ψ0,1 , . . . , φ0,(N −3)/2 , ψ0,(N −3)/2 , φ0,(N −1)/2 ).

Since all functions are assumed to have period N , we have that 1 1 φ0,(N −1)/2 = √ (φ1,N −1 + φ1,N ) = √ (φ1,0 + φ1,N −1 ).

2 2 This says that the last column in the change of coordinate matrix from C1 to φ1 (i.e. the IDWT matrix) equals √12 (1, 0, ..., 0, 1). In particular, for N = 3, the IDWT matrix equals   1 1 1 1  √ 1 −1 0 , 2 0 0 1 The DWT matrix, i.e. the inverse of this, is   1 1 −1 1 √ 1 −1 −1 .

2 0 0 2 a) Explain from this that, when N is odd, the DWT matrix can be constructed from the DWT matrix with N − 1 columns by adding √12 (−1, −1, 0, . . . , 0, 2)T as the last columns, and adding zeros in the last row.

Solution.

Note that the (N − 1) × (N − 1)-IDWT matrix equals the upper left block of the N × N -IDWT matrix. To be more precise we have that √

IDWTN −1 e0 /√ 2 IDWTN = .

0 1/ 2 Subtracting the last row from the first we obtain √

IDWTN −1 0√ I −e0 IDWTN −1 e0 /√ 2 = .

0 1 0 1/ 2 0 1/ 2 Thus

IDWTN =

I e0 0 1

IDWTN −1 0√ 0 1/ 2

Both matrices here are easily inverted and we obtain

102

4 Motivation for wavelets and some simple examples

√ √   √ 1/√2 1 √2 0 0 0 · · · 0 −1/√2 0 0 · · · 0 −1/ 2

  1/ 2 −1 2 0√ √ DWTN −1 √0 I −e0  0 0 1/ 2 1 2 0 · · · 0 0  DWTN = =  0 1 0 2  . .. .. .. .. . . .. ..   .. . . . . . . √.  2 0 0 0 0 0 ··· 0 where multiplication with the matrix to the right corresponded to subtracting the first column from the last. This gives the stated expression for the last column. b) Explain that the DWT matrix is orthogonal if and only if N is even. Also explain that it is only the last column which spoils the orthogonality.

Solution. √ √ 1/√2 1 √2 When N is even orthogonality follows from the orthogonality of . If 1/ 2 −1 2 DWTN was orthogonal for N odd, the last row of the IDWT-matrix would be equal to the last column of the DWT matrix. But the last row of the IDWT matrix has one nonzero entry, while the last column of the DWT matrix has three nonzero entries. This proves that DWTN is not orthogonal for N odd.

Exercise 4.8: General kernel transformations In the next sections we will define other functions φ and ψ. For these we will also define φm,n (t) = 2m/2 φ(2m t − n) and ψm,n (t) = 2m/2 ψ(2m t − n), and we will define resolutionand detail spaces using these instead, and find similar properties to those we found in this section. These new functions give rise to new kernel transformations G and H. From this exercise it will be clear that, also in this more general setting, it causes no confusion to suppress the resolution m in the notation for the kernel transformations (i.e. H and G), as we have done. a) Show that if

φ(t) =

X

an φ1,n (t)

ψ(t) =

n

X

bn φ1,n (t),

n

then also

φm,k (t) =

X n

Solution.

We have that

an φm+1,2k+n (t)

ψm,k (t) =

X n

bn φm+1,2k+n (t).

4 Motivation for wavelets and some simple examples

103

φm,k (t) = 2m/2 φ(2m t − k) = 2m/2

X

an φ1,n (2m t − k)

n

=

X

(m+1)/2

an 2

φ(2

m+1

t − 2k − n)

n

=

X

an φm+1,2k+n (t).

n

A similar computation can be done for ψ. b) Show that if

φ1,0 (t) =

X

(cn φ0,n (t) + dn ψ0,n (t))

n

φ1,1 (t) =

X

(en φ0,n (t) + fn ψ0,n (t)) ,

n

then also

φm+1,2k (t) =

X

(cn φm,n+k (t) + dn ψm,n+k (t))

n

φm+1,2k+1 (t) =

X

(en φm,n+k (t) + fn ψm,n+k (t)) .

n

Solution.

We have that φm+1,2k (t) = 2(m+1)/2 φ(2m+1 t − 2k) = 2m/2 21/2 φ(2(2m t − k)) = 2m/2 φ1,0 (2m t − k)

X = bn 2m/2 φ0,n (2m t − k) + cn 2m/2 ψ0,n (2m t − k) n

=

X

(bn φm,n+k (t) + cn ψm,n+k (t))

n

A similar computation is done for φm+1,2k+1 (t).

Exercise 4.13: Finding N Assume that you run an m-level DWT on a vector of length r. What value of N does this correspond to?

104

4 Motivation for wavelets and some simple examples

Solution.

The functions φm,n are 2−m apart, and since there are r of them we must have that N = r2−m . In particular, if r is not divisible by 2m , N will not be an integer.

Exercise 4.14: Different DWT’s for similar vectors In Figure 4.1 we have plotted the DWT’s of two vectors x1 and x2 . In both vectors we have 16 ones followed by 16 zeros, and this pattern repeats cyclically so that the length of both vectors is 256. The only difference is that the second vector is obtained by delaying the first vector with one element.

1.2 1.0 0.8 0.6 0.4 0.2 0.0

50

100

150

200

250

1.2 1.0 0.8 0.6 0.4 0.2 0.0

50

100

150

200

250

Fig. 4.1 2 vectors x1 and x2 which seem equal, but where the DWT’s are very different.

You see that the two DWT’s are very different: For the first vector we see that there is much detail present (the second part of the plot), while for the second vector there is no detail present. Attempt to explain why this is the case. Based on your answer, also attempt to explain what can happen if you change the point of discontinuity for the piecewise constant function to something else.

Solution.

The basis vectors in φ1 have support intervals of the form [n/2, (n + 1)/2), while the basis vectors in φ0 have support intervals of the form [n, n + 1). The functions in V1 with zero detail are thus exactly the functions which are constant on intervals of the form [n, n + 1). It is straightforward to check that the function corresponding to the first vector is non-zero and constant on [0, 8), [16, 24), and so on, while for the second vector we obtain [0.5, 8.5), [16.5, 24.5), and so on. The second vector therefore has detail associated with it, while the first one has not.

If the function φ was changed so that its support is [0.5, 1.5), the appearance of the functions in φ1 without detail would change accordingly. In this case you are invited to check that the first vector above has detail, while the second does not.

4 Motivation for wavelets and some simple examples

105

Exercise 4.15: Construct a sound Construct a (nonzero) sound where the low resolution approximations equal the sound itself for m = 1, m = 2.

Solution.

It is enough to consider a sound where the samples satisfy x4r = x4r+1 = x4r+2 = x4r+3 for any r. This makes the function in V2 with coordinates x in φ2 constant on intervals of the form [n, n + 1), so that it is in V0 , and therefore has no detail associated with it.

It follows that both low resolution approximations equal the sound itself.

Exercise 4.16: Exact computation of wavelet coefficients For the following functions compute the detail coefficients analytically, i.e. compute the RN quantities wm,n = 0 f (t)ψm,n (t)dt similarly to how this was done in Example 4.12 in the book: a) The functions in Example 4.11 in the book.

Solution.

Note first that Z N

f (t)ψm,n (t)dt = 2m/2

Z (n+1)2−m

Z (n+1/2)2−m f (t)dt − n2−m

0

! f (t)dt .

(n+1/2)2−m

With f (t) = 1 − 2|1/2 − t/N | we have two possibilities: when n < N 2m−1 we have that [n2−m , (n + 1)2−m ) ⊂ [0, N/2], so that f (t) = 2t/N , and we get

wm,n = 2m/2

Z (n+1/2)2−m

Z (n+1)2−m 2t/N dt −

n2−m (n+1/2)2−m

= 2m/2 [t2 /N ]n2−m

!

2t/N dt

(n+1/2)2−m (n+1)2−m

− 2m/2 [t2 /N ](n+1/2)2−m

2−3m/2 2−3m/2−1 2(n + 1/2)2 − n2 − (n + 1)2 = − .

N N RN When n ≥ N 2m−1 we have that f (t) = 2 − 2t/N , and using that 0 ψm,n (t)dt = 0 we =

−3m/2−1

must get that wm,n = 2 N .

For f (t) = 1/2 + cos(2πt/N )/2, note first that this has the same coefficients as RN cos(2πt/N )/2, since 0 ψm,n (t)dt = 0. We now get

106

4 Motivation for wavelets and some simple examples

Z (n+1/2)2−m

m/2

Z (n+1)2−m cos(2πt/N )/2dt −

wm,n = 2

n2−m

cos(2πt/N )/2dt (n+1/2)2−m

(n+1/2)2−m

= 2m/2 [N sin(2πt/N )/(4π)]n2−m =

!

(n+1)2−m

− 2m/2 [N sin(2πt/N )/(4π)](n+1/2)2−m

2m/2−2 N 2 sin(2π(n + 1/2)2−m /N ) − sin(2πn2−m /N ) − sin(2π(n + 1)2−m /N ) . π

There seems to be no more possibilities for simplification here. k b) The functions f (t) = Nt .

Solution.

We get

m/2

Z (n+1/2)2−m

k

Z (n+1)2−m

(t/N ) dt −

wm,n = 2

n2−m

(t/N ) dt (n+1/2)2−m

(n+1/2)2−m

= 2m/2 [tk+1 /((k + 1)N k )]n2−m =

! k

(n+1)2−m

− 2m/2 [tk+1 /((k + 1)N k )](n+1/2)2−m

2−m(k+1/2) 2(n + 1/2)k+1 − nk+1 − (n + 1)k+1 . k (k + 1)N

The leading term nk+1 will here cancel, but the others will not, so there is no room for further simplification here.

Exercise 4.17: Computing the DWT of a simple vector Suppose that we have the vector x with length 210 = 1024, defined by xn = 1 for n even, xn = −1 for n odd. What will be the result if you run a 10-level DWT on x? Use the function dwt_impl to verify what you have found.

Hint. √ We defined ψ by ψ(t) = (φ it follows that √1,0 (t) − φ1,1 (t))/ 2. From this connection √ ψ9,n = (φ10,2n − φ10,2n+1 )/ 2, and thus φ10,2n − φ10,2n+1 = 2ψ9,n . Try to couple this identity with the alternating sign you see in x.

Solution.

P1023 The vector x is the coordinate vector of the function f (t) = n=0 (−1)n φ10,n in the √ P511 √ basis φ10 for V10 . Since φ10,2n − φ10,2n+1 = 2ψ9,n , we can write f (t) = n=0 2ψ9,n .

Since a 10-level-DWT gives as a result the coordinate vector of f in (φ0 , ψ0 , ψ1 , ψ2 , ψ3 , ψ4 , ψ5 , ψ6 , ψ7 , ψ8 , ψ9 ),

4 Motivation for wavelets and some simple examples

107

(the DWT is √ nothing but the change of coordinates from φ10 to this basis), and√since P511 f (t) = n=0 2ψ9,n , it is clear that the coordinate vector of f in this basis has 2 in the second part (the ψ9 -coordinates), and 0 elsewhere. The 10-level DWT of x √therefore gives the vector of length 1024 which is 0 on the first half, and equal to 2 on the second half. m = 10 is here arbitrarily chosen: The result would have been the same for m = 1, m = 2, and so on. The following code verifies the result: dwt_impl(repmat([1; -1],512,1), ’Haar’, 10)

Exercise 4.18: Implementing the DWT when N is odd Use the results from Exercise 4.7 to rewrite the implementations dwt_kernel_haar and idwt_kernel_haar so that they also work in the case when N is odd.

Solution.

The library versions of these functions are as follows function x=dwt_kernel_haar(x, bd_mode) x = x/sqrt(2); N = size(x, 1); if mod(N,2) == 1 x(1:2, :) = [x(1, :) + x(2, :) - x(N, :);x(1, :) - x(2, :) - x(N, :)]; x(N, :) = 2*x(N, :); else x(1:2, :) = [x(1, :) + x(2, :); x(1, :) - x(2, :)]; end for k = 3:2:(N-1) x(k:(k+1), :) = [x(k, :) + x(k+1, :); x(k, :) - x(k+1, :)]; end end function x=idwt_kernel_haar(x, bd_mode) x = x/sqrt(2); N = size(x, 1); if mod(N,2) == 1 x(1:2, :) = [x(1, :) + x(2, :) + x(N, :); x(1, :) - x(2, :)]; else x(1:2, :) = [x(1, :) + x(2, :); x(1, :) - x(2, :)]; end for k = 3:2:(N-1) x(k:(k+1), :) = [x(k, :) + x(k+1, :); x(k, :) - x(k+1, :)]; end end

Since find_kernel finds and returns the kernels supported by the library, dwt_kernel_haar is defined within find_kernel.m in the library.

Exercise 4.19: In-place DWT and partial bit-reversal The kernel transformations in this book can be computed in-place. A DWT is also required to reorder coordinates as in the (φ0 , ψ0 , ..., ψm−1 )-basis, however, and this

108

4 Motivation for wavelets and some simple examples

exercise addresses how this reordering also can be computed in-place. As a result the entire DWT can be computed in-place. a) Show that the coordinates in φ0 after m iterations of a kernel transformation end up at indices k2m , k = 0, 1, 2, . . ., and that the coordinates in ψ0 end up at indices 2m−1 + k2m , k = 0, 1, 2, . . ..

Solution.

This is easily shown by induction. For m = 1 the statement says that the φ0 -coordinates are at the even indices, and the ψ0 -coordinates are at the odd indices. Both these statements are clearly true. For the induction step, if the coordinates in φ1 after m − 1 iterations of the kernel are at the indices k2m−1 , after the next iteration every second element of these (i.e. the ones at indices 2m−1 · 2k = k2m ) will be coordinates in φ0 . while the ones at indices 2m−1 (2k + 1) = 2m−1 + k2m will be coordinates in ψ0 . b) After m iterations of a kernel transformation, show that if the indices are rearranged so that the last m bits are placed in front in reverse order, then coordinates from φ0 are placed first, while coordinates from ψi are placed before those from {ψj }j>i . This is also called a partial bit-reversal. When an m-level DWT on a vector of length 2m is computed, the partial bit-reversal is actually a full bit-reversal.

After a partial bit-reversal, what can you say about the internal ordering in {ψj }j>i ?

Solution.

As proved in a), the ψm−n -coordinates end up at indices 2n−1 + k2n . After a partial bit-reversal, their bit representations have 0 · · · 01 as the first n bits. Lower n thus implies higher numbers, so that coordinates from ψi are placed before those from {ψj }j>i . Since φ0 -coordinates end up at indices k2m , their first m bits are 0 · · · 0, so that these are placed first.

The internal ordering of ψj will be violated by a partial bit-reversal, however. If one requires an in-place ordering that produces the same order as (φ0 , ψ0 , ..., ψm−1 ), one should secure that, for the ψm−j -coordinates, where the last j bits are 0 · · · 01, only those j bits should be placed in front in reverse order. Thus, partial bit-reversal reverses too many bits, violating the internal order in the ψm−j . Reversing the last bits and placing them in front should thus be replaced with a more refined version which adapts to the different resolutions. c) Write a function partial_bit_reversal(x, m) which computes a partial bitreversal of the vector x in-place. Assume that x has length 2n with n ≥ m.

Hint.

Partial bit-reversal can be split in two: First a full bit-reversal, and then reversing the last n − m bits again. It turns out that reversal of the last bits can be computed easily using bit-reversal of n − m bits. Due to this we can easily implement partial bit-reversal by adapting our previous function for bit-reversal.

4 Motivation for wavelets and some simple examples

109

Solution.

The general code can be found in the solution to d). d) Contrary to full bit-reversal, partial bit-reversal is not its own inverse. It is straightforward, however, to compute the inverse following the same lines as in c). Extend the function partial_bit_reversal so that it takes takes a third parameter forward, which indicates whether a forward- or reverse partial bit-reversal should be computed.

Solution.

The code can look as follows. The code calls a function which is a slight modification of bit_reversal, adapated to the current setting. function x=partial_bit_reversal(x, m, forward) N = size(x,1); if forward x=bit_reversal_last_bits(x, log2(N)); x=bit_reversal_last_bits(x, m); else x=bit_reversal_last_bits(x, m); x=bit_reversal_last_bits(x, log2(N)); end end function x=bit_reversal_last_bits(x, num_bits) N = 2^num_bits; j=0; for i = 0:2:(N/2 - 2) if (j > i) temp = x((j + 1):2^num_bits:end, :); x((j + 1):2^num_bits:end, :) = x((i + 1):2^num_bits:end, :); x((i + 1):2^num_bits:end, :) = temp; temp = x((N-i):2^num_bits:end, :); x((N-i):2^num_bits:end, :) = x((N-j):2^num_bits:end, :); x((N-j):2^num_bits:end, :) = temp; end temp = x((i+2):2^num_bits:end, :); x((i+2):2^num_bits:end, :) = x((j + N/2 + 1):2^num_bits:end, :); x((j + N/2 + 1):2^num_bits:end, :) = temp; m = N/4; while (m >= 1 && j >= m) j = j - m; m = m/2; end j = j + m; end end

Exercise 4.22: The vector of samples is the coordinate vector Show that, for f ∈ V0 we have that [f ]φ0 = (f (0), f (1), . . . , f (N − 1)). This shows that, also for the piecewise linear wavelet, there is no loss in working with the samples of f rather than f itself.

110

4 Motivation for wavelets and some simple examples

Solution.

Let us write f (t) =

PN −1

n=0 cn φ0,n (t). If k is an integer we have that

f (k) =

N −1 X

cn φ0,n (k) =

n=0

N −1 X

cn φ(k − n).

n=0

Clearly the only integer for which φ(s) 6= 0 is s = 0 (since φ(0) = 1), so that the only n which contributes in the sum is n = k. This means that f (k) = ck , so that [f ]φ0 = (f (0), f (1), . . . , f (N − 1)).

Exercise 4.23: The piecewise linear wavelet is not orthogonal Show that

hφ0,n , φ0,n i =

2 3

hφ0,n , φ0,n±1 i =

1 6

hφ0,n , φ0,n±k i = 0 for k > 1.

As a consequence, the {φ0,n }n are neither orthogonal, nor have norm 1.

Solution.

We have that Z n+1 hφ0,n , φ0,n i =

(1 − |t − n|)2 dt

n−1

Z n+1 =

1 − 2|t − n| + (t − n)2 dt

n−1

=2−2+

n+1 2 1 = . (t − n)3 3 3 n−1

We also have Z n+1 hφ0,n , φ0,n+1 =

Z 1 (1 − (t − n))(1 + (t − n − 1))dt =

n Z 1

= 0

(1 − u)(1 + u − 1)du 0

(t − t2 )dt =

1 1 1 − = .

2 3 6

Finally, the supports of φ0,n and φ0,n±k are disjoint for k > 1, so that we must have hφ0,n , φ0,n±k i = 0 in that case.

4 Motivation for wavelets and some simple examples

111

Exercise 4.24: Implementation of elementary lifting of odd type Write a function lifting_odd_symm(lambda, x, bd_mode)

which applies an elementary lifting matrix of odd type (equation (4.28) in the book) to x. Assume that N is even. The parameter bd_mode should do nothing, as we will return to this parameter later. The function should not allocate the full matrix, and apply as few multiplications as possible. How many local variables are needed in your code? Compare with the corresponding number you found for filters in Exercise 3.26.

Solution.

The library version of this function is as follows. function x=lifting_odd_symm(lambda, x, bd_mode) N = size(x, 1); x(2:2:(N-1), :) = x(2:2:(N-1),:) + lambda*(x(1:2:(N-2),:) + x(3:2:N,:)); if mod(N,2)==0 % last must also be included if strcmpi(bd_mode, ’symm’) x(N, :) = x(N, :) + 2*lambda*x(N-1, :); % Symmetric extension elseif strcmpi(bd_mode, ’per’) x(N, :) = lambda*(x(1, :) + x(N-1, :)) + x(N, :); elseif strcmpi(bd_mode, ’bd’) || strcmpi(bd_mode, ’none’) x(N, :) = lambda*x(N-1, :) + x(N, :); end end end

Since λ appears twice in each row, note that we have added entries before multiplication, to avoid extra multiplication. We also see that there is no need for local variables in the code.

Exercise 4.25: Computing projections In this exercise we will show how the projection of φ1,1 onto V0 can be computed.

We will see from this that it is nonzero, and that its support is the entire [0, N ]. Let f = projV0 φ1,1 , and let xn = f (n) for 0 ≤ n < N . This means that, on (n, n + 1), f (t) = xn + (xn+1 − xn )(t − n).

R n+1 a) Show that n f (t)2 dt = (x2n + xn xn+1 + x2n+1 )/3.

Solution.

We have that

112

4 Motivation for wavelets and some simple examples

Z n+1

f (t)2 dt =

Z n+1

n

(xn + (xn+1 − xn )(t − n))2 dt =

n

Z 1 =

Z 1

(xn + (xn+1 − xn )t)2 dt

0

(x2n + 2xn (xn+1 − xn )t + (xn+1 − xn )2 t2 )dt

0

1 = x2n t + xn (xn+1 − xn )t2 + (xn+1 − xn )2 t3 /3 0 1 = x2n + xn (xn+1 − xn ) + (xn+1 − xn )2 /3 = (x2n + xn xn+1 + x2n+1 ).

3 b) Show that Z 1/2

√ (x0 + (x1 − x0 )t)φ1,1 (t)dt = 2 2

1 1 x0 + x1 12 24

√ (x0 + (x1 − x0 )t)φ1,1 (t)dt = 2 2

1 1 x0 + x1 .

24 12

0

Z 1 1/2

Solution.

We have that Z 1/2 (x0 + (x1 − x0 )t)φ1,1 (t)dt 0

Z 1/2 =

√ Z 1/2 (x0 t + (x1 − x0 )t2 )dt (x0 + (x1 − x0 )t)2 2tdt = 2 2 √

0

0

1/2

√ √ 1 1 1 1 =2 2 x0 + (x1 − x0 )) = 2 2 x0 t2 + (x1 − x0 )t3 2 3 8 24 0

√ 1 1 =2 2 x0 + x1 .

12 24 In the same way Z 1 (x0 + (x1 − x0 )t)φ1,1 (t)dt 1/2

Z 1 =

√ Z 1 (x0 + (x1 − 2x0 )t − (x1 − x0 )t2 )tdt (x0 + (x1 − x0 )t)2 2(1 − t)tdt = 2 2

1/2

√

1/2

1

√ √ 1 1 1 3 7 = 2 2 x0 t + (x1 − 2x0 )t2 − (x1 − x0 )t3 =2 2 x0 + (x1 − 2x0 ) − (x1 − x0 ) 2 3 2 8 24 1/2

√ 1 1 =2 2 x0 + x1 .

24 12 c) Use the fact that

4 Motivation for wavelets and some simple examples

Z N (φ1,1 (t) − 0

xn φ0,n (t))2 dt

n=0

Z 1 =

φ1,1 (t)2 dt − 2

N −1 Z n+1 X n=0

Z 1

Z 1/2

(x0 + (x1 − x0 )t)φ1,1 (t)dt

(x0 + (x1 − x0 )t)φ1,1 (t)dt − 2 1/2

0

0

+

N −1 X

113

(xn + (xn−1 − xn )t)2 dt

n

and a) and b) to find an expression for kφ1,1 (t) −

PN −1

2 n=0 xn φ0,n (t)k .

Solution.

Using a) and b) we see that the above can be written as

N −1 √ √ 2 X 1 2 1 1 1 1 + (xn + xn xn+1 + x2n+1 ) − 2 2 2 x0 + x1 − 2 2 x0 + x1 3 n=0 3 12 24 24 12 √ N −1 N −1 2 2 X 2 1 X 2 xn + xn xn+1 − = + (x0 + x1 ).

3 3 n=0 3 n=0 2 d) To find the minimum least squares error, we can set the gradient of the expression in c) to zero, and thus find the expression for the projection of φ1,1 onto V0 . Show that −1 1 the values {xn }N n=0 can be found by solving the equation Sx = b, where S =√3 {1, 4, 1} is an N × N symmetric filter, and b is the vector with components b0 = b1 = 2/2, and bk = 0 for k ≥ 2.

Solution.

We see that the partial derivatives of the function in c) are √ 1 4 1 ∂f 2 = xN −1 + x0 + x1 − ∂x0 3 3 3 2 √ ∂f 1 4 1 2 = x0 + x1 + x1 − ∂x1 3 3 3 2 ∂f 1 4 1 = xi−1 + xi + xi+1 2 ≤ i < N − 1 ∂xi 3 3 3 1 4 1 ∂f = xN −2 + xN −1 + x0 . ∂xN −1 3 3 3 √

Moving the two terms 22 over to the right hand side, setting the gradient equal to zero is the same as solving the system Sx = b which we stated. e) Solve the system in d. for some values of N to verify that the projection of φ1,1 onto V0 is nonzero, and that its support covers the entire [0, N ].

114

4 Motivation for wavelets and some simple examples

Solution.

The following code can be used N = 16; S=zeros(N); S(1,N)=1/3; S(1,1)=4/3; S(1,2)=1/3; % First row for k=2:(N-1) S(k,(k-1):(k+1)) = [1/3 4/3 1/3]; end S(N,N-1)=1/3; S(N,N)=4/3; S(N,1)=1/3; % Last row b=zeros(N,1); b(1)=sqrt(2)/2; b(2)=sqrt(2)/2; plot(0:(N-1),S\b) %Plots the projection

Exercise 4.26: Convolution relation between the piecewise constant- and piecewise linear scaling functions The convolution of two functions defined on (−∞, ∞) is defined by Z ∞ (f ∗ g)(x) = f (t)g(x − t)dt. −∞

Show that we can obtain the piecewise linear φ we have defined as φ = χ[−1/2,1/2) ∗ χ[−1/2,1/2) (recall that χ[−1/2,1/2) is the function which is 1 on [−1/2, 1/2) and 0, see equation (1.16) in the book). This gives us a nice connection between the piecewise constant scaling function (which is similar to χ[−1/2,1/2) ) and the piecewise linear scaling function in terms of convolution.

Solution.

We have that Z ∞ χ[−1/2,1/2) ∗ χ[−1/2,1/2) (x) =

χ[−1/2,1/2) (t)χ[−1/2,1/2) (x − t)dt. −∞

The integrand here is 1 when −1/2 < t < 1/2 and −1/2 < x − t < 1/2, or in other words when max(−1/2, −1/2 + x) < t < min(1/2, 1/2 + x) (else it is 0). When x > 0 this happens when −1/2 + x < t < 1/2, and when x < 0 this happens when −1/2 < t < 1/2 + x. This means that (R 1/2 dt = 1 − x , x > 0 χ[−1/2,1/2) ∗ χ[−1/2,1/2) (x) = R−1/2+x 1/2+x dt = 1 + x , x < 0. −1/2 But this is by definition φ.

4 Motivation for wavelets and some simple examples

115

Exercise 4.29: Implementation of elementary lifting of even type Write a function lifting_even_symm(lambda, x, bd_mode)

which applies an elementary lifting matrix of even type (equation (4.36) in the book) to x. As before, assume that N is even, and that the parameter bd_mode does nothing.

Solution.

The library version of this function is as follows. function x=lifting_even_symm(lambda, x, bd_mode) N = size(x, 1); if strcmpi(bd_mode, ’per’) assert(mod(N,2) == 0) end if strcmpi(bd_mode, ’symm’) x(1, :) = x(1, :) + 2*lambda*x(2, :); % Symmetric extension elseif strcmpi(bd_mode, ’per’) x(1, :) = lambda*(x(2, :) + x(N, :)) + x(1, :); elseif strcmpi(bd_mode, ’bd’) || strcmpi(bd_mode, ’none’) x(1, :) = lambda*x(2, :) + x(1, :); end x(3:2:(N-1), :) = x(3:2:(N-1),:) + lambda*(x(2:2:(N-2),:) + x(4:2:N,:)); if mod(N,2) == 1 % last must also be included if strcmpi(bd_mode, ’symm’) x(N, :) = x(N, :) + 2*lambda*x(N-1, :); % Symmetric extension elseif strcmpi(bd_mode, ’bd’) || strcmpi(bd_mode, ’none’) x(N, :) = x(N, :) + lambda*x(N-1, :); end end end

Since also here λ appears twice in each row, we have added entries before multiplication, to avoid extra multiplication.

Exercise 4.30: Two vanishing moments In this exercise we will show that there is a unique function on the form given by equation (4.30) in the book which has two vanishing moments. a) Show that, when ψ̂ is defined by equation (4.30) in the book, we have that   −αt − α for − 1 ≤ t < 0      (2 + α − β)t − α for 0 ≤ t < 1/2  ψ̂(t) = (α − β − 2)t − α + 2 for 1/2 ≤ t < 1    βt − 2β for 1 ≤ t < 2    0 for all other t

116

4 Motivation for wavelets and some simple examples

Solution.

The function ψ̂ is a sum of the functions ψ = √12 φ1,1 , φ, and φ0,1 (i.e. we have set n = 0 in equation (4.30) in the book). All these are continuous and piecewise linear, and we can write   0 ≤ t < 1/2 2t ψ(t) = 2 − 2t 1/2 ≤ t < 1   0 elsewhere   1 + t −1 ≤ t < 0 φ0,0 (t) = 1 − t 0 ≤ t < 1   0 elsewhere   0≤t<1 t φ0,1 (t) = 2 − t 1 ≤ t < 2 .   0 elsewhere It follows that ψ̂(t) = ψ(t) − αφ(t) − βφ1,1 is piecewise linear, and linear on the segments [−1, 0], [0, 1/2], [1/2, 1], [1, 2].

On the segment [−1, 0] only the function φ is seen to be nonzero, and since φ(t) = 1+t here, we have that ψ̂(t) = −α(1 + t) = −α − αt here.

On the segment [0, 1/2] all three functions are nonzero, and

φ1,1 (t) = 2t φ0,0 (t) = 1 − t φ0,1 (t) = t on this interval. This means that ψ̂(t) = 2t − α(1 − t) − βt = (2 + α − β)t − α on [0, 1/2].

On the segment [0, 1/2] all three functions are nonzero, and φ1,1 (t) = 2 − 2t φ0,0 (t) = 1 − t φ0,1 (t) = t on this interval. This means that ψ̂(t) = 2 − 2t − α(1 − t) − βt = (α − β − 2)t − α + 2 on [1/2, 1].

On the segment [1, 2] only the function φ0,1 is seen to be nonzero, and since φ0,1 (t) = 2 − t here, we have that ψ̂(t) = −β(2 − t) = βt − 2β here. For all other values of t, ψ̂ is zero. This proves the formulas for ψ̂ on the different intervals. b) Show that Z N ψ̂(t)dt = 0

1 − α − β, 2

Z N tψ̂(t)dt = 0

1 − β.

4

4 Motivation for wavelets and some simple examples

117

Solution.

We can write Z N

Z 2

Z 0

ψ̂(t)dt =

ψ̂(t)dt = −1

0

Z 1/2 ψ̂(t)dt +

−1

ψ̂(t)dt + 0

Z 2 ψ̂(t)dt +

1/2

ψ̂(t)dt 1

Z 1/2

Z 0

(2 + α − β)t − α)dt

(−α − αt)dt +

=

Z 1

−1 Z 1

0

Z 2 (βt − 2β)dt

((α − β − 2)t − α + 2)dt +

+ 1/2

1

0 1/2

1 2 1 2 = −αt − αt + (2 + α − β)t − αt 2 2 −1 0 1

2

1 2 1 2 + βt − 2βt + (α − β − 2)t + (2 − α)t 2 2 1/2 1 1 1 3 1 3 1 = −α + α + (2 + α − β) − α + (α − β − 2) + (2 − α) + β − 2β 2 8 2 8 2 2 1 = − α − β, 2 RN 0

tψ̂(t)dt is computed similarly, so that we in the end arrive at 14 − β.

c) Explain why there is a unique function on the form given by equation (4.30) in the book which has two vanishing moments, and that this function is given by equation (4.32) in the book.

Solution.

The equation system 1 −α−β =0 2 1 −β =0 4 has the unique solution α = β = 14 , which we already have found.

Exercise 4.31: More than two vanishing moments In the previous exercise we ended up with a lot of calculations to find α, β in equation (4.30) in the book. Let us try to make a program which does this for us, and which also makes us able to generalize the result. a) Define

118

4 Motivation for wavelets and some simple examples

Z 1 ak =

tk (1 − |t|)dt,

Z 2 bk =

−1

tk (1 − |t − 1|)dt,

0

Z 1 ek =

tk (1 − 2|t − 1/2|)dt,

0

for k ≥ 0. Explain why finding α, β so that we have two vanishing moments in equation (4.30) in the book is equivalent to solving the following equation:

a0 b0 α e = 0 a1 b1 β e1 Write a program which sets up and solves this system of equations, and use this program to verify the values for α, β we previously have found.

Hint. you can integrate functions in MATLAB with the function quad. As an example, the function φ(t), which is nonzero only on [−1, 1], can be integrated as follows: quad(@(t)t.^k.*(1-abs(t)),-1,1)

Solution.

In order for ψ to have vanishing moments we must have that Substituting ψ̂ = ψ − αφ0,0 − βφ0,1 we see that, for k = 0, 1, Z Z k t (αφ0,0 + βφ0,1 ) dt = tk ψ(t)dt.

R

ψ̂(t)dt =

R

tψ̂(t)dt = 0

The left hand side can here be written Z

tk (αφ0,0 + βφ0,1 ) dt = α

Z

tk φ0,0 dt + β

Z

tk φ0,1 (t)dt Z 1 Z 2 =α tk (1 − |t|)dt + β tk (1 − |t − 1|)dt = αak + βbk . −1

0

The right hand side is Z

k

t ψ(t)dt =

Z

k

Z 1 (1 − 2|t − 1/2|)dt = ek .

t φ1,1 (t)dt = 0

The following program sets up the corresponding equation systems, and solves it by finding α, β.

A = zeros(2); b = zeros(2, 1); for k = 0:1 A(k + 1, :) = [quad(@(t)t.^k.*(1 - abs(t)), -1, 1)... quad(@(t)t.^k.*(1 - abs(t - 1)), 0, 2)]; b(k + 1) = quad(@(t)t.^k.*(1 - 2*abs(t - 1/2)), 0, 1); end A\b;

4 Motivation for wavelets and some simple examples

119

b) The procedure where we set up a matrix equation in a) allows for generalization to more vanishing moments. Define ψ̂ = ψ0,0 − αφ0,0 − βφ0,1 − γφ0,−1 − δφ0,2 .

We would like to choose α, β, γ, δ so that we have 4 vanishing moments. Define also Z 0 gk =

tk (1 − |t + 1|)dt,

Z 3 dk =

−2

tk (1 − |t − 2|)dt

1

for k ≥ 0. Show that α, β, γ, δ must solve the equation      a0 b0 g0 d0 α e0 a1 b1 g1 d1  β  e1       a2 b2 g2 d2   γ  = e2  , a3 b3 g3 d3 δ e3 and solve this with your computer.

Solution.

Similarly to a), the equation in b) gives that Z Z tk (αφ0,0 + βφ0,1 + γφ0,−1 + δφ0,2 ) dt = tk ψ(t)dt.

The corresponding equation system is deduced exactly as in a). The following program sets up the corresponding equation systems, and solves it by finding α, β, γ, δ.

A = zeros(4); b = zeros(4, 1); for k = 0:3 A(k + 1, :) = [quad(@(t)t.^k.*(1 - abs(t)), - 1, 1)... quad(@(t)t.^k.*(1 - abs(t - 1)), 0, 2)... quad(@(t)t.^k.*(1 - abs(t + 1)), -2, 0)... quad(@(t)t.^k.*(1 - abs(t - 2)), 1, 3)]; b(k + 1) = quad(@(t)t.^k.*(1 - 2*abs(t - 1/2)), 0, 1); end coeffs=A\b;

c) Plot the function defined by the equation from b).

Hint.

If t is the vector of t-values, and you write (t >= 0).*(t <= 1).*(1-2*abs(t-0.5))

you get the points φ1,1 (t).

Solution.

The function ψ̂ now is supported on [−2, 3], and can be plotted as follows:

120

4 Motivation for wavelets and some simple examples

t=linspace(-2,3,100); plot(t, ( t>= 0).*(t <= 1).*(1-2*abs(t - 0.5)) ...

-coeffs(1)*(t >= -1).*(t <= 1).*(1 - abs(t))...

-coeffs(2)*(t >= 0).*(t <= 2).*(1 - abs(t - 1))...

-coeffs(3)*(t >= -2).*(t <= 0).*(1 - abs(t + 1))...

-coeffs(4)*(t >= 1).*(t <= 3).*(1 - abs(t - 2)))

d) Explain why the coordinate vector of ψ̂ in the basis (φ0 , ψ0 ) is [ψ̂](φ0 ,ψ0 ) = (−α, −β, −δ, 0, . . . , 0 − γ) ⊕ (1, 0, . . . , 0).

Hint.

The placement of −γ may seem a bit strange here, and has to do with that φ0,−1 −1 is not one of the basis functions {φ0,n }N n=0 . However, we have that φ0,−1 = φ0,N −1 , i.e. φ(t + 1) = φ(t − N + 1), since we always assume that the functions we work with have period N .

Solution.

The coordinates of φ0,0 , φ0,1 , and φ0,2 in ψ̂ are −α, −β, and −δ, respectively. Since these three basis functions occur first in (φ0 , ψ0 ), they appear first in [ψ̂](φ0 ,ψ0 ) . Since φ0,−1 = φ0,N −1 , and since this is the last basis function in the first half of (φ0 , ψ0 ), −γ (the corresponding coordinate) is placed accordingly. Finally, ψ0,0 is th first basis function in the second half of (φ0 , ψ0 ), so that a 1 (the coordinate) should be placed at that location. e) Compute the coordinates of ψ̂ in the basis φ1 (i.e. [ψ̂]φ1 ) with N = 8, i.e. compute the IDWT of [ψ̂](φ0 ,ψ0 ) = (−α, −β, −δ, 0, 0, 0, 0, −γ) ⊕ (1, 0, 0, 0, 0, 0, 0, 0), which is the coordinate vector you computed in d). For this, you should use the function idwt_impl, with the kernel of the piecewise linear wavelet without symmetric extension as input.

Solution.

The code which can be used looks like this: g1=idwt_impl([-coeffs(1);-coeffs(2);-coeffs(4);0;0;0;0;-coeffs(3);...

1; 0; 0; 0; 0; 0; 0; 0], ’pwl0’, 1, ’per’); g1 = [g1(14:16); g1(1:6)]; % Compact filter notation

Note that we have used a kernel which does not make symmetric extensions. f) Sketch a more general procedure than the one you found in b), which can be used to find wavelet bases where we have even more vanishing moments.

4 Motivation for wavelets and some simple examples

121

Solution.

If we define ψ̂ = ψ0,0 −

K X

(αk φ0,−k − βk φ0,k+1 ) ,

k=0

we have 2k unknowns. These can be determined if we require 2k vanishing moments.

Exercise 4.32: Two vanishing moments for the Haar wavelet Let φ be the scaling function of the Haar wavelet. a) Compute projV0 f , where f (t) = t2 , and where f is defined on [0, N ).

Solution.

We have that

projV0 f =

N −1 X

ht2 , φ0,n iφ0,n =

n=0

n=0

=

N −1 Z n+1 X

t2 dtφ0,n

n

N −1 X 1 ((n + 1)3 − n3 )φ0,n = (n2 + n + 1/3)φ0,n .

3 n=0 n=0

N −1 X

b) Find constants α, β so that ψ̂ = ψ − αφ0,0 − βφ0,1 has two vanishing moments, i.e. so that hψ̂, 1i = hψ̂, ti = 0. Plot also the function ψ̂.

Hint.

Start with computing the integrals R R tφ0,0 (t)dt, tφ0,1 (t)dt.

Solution.

We get that

R

ψ(t)dt,

R

tψ(t)dt,

R

φ0,0 (t)dt,

R

φ0,1 (t)dt, and

122

4 Motivation for wavelets and some simple examples

Z ψ(t)dt = 0 Z 1

Z 1/2

Z

tdt −

tψ(t)dt = Z

1/2

tdt = [t2 /2]0

− [t2 /2]11/2 = 1/8 − 1/2 + 1/8 = −1/4

1/2

0

Z φ0,0 (t)dt =

φ0,1 (t)dt = 1 Z 1

Z tφ0,0 (t)dt =

tdt = 1/2 0

Z 2

Z tφ0,1 (t)dt =

tdt = 3/2.

1

hψ̂, 1i = hψ̂, ti = 0 can be written as the system αhφ0,0 , 1i + βhφ0,1 , 1i = hψ, 1i αhφ0,0 , ti + βhφ0,1 , ti = hψ, ti so that

α+β =0

α/2 + 3β/2 = −1/4

which has the solution α = 1/4, β = −1/4. ψ̂ = ψ − φ0,0 /4 + βφ0,1 /4 can be plotted as follows. t=linspace(0,4,100); phi = @(t) (0<=t).*(t<1); psi = @(t) (0<=t).*(t<1/2) - (1/2<=t).*(t<1); plot(t,psi(t)-phi(t)/4+phi(t-1)/4)

c) Express φ and ψ̂ with the help of functions from φ1 , and use this to write down the change of coordinate matrix from (φ0 , ψ̂0 ) to φ1 .

Solution.

We have that ψ̂ = ψ − φ0,0 /4 + βφ0,1 /4 1 1 1 = √ (φ1,0 − φ1,1 ) − √ (φ1,0 + φ1,1 ) + √ (φ1,2 + φ1,3 ) 2 4 2 4 2 3 5 1 1 = √ φ1,0 − √ φ1,1 + √ φ1,2 + √ φ1,3 4 2 4 2 4 2 4 2 This means that the last half of the columns in the change of coordinates matrix are the 1 translates of 4√ (3, −5, 1, 1). The remaining columns coincide with those of the matrix 2 for the Haar wavelet.

4 Motivation for wavelets and some simple examples

123

Exercise 4.33: More than two vanishing moments for the Haar wavelet It is also possible to add more vanishing moments to the Haar wavelet. Define ψ̂ = ψ0,0 − a0 φ0,0 − · · · − ak−1 φ0,k−1 .

R l+1 r R1 Define also cr,l = l t dt, and er = 0 tr ψ(t)dt. a) Show that ψ̂ has k vanishing moments if and only if a0 , . . . , ak−1 solves the system      e0 a0 c0,0 c0,1 · · · c0,k−1  c1,0 c1,1 · · · c1,k−1   a1   e1        ..   ..  =  ..  .. .. ..     .  .  . . . . ck−1,0 ck−1,1 · · · ck−1,k−1

ak−1

ek−1

Solution.

Multiplying with tr and computing the integral we obtain hψ̂, tr i = hψ0,0 , tr i − a0 hφ0,0 , tr i − · · · − ak−1 hφ0,k−1 , tr i.

Setting those to zero we obtain 0 = er − a0 cr,0 − · · · − ak−1 cr,k−1 , which is row r in the stated equation system. b) Write a function vanishing_moms_haar which takes k as input, solves the system in a), and returns the vector a = (a0 , a1 , . . . , ak−1 ).

Solution.

1 ((l + 1)r+1 − lr+1 ), and We first find that cr,l = r+1

1 (2−r − 1). r+1 The following code can be used. If you set k = 2 the values from the previous exercise are obtained.

1/2

er = [tr+1 /(r + 1)]0

− [tr+1 /(r + 1)]11/2 =

function sol=vanishing_moms_haar(k) l=0:(k-1); r=l’; C = ((l+1).^(r+1)-l.^(r+1))./(r+1); e = (2.^(-r)-1)./(r+1); sol = C\e; end

124

4 Motivation for wavelets and some simple examples

Exercise 4.34: Listening experiments Run the function forw_comp_rev_dwt1 for different m for the Haar wavelet, the piecewise linear wavelet, and the alternative piecewise linear wavelet, but listen to the detail components instead. Describe the sounds you hear for different m, and try to explain why the sound seems to get louder when you increase m.

Solution.

The following code can be used: [x, fs] = forw_comp_rev_dwt1(m, ’Haar’); playerobj = audioplayer(x, fs); playblocking(playerobj)

[x, fs] = forw_comp_rev_dwt1(m, ’pwl0’); playerobj = audioplayer(x, fs); playblocking(playerobj)

[x, fs] = forw_comp_rev_dwt1(m, ’pwl2’); playerobj = audioplayer(x, fs); playblocking(playerobj)

Chapter 5

The filter representation of wavelets

Exercise 5.4: Symmetry of MRA matrices vs. symmetry of filters a) Find two symmetric filters, so that the corresponding MRA-matrix is not symmetric.

Solution.

You can set for instance H0 = {1/4, 1/2, 1/4}, and H1 = {1} (when you write down the corresponding matrix you will see that A0,1 = 1/2, A1,0 = 0, so that the matrix is not symmetric) b) Assume that an MRA-matrix is symmetric. Are the corresponding filters H0 , H1 , G0 , G1 also symmetric? If not, find a counterexample.

Solution.

The Haar wavelet is a counterexample.

Exercise 5.5: Passing between the MRA-matrices and the filters a) Assume that we have the forward filter bank transform   1/5 1/5 1/5 0 0 0 · · · 0 1/5 1/5 −1/3 1/3 −1/3 0 0 0 ··· 0 0 0     1/5 1/5 1/5 1/5 1/5 0 · · · 0 0 0  H=   0 0 −1/3 1/3 −1/3 0 · · · 0 0 0    .. .. .. .. .. .. .. .. .. .. . . . . . . . . . .

Write down the corresponding filters H0 , H1 , and compute and plot the frequency responses. Are the filters symmetric?

125

126

5 The filter representation of wavelets

Solution.

We have that H0 = 15 {1, 1, 1, 1, 1}, and H1 = 13 {−1, 1, −1}. The frequency responses are 1 2iω 1 iω 1 1 −iω 1 −2iω e + e + + e + e 5 5 5 5 5 2 1 2 = cos(2ω) + cos ω + 5 5 5 1 iω 1 1 −iω 2 1 λH1 (ω) = − e + − e = − cos ω + .

3 3 3 3 3 λH0 (ω) =

Both filters are symmetric. b) Assume that we have the reverse filter bank transform   1/2 −1/4 0 0 ··· 1/4 3/8 1/4 1/16 · · ·    0 −1/4 1/2 −1/4 · · ·    0 1/16 1/4 3/8 · · ·    0 0 0 −1/4 · · ·   G= 0 0 0 1/16 · · ·    0 0 0 0 · · ·    .. .. .. .. ..   . . . . .     0 0 0 0 · · · 1/4 1/16 0 0 ··· Write down the filters G0 , G1 , and compute and plot the frequency responses. Are the filters symmetric?

Solution.

We have that G0 = {1/4, 1/2, 1/4}, and G1 = {1/16, −1/4, 3/8, −1/4, 1/16}. The frequency responses are 1 iω 1 1 −iω e + + e 4 2 4 1 1 = cos(ω) + 2 2 1 2iω 1 iω 3 1 −iω 1 −2iω λG1 (ω) = e − e + − e e 16 4 8 4 16 1 1 3 = cos(2ω) − cos ω + .

8 2 8 λG0 (ω) =

Both filters are symmetric. c) Assume that H0 = {1/16, 1/4, 3/8, 1/4, 1/16}, and H1 = {−1/4, 1/2, −1/4}. Plot the frequency responses of H0 and H1 , and verify that H0 is a low-pass filter, and that H1 is a high-pass filter. Also write down the corresponding forward filter bank transform H.

5 The filter representation of wavelets

127

Solution.

The frequency responses are 1 2iω 1 iω 3 1 −iω 1 −2iω e + e + + e e 16 4 8 4 16 1 3 1 = cos(2ω) + cos ω + 8 2 8 1 iω 1 1 −iω λH1 (ω) = − e + − e 4 2 4 1 1 = − cos(ω) + .

2 2

λH0 (ω) =

The two first rows in H are

3/8 1/4 1/16 0 · · · 1/16 1/4 −1/4 1/2 −1/4 0 · · · 0 0

The remaining rows are obtained by translating these in alternating order. d) Assume that G0 = 13 {1, 1, 1}, and G1 = 15 {1, −1, 1, −1, 1}. Plot the frequency responses of G0 and G1 , and verify that G0 is a low-pass filter, and that G1 is a high-pass filter. Also write down the corresponding reverse filter bank transform G.

Solution.

The frequency responses are 2 1 1 iω 1 1 −iω e + + e = cos ω + 3 3 3 3 3 1 2iω 1 iω 1 1 −iω 1 −2iω λG1 (ω) = e − e + − e e 5 5 5 5 5 2 2 1 = cos(2ω) − cos ω + 5 5 5 λG0 (ω) =

The two first columns in G are 

 1/3 −1/5 1/3 1/5     0 −1/5    0 1/5     0 0     .. ..   . .     0 0  1/3 1/5 The remaining columns are obtained by translating these in alternating order.

128

5 The filter representation of wavelets

Exercise 5.6: Computing by hand In Exercise 4.14 we computed the DWT of two very simple vectors x1 and x2 , using the Haar wavelet. a) Compute H0 x1 , H1 x1 , H0 x2 , and H1 x2 , where H0 and H1 are the filters used by the Haar wavelet.

Solution.

Recall that x1 consisted of 16 consecutive ones followed by 16 consecutive zeros, repeated so that the final vector has length 256. Also, x2 was x1 delayed by 1. Using the expressions for the filters H0 and H1 of the Haar wavelet we obtain √ √ √ √ H0 x1 = (2/ 2, ..., 2/ 2, 1/ 2, 0, ..., 0, 1/ 2, ...) | {z } | {z } 16 16 √ √ H1 x1 = (−1/ 2, 0, ..., 0, 1/ 2, 0, ..., 0, ...) {z } | {z } | 16 16 √ √ √ √ H0 x2 = (1/ 2, 2/ 2, ..., 2/ 2, 1/ 2, 0, ..., 0, ...) | {z } | {z } 16 16 √ √ H1 x2 = (0, −1/ 2, 0, ..., 0, 0, 1/ 2, 0, ..., 0, ...), {z } | {z } | 16

16

where the underbraced vectors are repeated 8 times. b) Compare the odd-indexed elements in H1 x1 with the odd-indexed elements in H1 x2 .

From this comparison, attempt to find an explanation to why the two vectors have very different detail components.

Solution.

In H1 x1 the only nonzero entries are at indices 0, 16, 32, and so on, while for H1 x2 they are at indices 1, 17, 33, and so on. Since the odd indices give the detail components, it follows that the detail of x1 is zero, while the detail of x2 is non-zero.

Exercise 5.7: Code for applying wavelets to sound Suppose that we run the following algorithm on the sound represented by the vector x: N=size(x,1); c = (x(1:2:N, :) + x(2:2:N, :))/sqrt(2); w = (x(1:2:N, :) - x(2:2:N, :))/sqrt(2); newx = [c; w]; newx = newx/max(abs(newx)); playerobj=audioplayer(newx,44100); playblocking(playerobj)

5 The filter representation of wavelets

129

a) Comment the code and explain what happens. Which wavelet is used? What do the vectors c and w represent? Describe the sound you believe you will hear.

Solution. c and w represent the coordinates in the wavelet bases φ0 and ψ0 . The code runs a Haar wavelet transform. The sound is normalized so that the sound samples lie in the range between −1 and 1, and the resulting sound is played. The sound is split into two parts, and c represents a low-resolution version of the sound (with half the number of samples), so that we first will hear the sound played at double pace. After this we will hear the detail w in the sound, also played at double pace. We should also be able to recognize the sound from this detail. b) Assume that we add lines in the code above which sets the elements in the vector w to 0 before we compute the inverse operation. What will you hear if you play this new sound?

Solution.

This corresponds to reconstructing a low-resolution approximation of the sound.

Exercise 5.8: Computing filters and frequency responses a) Write down the filter coefficients for the corresponding filter G1 obtained in Exercise 4.31, and plot its frequency response.

Solution.

The code can look as follows: omega = linspace(0,2*pi,100); plot(omega, g1(5) + g1(6)*2*cos(omega) + g1(7)*2*cos(2*omega)... + g1(8)*2*cos(3*omega) + g1(9)*2*cos(4*omega))

b) Write down the corresponding filters G0 and G1 for Exercise 4.32. Plot their frequency responses, and characterize the filters as low-pass- or high-pass.

Solution.

The filter G0 is the same as for the Haar wavelet, and this frequency response has already 1 been plotted. In Exercise 4.32 we obtained G1 = 4√ (3, −5, 1, 1), and the magnitude of 2 this frequency response can be plotted as follows. omega=linspace(0,2*pi,100); plot(omega, abs(fft([[3; -5; 1; 1]/(4*sqrt(2)); zeros(96, 1)])));

130

5 The filter representation of wavelets

c) Repeat b) for the Haar wavelet as in Exercise 4.33, and plot the corresponding frequency responses for k = 2, 4, 6.

Solution.

Using the function we implemented in Exercise 4.33, the following code can be used. for k = [2 4 6] x = vanishing_moms_haar(k); x = idwt_impl([-x; zeros(50-length(x),1); 1; zeros(49,1)],’haar’,1,’per’); figure() plot(omega, abs(fft(x))); end

Exercise 5.9: Perfect reconstruction for forward and reverse filter bank transforms Combine equations (5.4) and (5.5) in Theorem 3 in the book to show that 1 GHφr = (λH0 ,r λG0 ,r + λH1 ,r λG1 ,r )φr 2 1 + (λH0 ,r λG0 ,r+N/2 − λH1 ,r λG1 ,r+N/2 )φr+N/2 .

2 Conclude that, in terms of continuous frequency responses, we have alias cancellation if and only if λH0 (ω)λG0 (ω + π) = λH1 (ω)λG1 (ω + π), and perfect reconstruction if and only if in addition λH0 (ω)λG0 (ω) + λH1 (ω)λG1 (ω) = 2.

These are also called the alias cancellation- and perfect reconstruction conditions, respectively.

Solution.

We have that

5 The filter representation of wavelets

131

GHφr 1 1 = (λH0 ,r + λH1 ,r )Gφr + (λH0 ,r − λH1 ,r )Gφr+N/2 2 2

1 1 1 = (λH0 ,r + λH1 ,r ) (λG0 ,r + λG1 ,r )φr + (λG0 ,r+N/2 − λG1 ,r+N/2 )φr+N/2 2 2 2

1 1 1 + (λH0 ,r − λH1 ,r ) (λG0 ,r+N/2 + λG1 ,r+N/2 )φr+N/2 + (λG0 ,r − λG1 ,r )φr ) 2 2 2 1 = ((λH0 ,r + λH1 ,r )(λG0 ,r + λG1 ,r ) + (λH0 ,r − λH1 ,r )(λG0 ,r − λG1 ,r )) φr 4

1 + (λH0 ,r + λH1 ,r )(λG0 ,r+N/2 − λG1 ,r+N/2 ) + (λH0 ,r − λH1 ,r )(λG0 ,r+N/2 + λG1 ,r+N/2 ) φr+N/2 4 1 1 = (λH0 ,r λG0 ,r + λH1 ,r λG1 ,r )φr + (λH0 ,r λG0 ,r+N/2 − λH1 ,r λG1 ,r+N/2 )φr+N/2 .

2 2

Exercise 5.10: Conditions for perfect reconstruction In this exercise we will prove that, when G0 and H0 are given filters which satisfy a certain property, we can define unique (up to a constant) filters H1 and G1 so that all four filters together give perfect reconstruction. In Section 7.1 in the book we will also prove that, as long as all filters are FIR, these conditions are also necessary. a) Show that, if there exist α ∈ R and d ∈ Z so that (H1 )n = (−1)n α−1 (G0 )n−2d (G1 )n = (−1)n α(H0 )n+2d 2 = λH0 ,n λG0 ,n + λH0 ,n+N/2 λG0 ,n+N/2 then we also have that λH1 (ω) = α−1 e−2idω λG0 (ω + π) λG1 (ω) = αe2idω λH0 (ω + π) 2 = λH0 (ω)λG0 (ω) + λH0 (ω + π)λG0 (ω + π)

Solution.

We have that

λH1 (ω) =

X

(H1 )k e−ikω =

k

=α =α

−1

X (−1)k α−1 (G0 )k−2d e−ikω k

X

(−1) (G0 )k e−i(k+2d)ω = α−1 e−2idω

k −1 −2idω

e

k

X (G0 )k e−ik(ω+π) k

λG0 (ω + π),

132

5 The filter representation of wavelets

which proves the first equation. A similar computation proves the second equation. The last equation is simply a restatement in terms of continuous frequency responses. b) Show that the first two conditions in a) implies the alias cancellation condition from Exercise 5.9, and that the last condition implies the perfect reconstruction condition from Exercise 5.9.

Solution.

The first two conditions imply the alias cancellation condition is met since λH1 (ω)λG1 (ω + π) =α−1 e−2idω λG0 (ω + π)(α)e2id(ω+π λH0 (ω) =λH0 (ω)λG0 (ω + π).

Inserting this in the third condition we get

2 = λH0 (ω)λG0 (ω) + λG0 (ω + π)λH0 (ω + π) = λH0 (ω)λG0 (ω) + α−1 e−2idω λG0 (ω + π)αe2idω λH0 (ω + π) = λH0 (ω)λG0 (ω) + λH1 (ω)λG1 (ω), so that the perfect reconstruction condition also is met. c) Show that if the conditions in a) are met, and the H0 , H1 , G0 , G1 are non-trivial symmetric FIR filters, then necessarily d = 0. It follows that the conditions take the form λH1 (ω) = α−1 λG0 (ω + π) λG1 (ω) = αλH0 (ω + π) 2 = λH0 (ω)λG0 (ω) + λH0 (ω + π)λG0 (ω + π),

Solution.

Since H0 is symmetric, (H0 )n = (H0 )−n , and from the equations in a) it follows that (G1 )n−2d = (−1)n−2d α(H0 )n = (−1)n α−1 (H0 )−n = (−1)(−n−2d) α−1 (H0 )(−n−2d)+2d = (G1 )−n−2d This shows that G1 is symmetric about both −2d, in addition to being symmetric about 0 (by assumption). This means that the filter coefficients take the form of a symmetric extension if d 6= 0, as there two different symmetry points then. Mirroring about the symmetry points it is clear that there must be infinitely many filter coefficients, which is a contradiction. We must thus have that d = 0.

5 The filter representation of wavelets

133

Exercise 5.11: Finding the DWT filter components from the IDWT filter components Assume that H and G give perfect reconstruction. It may be that only one of H and G is known, and that we would like to find an expression for the other. To this end we will assume that the conditions in Exercise 5.10a) are met for some d and α. a) Assume that G0 , G1 are known and symmetric (so that d = 0). Deduce that X α= (−1)n (G0 )n (G1 )n . n

Deduce a similar formula for the case when H0 and H1 are known.

Solution.

From the equations in Exercise 5.10a) it follows that X X X 1= (G1 )n (H1 )n = (G1 )n α−1 (−1)n (G0 )n = α−1 (−1)n (G0 )n (G1 )n , n

n

n

so that α = n (−1)n (G0 )n (G1 )n . On the other hand, if H0 , H1 are known instead, we have similarly that P

1=

X X X (G1 )n (H1 )n = α(−1)n (H0 )n (H1 )n = α (−1)n (H0 )n (H1 )n , n

n

n

P so that α = 1/( n (−1)n (H0 )n (H1 )n ). b) Use a) to verify the expressions for the filters H0 and H1 of the alternative piecewise linear wavelet from Example 5.3 in the book.

Solution.

All filters for the alternative piecewise linear wavelet are symmetric. We get that

X 1 1 1 3 1 1 1 n α= (−1) (G0 )n (G1 )n = − − +1· − − = .

2 2 4 4 2 4 2 n We now get (H0 )n = α−1 (−1)n (G1 )n = 2(−1)n (G1 )n (H1 )n = α−1 (−1)n (G0 )n = 2(−1)n (G0 )n , so that

H0 = H1 =

√ √

2{−1/8, 1/4, 3/4, 1/4, −1/8} 2{−1/2, 1, −1/2},

134

5 The filter representation of wavelets

which is what we found in Example 5.3 in the book.

Exercise 5.12: Classical QMF filter bank transforms A classical QMF filter bank is one where G0 = H0 and G1 = H1 (i.e. the filters in the forward and reverse transforms are equal), and λH1 (ω) = λH0 (ω + π).

QMF is short for Quadrature Mirror Filter. a) Show that, for a classical QMF filter bank transform, the two alias cancellation conditions in Exercise 5.10a) are satisfied with α = 1, d = 0.

Solution.

We have that

λH1 (ω) = λH0 (ω + π) = λG0 (ω + π) λG1 (ω) = λH1 (ω) = λH0 (ω + π) Comparing with the alias cancellation conditions in Exercise 5.10a), we see that we can set α = 1 and d = 1. b) Show that, for classical QMF filter bank transforms, the perfect reconstruction condition in Exercise 5.10a) can be written as λH0 (ω)2 + λH0 (ω + π)2 = 2.

Solution.

This follows from λH0 (ω)λG0 (ω) + λH0 (ω + π)λG0 (ω + π) = λH0 (ω)2 + λH1 (ω)λH0 (ω + π) = λH0 (ω)2 + λH0 (ω + π)2 . c) Show that it is impossible to find classical QMF filter banks with non-trivial FIRfilters, and which give perfect reconstruction.

Solution.

Let tk be the nonzero filter coefficient with smallest index k. We have that

5 The filter representation of wavelets

135

λH0 (ω)2 = t2k e−2ikω +

X

pr e−irω

r<−2k

λH0 (ω + π)

2

= t2k e−2ikω +

X

qr e−irω ,

r<−2k

where pr and qr are numbers. Adding these equations we obtain X λH0 (ω)2 + λH0 (ω + π)2 = 2t2k e−2ikω + (pr + qr )e−irω . r<−2k −2ikω

Since the term e contributes, in order for this to sum to 2 we must have that k = 0, so that t0 is the nonzero filter coefficient with smallest index k. By instead considering the nonzero filter coefficient with largest index k, we prove in the same way that this also is t0 , so that t0 can be the only nonzero filter coefficient, i.e. the filter is trivial, which is a contradiction.

Despite this negative result, classical QMF filter banks are still useful, as one can construct such filter banks which give near-perfect reconstruction. We shall see in Section 7.3 in the book that the MP3 standard take use of such filters, and this explains our previous observation that the MP3 standard does not give perfect reconstruction.

Exercise 5.13: Alternative QMF filter bank transforms An alternative QMF filter bank is one where G0 = (H0 )T and G1 = (H1 )T (i.e. the filter coefficients in the forward and reverse transforms are reverse of one-another), and λH1 (ω) = λH0 (ω + π).

There is some confusion in the literature between classical and alternative QMF filter banks. a) Show that, for an alternative QMF filter bank, the two alias cancellation conditions in Exercise 5.10a) are satisfied with α = 1, d = 0.

Solution.

If we in the first alias cancellation condition substitute G0 = (H0 )T we get λH1 (ω) = α−1 e−2idω λG0 (ω + π) = α−1 e−2idω λH0 (ω + π).

If we set α = 1, d = 0, we get equality here. A similar computation follows for the second alias cancellation condition. b) Show that the perfect reconstruction condition for alternative QMF filter banks can be written as |λH0 (ω)|2 + |λH0 (ω + π)|2 = 2.

136

5 The filter representation of wavelets

Solution.

We have that

2 = λH0 (ω)λG0 (ω) + λH1 (ω)λG1 (ω) = λH0 (ω)λH0 (ω) + λH0 (ω + π)λH0 (ω + π) = |λH0 (ω)|2 + |λH0 (ω + π)|2 .

We see that the perfect reconstruction conditions of the two definitions of QMF filter banks only differ in that the latter takes absolute values. It turns out that the latter also has many interesting solutions, as we will see in Chapter 6 in the book.

Exercise 5.14: Orthogonal filter bank transforms Show that H is orthogonal (i.e. G = H T ) if and only if the filters satisfy G0 = (H0 )T , G1 = (H1 )T , In the literature, a wavelet is called orthonormal if the filters satisfy G0 = (H0 )T , G1 = (H1 )T . Thus, alternative QMF filter banks with perfect reconstruction are examples of orthonormal wavelets.

Solution.

This can be proved simply by observing that, if we transpose the DWT-matrix we get an IDWT matrix with filters (H0 )T , (H1 )T , and this is equal to the IDWT if and only if G0 = (H0 )T , G1 = (H1 )T .

Exercise 5.15: The Haar wavelet as an alternative QMF filter bank transform Show that the Haar wavelet satisfies λH1 (ω) = −λH0 (ω + π), and G0 = (H0 )T , G1 = (H1 )T . The Haar wavelet thus differs from an alternative QMF filter bank only in a sign.

Despite this sign difference, the Haar wavelet is often called an alternative QMF filter bank transform in the literature. The additional sign leads to an orthonormal wavelet with α = −1, d = 0 instead.

Solution.

We have computed that 1 λH0 (ω) = √ (1 + eiω ) 2 It follows that

1 λH1 (ω) = √ (e−iω − 1).

2

5 The filter representation of wavelets

137

1 1 λH0 (ω + π) = √ (1 − eiω ) = √ (1 − e−iω ) = −λH1 (ω).

2 2 This proves the result.

Exercise 5.16: The support of the scaling function and the mother wavelet The scaling functions and mother wavelets we encounter will always turn out to be functions with compact support (for a general proof of this, see [1]). An interesting consequence of equations (5.2) and (5.3) in the book is that the corresponding filters then are FIR, and that one can find a connection between the supports of the scaling function and mother wavelet, and the number of filter coefficients. In the following we will say that the support of a filter is [E, F ] if tE , ..., tF are the only nonzero filter coefficients. a) Assume that G0 has support [M0 , M1 ], G1 has support [N0 , N1 ]. Show that Supp(φ) = [M0 , M1 ], and Supp(ψ) = [(M0 + N0 + 1)/2, (M1 + N1 + 1)/2].

Solution.

Let [m0 , m1 ] be the support of φ. Then φ1,n clearly has support [(m0 + n)/2, (m1 + n)/2].

In the equation φ(t) =

M1 X

(G0 )n φ1,n ,

n=M0

the function with the leftmost support is φ1,M0 , while the one with the rightmost one is φ1,M1 . This gives the support [(m0 + M0 )/2, (m1 + M1 )/2] for the sum. In order for the supports of the two sides to match we clearly must have

m0 = (m0 + M0 )/2

m1 = (m1 + M1 )/2.

It follows that m0 = M0 and m1 = M1 , so that the support of φ is [M0 , M1 ]. Similarly, let [n0 , n1 ] be the support of ψ. We have that ψ(t) =

NX 1 +1

(G1 )n−1 φ1,n ,

n=N0 +1

and we get in the same way

n0 = (m0 + N0 + 1)/2

n1 = (m1 + N1 + 1)/2.

It follows that n0 = (M0 + N0 + 1)/2. n1 = (M1 + N1 + 1)/2, so that the support of ψ is ((M0 + N0 + 1)/2, (M1 + N1 + 1)/2). b) Assume that all the filters are symmetric. Show that Supp(φ) is on the form [−M1 , M1 ] (i.e. symmetric around 0), and that Supp(ψ) is on the form 1/2 + [−(M1 +

138

5 The filter representation of wavelets

N1 )/2, (M1 + N1 )/2], i.e. symmetric around 1/2. Verify that the alternative piecewise linear wavelet has Supp(φ) = [−1, 1], and Supp(ψ) = [−1, 2] (as we already know from Figure 4.18 in the book), and that the piecewise linear wavelet has Supp(ψ) = [0, 1].

Solution.

Supp(φ) is on the stated form due to a), and since the support of a symmetric G0 has the form [−M1 , M1 ]. Since N0 = −N1 as well when G1 is symmetric, we have that Supp(ψ) = [(M0 + N0 + 1)/2, (M1 + N1 + 1)/2] = [(−M1 − N1 + 1)/2, (M1 + N1 + 1)/2] = 1/2 + [−(M1 + N1 )/2, (M1 + N1 )/2].

For both piecewise linear wavelets M1 = 1 so that Supp(φ) = [−1, 1]. For the first piecewise linear wavelet we had N0 = N1 = 0 so that Supp(ψ) = 1/2 + [−(M1 + N1 )/2, (M1 + N1 )/2] = 1/2 + [−1/2, 1/2] = [0, 1].

For the alternative piecewise linear wavelet we had N0 = −2, N1 = 2, so that Supp(ψ) = 1/2 + [−(M1 + N1 )/2, (M1 + N1 )/2] = 1/2 + [−3/2, 3/2] = [−1, 2]. c) The wavelet with symmetric filters with most filter coefficients we will consider has 7 and 9 filter coefficients, respectively. Explain why plotting over [−4, 4] captures the entire support of both the scaling function and the mother wavelet when there are at most this number of filter coefficients. This explains why [−4, 4] has been used as the plotting interval in many of the plots.

Solution.

With 7 and 9 filter coefficients it is easily seen that the support of φ is [−3, 3], and the support of ψ is [−3, 4]. Both these are contained in [−4, 4]. d) Verify that, for the Haar wavelet, G0 has filter coefficients evenly distributed around 1/2, G1 has equally many, and evenly distributed around −1/2, and the supports of both φ and ψ are symmetric around 1/2. The Haar wavelet is the simplest orthonormal wavelet, and these properties will be used as a template for the other orthonormal wavelets.

Solution.

We know that both φ and ψ for the Haar wavelet are supported on [0, 1), so that 1/2 is the midpoint of the support. The symmetries around 1/2 and −1/2 follows from G0 = √12 {1, 1} and G1 = √12 {1, −1}.

5 The filter representation of wavelets

139

e) We will only consider orthonormal wavelets with at most 8 filter coefficients. Explain again why [−4, 4] as plotting interval is desirable.

Solution.

This number of filter coefficients is easily seen to give the support [−3, 4], which again is contained within [−4, 4].

Exercise 5.19: Using the cascade algorithm In Exercise 4.31 we constructed a new mother wavelet ψ̂ for piecewise linear functions by finding constants α, β, γ, δ so that ψ̂ = ψ − αφ0,0 − βφ0,1 − δφ0,2 − γφ0,N −1 .

Use the cascade algorithm to plot ψ̂. Do this by using the wavelet kernel for the piecewise linear wavelet (do not use the code above, since we have not implemented kernels for this wavelet yet).

Solution.

Assuming that the vector coeffs has been set as in Exercise 4.31, the code can look as follows m = 10; t = linspace(-2, 6, 8*2^m); coordsvm=2^(m/2)*idwt_impl([-coeffs(1); -coeffs(2); -coeffs(4); 0;...

0; 0; 0; -coeffs(3); 1; 0; 0; 0; 0; 0;...

0; 0; zeros(8*2^m-16, 1)], ... ’pwl0’, m, ’per’); plot(t, coordsvm([(6*2^m+1):(8*2^m) 1:(6*2^m)]))

Exercise 5.20: Implementing the dual filter bank transforms a) Show that ATλ = Bλ and BλT = Aλ , i.e. that the transpose of an elementary lifting matrix of even/odd type is an elementary lifting matrix of odd/even type.

Solution.

This follows from the fact that the nonzero off-diagonal elements in Aλ come at the indices (2i, (2i ± 1) mod N ), while the nonzero off-diagonal elements in Bλ come at the indices ((2i ± 1) mod N, 2i). b) Assume that we have a factorization of H in terms of elementary lifting matrices.

Use a) to show that the dual forward transform is obtained by replacing each Aλ with B−λ , Bλ with A−λ , and k with 1/k in the multiplication step. These replacements are

140

5 The filter representation of wavelets

done by the function set_wav_props in the file find_wav_props.m in the library, on an object which is accessed by the general kernel.

Solution.

The kernel of the dual forward transform is GT = (H −1 )T . Transposing and inverting means that the terms are reversed twice, so that the order is the same. Transposing transforms odd lifting steps to even lifting steps and vice versa (according to a)), and inversion adds the minus sign. The result follows. c) Previously we expressed the forward and reverse transforms of the piecewise linear wavelets in terms of elementary lifting operations. Use b) to write down the dual forward and reverse transforms of these two wavelets in terms of lifting matrices.

Solution.

For the piecewise linear wavelet we had

H=

√

1 G = √ B1/2 .

2

2B−1/2

This implies that the forward and reverse transforms for the dual wavelet are 1 1 H̃ = GT = √ (B1/2 )T = √ A1/2 2 2 √ √ T T G̃ = H = 2(B−1/2 ) = 2A−1/2 .

For the alternative piecewise linear wavelet we had

H=

√

2A1/4 B−1/2

1 G = √ B1/2 A−1/4 , 2

and we get similarly 1 1 H̃ = GT = √ (B1/2 A−1/4 )T = √ B−1/4 A1/2 2 2 √ √ T T G̃ = H = 2(A1/4 B−1/2 ) = 2A−1/2 B1/4 .

Exercise 5.21: Transpose of the DWT and IDWT Explain why • The transpose of the DWT can be computed with an IDWT with the kernel of the dual IDWT • The transpose of the dual DWT can be computed with an IDWT with the kernel of the IDWT

5 The filter representation of wavelets

141

• The transpose of the IDWT can be computed with a DWT with the kernel of the dual DWT • The transpose of the dual IDWT can be computed with a DWT with the kernel of the DWT

Solution.

Assume that the kernel transformations of the DWT and the IDWT are H and G, respectively. For the transpose of the DWT the point is that, while its kernel transformation is H T , we compose the kernel with a permutation matrix when we compute the DWT. When we transpose, the order of the kernel and the permutation changes, so the transpose must use an IDWT implementation instead. The remaining statements follow similarly, since the kernel of the transpose of the dual DWT is, G that of the transpose of the IDWT is GT , and that of the transpose of the dual IDWT is H.

Exercise 5.22: Symmetric filters preserve symmetric extensions Prove that symmetric filters preserve symmetric vectors in the sense of Definition 10 in the book also. Show also more generally that, if x ∈ R2N is symmetric around any "half-number" (i.e. 1/2, 1, 3/2, 2,..,), then Sx is symmetric around the same half-number.

Use Theorem 14 in the book as a guide.

Solution.

If x has symmetry around t, i.e. (xt−n = xt+n ), we obtain zt+n =

X

tk xt+n−k = t0 xt+n +

X

tk (xt+n−k + xt+n+k )

k≥1

k

zt−n =

X

tk xt−n−k = t0 xt−n +

k

= t0 xt+n +

X

tk (xt−n−k + xt−n+k )

k≥1

X

tk (xt+n+k + xt+n−k ),

k≥1

and we see that z has the same symmetry. This argument also works when t is a half-number. Then everything makes sense when requiring that n is a half-number.

Exercise 5.23: Implementing symmetric extensions a) The function you implemented in Exercise 3.28 computed circular convolution, i.e. it assumed a periodic extension of the input. Reimplement that function so that it also takes a third parameter bd_mode. If this equals "per" a periodic extension should be assumed. If it equals "symm" a symmetric extension should be assumed, as described

142

5 The filter representation of wavelets

above. Again, the function filter_impl in the library can serve as a template for your code.

Solution.

The implementation of filter_impl in the library looks as follows, and handles other boundary modes which we do not cover here as well: function x=filter_impl(t, x, bd_mode) N0 = (length(t) - 1)/2; N = size(x, 1); szx = size(x); n = prod(szx(2:end)); if strcmpi(bd_mode, ’symm’) y = [x((N0+1):(-1):2, :) ; x(:,:); x((N-1):(-1):(N - N0), :)]; elseif strcmpi(bd_mode, ’per’) y = [x((N - N0 + 1):N, :); x(:,:); x(1:N0, :)]; elseif strcmpi(bd_mode, ’none’) || strcmpi(bd_mode, ’bd’) y = [ zeros(N0, n); x(:, :); zeros(N0, n)]; end for k=1:size(y,2) z = conv(t, y(:,k)); x(:,k) = z((2*N0+1):(length(z)-2*N0)); end end

We see that the function conv is called b) Implement functions dwt_kernel_filters(x, bd_mode, wav_props) idwt_kernel_filters(x, bd_mode, wav_props)

which return DWT and IDWT kernels that apply the function filter_impl from a) together with Theorem 2 in the book. The parameter wav_props should be an object which gives access to the filters by means of writing wav_props.h0, wav_props.h1, wav_props.g0, and wav_props.g1.

Solution.

The implementation of these functions in the library is as follows: function x=dwt_kernel_filters(x, bd_mode, dual_wav_props) if strcmpi(bd_mode, ’bd’) y1 = dual_wav_props.A_L’*x(1:size(dual_wav_props.A_L,1), :); y2 = dual_wav_props.A_R’*x((end-size(dual_wav_props.A_R,1)+1):end, :); end x0 = filter_impl(dual_wav_props.g0, x, bd_mode); x1 = filter_impl(dual_wav_props.g1, x, bd_mode); x(1:2:end,:) = x0(1:2:end,:); x(2:2:end,:) = x1(2:2:end,:); if strcmpi(bd_mode, ’bd’) x(1:size(dual_wav_props.A_L,2), :) ... = x(1:size(dual_wav_props.A_L,2), :) + y1; x((end-size(dual_wav_props.A_R,2)+1):end, :) ... = x((end-size(dual_wav_props.A_R,2)+1):end, :) + y2; end end

5 The filter representation of wavelets

143

function x=idwt_kernel_filters(x, bd_mode, wav_props) if strcmpi(bd_mode, ’bd’) y1 = wav_props.A_L*x(1:size(wav_props.A_L,2), :); y2 = wav_props.A_R*x((end-size(wav_props.A_R,2)+1):end, :); end x0 = x; x0(2:2:end,:) = 0; x1 = x; x1(1:2:end,:) = 0; x0 = filter_impl(wav_props.g0, x0, bd_mode); x1 = filter_impl(wav_props.g1, x1, bd_mode); x = x0 + x1; if strcmpi(bd_mode, ’bd’) x(1:size(wav_props.A_L,1), :) ... = x(1:size(wav_props.A_L,1), :) + y1; x((end-size(wav_props.A_R,1)+1):end, :) .... = x((end-size(wav_props.A_R,1)+1):end, :) + y2; end end

Using these functions one can define standard DWT and IDWT kernels in the following way, assuming that the wav_props has been computed: dwt_kernel = @(x, bd_mode) dwt_kernel_filters(x, bd_mode, wav_props); idwt_kernel = @(x, bd_mode) idwt_kernel_filters(x, bd_mode, wav_props);

Exercise 5.24: The boundary mode in the cascade algorithm In Exercise 5.19 we did not use symmetric extensions (the bd_mode-argument was set to per). Attempt to use symmetric extensions instead (set the bd_mode-argument to "symm"), and observe the new plots you obtain. Can you explain why these new plots do not show the correct functions, while the previous plots are correct?

Solution.

When using symmetric extensions, we really consider a function which is made symmetric at the boundaries, and this will alter the scaling function or the mother wavelet we are interested in, so that the resulting plot may differ if the function hits the boundaries. A periodic extension on the other hand should be unproblematic, as long as the support fits into the interval we plot over. The support is simply split into a left and a right part if the function hits a boundary in this case.

Exercise 5.25: Restricted matrices for elementary lifting Show that

144

5 The filter representation of wavelets

  1 2λ 0 0 · · · 0 0 0 0 1 0 0 · · · 0 0 0    0 λ 1 λ · · · 0 0 0    (Aλ )r =  . . . . . . . .   .. .. .. .. .. .. .. ..    0 0 0 0 · · · λ 1 λ 0 0 0 0 ··· 0 0 1   1 0 0 0 ··· 0 0 0 λ 1 λ 0 · · · 0 0 0    0 0 1 0 · · · 0 0 0   (Bλ )r =  . . . . . . . .  .  .. .. .. .. .. .. .. ..     0 0 0 0 · · · 0 1 0 0 0 0 0 · · · 0 2λ 1 Also, change the implementations of lifting_even_symm and lifting_odd_symm so that these expressions are used (rather than Aλ , Bλ ) when the bd_mode parameter equals "symm".

Solution.

Elementary lifting matrices can be viewed as wavelet transforms where the filters are {λ, 1, λ} and {1}. Since these are symmetric, Theorem 11 in the book guarantees that they preserve symmetric extensions.

Exercise 5.26: Expression for Sr Show that, with

S1 S2 S= ∈ R2N −2 × R2N −2 S3 S4 a symmetric filter, with S1 ∈ RN × RN , S2 ∈ RN × RN −2 , we have that

Sr = S1 + 0 (S2 )↔ 0 .

Use the proof of Theorem 15 in the book as a guide.

Solution.

We compute

5 The filter representation of wavelets

145

 x0  ..   .            x0 xN −2 x0 x1 xN −2     .   .   .  ↔ .   Sr x = S1 S2  xN −1  = S1  ..  + S2  ..  = S1  ..  + (S2 )  ..  xN −2  xN −1 x1 xN −1 xN −2    .   ..  x1     x0 x0 

   = S1  ...  + 0 (S2 )↔ 0  ...  = (S1 + 0 (S2 )↔ 0 )x, 

xN −1

xN −1

so that

Sr = S1 + 0 (S2 )↔ 0 .

Exercise 5.27: Orthonormal basis for the symmetric extensions In this exercise we will establish an orthonormal basis for the symmetric extensions in the sense of Definition 10 in the book. This parallels Theorem 23 in the book. a) Show that

2π · 0 · k 1 cos , 2N − 2 2N − 2

N −2 1 2πnk √ , cos 2N − 2 N −1 n=1

1 2π(N − 1)k √ cos 2N − 2 2N − 2 √

is an orthonormal basis for the symmetric extensions in R2N −2 .

Solution.

We compute the IDFT for all vectors in b). Since the IDFT is unitary, this will give us an orthonormal basis for the symmetric vectors in R2N −2 . Since (FN )H φn = en we get that

146

5 The filter representation of wavelets

1 0 (FN )H e0 = φ0 = √ cos 2π k 2N − 2 2N − 2

1 1 (FN )H √ (en + e2N −2−n ) = √ (φn + φ2N −2−n ) 2 2

1 1 =√ √ e2πikn/(2N −2) + e−2πikn/(2N −2) 2 2N − 2

1 n =√ cos 2π k 2N − 2 N −1

1 N −1 H (FN ) eN −1 = φN −1 = √ cos 2π k .

2N − 2 2N − 2 These coincide with the vectors listed in the exercise. b) Assume that S is symmetric. Show that the vectors listed in a) are eigenvectors for Sr , when the vectors are viewed as vectors in RN , and that they are linearly independent.

This shows that Sr is diagonalizable.

Solution.

Since S is symmetric, it preserves vectors which are symmetric around N − 1. In the frequency domain, applying S to a vector listed in c) corresponds to multiplying the vectors listed in b). with the frequency response. Since this does not introduce any more components, it is clear that the new vector must be a multiple of the same vector, so that these vectors indeed are eigenvectors. But then the vectors restricted to RN are also eigenvectors for Sr , since this is simply S when viewed on the first N elements. Since the vectors in R2N −2 are linearly independent, it is immediate that the corresponding vectors in RN also are linearly independent, since the second part of the vectors mirror the first part.

Exercise 5.28: Diagonalizing Sr Let us explain how the matrix Sr can be diagonalized, similarly to how we previously diagonalized using the DCT. In Exercise 5.27 we showed that the vectors

cos

2πnk 2N − 2

N −1 n=0

in RN is a basis of eigenvectors for Sr when S is symmetric. Sr itself is not symmetric, however, so that this basis can not possibly be orthogonal (S is symmetric if and only if it is orthogonally digonalizable). However, when the vectors are viewed in R2N −2 we showed in Exercise 5.27a) that

5 The filter representation of wavelets 2N −3 X

147

2πn1 k 2πn2 k cos 2N − 2 2N − 2 k=0   2 if n1 = n2 ∈ {0, N − 1} = (N − 1) × 1 if n1 = n2 6∈ {0, N − 1} .   0 if n1 = 6 n2 cos

a) Show that   1

if n1 = n2 ∈ {0, N − 1} if n1 = n2 6∈ {0, N − 1}  0 if n1 6= n2

1 2πn1 · 0 2πn2 · 0 = cos cos 2 2N − 2 2N − 2

N −2 X 2πn1 k 2πn2 k + cos cos 2N − 2 2N − 2 k=1

2πn1 (N − 1) 2πn2 (N − 1) 1 + cos cos .

2 2N − 2 2N − 2 (N − 1) ×

1 2

Hint.

Use that cos x = cos(2π − x) to pair the summands k and 2N − 2 − k.

Solution.

Using that cos x = cos(2π − x) we can here pair the summands k and 2N − 2 − k to obtain 2N −3 X

n2 n1 cos 2π k cos 2π k 2N − 2 2N − 2 k=0

n1 n2 = cos 2π · 0 cos 2π ·0 2N − 2 2N − 2

N −2 X n1 n2 +2 cos 2π k cos 2π k 2N − 2 2N − 2 k=1

n1 n2 + cos 2π (N − 1) cos 2π (N − 1) .

2N − 2 2N − 2

If we divide by 2 and combine these equations we get the result. −1 b) Define the vectors {xn }N n=0 by

148

5 The filter representation of wavelets

(

1 √ cos 2

2πn · 0 2N − 2

N −2 ) 2πn(N − 1) 2πnk 1 , , cos , √ cos 2N − 2 2N − 2 2 k=1

(I)

−1 and the vectors {dn }N n=0 by

q 1  x0   q N −1 (I) 2 dn = N −1 xn  q   1  N −1 xN −1

n=0 1≤n<N −1 n = N − 1.

(I)

Explain that the vectors dn are orthonormal, and that the matrix    √  √ 1/ 2 0 · · · 0 0 1/ 2 0 · · · 0 0   0 1 ··· 0 0   r 

 0 1 · · · 0 0  2   .. .. .. .. ..   .. .. .. .. ..  2πnk  cos 2N −2  . . . . .     N −1 . . . . .   0 0 ··· 1 0   0 0 ··· 1 0  √ √ 0 0 · · · 0 1/ 2 0 0 · · · 0 1/ 2 (I)

has the dn as columns, and is therefore orthogonal. Orthogonality in RN could thus be achieved by a scaling of the first and last components in the vectors. This orthogonal (I) (II) matrix is denoted DCTN . It is much used, just as the DCTN of Section 2.4 in the book.

Solution.

(I)

(I)

(I)

(I)

In a) we computed the inner products hdn1 , dn2 i, and the vectors are orthogonal since we obtained zero whenever n1 6= n2 . By a) also (I)

(I)

hd0 , d0 i = hdN −1 , dN −1 i = N − 1, p (I) (I) which explains why d0 and dN −1 both must be scaled bt 1/(N − 1). for the other n p (I) (I) a) said that hdn , dn i = (N − 1)/2, which explains the scaling with 2/(N − 1).

The pre- and post-multiplication with the stated diagonal √ matrix multiplies the first and last row, as well as the first and last column, with 1/ 2 (the four corners of the matrix will thus be multiplied by 1/2, since they appear in both the first/last rows, and the first/last columns). For the columns between 1 and N − 2 this clearly gives the p (I) −2 vectors {dn }N factor 2/(N − 1)pis added. For the p first and last n=1 , when the leading √ columns, taking out a factor of 1/ 2 and pairing with 2/(N − 1) gives 1/(N − 1), so that the right normalizing factor is obtained.

−1 2πnk c) Prove that cos 2N can be written as −2     1/2 0 · · · 0 0 1/2 0 · · · 0 0  0 1 ··· 0 0    

 0 1 · · · 0 0  2   .. .. .. .. ..    . . . . .

2πnk   cos 2N  .. .. .. .. ..  −2   N −1 . . . . .   0 0 ··· 1 0   0 0 ··· 1 0  0 0 · · · 0 1/2 0 0 · · · 0 1/2

5 The filter representation of wavelets

149

−1 2πnk 2πnk Since Sr can be diagonalized as cos 2N D cos , we can substitute −2 2N −2

2πnk here with a simple expression for the inverse. Thus, even if the matrix cos 2N −2 was not orthogonal, this represents no problem in the diagonalization/transition to the frequency domain.

Solution.

Since the product we obtained in b) is an orthogonal matrix, its transpose equals its inverse. The transpose of the product in b) clearly gives the same matrix back, and the inverse is √  √  2 0 ··· 0 0 2 0 ··· 0 0  0 1 ··· 0 0    r 

−1  0 1 · · · 0 0  N −1    .. .. .. .. ..  . . . . .

2πnk   cos 2N  .. .. .. .. ..  . −2   2  . . . . .   0 0 ··· 1 0   0 0 ··· 1 0  √ √ 0 0 ··· 0 2 0 0 ··· 0 2 Multiplying with N 2−1 , and then with  √  1/ 2 0 · · · 0 0  0 1 ··· 0 0     .. .. .. .. ..   . . . . .     0 0 ··· 1 0  √ 0 0 · · · 0 1/ 2 on both sides proves the result.

Exercise 5.29: Verifying a connection between the tables in the MP3 standard In addition to the function mp3_ctable(), the library has a function mp3_dtable() which returns the table D used by the reverse transform. Verify the connection we have stated between the tables C and D, i.e. that Di = 32Ci for all i.

Solution.

This can be verified as follows.

C = mp3_ctable(); D = mp3_dtable(); max(abs(D-32*C))

150

5 The filter representation of wavelets

Exercise 5.30: Implementing forward and reverse filter bank transforms It is not too difficult to implement the encoding and decoding steps as explained in the MP3 standard. Write a function which implements the encoding steps, and one which implements the decoding steps. In your code, assume for simplicity that the input and output vectors to your methods all have lengths which are multiples of 32, and use the functions mp3_ctable, mp3_dtable.

In the library the functions mp3_forward_fbt and mp3_reverse_fbt implement these encoding- and decoding steps. Use these as a template for your code.

Solution.

The library versions of mp3_forward_fbt and mp3_reverse_fbt are as follows. function z = mp3_forward_fbt(x) N = length(x); z = zeros(N,1); C = mp3_ctable(); % The analysis window; x = flipud(x); x = [x; zeros(512-32,1)]; % The 32x64 matrix M M = cos((2*((0:31)’)+1)*((0:63)-16)*pi/64); start = length(x) - 511; n = 1; for n = 1:(N/32) X = x(start:(start + 511)); Z = C.*X; Y = zeros(64, 1); for j = 0:7 Y = Y + Z((64*j + 1):(64*(j + 1))); end z((1+(n-1)*32):(n*32)) = M*Y; start = start - 32; end

function x = mp3_reverse_fbt(z) Ns = length(z)/32; x = zeros(32*Ns, 1); D = mp3_dtable(); % The synthesis window.

V = zeros(1024,1); % The 64x32 matrix N N = cos((16+((0:63)’))*(2*(0:31)+1)*pi/64); U = zeros(512,1); for n = 1:Ns; V(65:1024) = V(1:(1024-64)); V(1:64) = N*z((1+(n-1)*32):(n*32)); for i = 0:7 U((i*64 + 1):(i*64 + 32)) = V((i*128 + 1):(i*128 + 32)); U((i*64 + 33):((i + 1)*64)) = V((i*128 + 97):((i+1)*128)); end W = U.*D; for i = 0:15 x(((n-1)*32 + 1):(n*32)) = x(((n-1)*32 + 1):(n*32)) ... + W((32*i + 1):(32*(i + 1))); end end

5 The filter representation of wavelets

151

Exercise 5.31: Perfect reconstruction for forward and reverse M -channel filter bank transforms In this exercise H and G are forward and reverse M -channel filter bank transforms, respectively. a) Generalize the proof of Theorem 3 in the book to show that M −1 M −1 X X

1 Hφr = M

! λHs ,r e

−2πits/M

φr+tN/M

t=0 s=0 M −1 M −1 X X

1 Gφr = M

! λGs ,r+tN/M e

−2πist/M

φr+tN/M

t=0 s=0

where the φr are the Fourier basis vectors. This shows that the output from φr is a sum −1 of the {φr+sN/M }M s=0 , so that at most M − 1 frequencies can contribute with aliasing.

Solution.

Following the lines in the proof we first get  λH0 ,r e2πirk/N     λH1 ,r e2πirk/N (H(e2πirk/N ))k = . ..      λHM −1 ,r e2πirk/N

k mod M = 0 k mod M = 1 .. . k mod M = M − 1

Component n in the DFT of H(e2πirk/N ) is M −1 N/M X X−1 s=0

=

k=0

M −1 N/M X X−1 s=0

=

λHs ,r e2πir(M k+s)/N e−2πi(M k+s)n/N

M −1 X

λHs ,r e2πi(r−n)(M k+s)/N

k=0

λHs ,r e

2πi(r−n)s/N

s=0

! N/M −1 X

e2πi(r−n)k/(N/M ) .

k=0

PN/M −1 2πi(r−n)k/(N/M ) Clearly, k=0 e = N/M if n = r + tN/M for some t, and 0 else. It follows that !

M −1 M −1 X X N DFTN (H(e2πirk/N )) = λHs ,r e−2πits/M er+tN/M .

M t=0 s=0 Taking an IDFT gives M −1 M −1

H(e2πirk/N ) =

1 X X λHs ,r e−2πits/M e2πi(r+tN/M )k/N , M t=0 s=0

152

5 The filter representation of wavelets

and substituting with φr gives the first equation. For the reverse filter bank transform, recall Exercise 2.18b), which said that the vector e2πirk/N with all components with indices different from kM + s zeroed out (with s fixed), can be written as M −1

1 X −2πist/M 2πi(r+tN/M )k/N e e M t=0 so that M −1

1 X G G(e2πirk/N ) = M s=0

M −1 X

M −1 X

1 = Gs M s=0

! e−2πist/M e2πi(r+tN/M )k/N

t=0 M −1 X

! e

−2πist/M 2πi(r+tN/M )k/N

e

t=0

M −1 M −1

=

1 X X −2πist/M e λGs ,r+tN/M e2πi(r+tN/M )k/N M s=0 t=0

=

1 X X −2πist/M e λGs ,r+tN/M e2πi(r+tN/M )k/N .

M t=0 s=0

M −1 M −1

This proves the second equation. b) Find a formula for GHφr , and conclude that, in terms of continuous frequency responses, we have alias cancellation if and only if M −1 X

λHs (ω + 2πq/M )λGs (ω)e2πiqs/M = 0

s=0

for all 0 ≤ q < M/2, and perfect reconstruction if and only if in addition λH0 (ω)λG0 (ω) + · · · + λHM −1 (ω)λGM −1 (ω) = M. These expressions clearly generalize the conditions obtained for M = 2 in Exercise 5.9.

Solution.

Note that what we found in a) implies that, for any r, the space spanned by B = −1 0 {φr+tN/M }M t=0 is invariant under H and G. Replacing r with r + t N/M we obtain M −1 M −1

Hφr+t0 N/M =

1 X X λHs ,r+t0 N/M e−2πits/M φr+(t+t0 )N/M M t=0 s=0

=

0 1 X X λHs ,r+t0 N/M e−2πi(q−t )s/M φr+qN/M M q=0 s=0

=

0 1 X X −2πiqs/M e λHs ,r+t0 N/M e2πit s/M φr+qN/M M q=0 s=0

M −1 M −1

M −1 M −1

5 The filter representation of wavelets

153

M −1 M −1

Gφr+t0 N/M =

1 X X λGs ,r+(t+t0 )N/M e−2πist/M φr+(t+t0 )N/M M t=0 s=0 M −1 M −1

=

0 1 X X λGs ,r+qN/M e−2πis(q−t )/M φr+qN/M M q=0 s=0

=

0 1 X X λGs ,r+qN/M e−2πisq/M e2πist /M φr+qN/M , M q=0 s=0

M −1 M −1

where we did the substitution q = t + t0 . From this we can see that the matrices of H −1 and G relative to {φr+tN/M }M t=0 are  [H]B =

1  DFTM  M 

[G]B =

λH0 ,r+0·N/M e2πi0·0/M .. .

··· .. .

λH0 ,r+(M −1)N/M e2πi0·(M −1)/M .. .

 

λHM −1 ,r+0·N/M e2πi(M −1)·0/M · · · λHM −1 ,r+(M −1)N/M e2πi(M −1)(M −1)/M  λG0 ,r+0·N/M e−2πi0·0/M ··· λGM −1 ,r+0·N/M e−2πi0·(M −1)/M  .. .. ..  DFTM . . . .

1   M λG0 ,r+(M −1)N/M e−2πi(M −1)·0/M · · · λGM −1 ,r+(M −1)N/M e−2πi(M −1)(M −1)/M

Note that DFTM DFTM is simply M times the identity matrix Thus, in order for perfect reconstruction, we must have that the product of the two modulated matrices above must also be M times the identity matrix. For the diagonal entries to be nonzero we must have that (the modulating complex exponentials cancel in this case) λH0 (ω)λG0 (ω) + · · · + λHM −1 (ω)λGM −1 (ω) = M (for M different values of ω). For alias cancellation we must have that M −1 X

λHs ,r+q1 N/M λGs ,r+q2 N/M e2πi(q1 −q2 )s/M = 0

s=0

for all q1 6= q2 . Rewriting this in terms of angular frequency proves the result.

Exercise 5.32: Time domain view of perfect reconstruction Assume that Hi and Gi are the filters of forward and reverse M -channel filter bank transforms. Show that GH can be computed with the formula ! −1 X M X (n−k) (k−l) zn = Gk ∗ Hk xl , l (n−k) where Gk



k=0

0 (n−k)

is polyphase component n − k of Gk , i.e. (Gk )r = (Gk )rM +n−k .

This can be though of as a time domain version of the previous exercise, in that alias cancellation and perfect reconstruction can be spotted in terms of time domain polyphase components. We also obtain an expression for the matrix GH as

154

5 The filter representation of wavelets M −1 X

(GH)n,l =

! (n−k) (k−l) (Gk ) ∗ (Hk )

k=0

.

0

Solution.

We have that  ..  .   0    (Hk x)0  M −1 M −1   X X   zn = Gk z k = Gk  ...    k=0 k=0  0    (Hk x)1    .. . 

=

M −1 X X

(Gk )n−(rM +k) (Hk x)rM +k

k=0

=

M −1 X X X k=0

=

=

r

r

(Gk )n−(rM +k) (Hk )rM +k−l xl

l

M −1 X X X k=0

r

X

M −1 X

l

k=0

(n−k)

(Gk

(k−l)

)−r (Hk

)r xl

l

! (n−k) (k−l) Gk ∗ Hk

xl .

0

Exercise 5.33: Connection between cosine-modulated forward and reverse transforms Show that, if H is a cosine-modulated forward transform with prototype {hr }511 r=0 , then G = E481 H T is a cosine-modulated reverse transform with prototype {gr }511 r=0 , with gr = h512−r (i.e. the prototype filter is reversed). Deduce also from this that H = GT E481 .

In Exercise 7.13 we construct an example showing that the forward and reverse transforms of this section do not invert each other exactly. In Section 7.3 in the book, we also will see how one can construct similar filter banks with perfect reconstruction. It may seem strange that the MP3 standard does not do this. A partial explanation may be that, while perfect reconstruction is important, other design criteria may be even more important.

Solution.

We know that H T is a reverse filter bank transform with filters

5 The filter representation of wavelets

155

(Hi )T = (Ei−31 h(i) )T = E31−i (h(i) )T . (h(i) )T has filter coefficients cos((2i + 1)(−k − 16)π/64))h−k . If we delay all (Hi )T with 481 = 512−31 elements as in the theorem, we get a total delay of 512−31+31−i = 512−i elements, so that we get the filter E512−i {cos((2i + 1)(−k − 16)π/64))h−k }k = E−i {cos((2i + 1)(−(k − 512) − 16)π/64))h−(k−512) }k = E−i {cos((2i + 1)(k + 16)π/64))h−(k−512) }k .

Now, we define the prototype filter g with elements gk = h512−k . This has, just as h, its support on [1, 511], and consists of the elements from h in reverse order. If we define g (i) as the filter with coefficients cos((2i + 1)(k + 16)π/64))gk , we see that E481 H T is a reverse filter bank transform with filters E−i g (i) . Since g (k) now has been defined as for the MP3 standard, and its elements are the reverse of those in h, the result follows.

Chapter 6

Constructing interesting wavelets

Exercise 6.1: Properties of the CTFT Prove the properties of the Continuous time Fourier transform listed in Theorem 3 in the book.

Solution.

To prove the first property write 1 fˆ(ν) = √ 2π

Z ∞ f (t)e

−iνt

−∞

1 dt = √ 2π

Z ∞

f (t)e−(−ν)it dt = fˆ(−ν).

−∞

R∞ If f is symmetric the substitution u = −t can be used to prove that −∞ f (t)e−iνt dt = R∞ f (t)eiνt dt, so that f (ν) = f (−ν) = f (ν) due to the first property. It follows that −∞ f (ν) is real. The third property follows similarly. For the fourth property, Z ∞ Z ∞ Z ∞ −iνt −iν(t+d) −iνd f (t − d)e dt = f (t)e dt = e f (t)e−iνt dt, −∞

and ĝ(ν) = e

−∞

−∞

−iνd ˆ

f (ν) follows. The fifth property follows from Z ∞ Z ∞ eidt f (t)e−iνt dt = f (t)e−i(ν−d)t dt. −∞

−∞

Exercise 6.2: The CTFT of a pulse a) Show that the CTFT of χ[−a,a] (t) is √12π sin(νa) for any a. ν/2 Solution.

We have that

157

158

6 Constructing interesting wavelets

1 √ 2π

a 1 1 −iνt 1 1 sin(νa) √ e−iνt dt = √ e = (e−iνa − eiνa ) = √ .

2π −iν −i 2πν 2π ν/2 −a −a

Z a

s) s b) Use a) to show that the CTFT of sin(πt/T is √T2π χ[−π/Ts ,π/Ts ] (t). πt/Ts

Solution.

Setting a = π/Ts and taking the ICTFT of the expression we obtained in a) we get Z ∞ 1 1 sin(πν/Ts ) iνt √ χ[−π/Ts ,π/Ts ] (t) = √ e dν ν/2 2π −∞ 2π so that 1 √ 2π

Z ∞

sin(πν/Ts ) −iνt Ts e dν = √ χ[−π/Ts ,π/Ts ] (−t). πν/Ts 2π −∞

The result follows.

Exercise 6.3: Mixed continuous-discrete expressions P Let f (t) = k xk h(t − nTs ). Show that fˆ(ν) = x̂(νTs )ĥ(ν). This can be considered a mixed continuous-discrete expression for that convolution in time to corresponds to multiplication in frequency.

Solution.

We have that Z ∞X Z ∞ X 1 1 fˆ(ν) = √ xk h(t − nTs )e−iνt dt = xk √ h(t − nTs )e−iνt dt 2π −∞ k 2π −∞ k Z ∞ X 1 h(t)e−iνt dt = x̂(νTs )ĥ(ν). = xk e−iνnTs √ 2π −∞ k

Exercise 6.4: The frame inequalities and singular values Show that Akf k2 ≤ kU T f k2 ≤ Bkf k2 . for some constants A, B > 0 if and only if U U T is non-singular. Show also in this case 2 that the optimal values for A and B are A = σm and B = σ12 , where σi are the singular T values of U .

6 Constructing interesting wavelets

159

Hint.

Consult Appendix A in the book about the singular value decomposition.

Solution.

Let U T = U0 Σ(V0 )H be a singular value decomposition of U T . Since U0 and V0 are unitary they preserve norm, i.e. kU0 f k = kf k and kV0 f k = kf k for any f Rn or Rm , respectively. Using this and replacing U T with U0 Σ(V0 )H , and f with V0 f in the frame inequalities we obtain that Akf k2 ≤ kU0 Σf k2 = kΣf k2 ≤ Bkf k2 .

Here we have that     σ1 0 · · · 0 σ1 f1  0 σ2 · · · 0   σ2 f2       .. .. .. ..   ..  . . . .   .         Σf =   0 0 · · · σm  f = σm fm  ,  0 0 ··· 0   0      . . . .   .   .. .. .. ..   ..  0 0 ··· 0

0

so that 2 σm kf k2 ≤ kΣf k2 =

m X

σi2 fi2 ≤ σ12 kf k2 .

i=1 2 It follows that A = σm and B = σ12 can be used as bounds in the frame inequalities. It

is straightforward to see that these bounds are optimal.

In particular σm > 0 is the same as A > 0. According to Appendix A in the book T the eigenvalues of U U T are {σi2 }m if and only i=1 , so that 0 is not an eigenvalues in U U T σm > 0, and this is again equivalent to U U being non-singular.

Exercise 6.5: Finding a dual frame Assume that U U T is non-singular, so that {un }n is a frame. From the previous exercise it follows from this that σm > 0. From Appendix A in the book it also follows that the rank of U is m, i.e. U has full column rank. Show that Ũ = (U T )† satisfies the requirements of a dual frame, where (U T )† is the generalized inverse of U T , defined in the appendix. The appendix then says that Ũ = (U T )† = (U U T )−1 U . It follows that (U U T )−1 maps the frame vectors uk to the dual frame vectors ũk , and that the frame operator U U T maps the dual frame vectors to the frame vectors.

Solution.

First we have that

160

6 Constructing interesting wavelets

Ũ U T = (U T )† U T = Im , where we used Appendix A in the book. Also, if σi are the singular values of U T , 1/σi are the singular values of (U T )† . Its smallest singular value is thus 1/σ1 , and its largest singular value is 1/σm . Again with U T = U0 Σ(V0 )H a singular value decomposition of U T , we have that Ũ T = U0 Σ −1 (V0 )H . Following the deductions in the previous exercises we get that 1 1 kf k2 ≤ kŨ T f k2 =≤ 2 kf k2 . σ12 σm Since 1/σ1 = 1/B and 1/σm = 1/A the result follows.

Exercise 6.6: Tight frames in finite-dimensional spaces Let {ui }ni=1 be a set of vectors in Rm . Show that {ui }ni=1 is a tight frame with frame bound A is equivalent to any of the two following statements. a) U U T = AIm .

Solution.

Using the singular value decomposition again, a tight frame is equivalent to that σ1 = · · · = σm , i.e. that U T = σ1 U0 (V0 )H in the singular value decomposition of U T .

But this implies that U U T = σ12 V0 (U0 )H U0 (V0 )H = σ12 V0 (V0 )H = σ12 Im , since V0 is unitary. On the other hand, if U U T = AIm , we have that U U T = 2 2 V0 Σ12 (V0 )H = AIm , from which it √ follows that Σ1 = AIm , so that σi = A for all i. It follows that σ1 = · · · = σm = A, so that the frame is tight.

Pn b) x = A1 i=1 hx, ui iui for all x ∈ Rm .

Solution.

This statement is equivalent to that x = A1 U U T x for all x, i.e. that U U T = AIm , which is what we considered in a).

Exercise 6.7: Riesz bases for finite-dimensional spaces m Let {ui }m i=1 be a Riesz basis for R . −1 a) Show that the dual frame of {ui }m . i=1 are the rows of U

6 Constructing interesting wavelets

161

Solution. † −1 Since {ui }m . Since Ũ = (U T )† = i=1 is a Riesz basis U is non-singular, so that U = U −1 T (U ) , the dual frame vectors (i.e. the columns of Ũ ) are the rows of U −1 .

b) Show that {ui }m i=1 is a tight frame if and only if the vectors are orthogonal, and that the dual frame is a scaled version of the frame itself.

Solution.

From the previous exercise we have that {ui }m i=1 is a tight frame is the same as U U T = AIm for some scalar A, which is equivalent to U T U = AIm . This is clearly again equivalent to the columns of U being orthogonal. From a) we also have that the dual frame vectors are the rows of U −1 = A1 U T , i.e. the columns of A1 U . The dual frame is thus obtained from the frame by scaling with 1/A.

Exercise 6.8: Computing filters Compute the filters H0 , G0 in Theorem 8 in the book when N = N1 = N2 = 4, and Q1 = Q(4) , Q2 = 1. Compute also filters H1 , G1 so that we have perfect reconstruction (note that these are not unique).

Solution.

We have that N1 /2

1 1 (1 + cos ω) Q1 (1 − cos ω) 2 2

2

1 1 (4) = (1 + cos ω) Q (1 − cos ω) 2 2 N2 /2

2

1 1 1 (1 + cos ω) Q2 (1 − cos ω) = (1 + cos ω) λG0 (ω) = 2 2 2

2 1 1 1 1 2iω = 1 + eiω + e−iω = (e + 4eiω + 6 + 4e−iω + e−2iω ).

4 2 2 16

λH0 (ω) =

1 Therefore G0 = 16 {1, 4, 6, 4, 1}. We do not recommend to compute H0 by hand. With symbolic math toolbox in MATLAB you can do as follows to compute H0 .

syms x expand( ((1+x/2+1/(2*x))/2)^2*...

(2+8*((1-x/2-1/(2*x))/2)+20*((1-x/2-1/(2*x))/2)^2+...

40*((1-x/2-1/(2*x))/2)^3) )

2 Here we have substituted x for eiω , 1/x for e−iω . The first part represents 12 (1 + cos ω) , the second part represents Q(4) (u) = 2 + 8u + 20u2 + 40u3 with u = 12 (1 − cos ω) =

1 1 iω 1 −iω − 2e . This gives 2 1 − 2e

162

6 Constructing interesting wavelets

1 {−5, 20, −1, −96, 70, 280, 70, −96, −1, 20, −5}.

128 Using Exercise 5.10 with α = 1, d = 0, we get H0 =

1 {1, −4, 6, −4, 1} 16 1 G1 = {5, 20, 1, −96, −70, 280, −70, −96, 1, 20, 5} 128

H1 =

Exercise 6.9: Plotting frequency responses of Spline wavelets The library represents a Spline wavelet with x and y vanishing moments (i.e. N1 and N2 ) with the name "splinex.y". a) Listen to the low-resolution approximations and detail components in sound for the "spline4.4" wavelet.

Solution.

The following code can be used for listening to the low-resolution approximations for a given value of m. m = 1; [x, fs] = forw_comp_rev_dwt1(m, ’spline4.4’); playerobj = audioplayer(x, fs); playblocking(playerobj)

[x, fs] = forw_comp_rev_dwt1(m, ’spline4.4’, 0); playerobj = audioplayer(x, fs); playblocking(playerobj)

b) Plot all scaling functions and mother wavelets (using the cascade algorithm), and frequency responses for the "spline4.4" wavelet.

Solution.

The code to plot the functions can look as follows. m=10; cascade_alg(m, -4, 4, ’spline4.4’, 1, 0) cascade_alg(m, -4, 4, ’spline4.4’, 0, 0)

The frequency response can be plotted as follows freqresp_alg(’spline4.4’, lowpass, dual)

6 Constructing interesting wavelets

163

Exercise 6.10: Wavelets based on higher degree polynomials Show that Br (t) = ∗rk=1 χ[−1/2,1/2) (t), the B-spline of order r, is r −2 times differentiable, and equals a polynomial of degree r − 1 on subintervals of the form [n, n + 1]. Explain why these functions can be used as bases for the resolution spaces from Definition 11 in the book.

Solution.

The case for r = 2 follows from Exercise 4.26. Assuming that the result holds for a given r, it is straightforward to prove by induction that the result also holds for r + 1. That it equals a polynomial of one degree higher is easiest to prove, and follows by splitting up the integral over the different intervals [n, n + 1], using a polynomial expression on each.

That the differentiability is increased by one follows from the fundamental theorem of calculus (i.e. the integral of a continuous function is differentiable), from a result which says that it is allowed to differentiate under the integral sign, and by writing (for an r − 2 times differentiable function f ) Z ∞

Z 1/2 χ[−1/2,1/2) (t)f (x − t)dt =

−∞

f (x − t)dt. −1/2

Chapter 7

The polyphase representation of filter bank transforms

Exercise 7.1: The frequency responses of the polyphase components Let H and G be 2-channel forward and reverse filter bank transforms, with filters H0 , H1 , G0 , G1 , and polyphase components H (i,j) , G(i,j) . a) Show that λH0 (ω) = λH (0,0) (2ω) + eiω λH (0,1) (2ω) λH1 (ω) = λH (1,1) (2ω) + e−iω λH (1,0) (2ω) λG0 (ω) = λG(0,0) (2ω) + e−iω λG(1,0) (2ω) λG1 (ω) = λG(1,1) (2ω) + eiω λG(0,1) (2ω).

Solution.

G(0,0) , G(1,1) are the even-indexed filter coefficients of G0 , G1 , respectively, so that λG(0,0) (2ω), λG(1,1) (2ω) represents the half of λG0 (ω), λG1 (ω), respectively, from the even filter coefficients. G(1,0) are the odd-indexed filter coefficients of G0 . Since coefficient 0 in G(1,0) equals coefficient 1 in G0 , it is clear that e−iω λG(1,0) (2ω) represents the half of λG0 (ω) from the odd filter coefficients. This proves the first formula. The second formula follows from the same kind of reasoning.

If we transpose H (also in polyphase form), we get an MRA-matrix where the columns are given by the filters (H0 )T , (H1 )T . Inserting these in the formulas we just proved we get that λ(H0 )T (ω) = λ(H (0,0) )T (2ω) + e−iω λ(H (0,1) )T (2ω) λ(H1 )T (ω) = λ(H (1,1) )T (2ω) + eiω λ(H (1,0) )T (2ω).

If we conjugate these expressions we get

165

166

7 The polyphase representation of filter bank transforms

λH0 (ω) = λH (0,0) (2ω) + eiω λH (0,1) (2ω) λH1 (ω) = λH (1,1) (2ω) + e−iω λH (1,0) (2ω), and the proof is done. b) In the proof of the last part of Exercise 5.10, we deferred the last part, namely that λH1 (ω) = α−1 e−2idω λG0 (ω + π) λG1 (ω) = αe2idω λH0 (ω + π). follow from

G(0,0) G(0,1) G(1,0) G(1,1)

αE−d H (1,1) −αE−d H (0,1) . −αE−d H (1,0) αE−d H (0,0)

=

Prove this based on the result from a).

Solution.

The first column in the matrix on the left hand side gives the filter G0 . On the right hand side, a) states that the even-indexed columns are taken from the filter with frequency response λαE−d H (1,1) (2ω) + e−iω λ−αE−d H (1,0) (2ω)

= αλE−d (2ω) λH (1,1) (2ω) − e−iω λH (1,0) (2ω)

= αe2idω λH (1,1) (2(ω + π)) + e−i(ω+π) λH (1,0) (2(ω + π)) = αe2idω λH1 (ω + π).

This shows that λG0 (ω) = αe2idω λH1 (ω+π). We obtain the first equation easily from this.

Now, the second column in the matrix on the left hand side gives the filter coefficients of G1 . On the right hand side, a) states that the odd-indexed columns are taken from the filter with frequency response λαE−d H (0,0) (2ω) + eiω λ−αE−d H (0,1) (2ω)

= αλE−d (2ω) λH (0,0) (2ω) − eiω λH (0,1) (2ω)

= αe2idω λH (0,0) (2(ω + π)) + ei(ω+π) λH (0,1) (2(ω + π)) = αe2idω λH0 (ω + π).

This shows that λG1 (ω) = αe2idω λH0 (ω + π), which is the second equation.

Exercise 7.2: Polyphase view of time delay a) Write down the polyphase representation for an arbitrary delay Ed , and for any given M .

7 The polyphase representation of filter bank transforms

167

Solution.

Write d = kM + s for some 0 ≤ s < M . The polyphase representation will be a block matrix with only one nonzero block-diagonal. The nonzero blocks will be at positions (s, 0), (s + 1, 1), and so on, and each nonzero block will equal the delay Ek . b) Assume that d is a multiple of M . Show that Ed commutes with any M -channel filter bank transform. Will this be the case if d is not a multiple of M ?

Solution.

If d is a multiple of M , the nonzero block-diagonal from a) will actually be the main block-diagonal. This means that   Ek 0 · · · 0 0  0 Ek · · · 0 0      Ed =  ... ... . . . ... ...  .    0 0 · · · Ek 0  0 0 · · · 0 Ek Since Ed commutes with any filter, and since the same element repeats on the main block-diagonal, such Ed will commute with all filter bank transforms. When d is not a multiple of M , the nonzero block diagonal will not be the main one. Matrices with one nonzero off-diagonal do not commute with all other matrices, even if it is constant on that diagonal. In particular this will be the case for filter bank transforms as well (even if Ed commutes with filters).

Exercise 7.3: Polyphase view of alias cancellation How we can spot alias cancellation in terms of the polyphase representation will be useful to us in the next section when we explain why the filter bank transform of the MP3 standard provides such cancellation. We will there end up with a polyphase representation on the form given in a) of this exercise. a) Assume that the polyphase representation of S is a k × k-block diagonal matrix where the diagonal elements are M × M -filters which are all equal. Show that S is a (kM ) × (kM )-filter. Thus, if GH has this form for forward and reverse filter bank transforms H and G, we have alias cancellation.

Solution.

Since S has a k × k-block diagonal polyphase representation with equal diagonal blocks, the only nonzero elements in the filter representation are at indices (i + M r1 , i + M r2 ) with 0 ≤ i < M , 0 ≤ r1 , r2 < k. Increasing i with 1 just switches to the next block on the diagonal, and we get the same value since the blocks are assumed equal. Also, increasing both r1 and r2 with one clearly gives a new element on the same diagonal in the same block, which has the same value by the assumption that each block entry is

168

7 The polyphase representation of filter bank transforms

a filter. It follows that S is constant on diagonals 0, ±M , ±2M , and so on, and zero elsewhere. It follows that S is a (kM ) × (kM )-filter. b) If S is a general (kM ) × (kM )-filter, give a general description of its polyphase representation.

Solution.

Each block in the polyphase representation is a k × k-filter. Also, the filters S (i,j) and S (i+k,j+k) must be equal for any i, j, k.

Exercise 7.4: Polyphase components for symmetric filters Assume that the filters H0 , H1 of a forward filter bank transform are symmetric, and let H (i,j) be the corresponding polyphase components. Show that • H (0,0) and H (1,1) are symmetric filters, • the filter coefficients of H (1,0) have symmetry about −1/2, • the filter coefficients of H (0,1) have symmetry about 1/2.

Also show a similar statement for reverse filter bank transforms.

Solution.

Let H0 = {..., t−1 , t0 , t1 , ...} and H1 = {..., r−1 , r0 , r1 , ...}. We clearly have that H (0,0) = {..., t−4 , t−2 , t0 , t2 , t4 , ...} H (1,1) = {..., r−4 , r−2 , r0 , r2 , r4 , ...} H (1,0) = {..., r−3 , r−1 , r1 , r3 , r5 , ...} H (0,1) = {..., t−5 , t−3 , t−1 , t1 , t3 , ...} The symmetry statements now follow from the symmetries in tk and rk .

Exercise 7.5: Classical QMF filter banks Recall from Exercise 5.12 that we defined a classical QMF filter bank as one where M = 2, G0 = H0 , G1 = H1 , and λH1 (ω) = λH0 (ω + π). Show that the forward and reverse filter bank transforms of a classical QMF filter bank have polyphase representations of the form

H=G=

A −E1 B .

B A

7 The polyphase representation of filter bank transforms

169

Solution.

We have that λG0 (ω) = λG(0,0) (2ω) + e−iω λG(1,0) (2ω) λG1 (ω) = eiω λG(0,1) (2ω) + λG(1,1) (2ω). λG1 (ω) = λG0 (ω + π) thus implies that λG(1,1) (2ω) = λG(0,0) (2(ω + π)) = λG(0,0) (2ω) λG(0,1) (2ω) = −e−2iω λG(1,0) (2(ω + π)) = −e−2iω λG(1,0) (2ω).

The first equation says that G(1,1) = G(0,0) , the second says that G(0,1) = −E1 G(1,0) .

Exercise 7.6: Alternative QMF filter banks Recall from Exercise 5.13 that we defined an alternative QMF filter bank as one where M = 2, G0 = (H0 )T , G1 = (H1 )T , and λH1 (ω) = λH0 (ω + π). a) Show that the forward and reverse filter bank transforms of an alternative QMF filter bank have polyphase representations of the form

H=

AT B T −B A

G=

A −B T B AT

=

AT B T −B A

T .

Solution.

We have that λG0 (ω) = λG(0,0) (2ω) + e−iω λG(1,0) (2ω) λG1 (ω) = eiω λG(0,1) (2ω) + λG(1,1) (2ω). λG1 (ω) = λG0 (ω + π) thus implies that λG(1,1) (2ω) = λG(0,0) (2ω)

λG(0,1) (2ω) = −λG(1,0) (2ω),

so that G(1,1) = (G(0,0) )T and G(0,1) = −(G(1,0) )T . The result follows. b) Show that A and B give rise to an alternative QMF filter bank with perfect reconstruction as above if and only if AT A + B T B = I.

170

7 The polyphase representation of filter bank transforms

Solution.

We have that

AT B T −B A

A −B T B AT

T

A A + BT B 0 = , 0 AT A + B T B

and the result follows. c) Consider alternative QMF filter banks where we use an additional sign, so that λH1 (ω) = −λH0 (ω + π) (the Haar wavelet was an example of such a filter bank, see exercise 5.15). Show that such forward and reverse filter bank transforms have polyphase representations of the form

H=

T T A B B −A

G=

A BT B −AT

=

AT B T B −A

T .

This sign change does not substantially change the properties of alternative QMF filter banks, so that we will also call these new filter banks as alternative QMF filter banks.

Solution.

This follows as in a), just incorporating the additional minus sign.

Exercise 7.7: View of lifting as altering the filters This exercise will show that one can think of the steps in a lifting factorization as altering the low-pass/high-pass filters in alternating order. Let S be a general filter. a) Show that

G

I 0 SI

is an MRA matrix with filters Ĝ0 , G1 , where λGˆ0 (ω) = λG0 (ω) + λS (2ω)e−iω λG1 (ω), Solution.

We have that

G(0,0) G(0,1) G(1,0) G(1,1)

(0,0)

I 0 G + SG(0,1) G(0,1) = .

SI G(1,0) + SG(1,1) G(1,1)

Using Exercise 7.1a), the even-indexed columns in this matrix are taken from the filter with frequency response

7 The polyphase representation of filter bank transforms

171

λG(0,0) +SG(0,1) (2ω) + e−iω λG(1,0) +SG(1,1) (2ω)

= λG(0,0) (2ω) + e−iω λG(1,0) (2ω) + λS (2ω) λG(0,1) (2ω) + e−iω λG(1,1) (2ω)

= λG0 (ω) + λS (2ω)e−iω λG(1,1) (2ω) + eiω λG(0,1) (2ω) = λG0 (ω) + λS (2ω)e−iω λG1 (ω). b) Show that

I S G 0I

is an MRA matrix with filters G0 , Ĝ1 , where λGˆ1 (ω) = λG1 (ω) + λS (2ω)eiω λG0 (ω), Solution.

We have that

G(0,0) G(0,1) G(1,0) G(1,1)

I S 0I

(0,0)

G SG(0,0) + G(0,1) = , G(1,0) SG(1,0) + G(1,1)

so that the odd-indexed columns in this matrix are taken from the filter with frequency response λSG(1,0) +G(1,1) (2ω) + eiω λSG(0,0) +G(0,1) (2ω) = λG(1,1) (2ω) + eiω λG(0,1) (2ω) + λS (2ω) λG(1,0) (2ω) + eiω λG(0,0) (2ω)

= λG1 (ω) + λS (2ω)eiω λG(0,0) (2ω) + e−iω λG(1,0) (2ω)

= λG1 (ω) + λS (2ω)eiω λG0 (ω). c) Show that

I 0 H SI

is an MRA-matrix with filters H0 , Ĥ1 , where λĤ1 (ω) = λH1 (ω) + λS (2ω)e−iω λH0 (ω).

Solution.

We transpose the expression to obtain HT

I ST 0 I

Since H T has filters (H0 )T and (H1 )T in the columns, from b) it follows that

I ST HT 0 I

172

7 The polyphase representation of filter bank transforms

has columns given by (H0 )T and the filter with frequency response λ(H1 )T (ω) + λS T (2ω)eiω λ(H0 )T (ω) = λH1 (ω) + λS (2ω)e−iω λH0 (ω), so that

I 0 H SI

has row filters H0 and a filter Ĥ1 with frequency response λĤ1 (ω) = λH1 (ω) + λS (2ω)e−iω λH0 (ω). d)

I S H 0I

is an MRA-matrix with filters Ĥ0 , H1 , where λĤ0 (ω) = λH0 (ω) + λS (2ω)eiω λH1 (ω).

Solution.

We transpose the expression to obtain HT

I 0 .

ST I

Since H T has filters (H0 )T and (H1 )T in the columns, using a) we see that

I 0 HT ST I has columns given by the filter with frequency response λ(H0 )T (ω) + λS T (2ω)e−iω λ(H1 )T (ω) = λH0 (ω) + λS (2ω)eiω λH1 (ω), and (H1 )T , so that

I S H 0I

has a row filter Ĥ0 with frequency response λĤ0 (ω) = λH0 (ω) + λS (2ω)eiω λH1 (ω), and H1 .

7 The polyphase representation of filter bank transforms

173

Exercise 7.8: Lifting-based implementations of the Spline 5/3 and CDF 9/7 wavelets Let us use the different lifting factorizations obtained in this chapter to implement the corresponding wavelet kernels. Your functions should call the functions from Exercise 4.24 and 4.29. You will need equations (7.8)-(7.11) here, in order to complete the kernels. a) Write functions dwt_kernel_53(x, bd_mode) idwt_kernel_53(x, bd_mode)

which implement the DWT and IDWT kernels for the Spline 5/3 wavelet. Use the lifting factorization obtained in Example 7.2.3 in the book.

Solution.

The code can look like this: function x=dwt_kernel_53(x, bd_mode) lambda1 = -1; lambda2 = 0.125; alpha = 2; beta = 0.5; x(1:2:end, :) = x(1:2:end, :)*alpha; x(2:2:end, :) = x(2:2:end, :)*beta; x = lifting_odd_symm(-lambda2, x, bd_mode); x = lifting_even_symm(-lambda1, x, bd_mode); end

function x=idwt_kernel_53(x, bd_mode) lambda1 = -1; lambda2 = 0.125; alpha = 2; beta = 0.5; x = lifting_even_symm(lambda1, x, bd_mode); x = lifting_odd_symm(lambda2, x, bd_mode); x(1:2:end, :) = x(1:2:end, :)/alpha; x(2:2:end, :) = x(2:2:end, :)/beta; end

b) Write functions dwt_kernel_97(x, bd_mode) idwt_kernel_97(x, bd_mode)

which implement the DWT and IDWT kernels for the CDF 9/7 wavelet. Use the lifting factorization obtained in Example 7.2.4 in the book.

Solution.

The code can look like this:

174

7 The polyphase representation of filter bank transforms

function x=dwt_kernel_97(x, bd_mode) lambda1 = -0.586134342059950; lambda2 = -0.668067171029734; lambda3 = 0.070018009414994; lambda4 = 1.200171016244178; alpha = -1.149604398860250; beta = -0.869864451624777; x(1:2:end, :) = x(1:2:end, :)*alpha; x(2:2:end, :) = x(2:2:end, :)*beta; x = lifting_odd_symm(-lambda4, x, bd_mode); x = lifting_even_symm(-lambda3, x, bd_mode); x = lifting_odd_symm(-lambda2, x, bd_mode); x = lifting_even_symm(-lambda1, x, bd_mode); end

function x=idwt_kernel_97(x, bd_mode) lambda1 = -0.586134342059950; lambda2 = -0.668067171029734; lambda3 = 0.070018009414994; lambda4 = 1.200171016244178; alpha = -1.149604398860250; beta = -0.869864451624777; x = lifting_even_symm(lambda1, x, bd_mode); x = lifting_odd_symm(lambda2, x, bd_mode); x = lifting_even_symm(lambda3, x, bd_mode); x = lifting_odd_symm(lambda4, x, bd_mode); x(1:2:end, :) = x(1:2:end, :)/alpha; x(2:2:end, :) = x(2:2:end, :)/beta; end

c) In Chapter 4 in the book, we listened to the low-resolution approximations and detail components in sound for three different wavelets. Repeat these experiments with the Spline 5/3 and the CDF 9/7 wavelet, using the new kernels you implemented in a) and b).

Solution.

The following code can be used for listening to the low-resolution approximations for a given value of m. m = 1; [x, fs] = forw_comp_rev_dwt1(m, ’cdf53’); playerobj = audioplayer(x, fs); playblocking(playerobj)

m = 1; [x, fs] = forw_comp_rev_dwt1(m, ’cdf97’); playerobj = audioplayer(x, fs); playblocking(playerobj)

d) Plot all scaling functions and mother wavelets for the Spline 5/3 and the CDF 9/7 wavelets, using the cascade algorithm and the kernels you have implemented.

7 The polyphase representation of filter bank transforms

175

Solution.

The code can look as follows. m=10; cascade_alg(m, -4, 4, ’cdf53’, 1, 0) cascade_alg(m, -4, 4, ’cdf53’, 0, 0) cascade_alg(m, -4, 4, ’cdf97’, 1, 0) cascade_alg(m, -4, 4, ’cdf97’, 0, 0)

In the plot for the CDF 9/7 wavelet, it is seen that the functions and their dual counterparts are close to being equal. This reflects the fact that this wavelet is close to being orthogonal.

Exercise 7.9: Lifting-based implementation of orthonormal wavelets a) Write functions lifting_even(lambda1, lambda2, x, bd_mode) lifting_odd(lambda1, lambda2, x, bd_mode)

which apply the elementary lifting matrices (7.14) in the book to x. Assume that N is even.

Solution.

The library versions of these functions are as follows. function x=lifting_even(lambda1, lambda2, x, bd_mode) N = size(x, 1); if strcmpi(bd_mode, ’per’) x(1, :) = lambda1*x(2, :) + x(1, :) + lambda2*x(N, :); elseif strcmpi(bd_mode, ’bd’) || strcmpi(bd_mode, ’none’) x(1, :) = lambda1*x(2, :) + x(1, :); end x(3:2:(N-1), :) = ... lambda1*x(4:2:N, :) + x(3:2:(N-1), :) + lambda2*x(2:2:(N-2), :); if strcmpi(bd_mode, ’bd’) || strcmpi(bd_mode, ’none’) if mod(N,2) == 1 % last must also be included x(N, :) = x(N, :) + lambda2*x(N-1, :); end end end

function x=lifting_odd(lambda1, lambda2, x, bd_mode) N = size(x, 1); x(2:2:(N-1), :) = ... lambda1*x(3:2:N, :) + x(2:2:(N-1), :) + lambda2*x(1:2:(N-2), :); if mod(N,2) == 0 % last must also be included if strcmpi(bd_mode, ’per’) x(N, :) = lambda1*x(1, :) + x(N, :) + lambda2*x(N-1, :); elseif strcmpi(bd_mode, ’bd’) || strcmpi(bd_mode, ’none’) x(N, :) = x(N, :) + lambda2*x(N-1, :); end

176

7 The polyphase representation of filter bank transforms

end end

b) Write functions dwt_kernel_ortho(x, bd_mode, wav_props) idwt_kernel_ortho(x, bd_mode, wav_props)

which apply the DWT and IDWT kernels for orthonormal wavelets, using the functions lifting_even and lifting_odd. Assume that you can access lifting steps so that the lifting factorization (7.7) in the book holds through the object wav_props by means of writing wav_props.lambdas, wav_props.alpha, and wav_props.beta, and that wav_props.last_even indicates whether the last lifting step is even. wav_props.lambdas is an n × 2-matrix so that the filter coefficients {λ1 , λ2 } or {λ1 , λ2 } in the i’th lifting step are found in row i.

You can now define standard DWT and IDWT kernels in the following way, once the wav_props object has been computed for a given N : dwt_kernel = @(x, bd_mode) dwt_kernel_ortho(x, bd_mode, wav_props);

Solution.

The library versions of these functions are as follows, and again support more general boundary modes: function x=dwt_kernel_ortho(x, bd_mode, dual_wav_props) if strcmpi(bd_mode, ’bd’) y1 = dual_wav_props.A_L’*x(1:size(dual_wav_props.A_L,1), :); y2 = dual_wav_props.A_R’*x((end-size(dual_wav_props.A_R,1)+1):end, :); end x(1:2:end, :) = x(1:2:end, :)/dual_wav_props.alpha; x(2:2:end, :) = x(2:2:end, :)/dual_wav_props.beta; iseven = ~dual_wav_props.last_even; for stepnr = (size(dual_wav_props.lambdas, 1)):(-1):1 if iseven x = lifting_even(dual_wav_props.lambdas(stepnr,2), ... dual_wav_props.lambdas(stepnr,1), x, bd_mode); else x = lifting_odd(dual_wav_props.lambdas(stepnr,2), ... dual_wav_props.lambdas(stepnr,1), x, bd_mode); end iseven = ~iseven; end if strcmpi(bd_mode, ’bd’) x(1:size(dual_wav_props.A_L,2), :) = ... x(1:size(dual_wav_props.A_L,2), :) + y1; x((end-size(dual_wav_props.A_R,2)+1):end, :) = ... x((end-size(dual_wav_props.A_R,2)+1):end, :) + y2; end end

function x=idwt_kernel_ortho(x, bd_mode, wav_props) if strcmpi(bd_mode, ’bd’) y1 = wav_props.A_L*x(1:size(wav_props.A_L,2), :); y2 = wav_props.A_R*x((end-size(wav_props.A_R,2)+1):end, :); end iseven = ( mod(size(wav_props.lambdas, 1), 2) == wav_props.last_even ); for stepnr = 1:(size(wav_props.lambdas, 1))

7 The polyphase representation of filter bank transforms

177

if iseven x = lifting_even(wav_props.lambdas(stepnr, 1), ... wav_props.lambdas(stepnr, 2), x, bd_mode); else x = lifting_odd(wav_props.lambdas(stepnr, 1), ... wav_props.lambdas(stepnr, 2), x, bd_mode); end iseven = ~iseven; end x(1:2:end, :)=x(1:2:end, :)/wav_props.alpha; x(2:2:end, :)=x(2:2:end, :)/wav_props.beta; if strcmpi(bd_mode, ’bd’) x(1:size(wav_props.A_L,1), :) = ... x(1:size(wav_props.A_L,1), :) + y1; x((end-size(wav_props.A_R,1)+1):end, :) = ... x((end-size(wav_props.A_R,1)+1):end, :) + y2; end end

c) Listen to the low-resolution approximations and detail components in sound for orthonormal wavelets for N = 1, 2, 3, 4.

Solution.

The following code can be used for listening to the low-resolution approximations for a given value of m. [x, fs] = forw_comp_rev_dwt1(m, ’db2’); playerobj = audioplayer(x, fs); playblocking(playerobj) [x, fs] = forw_comp_rev_dwt1(m, ’db3’); playerobj = audioplayer(x, fs); playblocking(playerobj) [x, fs] = forw_comp_rev_dwt1(m, ’db4’); playerobj = audioplayer(x, fs); playblocking(playerobj)

d) Plot all scaling functions and mother wavelets for the orthonormal wavelets for N = 1, 2, 3, 4, using the cascade algorithm. Since the wavelets are orthonormal, we should have that φ = φ̃, and ψ = ψ̃. In other words, you should see that the bottom plots equal the upper plots.

Solution.

The code can look as follows. m=10; cascade_alg(m, -4, 4, ’db2’, 1) cascade_alg(m, -4, 4, ’db2’, 0) cascade_alg(m, -4, 4, ’db3’, 1) cascade_alg(m, -4, 4, ’db3’, 0) cascade_alg(m, -4, 4, ’db4’, 1) cascade_alg(m, -4, 4, ’db4’, 0)

178

7 The polyphase representation of filter bank transforms

Exercise 7.10: Piecewise linear wavelet with 4 vanishing moments In Exercise 4.31 we found constants α, β, γ, δ which give the coordinates of ψ̂ in (φ1 , ψ̂1 ), where ψ̂ was a mother wavelet in the MRA for piecewise linear functions with four vanishing moments. a) Show that the polyphase representation of G when ψ̂ is used as mother wavelet can be factored as

1 I 0 I {−γ, −α, −β, −δ} √ .

0 I 2 {1/2, 1/2} I Find also a lifting factorization of H.

Solution.

We have found constants α, βγδ so that [ψ̂](φ0 ,ψ0 ) = (−α, −β, −δ, 0, 0, 0, 0, −γ) ⊕ (1, 0, 0, 0, 0, 0, 0, 0), From this it is clear that

P(φ1 ,ψ1 )←(φ1 ,ψ̂1 ) =

I S2 0 I

where S2 = {−γ, −α, −β, −δ} This gives as before the lifting factorization

1 I 0 I {−γ, −α, −β, −δ} .

PD1 ←(φ1 ,ψ̂1 ) = √ 0 I 2 S1 I

(7.1)

where S1 = {1/2, 1/2} as before.

Exercise 7.11: Wavelet based on piecewise quadratic scaling function In Exercise 6.8 you should have found the filters 1 {−5, 20, −1, −96, 70, 280, 70, −96, −1, 20, −5} 128 1 H1 = {1, −4, 6, −4, 1} 16 1 G0 = {1, 4, 6, 4, 1} 16 1 G1 = {5, 20, 1, −96, −70, 280, −70, −96, 1, 20, 5}.

128

H0 =

Show that

7 The polyphase representation of filter bank transforms

1 {5, −29, −29, 5} I − 128 0 I

I 0 −{1, 1} I

179

1 0 I − 41 {1, 1} G= 4 , 0 I 04

and derive a lifting factorization of G from this.

Solution.

The polyphase representation of the IDWT is

1 1 16 {1, 6, 1} 128 {5, 1, −70, −70, 1, 5} .

1 1 16 {4, 4} 128 {20, −96, 280, −96, 20} We can first apply an even lifting step:

1 1 I − 14 {1, 1} 16 {1, 6, 1} 128 {5, 1, −70, −70, 1, 5} 1 1 0 I 16 {4, 4} 128 {20, −96, 280, −96, 20} 1

1 {4} 128 {20, −116, −116, 20} 16 = 1 .

1 16 {4, 4} 128 {20, −96, 280, −96, 20}

We can now apply an odd lifting step

1 1

1 1 I 0 16 {4} 128 {20, −116, −116, 20} 4 128 {20, −116, −116, 20} = 1 1 −{1, 1} I 0 4 16 {4, 4} 128 {20, −96, 280, −96, 20}

Since

1 1 1 1 I − 512 {20, −116, −116, 20} 4 128 {20, −116, −116, 20} = 4 0 , 0 I 0 4 04

it follows that

1 1 I 0 I − 128 {5, −29, −29, 5} I − 41 {1, 1} 0 G= 4 . −{1, 1} I 0 I 0 I 04

Exercise 7.12: Connection between forward and reverse filter bank transforms based on prototypes In Exercise 5.33 we used a direct approach to relate G and H T . Insert instead (7.27) in the book in equations (7.27) in the book and (7.23) in the book to show that G = E480 H T and H = E480 GT . Also, combine these with GH = 16E480 to show that H T H = GT G = 16. Thus, up to a scalar, the forward and reverse filter bank transforms are orthogonal.

Solution.

We have that

180

7 The polyphase representation of filter bank transforms

GT G = E−480 HG = E−480 16E480 = 16 H T H = E−480 GH = E−480 16E480 = 16

Exercise 7.13: Run forward and reverse transform Run the forward and then the reverse transform from Exercise 5.30 on the vector (1, 2, 3, . . . , 8192). Verify that there seems to be a delay on 481 elements, as promised in Section 7.3.4 in the book. Do you get the exact same result back?

Solution.

The following code can be used: x = (1:8192)’; x = mp3_reverse_fbt(mp3_forward_fbt(x)); plot(x) x = (1:8192)’; x = mp3_reverse_fbt(x); x(1:size(x)-481) = x(482:size(x)); x = mp3_forward_fbt(x); plot(x)

There are some small errors from the original vector in the resulting vector, when one compensates for the delay of 481 elements.

Exercise 7.14: Verify statement of filters Use your computer to verify the following symmetries for the prototype filters: ( −C512−i i 6= 64, 128, . . . , 448 Ci = C512−i i = 64, 128, . . . , 448.

Explain also that this implies that hi = h512−i for i = 1, . . . , 511. Recall that the modified version had the symmetry hi = h511−i . When the filters V (m) are defined as in this section, explain why (7.26) in the book should be replaced by V (64−k) = −E14 (V (k) )T in the MP3 standard

Solution.

The following code can be used ctable = mp3_ctable(); ctable = ctable(2:end); ctable(64:64:255) = -ctable(64:64:255); res = ctable + ctable(end:-1:1);

7 The polyphase representation of filter bank transforms

181

res(256)=0; max(abs(res))

Note that the middle element of the table is set to 0 in this verification. The reason is that, for i = 256, there is nothing to check. So, since the verification works by producing zero when there is symmetry, the element corresponding to i = 256 is set to zero. Also, every 64’th element in one half of the table changed sign, in order to establish symmetry for the corresponding entries, rather than anti-symmetry as for the other entries.

Exercise 7.15: Verify near perfect reconstruction in the MP3 standard In Section 7.3.3 in the book we saw that the polyphase representation of GH is   S0 · · · 0   16  ... . . . ...  E480 , 0 · · · S31 where Si = (V (16+i) )T V (16+i) + (V (48+i) )T V (48+i) . As explained in Section 7.3.4 in the book, the MP3 standard chooses the prototype filter so that the Si are equal (so that the block matrix above is a filter), but only near perfect reconstruction is possible. a) With Si = {..., t−1 , t0 , t1 , ...}, explain why the (magnitude of the) frequency response of (the filter) GH equals 16

X

tk e−iM kω .

k

Solution.

Disregarding the delay E480 , which does not affect the absolute value of the frequency response, the filter coefficients of GH come at indices 0, M , 2M , and so on, and are 16t0 , 16t1 , 16t2 , and so on. This gives

|λGH (ω)| = 16

X

tk e−iM kω .

k

b) Use the values in the table C to find the filter coefficients tk for S1 = (V (17) )T V (17) + (V (49) )T V (49) . (use (7.20) in the book, even though this filter bank does not give perfect reconstruction), and plot this frequency response. If you get an almost flat frequency response, you have verified near perfect reconstruction in the MP3 standard. It is important that you scale the vertical axis in order to see that the frequency response is close to constant.

182

7 The polyphase representation of filter bank transforms

Solution.

The following code can be used.

C = mp3_ctable(); V1=zeros(15,1); V1(1:2:end)=C(18:64:end); V2=zeros(15,1); V2(1:2:end)=C(50:64:end); diagentry = conv(flip(V1),V1)+conv(flip(V2),V2); S = zeros(32*length(diagentry),1); S(1:32:end) = diagentry; % upsampled S = [S; zeros(16384,1)]; N = length(S); n = (0:(N-1))’; plot(2*pi*n/N, abs(fft(S)), ’k-’) axis([0 2*pi 0 1.5/512])

c) Verify that the frequency response of S1 in b) should be close to 1/(16 · 32) = 1/512.

The factor 32 has to do with the relative scaling of the C and D tables.

Solution.

Since the values in the table D are 32 times those in the table C, we will instead have λS (ω) = (16 · 32)

X k

tk e−iM kω = 512

X

tk e−iM kω

k

(recall that SiPrepresents values from the table C). For this to be close to one we must −iM kω clearly have ≈ 1/512. k tk e

Exercise 7.16: Lifting factorization of M -channel filter bank transforms The fact that step 2 and 3 in the MP3 encoding procedure can be computed in-place, corresponds in our modified scheme to multiplying with the rightmost matrix in (7.21) in the book in-place. This exercise addresses how this can be done. We omit some details in that we make some simplifications, and don’t write down any code. a) Use a lifting factorization of each of the 16 subblocks to show that (7.21) in the book can be factored as a product of a diagonal matrix with entries on the form αi Edi on the diagonal, a permutation matrix, and matrices on the form

7 The polyphase representation of filter bank transforms

183

   I ··· 0 0 ··· 0 I · · · 0 0 · · · S15  .. . . .. .. . . ..   .. . . .. .. . . ..   . . . . . . . . . . . .       0 · · · I 0 · · · 0 0 · · · I S0 · · · 0    , and    0 · · · S0 I · · · 0 . 0 · · · 0 I · · · 0       . . . . . . . . . . .   .. . . .. .. . . . ..   .. . . .. .. . . ..  S15 · · · 0 0 · · · I 0 ··· 0 0 ··· I 

This shows that lifting factorizations can be useful in saving arithmetic operations for M -channel filter bank transforms as well.

Solution.

The mentioned permutation matrix is of size 16 × 16, and flips the numbers from 1 to 8 internally, and also the numbers from 9 to 16 internally. Using this it is clear that ···  ..  .   V (31) · · ·  −V (63) E1 · · ·   .. ..  . . 

0 .. .

0

V (16) .. .

0 0 .. .

 V (15) · · · 0 .. .. ..   . . .  (0)  0 ··· V  (P ⊗k I) 0 · · · V (32) E1    .. .. ..  . . .

· · · −V (48) E1 V (47) E1 · · ·

0

 ··· 0 0 · · · V (15) .. .. .. .. ..   ..   . . . . . .   (31) (0)  V ··· 0  0 ··· V  . = 0 · · · −V (63) E1 V (32) E1 · · · 0      .. .. .. .. .. ..   . . . . . .

(48) (47) −V E1 · · · 0 0 ··· V E1 

V

(16)

The reason for using this permutation matrix is that it places the diagonal entries in each subblock on the diagonal of the larger matrix. After this transformation the subblocks have changed, and the first subblock consists of the blocks at positions (0, 0), (0, 31), (31, 0), (31, 31). The other subblocks are found by going towards the middle of the matrix in the four diagonal directions. Combining an even lifting step for all subblocks gives a matrix on the form   I · · · 0 0 · · · S15  .. . . .. .. . . ..  . . . . . .    0 · · · I S0 · · · 0    0 · · · 0 I · · · 0  ,   . . . . . .   .. . . .. .. . . ..  0 ··· 0 0 ··· I while combining an odd lifting step for all subblocks gives a matrix on the form

184

7 The polyphase representation of filter bank transforms

 ··· 0 0 ··· 0 . . .. .. . . ..  . . . . .  · · · I 0 · · · 0 . · · · S0 I · · · 0  . . .. .. . . ..  . . . . . S15 · · · 0 0 · · · I



I  ..  .   0   0   .  ..

The reason that we can combine the lifting steps in this way is that each corresponds to elementary row operations which do not affect the other subblocks. Finally, after all the lifting steps each subblock is diagonal with entries on the form αi Edi on the diagonal. Clearly then the matrix with 32 × 32 subblocks also is diagonal, with αi Edi on the diagonal. b) Assume the the 16 subblocks have a lifting factorization in terms of elementary lifting matrices. Show that the filter representation of the first matrix listed in a) looks as follows around the diagonal   0 · · · λM/2−1 1 · · · 0 0 · · · λM/2−1  .. . .  .. .. . . .. .. . . ..  .  . . . . . . . .    λ0 · · ·  0 0 · · · 1 λ0 · · · 0  ,  0 ···  0 0 ··· 0 1 ··· 0    . .  . . . . . .. .. . . . .. .. . . . ..  .. . .  0 ···

0

0 ··· 0 0 ···

1

where the shown blocks are of size (M/2) × (M/2), and the ones are on the main diagonal. Convince yourself that M/2 consecutive rows are unaltered, while the other M/2 consecutive rows are altered by combining with the M/2 rows above and below, so that a composition of these matrices can be computed in-place. This also gives a reduction in the number of arithmetic operations with up to 50 percent, since this is the case for each subblock.

Solution.

For a block row of the first matrix listed in a), since Si = λi {1, 1} if even elementary lifting steps are assumed, the only nonzero entries in a row in the filter representation will be • a 1 on the diagonal, • λi 31 − i to the left of the diagonal, and • λi i + 1 to the right of the diagonal.

Writing this down for different i we obtain the matrix from the text. c) Show also that the matrices in a) can be inverted by adding a minus sign to all the Si -components.

7 The polyphase representation of filter bank transforms

185

Solution.

For elementary lifting matrices we know that inversion is achieved by adding a minus sign. Since the different subblocks don’t interact with the others in multiplication, we simply have to add a minus sign to all the Si to find the inverse.

This exercise can also be used in constructing useful cosine-modulated filter banks with perfect reconstruction: Simply let the λi in the “generalized lifting steps” in b) be parameters one needs to optimize in some way.

Exercise 7.17: DFT modulated filter banks −1 Assume that {Si }M i=0 are all-pass filters (see Exercise 3.44). Show that   S0 · · · 0   (DFTM ⊗k I)  ... . . . ... 

0 · · · SM −1 and (S0 )T · · ·  .. . .  . . 

0

0 .. .

· · · (SM −1 )T

  k  (IDFTM ⊗ I)

invert each other. By realizing this in terms of polyphase components, explain how one can assemble a prototype filter from all-pass filters, and forward and reverse DFTmodulated filter banks (i.e. where the filters are obtained by modulating the prototype with e2πikn/M ), so that we have perfect reconstruction.

Solution.

This follows from the fact hat (Si )T Si = I from allpass filters, and from that (DFTM ⊗k I)(IDFTM ⊗k I) = (DFTM IDFTM ) ⊗k I = I ⊗k I = I. This is now very similar to the previous construction. The difference is that, while we previously factored subblocks into lifting steps we now instead factor the all-pass filters on the diagonal into simpler ones, following for instance the procedure from Exercise 3.44.

Chapter 8

Digital images

Exercise 8.6: Generating black and white images Black and white images can be generated from grayscale images (with values between 0 and 255) by replacing each pixel value with the one of 0 and 255 which is closest. Use this strategy to generate the black and white image shown in the right part of Figure 8.2 in the book.

Solution.

The following code can be used X2 = 255.*(X1 >= 128); imshow(uint8(X2))

Exercise 8.7: Adjusting contrast a) Write a function which instead uses the function fn from Example 8.5 in the book to adjust the contrast (rather than g ). n should be a parameter to your function.

Solution.

The code could look as follows: function Z=contrast_adjust0(X, n) Z = X/255; % Maps the pixel values to [0,1] Z = atan(n*(Z-1/2))/(2*atan(n/2)) + 1/2; Z = Z*255; % Maps the values back to [0,255] end

b) Generate the left and right images in Figure 8.7 in the book on your own by using code which calls the function you have written, as well as contrast_adjust.

187

188

8 Digital images

Solution.

The following code can be used.

X1 = contrast_adjust0(img, 10); imshow(uint8(X1))

X2 = contrast_adjust(img, 0.01); imshow(uint8(X2))

Exercise 8.8: Adjusting contrast with another function a) Show that the function hn : R → R given by hn (x) = xn , maps the interval [0, 1] → [0, 1] for any n, and that h0n (1) → ∞ as n → ∞.

Solution.

We have that h0n (x) = nxn−1 , so that h0n (1) = n → ∞. b) The image secret.jpg, shown in Figure 8.1, contains some information that is nearly invisible to the naked eye on most monitors. Use the function hn , to reveal the secret message.

Fig. 8.1 Secret message.

8 Digital images

189

Hint.

Convert to a grayscale image before you adjust the contrast with hn .

Solution.

The following code can be used n = 100; X = double(imread(’images/secret.jpg’, ’jpg’)); X = (X(:,:,1) + X(:,:,2) + X(:,:,3))/3; X = map_to_01(X); X = X.^n; X = 255*X; imshow(uint8(X))

The resulting image is shown in Figure 8.2.

Fig. 8.2 Secret message revealed!

Exercise 8.13: Generating images Write code which calls the function tensor2_impl with appropriate filters to produce the following images: a) The right image in Figure 8.10 in the book.

Solution.

The following code can be used

190

8 Digital images

longmolecule = @(x, bd_mode) filter_impl([1, 6, 15, 20, 15, 6, 1]/64, ... x, bd_mode); X3 = tensor2_impl(X1, longmolecule, longmolecule, ’symm’); imshow(uint8(X3))

b) The right image in Figure 8.12 in the book.

Solution.

The following code can be used X3 = tensor2_impl(excerpt, donothing, diffxmolecule, ’per’); X3 = map_to_01(X3); X3 = X3*255; X3 = contrast_adjust0(X3,50); imshow(uint8(X3))

c) The images in Figure 8.14 in the book.

Solution.

The following code can be used excerpt=create_excerpt(); X1 = tensor2_impl(excerpt, donothing, diffxmolecule, ’per’); X2 = tensor2_impl(excerpt, diffymolecule, donothing, ’per’); X1 = X1.^2 + X2.^2; imshow(uint8(X1)) X2 = X1; X2 = map_to_01(X2); X2 = X2*255; imshow(uint8(X2)) X3 = X2; X3 = contrast_adjust0(X3, 50); imshow(uint8(X3))

d) The images in Figure 8.13 in the book.

Solution.

The following code can be used excerpt=create_excerpt(); X1 = tensor2_impl(excerpt, donothing, diffxmolecule, ’per’); X1 = map_to_01(X1); X1 = X1*255; X1 = contrast_adjust0(X1, 50); imshow(uint8(X1))

8 Digital images

191

diffymolecule = @(x, bd_mode) filter_impl([-1 0 1]/2, x, bd_mode); X2 = tensor2_impl(excerpt, diffymolecule, donothing, ’per’); X2 = map_to_01(X2); X2 = X2*255; X2 = contrast_adjust0(X2, 50); imshow(uint8(X2))

e) The images in Figure 8.15 in the book.

Solution.

The following code can be used excerpt=create_excerpt(); X1 = tensor2_impl(excerpt, donothing, diffxmolecule, ’per’); X1 = tensor2_impl(X1, donothing, diffxmolecule, ’per’); X1 = map_to_01(X1); X1 = X1*255; X1 = contrast_adjust0(X1, 100); imshow(uint8(X1))

X2 = tensor2_impl(excerpt, diffymolecule, diffxmolecule, ’per’); X2 = map_to_01(X2); X2 = X2*255; X2 = contrast_adjust0(X2, 100); imshow(uint8(X2))

X3 = tensor2_impl(excerpt, diffymolecule, donothing, ’per’); X3 = tensor2_impl(X3, diffymolecule, donothing, ’per’); X3 = map_to_01(X3); X3 = X3*255; X3 = contrast_adjust0(X3, 100); imshow(uint8(X3))

Exercise 8.14: Interpret tensor products Let the filter S be defined by S = {−1, 1}. Write down the 4 × 4-matrix X = (1, 1, 1, 1) ⊗ (0, 0, 1, 1), and compute (S ⊗ I)X and (I ⊗ S)X.

Solution.

We have that 

 0011 0 0 1 1  (1, 1, 1, 1) ⊗ (0, 0, 1, 1) =  0 0 1 1 .

0011 For the tensor products we first compute

192

8 Digital images

S(1, 1, 1, 1) = (0, 0, 0, 0)

S(0, 0, 1, 1) = (0, 0, −1, 0)

(assuming symmetric extensions). We then obtain   0000 0 0 0 0  (S ⊗ I)X = (S(1, 1, 1, 1)) ⊗ (0, 0, 1, 1) = (0, 0, 0, 0) ⊗ (0, 0, 1, 1) =  0 0 0 0 0000   0 0 −1 0 0 0 −1 0  (I ⊗ S)X = (1, 1, 1, 1) ⊗ (S(0, 0, 1, 1)) = (1, 1, 1, 1) ⊗ (0, 0, −1, 0) =  0 0 −1 0 .

0 0 −1 0

Exercise 8.15: Computational molecule of moving average filter Let S be the moving average filter of length 2L + 1, i.e. T = L1 {1, · · · , 1, 1, 1, · · · , 1}. | {z } 2L+1 times

What is the computational molecule of S ⊗ S?

Solution.

The molecule is a (2L + 1) × (2L + 1)-matrix with all entries equal to 1/L2 .

Exercise 8.16: Bilinearity of the tensor product Show that the mapping F (x, y) = x ⊗ y is bi-linear, i.e. that F (αx1 + βx2 , y) = αF (x1 , y) + βF (x2 , y), and F (x, αy1 + βy2 ) = αF (x, y1 ) + βF (x, y2 ).

Solution.

We have that F (αx1 + βx2 , y) = (αx1 + βx2 ) ⊗ y = (αx1 + βx2 )y T = αx1 y T + βx2 y T = α(x1 ⊗ y) + β(x ⊗ y) = αF (x1 , y) + βF (x1 , y).

The second statement follows similarly.

8 Digital images

193

Exercise 8.17: Attempt to write as tensor product Attempt to find matrices S1 : RM → RM and S2 : RN → RN so that the following mappings from LM,N (R) to LM,N (R) can be written on the form X → S1 X(S2 )T = (S1 ⊗ S2 )X. In all the cases it may be that no such S1 , S2 can be found. If this is the case, prove it. a) The mapping which reverses the order of the rows in a matrix.

Solution.

Multiplication with the matrix   0 0 0 ··· 0 0 1 0 0 0 · · · 0 1 0     S =  ... ... ... ... ... ... ...    0 1 0 · · · 0 0 0 1 0 0 ··· 0 0 0 reverses the elements in a vector. This means that ((S ⊗ I)(x ⊗ y))i,j = ((Sx) ⊗ y)i,j = (Sx)i yj = xM −1−i yj = (x ⊗ y)M −1−i,j .

This means that also ((S ⊗ I)X)i,j = XM −1−i,j for all X, so that S ⊗ I reverses rows, and thus is a solution to a). b) The mapping which reverses the order of the columns in a matrix.

Solution.

Similarly one shows that I ⊗ S reverses columns, and is thus a solution to b). c) The mapping which transposes a matrix.

Solution.

It turns out that it is impossible to find S1 and S2 so that transposing a matrix X corresponds to computing (S1 ⊗ S2 )X. To see why, S1 and S2 would need to fulfill (S1 ⊗ S2 )(ei ⊗ ej ) = (S1 ei ) ⊗ (S2 ej ) = ej ⊗ ei , since ej ⊗ ei is the transpose of ei ⊗ ej . This would require that S1 ei = ej for all i, j, which is impossible.

Exercise 8.18: Computational molecules Let the filter S be defined by S = {1, 2, 1}.

194

8 Digital images

a) Write down the computational molecule of S ⊗ S.

Solution.

The computational molecule of S ⊗ S is     121 1

(1, 2, 1) ⊗ (1, 2, 1) = (1, 2, 1) ⊗ (1, 2, 1) = 2 1 2 1 = 2 4 2 .

1 121 b) Let us define x = (1, 2, 3), y = (3, 2, 1), z = (2, 2, 2), and w = (1, 4, 2). Compute the matrix A = x ⊗ y + z ⊗ w.

Solution.

We get that     1 2

A = 2 3 2 1 + 2 1 4 2 3 2       321 284 5 10 5 = 6 4 2 + 2 8 4 =  8 12 6 .

963 284 11 14 7 c) Compute (S ⊗ S)A by applying the filter S to every row and column in the matrix the way we have learned. If the matrix A was more generally an image, what can you say about how the new image will look?

Solution.

We need to compute (S ⊗ S)A = SAS T , which corresponds to first applying S to every column in the image, and then applying S to every row in the resulting image. If we apply S to every column in the image we first get the matrix   29 46 23 SA = 32 48 24 .

35 50 25 If we apply the filter to the rows here we get   127 144 121 SAS T = 136 152 128 .

145 160 135 Since the filter which is applied is a low-pass filter, the new image should look a bit more smooth than the original image.

8 Digital images

195

Exercise 8.19: Computational molecules 2 Let S = 14 {1, 2, 1} be a filter. a) What is the effect of applying the tensor products S ⊗ I, I ⊗ S, and S ⊗ S on an image represented by the matrix X?

Solution.

Note first that the filter is a low-pass filter. We know that S ⊗ I corresponds to applying S to the columns of the matrix, so that we get the result by applying the low-pass filter to the columns of the matrix. The result of this is that horizontal edges are smoothed.

Similarly, the tensor product I ⊗ S corresponds to applying S to the rows of the matrix, so that vertical edges are smoothed. Finally, S ⊗ S corresponds to applying S first to the columns of the matrix, then to the rows. The result is that both horizontal and vertical edges are smoothed. You could also have computed the computational molecules for S ⊗ I, I ⊗ S, and S ⊗ S, by taking the tensor product of the filter coefficients 14 {1, 2, 1} with itself. From these molecules it is also clear that they either work on the columns, the rows, or on both rows and columns. b) Compute (S ⊗ S)(x ⊗ y), where x = (4, 8, 8, 4), y = (8, 4, 8, 4) (i.e. both x and y are column vectors).

Solution.

A 4 × 4 circulant Toeplitz matrix for S is   2101  1 1 2 1 0 .

4 0 1 2 1 1012 From this we can quickly compute that 

      2101 4 2+2+1 5  8 4 + 2 + 1 7 1 1 2 1 0   =  =  Sx =  4 0 1 2 1 8 4 + 2 + 1 7 1012 4 2+2+1 5        2101 8 4+1+1 6  4 2 + 2 + 2 6 1 1 2 1 0   =   =  .

Sy =  4 0 1 2 1 8 4 + 1 + 1 6 1012 4 2+2+2 6 From this it is clear that     5 30 30 30 30 7 42 42 42 42    (S ⊗ S)(x ⊗ y) = (Sx)(Sy)T =  7 6 6 6 6 = 42 42 42 42 .

5 30 30 30 30

196

8 Digital images

Exercise 8.20: Eigenvectors of tensor products Let vA be an eigenvector of A with eigenvalue λA , and vB an eigenvector of B with eigenvalue λB . Show that vA ⊗ vB is an eigenvector of A ⊗ B with eigenvalue λA λB .

Solution.

We have that (A ⊗ B)(vA ⊗ vB ) = (AvA ) ⊗ (BvB ) = (λA vA ) ⊗ (λB vB ) = λA λB (vA ⊗ vB ).

Exercise 8.21: The Kronecker product The Kronecker tensor product of two matrices A and B, written A ⊗k B, is defined as   a1,1 B a1,2 B · · · a1,M B a2,1 B a2,2 B · · · a2,M B    A ⊗k B =  . .. . . ..  ,  .. . . .  ap,1 B ap,2 B · · · ap,M B where the entries of A are ai,j . The tensor product of a p×M -matrix, and a q ×N -matrix is thus a (pq) × (M N )-matrix. Note that this tensor product in particular gives meaning for vectors: if x ∈ RM , y ∈ RN are column vectors, then x ⊗k y ∈ RM N is also a column vector. In this exercise we will investigate how the Kronecker tensor product is related to tensor products as we have defined them in this section. a) Explain that, if x ∈ RM , y ∈ RN are column vectors, then x ⊗k y is the column vector where the rows of x ⊗ y have first been stacked into one large row vector, and this vector transposed. The linear extension of the operation defined by x ⊗ y ∈ RM,N → x ⊗k y ∈ RM N thus stacks the rows of the input matrix into one large row vector, and transposes the result.

Solution.

We have that 

 x1 y  x2 y    x ⊗k y =  .   ..  xM y The rows of x ⊗ y are xi y T . Stacking these into one large row vector we obtain

x1 y T x2 y T · · · xM y T ,

8 Digital images

197

which is the transpose of what we obtained above. b) Show that (A ⊗k B)(x ⊗k y) = (Ax) ⊗k (By). We can thus use any of the defined tensor products ⊗, ⊗k to produce the same result, i.e. we have the commutative diagram shown in Figure 8.3, where the vertical arrows represent stacking the rows in the matrix, and transposing, and the horizontal arrows represent the two tensor product linear transformations we have defined. In particular, we can compute the tensor product in terms of vectors, or in terms of matrices, and it is clear that the Kronecker tensor product gives the matrix of tensor product operations.

x⊗y

x ⊗k y

A⊗B

/ (Ax) ⊗ (By)

/ (Ax) ⊗k (By),

A⊗k B

Fig. 8.3 Tensor products

Solution.

We have that (A ⊗k B)(x ⊗k y)      (a11 x1 + . . . + a1M xm )By a11 B a12 B · · · a1M B x1 y a21 B a22 B · · · a2M B   x2 y  (a21 x1 + . . . + a2M xm )By       = .  .. .. . . ..   ..  =     ..    . . . . . ap1 B ap2 B · · · apM B xM y   (Ax)1 By (Ax)2 By    =  = (Ax) ⊗k (By). ..   .

(Ax)p By

(ap1 x1 + . . . + apM xm )By

c) Using the Euclidean inner product on L(M, N ) = RM N , i.e. hX, Y i =

M −1 N −1 X X

Xi,j Yi,j .

i=0 j=0

and the correspondence in a) we can define the inner product of x1 ⊗ y1 and x2 ⊗ y2 by hx1 ⊗ y1 , x2 ⊗ y2 i = hx1 ⊗k y1 , x2 ⊗k y2 i.

Show that hx1 ⊗ y1 , x2 ⊗ y2 i = hx1 , x2 ihy1 , y2 i.

Clearly this extends linearly to an inner product on LM,N .

198

8 Digital images

Solution.

We have that     + M −1 * (x2 )0 y2 (x1 )0 y1 X     . . .. .. hx1 ⊗ y1 , x2 ⊗ y2 i =  (x1 )i (x2 )i hy1 , y2 i  = , i=0 (x2 )M −1 y2 (x1 )M −1 y1 = hy1 , y2 i

M −1 X

(x1 )i (x2 )i = hx1 , x2 ihy1 , y2 i.

i=0

d) Show that the FFT factorization can be written as

FN/2 DN/2 FN/2 I DN/2 = N/2 (I2 ⊗k FN/2 ).

FN/2 −DN/2 FN/2 IN/2 −DN/2 Also rewrite the sparse matrix factorization for the FFT from equation (2.16) in the book in terms of tensor products.

Solution.

We have that

FN/2 DN/2 FN/2 IN/2 DN/2 FN/2 0 IN/2 DN/2 = = (I2 ⊗k FN/2 ).

FN/2 −DN/2 FN/2 IN/2 −DN/2 0 FN/2 IN/2 −DN/2 Equation (2.16) in the book can clearly be rewritten as log2 N

DFTN =

Y

k=1

I DN/2k I2k−1 ⊗ I −DN/2k

P.

Exercise 8.24: Implementing DFT and DCT on blocks In this section we have used functions which apply the DCT and the DFT either to subblocks of size 8 × 8, or to the full image. a) Implement functions dft_impl8, idft_impl8, dct_impl8, and idct_impl8 which apply the DFT, IDFT, DCT, and IDCT, to consecutive blocks of length 8.

Solution.

The following code can be used. function x = dft_impl8(x, bd_mode) N = size(x, 1); for n = 1:8:N x(n:(n+7), :) = fft(x(n:(n+7), :));

8 Digital images

199

end end

function x = idft_impl8(x, bd_mode) N = size(x, 1); for n = 1:8:N x(n:(n+7), :) = ifft(x(n:(n+7), :)); end end

function x = dct_impl8(x, bd_mode) N = size(x, 1); for n = 1:8:N x(n:(n+7), :) = dct(x(n:(n+7), :)); end end

function x = idct_impl8(x, bd_mode) N = size(x, 1); for n = 1:8:N x(n:(n+7), :) = idct(x(n:(n+7), :)); end end

b) Implement the two-dimensional FFT, IFFT, DCT, and IDCT on images, with the help of their one-dimensional counterparts, as well as the function tensor2_impl.

Solution.

The following code can be used.

X = tensor2_impl(img, @(x, bd_mode) fft(x(:,:)), ... @(x, bd_mode) fft(x(:,:)), ’symm’); X = tensor2_impl(img, @(x, bd_mode) ifft(x(:,:)), ... @(x, bd_mode) ifft(x(:,:)), ’symm’);

X = tensor2_impl(img, @(x, bd_mode) dct(x(:,:)), ... @(x, bd_mode) dct(x(:,:)), ’symm’); X = tensor2_impl(img, @(x, bd_mode) idct(x(:,:)), ... @(x, bd_mode) idct(x(:,:)), ’symm’);

c) The function forw_comp_rev_2d in the library applies given transforms to the rows and columns of an input image, sets the coefficients below a certain threshold to zero, and transforms back. Run forw_comp_rev_2d for different threshold parameters on the sample image, and with the functions dct_impl8, idct_impl8, as well as DCT and IDCT applied to the entire input, as parameters. Check that this reproduces the DCT test images of this section, and that the correct numbers of values which have been discarded (i.e. which are below the threshold) are printed on screen.

200

8 Digital images

Solution.

Once X1 = forw_comp_rev_2d(X, @dct_impl8, @idct_impl8, 30); imshow(uint8(X1))

X2 = forw_comp_rev_2d(X, @dct_impl8, @idct_impl8, 50); imshow(uint8(X2))

X3 = forw_comp_rev_2d(X, @dct_impl8, @idct_impl8, 100); imshow(uint8(X3))

X1 = forw_comp_rev_2d(X, @(x, bd_mode) dct(x(:,:)), ... @(x, bd_mode) idct(x(:,:)), 30); imshow(uint8(X1))

X2 = forw_comp_rev_2d(X, @(x, bd_mode) dct(x(:,:)), ... @(x, bd_mode) idct(x(:,:)), 50); imshow(uint8(X2))

X3 = forw_comp_rev_2d(X, @(x, bd_mode) dct(x(:,:)), ... @(x, bd_mode) idct(x(:,:)), 100); imshow(uint8(X3))

Exercise 8.25: Code example Suppose that we have given an image by the matrix X. Consider the following code: threshold = 30; X = fft_impl(X, @fft_kernel_standard); X = X’; X = fft_impl(X, @fft_kernel_standard); X = X’; X = X.*(abs(X) >= threshold); X = fft_impl(X, @fft_kernel_standard, 0); X = X’; X = fft_impl(X, @fft_kernel_standard, 0); X = X’;

Comment what the code does. Comment in particular on the meaning of the parameter threshold, and its effect on the image.

Solution.

In the first part of the code one makes a change of coordinates with the DFT. More precisely, this is a change of coordinates on a tensor product, as we have defined it.

In the last part the change of coordinates is performed the opposite way. Both these change of coordinates is performed is performed the way we have described them, first

8 Digital images

201

on the rows in the matrix, then on the columns. The parameter threshold is used to discard DFT-coefficients which are below a certain value. We have seen that this can give various visual artifacts in the image, even though the main contents of the image still may be visible. If we increase threshold, these artifacts will be more dominating since we then discard many DFT-coefficients.

Chapter 9

Using tensor products to apply wavelets to images

Exercise 9.3: Alternative definition of the 2D DWT In terms of tensor products, the way we defined the two-dimensional DWT may not seem very natural. Why not define it as the tensor product of two m-level DWT’s, instead of iterated tensor products of 1-level DWT’s)? a) Explain that this alternative definition would alter the vectors listed in (9.10) in the book to {φ0,n1 ⊗ ψk2 ,n2 , ψk1 ,n1 ⊗ φ0,n2 , ψk1 ,n1 ⊗ ψk2 ,n2 }n1 ,n2 ,0≤k1 ,k2 <m .

Solution.

This simply amounts to combining functions from {φ0,n1 , ψ0,n1 , ..., ψm−1,n1 } with functions from {φ0,n2 , ψ0,n2 , ..., ψm−1,n2 } in all possible ways. The difference from the previous situation is that we now allow for combinations where the components have different resolutions k1 and k2 . b) Explain that, with this alternative definition, dwt2_impl_internal and idwt2_impl_internal could be implemented simply as f = @(x, bd_mode) dwt_impl(x, ’cdf97’, 4, bd_mode, ’none’, 1); img = tensor2_impl(img, f, f, ’symm’);

and invf = @(x, bd_mode) idwt_impl(x, ’cdf97’, 4, bd_mode, ’none’, 1); img = tensor2_impl(img, invf, invf, ’symm’);

This is much simpler than the implementation we have presented.

Solution.

This follows from that the new definition is a simple tensor product, for which tensor2_impl does the job.

There seems to be few comments in the literature on why the two-dimensional DWT is not defined as a tensor product of two m-level DWT’s.

203

204

9 Using tensor products to apply wavelets to images

Exercise 9.4: Higher order tensor products Tensor products can easily be generalized to higher dimensions. For 3D, and for x ∈ RM , y ∈ RN , z ∈ RK , we define x ⊗ y ⊗ z as the "3D matrix" with entries (x ⊗ y ⊗ z)i,j,k = xi yj zk , and if S1 , S2 and S3 are linear transformations on RM , RN , and RK , respectively, their tensor product is the unique linear transformation S1 ⊗ S2 ⊗ S3 defined on 3D matrices by (S1 ⊗ S2 ⊗ S3 )(x ⊗ y ⊗ z) = (S1 x) ⊗ (S2 y) ⊗ (S3 z), for all x, y, and z. a) Explain that S1 ⊗ S2 ⊗ S3 can be implemented by • replacing x0:(M −1),j,k by S1 (x0:(M −1),j,k ) for all j, k, • replacing xi,0:(N −1),k by S2 (xi,0:(N −1),k ) for all i, k, • replacing xi,j,0:(K−1) by S3 (xi,j,0:(K−1) ) for all j, k, and that the order of these three operations does not matter. In other words, higher order tensor products can also be split into a series of one-dimensional operations.

Solution.

Similarly to the 2D case we can write S1 ⊗ S2 ⊗ S3 = (S1 ⊗ I ⊗ I)(I ⊗ S2 ⊗ I)(I ⊗ I ⊗ S3 ), where the order of the three right hand factors does not matter. Also, by adding terms, we can write any 3D matrix as X (x0:(M −1),j,k ) ⊗ ej ⊗ ek , i,j

and applying (S1 ⊗ I ⊗ I) to this we obtain X (S1 x0:(M −1),j,k ) ⊗ ej ⊗ ek . i,j

This proves the claim that we can apply S1 to the different “(:, j, k)-segments”. The cases for I ⊗ S2 ⊗ I and I ⊗ I ⊗ S3 follow similarly. b) In particular we can extend the tensor product to wavelets in 3D. How many detail components will there be at each level in a 3D DWT, following the setup we did for the 2D DWT? How would you label the different components?

Solution.

Duplicating the theory for changes of coordinates in 2D, we see that the target basis for a 1-level 3D DWT consists of 23 = 8 different types of basis functions:

9 Using tensor products to apply wavelets to images

φ0,n1 ⊗ φ0,n2 ⊗ φ0,n3

205

ψ0,n1 ⊗ φ0,n2 ⊗ φ0,n3

φ0,n1 ⊗ ψ0,n2 ⊗ φ0,n3

φ0,n1 ⊗ φ0,n2 ⊗ ψ0,n3

ψ0,n1 ⊗ ψ0,n2 ⊗ φ0,n3

ψ0,n1 ⊗ φ0,n2 ⊗ ψ0,n3

φ0,n1 ⊗ ψ0,n2 ⊗ ψ0,n3

ψ0,n1 ⊗ ψ0,n2 ⊗ ψ0,n3 .

Using as before L for φ, H for ψ, these would be labeled LLL, HLL, LHL, LLH, HHL, HLH, LHH, and HHH, respectively. dwt3_impl_internal and idwt3_impl_internal are the functions in the library which compute the 3D DWT and the 3D IDWT. You are in particular encouraged to review the function tensor3_impl, which is used in the functions, and is a straightforward generalization of tensor2_impl. Also in the 3D case there is a simpler alternative definition for the DWT, following the setup from the previous exercise.

Exercise 9.9: Code example Assume that we have an image represented by the M × N -matrix X, and consider the following code: c = (X(1:2:M, :) + X(2:2:M, :))/sqrt(2); w = (X(1:2:M, :) - X(2:2:M, :))/sqrt(2); X = [c; w]; c = (X(:, 1:2:N) + X(:, 2:2:N))/sqrt(2); w = (X(:, 1:2:N) - X(:, 2:2:N))/sqrt(2); X = [c w];

a) Comment what the code does, and explain what you will see if you display X as an image after the code has run.

Solution.

The code runs a DWT over one level, and the Haar wavelet is used. Inside the forloops the DWT is applied to every row and column in the image. k=1 i the for-loop corresponds to applying the DWT to the columns, k=2 corresponds to applying the DWT to the rows. In the upper left corner we will see a low-resolution version of the image. In the other three corners you will see different types of detail: In the upper right corner you will see detail which corresponds to quick vertical changes, in the lower left corner you will see detail which corresponds to quick horizontal changes, and in the lower right corner you will see points where quick changes both vertically and horizontally occur simultaneously. b) The code above has an inverse transform, which reproduces the original image from the transformed values. Assume that you zero out the values in the lower left and the upper right corner of the matrix X after the code above has run, and that you reproduce the image by applying this inverse transform. What changes do you expect in the image?

206

9 Using tensor products to apply wavelets to images

Solution.

By zeroing out the two corners you remove detail which correspond to quick horizontal and vertical changes. But since we keep the lower right corner, we keep detail which corresponds to simultaneous changes vertically and horizontally. The result after the inverse transformation is that most edges have been smoothed, but we see no smoothing effect in points where quick changes occur both horizontally and vertically. In Example 9.5 in the book, this corresponds to that we emphasize the grid points in the chess pattern, mut that we smooth out the horizontal and vertical edges in the chess pattern.

Exercise 9.10: Applying the Haar wavelet to another chess pattern image The following code produces a chess pattern type image almost identical to that from Example 9.5 in the book.

N = 128; img=zeros(N); for x=0:(N-1) for y=0:(N-1) img(x+1,y+1)=255*( (mod(x,64)>=32) == (mod(y,64)>=32) ); end end imshow(uint8(img))

Let us now apply a 2D DWT to this image as well with the Haar wavelet: img2 = img; img2 = dwt_impl(img2, ’Haar’, 1, ’per’, ’none’, 2); img2 = map_to_01(img2); img2 = img2*255; imshow(uint8(img2))

The resulting images are shown in Figure 9.1

Fig. 9.1 A simple image before and after one level of the 2D DWT. The Haar wavelet was used.

9 Using tensor products to apply wavelets to images

207

There seem to be no detail components here, which is very different from what you saw in Example 9.5 in the book, even though the images are very similar. Attempt to explain what causes this to happen.

Hint.

Compare with Exercise 4.14.

Solution.

In Example 9.5 in the book, the borders in the chess pattern was chosen so that they occur at odd numbers. This means that the image can not be represented exactly in Vm−1 ⊗ Vm−1 , so that there is detail present in the image at all the borders in the chess pattern. In the new image, the borders in the chess pattern was chosen so that they occur at even numbers. This means that the image can be represented exactly in Vm−1 ⊗ Vm−1 , so that there is no detail components present in the image.

Exercise 9.11: Implementing the fingerprint compression scheme Write code which generates the images shown in figures 9.5 in the book, 9.5 in the book, and 9.23 in the book. Use the functions dwt_impl and idwt_impl with the CDF 9/7 wavelet.

Solution.

The following code generates the different parts of Figure 9.5 in the book. img=double(imread(’images/fingerprint_500x500.png’)); imgnew=255*ones(512,512,3); imgnew(1:500,1:500,:)=img; img=imgnew; l1 = 512; l2 = 512; l3 = 3; X0 = dwt_impl(img, ’cdf97’, 1); X1 = X0; X0 = map_to_01(X0); X0 = X0*255; imshow(uint8(X0))

X1(1:(l1/2), 1:(l2/2), :) = dwt_impl(X1(1:(l1/2), 1:(l2/2), :), ’cdf97’, 1); X1(1:(l1/2), (l2/2+1):end, :) = ... dwt_impl(X1(1:(l1/2), (l2/2+1):end, :), ’cdf97’, 1); X1((l1/2+1):end, 1:(l2/2), :) = ... dwt_impl(X1((l1/2+1):end, 1:(l2/2), :), ’cdf97’, 1); X1((l1/2+1):end, (l2/2+1):512, :) = ... dwt_impl(X1((l1/2+1):end, (l2/2+1):512, :), ’cdf97’, 1); X2 = X1; X1 = map_to_01(X1); X1 = X1*255; imshow(uint8(X1))

208

9 Using tensor products to apply wavelets to images

X2(1:(l1/4), 1:(l2/4), :) = dwt_impl(X2(1:(l1/4), 1:(l2/4), :), ’cdf97’, 1); X2(1:(l1/4), (l2/4+1):(l2/2), :) = ... dwt_impl(X2(1:(l1/4), (l2/4+1):(l2/2), :), ’cdf97’, 1); X2((l1/4+1):(l1/2), 1:(l2/4), :) = ... dwt_impl(X2((l1/4+1):(l1/2), 1:(l2/4), :), ’cdf97’, 1); X3 = X2; X2 = map_to_01(X2); X2 = X2*255; imshow(uint8(X2))

X3(1:64, 1:64, :) = dwt_impl(X3(1:64, 1:64, :), ’cdf97’, 1); X3(1:64, 65:128, :) = dwt_impl(X3(1:64, 65:128, :), ’cdf97’, 1); X3(1:64, 129:192, :) = dwt_impl(X3(1:64, 129:192, :), ’cdf97’, 1); X3(1:64, 193:256, :) = dwt_impl(X3(1:64, 193:256, :), ’cdf97’, 1); X3(65:128, 1:64, :) = dwt_impl(X3(65:128, 1:64, :), ’cdf97’, 1); X3(65:128, 65:128, :) = dwt_impl(X3(65:128, 65:128, :), ’cdf97’, 1); X3(65:128, 129:192, :) = dwt_impl(X3(65:128, 129:192, :), ’cdf97’, 1); X3(65:128, 193:256, :) = dwt_impl(X3(65:128, 193:256, :), ’cdf97’, 1); X3(129:192, 1:64, :) = dwt_impl(X3(129:192, 1:64, :), ’cdf97’, 1); X3(129:192, 65:128, :) = dwt_impl(X3(129:192, 65:128, :), ’cdf97’, 1); X3(193:256, 1:64, :) = dwt_impl(X3(193:256, 1:64, :), ’cdf97’, 1); X3(193:256, 65:128, :) = dwt_impl(X3(193:256, 65:128, :), ’cdf97’, 1); Z = X3; X3 = map_to_01(X3); X3 = X3*255; imshow(uint8(X3))

The following code generates Figure 9.5 in the book.

Z(1:32,1:32,:) = dwt_impl(Z(1:32,1:32,:), ’cdf97’, 1); Y0 = Z; Z = map_to_01(Z); Z = Z*255; Z = uint8(Z); imshow(Z)

The following code generates the parts of Figure 9.23 in the book.

Y0(17:end, :, :) = 0; Y0(1:16, 17:end, :) = 0; Y0 = idwt_impl(Y0, ’cdf97’, 5); Y1 = Y0; Y0 = map_to_01(Y0); Y0 = Y0*255; imshow(uint8(Y0))

Y1 = dwt_impl(img, ’cdf97’, 5); Y1(1:16, 1:16, :) = 0; Y1 = idwt_impl(Y1, ’cdf97’, 5); Y1 = map_to_01(Y1); Y1 = Y1*255; imshow(uint8(Y1))

References

[1] A. Cohen, I. Daubechies, and J-C. Feauveau. Biorthogonal bases of compactly supported wavelets. Communications on Pure and Appl. Math., 45(5):485–560, June 1992.

[2] P. Duhamel and H. Hollmann. ’split-radix’ FFT-algorithm. Electronic letters, 20(1):14–16, 1984.

[3] S. Foucart and H. Rauhut. A mathematical introduction to compressive sensing.

Birkhauser, 2013.

[4] F. J. Harris. On the use of windows for harmonic analysis with the discrete Fourier transform. Proceedings of the IEEE, 66(1), January 1978.

[5] M. Vetterli and H. J. Nussbaumer. Simple FFT and DCT algorithms with reduced number of operations. Signal Processing, 6:267–278, 1984.

[6] R. Yavne. An economical method for calculating the discrete Fourier transform.

Proc. AFIPS Fall Joint Computer Conf., 33:115–125, 1968.

209