SOLUTIONS MANUAL to Introduction to Machine Learning 4th Edition by Ethem Alpaydin

SOLUTION MANUAL

SOLUTION MANUAL

Chapter 1. Introduction Chapter 2. Supervised Learning Chapter 3. Bayesian Decision Theory Chapter 4. Parametric Methods Chapter 5. Multivariate Methods Chapter 6. Dimensionality Reduction Chapter 7. Clustering Chapter 8. Nonparametric Methods Chapter 9. Decision Trees Chapter 10. Linear Discrimination Chapter 11. Multilayer Perceptrons Chapter 12. Deep Learning Chapter 13. Local Models Chapter 14. Kernel Machines Chapter 15. Graphical Models Chapter 16. Hidden Markov Models Chapter 17. Bayesian Estimation Chapter 18. Combining Multiple Learners Chapter 19. Reinforcement Learning Chapter 20. Design and Analysis of Machine Learning Experiments

Introduction to Machine Learning Fourth Edition

Solutions Manual

Ethem Alpaydın The MIT Press

1

Introduction

1. Imagine we have two possibilities: We can scan and email the image, or we can use an optical character reader (OCR) and send the text file.

Discuss the advantage and disadvantages of the two approaches in a comparative manner. When would one be preferable over the other?

The text file typically is shorter than the image file but a faxed document can also contain diagrams, pictures, etc. After using an OCR, we lose properties such as font, size, etc (unless we also recognize and transmit such information) or the personal touch if it is handwritten text. OCR may not be perfect, and for ambiguous cases, OCR should identify those image blocks and transmit them as they are. A fax machine is cheaper and easier to find than a computer with scanner and OCR software.

OCR is good if we have high volume, good quality documents; for documents of few pages with small amount of text, it is better to transmit the image.

2. Let us say we are building an OCR and for each character, we store the bitmap of that character as a template that we match with the read character pixel by pixel. Explain when such a system would fail. Why are barcode readers still used?

Such a system allows only one template per character and cannot distinguish characters from multiple fonts, for example. There are standardized fonts such as OCR-A and OCR-B—the fonts we typically see on the packaging of stuff we buy—which are used with OCR software (the characters in these fonts have been slightly changed to minimize the similarities between them). Barcode readers are still used because reading barcodes is still a better (cheaper, more reliable, more avail-

2

1

Introduction

able) technology than reading characters in arbitrary font, size, and styles.

3. Assume we are given the task of building a system to distinguish junk email. What is in a junk email that lets us know that it is junk? How can the computer detect junk through a syntactic analysis? What would we like the computer to do if it detects a junk email—delete it automatically, move it to a different file, or just highlight it on the screen?

Typically, text-based spam filters check for the existence/absence of words and symbols. Words such as “opportunity,” ”viagra,” ”dollars,” and characters such as ’$’ and ’!’ increase the probability that the email is spam. These probabilities are learned from a training set of example past emails that the user has previously marked as spam. We see many algorithms for this in later chapters.

The spam filters do not work with 100 percent reliability and may make errors in classification. If a junk mail is not filtered, this is not good, but it is not as bad as filtering a good mail as spam. We discuss how we can take into account the relative costs of such false positives and false negatives later on.

Therefore, mail messages that the system considers as spam should not be automatically deleted but kept aside so that the user can see them if he/she wants to, especially in the early stages of using the spam filter when the system has not yet been trained sufficiently.

Spam filtering is probably one of the best application areas of machine learning where learning systems can adapt to changes in the ways spam messages are generated.

4. Let us say we are given the task of building an automated taxi. Define the constraints. What are the inputs? What is the output? How can we communicate with the passenger? Do we need to communicate with the other automated taxis, that is, do we need a “language”?

An automated taxi should be able to pick a passenger and drive him/her to a destination. It should have some positioning system (GPS/GIS) and should have other sensors (cameras) to be able to sense cars, pedestrians, obstacles, etc on the road. The output should be the sequence of actions to reach the destination in the smallest time with minimum inconvenience to the passenger. The automated taxi needs to communicate with the passenger to receive commands and may also need to interact with other automated taxis and possibly with a centralized

3 control to exhange information about road traffic or scheduling, load balancing, etc.

5. In basket analysis, we want to find the dependence between two items X and Y . Given a database of customer transactions, how can we find these dependencies? How would we generalize this to more than two items?

This is discussed in section 3.5 of the book.

6. In a daily newspaper, find five sample news reports for each category of politics, sports, and the arts. Go over these reports and find words that are used frequently for each category, which may help you discriminate between different categories. For example, a news report on politics is likely to include words such as “government,” “recession,” “congress,” and so forth, whereas a news report on the arts may include “album,” “canvas,” or “theater.” There are also words such as “goal” that are ambiguous.

I have checked the web page for the New York Times of Feb 10th, 2010 and found the following words. For politics: republican, party, senate, vote, administration; for sports: medal, athlete, freestyle, ski, snowboard; for the arts: show, celebrity, debut, vocal, resonance. News categorization systems have a preprocessing stage to handle suffixes, such as votes vs vote, or snowboarding vs snowboard. Note that sports is a metaphor used in politics and many ways of life that involve competition, so the use of few keywords is tricky and one needs to take context into account by employing hundreds/thousands of keywords.

In a class of students, it would be interesting to see the overlap of the words students find.

7. If a face image is a 100 × 100 image, written in row-major, this is a 10,000-dimensional vector. If we shift the image one pixel to the right, this will be a very different vector in the 10,000-dimensional space. How can we build face recognizers robust to such distortions?

Face recognition systems typically have a preprocessing stage for normalization where the input is centered and possibly resized before recognition. This is generally done by first finding the eyes and then translating the image accordingly. There are also recognizers that do not use the face image as pixels but rather extract structural features from the image, for example, the ratio of the distance between the two

4

1

Introduction

eyes to the size of the whole face. Such features would be invariant to translations and size changes.

8. Take, for example, the word “machine.” Write it ten times. Also ask a friend to write it ten times. Analyzing these twenty images, try to find features, types of strokes, curvatures, loops, how you make the dots, and so on, that discriminate your handwriting from that of your friend’s.

I leave this to the reader. One personal comment: Over the years, I have noticed that though many western nations use some version of the roman alphabet, there are noticable distinctions between their writing styles. I notice for example that french and italian handwritings are more cursive, connected, and flowery, whereas german, british and american handwritten characters are stroke-based, formed of separate characters. This may just be my impression and in a class formed of students from different nationalities, this may be an interesting thing to check.

9. In estimating the price of a used car, it makes more sense to estimate the percent depreciation over the original price than to estimate the absolute price. Why? Depreciation is generally measured as some percentage of the original price. Cars having different prices may suffer the same loss in the same amount of time or mileage; for example in Turkey, the general belief is that a car, on the average, loses 10-20 per cent of its value in its first year.

10. Consider your purchases from your local supermarket. What are the typical associations between the products you buy? How do they depend on the season? Try to devise general rules that explain your purchasing behavior.

I buy bread regularly irrespective of the season. When I buy beer, I also buy crips. I buy ice cream in summer and I buy more tea in winter than in summer.

11. List all the companies or institutions that have some data about you and what they know about you. Check their policies for data security and privacy.

My employer has my personal information. I’m very careful submitting my personal information over the web, and never share any with any

5 site that I do not know. Whenever I share confidential information, I make sure that the communication is secure.

12. What do you think is the difference between a “data engineer” and a “data scientist”‘?

The former is closer to implementation and the latter is more focused on theory. Neither one is superior to the other; we need both.

2

Supervised Learning

1. Let us say our hypothesis class is a circle instead of a rectangle. What are the parameters? How can the parameters of a circle hypothesis be calculated in such a case? What if it is an ellipse? Why does it make more sense to use an ellipse instead of a circle?

In the case of a circle, the parameters are the center and the radius (see figure 2.1). We then need to find S and G where S is the tightest circle that includes all the positive examples and G is the largest circle that includes all the positive examples and no negative example; any circle between them is a consistent hypothesis.

It makes more sense to use an ellipse because the two axes need not have the same scale and an ellipse has two separate parameters for the widths in the two axes rather than a single radius. Actually, price and engine power are positively correlated; the price of a car tends to increase as its engine power increases, and hence it makes more sense to use an oblique ellipse—we will see such models in chapter 5.

2. Imagine our hypothesis is not one rectangle but a union of two (or m > 1) rectangles. What is the advantage of such a hypothesis class? Show that any class can be represented by such a hypothesis class with large enough m.

In the case when there is a single rectangle, all the positive instances should form one single group; with two rectangles, for example (see figure 2.2), the positive instances can form two, possibly disjoint clusters in the input space. Note that each rectangle corresponds to a conjunction on the two input attributes, and having multiple rectangles corresponds to a disjunction. Any logical formula can be written

8

Supervised Learning

x2: Engine power

2

C cc

c2

r

c1

x1: Price

Figure 2.1 Hypothesis class is a circle with two parameters, the coordinates of its center and its radius.

x2: Engine power

as a disjunction of conjunctions. In the worst case (m = N), we have a separate rectangle for each positive instance.

h1

C

h2

x1: Price

Figure 2.2

Hypothesis class is a union of two rectangles.

3. In many applications, wrong decisions—namely, false positives and false

9 negatives—have a monetary cost, and these two costs may be different.

What is the relationship between the positioning of h between S and G and the relative costs of these?

We can see that S makes no false positives, but only false negatives; similarly, G makes no false negatives, only false positives. So if false positives and false negatives are equally bad, we want our h to be halfway; if false positives are costlier, we want h to be closer to S; if false negatives are costlier, h should be closer to G. 4. The complexity of most learning algorithms is a function of the training set. Can you propose a filtering algorithm that finds redundant instances?

The instances that affect the hypothesis are those that are in the vicinity of instances with a different label. A positive instance that is surrounded on all sides by many positive instances is not needed, nor is a negative instance surrounded by many negative instances. We discuss such neighbor-based methods in chapter 8.

5. If we have a supervisor who can provide us with the label for any x, where should we choose x to learn with fewer queries?

The region of ambiguity is between S and G. It would be best to be given queries there so that we can make this region of doubt smaller.

If a given instance there turns out to be positive, this means we can make S larger up to that instance; if it is negative, this means we can shrink G up until there.

6. In equation 2.13, we summed up the squares of the differences between the actual values and the estimated values. This error function is the one most frequently used, but it is one of several possible error functions. Because it sums up the squares of the differences, it is not robust to outliers. What would be a better error function to implement robust regression?

As we see in chapter 4, the squared error corresponds to assuming that there is Gaussian noise. If the noise comes from a distribution with long tails, then summing up squared differences cause a few, far away points, i.e., outliers, to corrupt the fitted line.

10

2

Supervised Learning

To decrease the effect of outliers, we can sum up the absolute value of differences instead of squaring them: N

E(g|X) =

1 X t |r − g(xt )| N t=1

but note that we lose from differentiability. Support vector regression which we discuss in chapter 14 uses an error function (see figure 14.6) that uses the absolute difference and also has a term that neglects error due to very small differences.

7. Derive equation 2.17.

We take the derivative of the sum of squared errors with respect to the two parameters, set them equal to 0, and solve these two equations in two unknowns: N

E(w1 , w0 |X) ∂E ∂w0 X rt t

w0 ∂E ∂w1 X

t t

r x

t

X

r x

X

r t xt

t t

t

t

X

r t xt

t

w1

= =

2 1 X t r − (w1 xt + w0 ) N i=1 X

r t − (w1 xt + w0 ) = 0 t

X

xt + Nw0

=

w1

=

X

=

X

=

w1

=

X X w1 (xt )2 + (r − w1 x) xt

= = =

t

t

t

r t /N − w1

X t

xt /N = r − w1 x

r t − (w1 xt + w0 ) xt = 0

X X xt (xt )2 + w0 t

t

t

t

 X X X xt w1  (xt )2 − x xt  + r 



w1  P

t

t

t



X (xt )2 − xNx + r Nx t

t t

r x − xr N Pt t )2 − Nx2 (x t

11 8. How can we estimate w2 , w1 , w0 for the quadratic model of equation 2.18 ?

We derive it just like we did in the previous exercise, but this time we will have three equations in three unknowns: N

E(w1 , w0 |X) =

i2 1 Xh t r − (w2 (xt )2 + w1 xt + w0 ) N i=1

Take derivatives wrt w2 , w1 , w0 , set them equal to 0, and solve for w2 , w1 , w0 .

9. Assume our hypothesis class is the set of lines, and we use a line to separate the positive and negative examples, instead of bounding the positive examples as in a rectangle, leaving the negatives outside (see figure 2.3). Show that the VC dimension of a line is 3.

x2

(2.1)

x1 Figure 2.3

A line separating positive and negative instances.

As we see in figure 2.4, for all possible labeling of three points, there exist a line to separate positive and negative examples. With four points, no matter how we place these four points in two dimensions, there is at least one labeling where we cannot draw a line such that on one side lie all the positives and on the other lie all the negatives.

10. Show that the VC dimension of the triangle hypothesis class is 7 in two dimensions. (Hint: For best separation, it is best to place the seven points equidistant on a circle.)

12

Supervised Learning

x2

x2

2

x1 All possible labelings of three points can be separated using a line.

x1 These four points cannot be separated using a line.

Figure 2.4 A line separating positive and negative instances.

x2

x2

As we can see in figure 2.5, for all possible labeling of seven points, we can draw a triangle to separate the positive and negative examples.

We cannot do the same when there are eight points.

x1 These seven points can be separated using a triangle no matter how they are labeled.

x1 These eight points with this labeling cannot be separated using a triangle.

Figure 2.5 A triangle can shatter seven points but not eight.

11. Assume as in exercise 8 that our hypothesis class is the set of lines.

Write down an error function that not only minimizes the number of misclassifications but also maximizes the margin.

We will need to write an error function that sums margins, that is, distances to the boundary, and not only 0/1 misclassifications, which only checks whether we are on the correct side of the boundary; but we do this only if an instance is on the wrong side of the boundary.

13 This is the hinge loss used in kernel machines and we will discuss it in chapter 13 (see figure 13.3).

12. One source of noise is error in the labels. Can you propose a method to find data points that are highly likely to be mislabeled?

If we have an instance that is surrounded by many instances with a different label, then it is most probably mislabeled. By using a neighborbased method, we can check for this; we discuss such a method for outlier detection in section 8.7.

13. In mutlilabel classification, labels are not mutually exclusive and exhaustive, and an instance may have zero or more labels. For example, an image may have the three labels “beach,” “vacation,” and “sea.”

Adapt the setting we discussed in section 2.5 to multiple labels.

When we have labels, the hypotheses we learn for each is independent of all others; that is, when we have L labels, we treat them as L independent two-class problems, seperating instances with label 0 from instances with label 1.

14. The rectangle we used in this chapter as our hypothesis class is axisaligned (figure 2.2). How can we define the hypothesis class of rectangles of arbitrary orientation? What are the advantages and disadvantages of this more general class?

We can define a rectangle with arbitrary orientation by using a rotation matrix. This is useful when attributes are correlated; for example, cars with more engine power cost more so we expect the data to exhibit a distribution with positive correlation; we will see examples of this in chapter 5.

3

Bayesian Decision Theory

1. Assume a disease so rare that it is seen in only one person out of every million. Assume also that we have a test that is effective in that if a person has the disease, there is a 99 percent chance that the test result will be positive; however, the test is not perfect, and there is a one in a thousand chance that the test result will be positive on a healthy person.

Assume that a new patient arrives and the test result is positive. What is the probability that the patient has the disease ?

Let us represent disease by d and test result by t. We are given the following: P (d = 1) = 10−6 , P (t = 1|d = 1) = 0.99, P (t = 1|d = 0) = 10−3 . We are asked P (d = 1|t = 1).

We use Bayes’ rule: P (d = 1|t = 1)

= = =

P (t = 1|d = 1)P (d = 1) P (t = 1) P (t = 1|d = 1)P (d = 1) P (t = 1|d = 1)P (d = 1) + P (t = 1|d = 0)P (d = 0) 0.99 · 10−6 = 0.00098902 0.99 · 10−6 + 10−3 · (1 − 10−6 )

That is, knowing that the test result is positive increased the probability of disease from one in a million to one in a thousand.

2. In a two-class problem, the likelihood ratio is p(x|C1 ) p(x|C2 ) Write the discriminant function in terms of the likelihood ratio.

16

3

Bayesian Decision Theory

We can define a discriminant function as P (C1 |x) and choose g(x) = P (C2 |x)

(

C1 C2

if g(x) > 1 otherwise

We can write the discriminant as the product of the likelihood ratio and the ratio of priors: g(x) =

p(x|C1 ) P (C1 ) p(x|C2 ) P (C2 )

If the priors are equal, the discriminant is the likelihood ratio.

3. In a two-class problem, the log odds is defined as log

P (C1 |x) P (C2 |x)

Write the discriminant function in terms of the log odds.

We define a discriminant function as P (C1 |x) g(x) = log and choose P (C2 |x)

(

C1 C2

if g(x) > 0 otherwise

Log odds is the sum of log likelihood ratio and log of prior ratio: g(x) = log

P (C1 ) p(x|C1 ) + log p(x|C2 ) P (C2 )

If the priors are equal, the discriminant is the log likelihood ratio.

4. In a two-class, two-action problem, if the loss function is λ11 = λ22 = 0, λ12 = 10, and λ21 = 5, write the optimal decision rule. How does the rule change if we add a third action of reject with λ = 1?

The loss table is as follows:

Action α1 : Choose C1 α2 : Choose C2

Truth C1 C2 0 10 5 0

17 Let us calculate the expected risks of the two actions: R(α1 |x)

R(α2 |x)

=

=

0 · P (C1 |x) + 10 · P (C2 |x) = 10 · (1 − P (C1 |x)) 5 · P (C1 |x) + 0 · P (C2 |x) = 5 · P (C1 |x)

We choose α1 if R(α1 |x)

<

P (C1 |x)

>

10 · (1 − P (C1 |x))

<

R(α2 |x)

5 · P (C1 |x)

2/3

If the two misclassifications were equally costly, the decision threshold would be at 1/2 but because the cost of wrongly choosing C1 is higher, we want to choose C1 only when we are really certain; see figure 3.1a and b.

If we add a reject option with a cost of 1, the loss table now becomes

Action α1 : Choose C1 α2 : Choose C2 αr : Reject

Truth C1 C2 0 10 5 0 1 1

Let us calculate the expected risks of the three actions: R(α1 |x)

=

0 · P (C1 |x) + 10 · P (C2 |x) = 10 · (1 − P (C1 |x))

R(αr |x)

=

1

R(α2 |x)

=

5 · P (C1 |x) + 0 · P (C2 |x) = 5 · P (C1 |x)

We choose α1 if R(α1 |x) < 1 ⇒ P (C1 |x) > 9/10 We choose α2 if R(α2 |x) < 1 ⇒ P (C1 |x) < 1/5, or equivalently if P (C1 |x) > 4/5 We reject otherwise, that is, if 1/5 < P (C1 |x) < 9/10; see figure 3.1c.

18

3

Bayesian Decision Theory

(a)

p(Ci|x)

1 C1

C2

0.5 0 −3

−2

−1

0 x (b)

1

2

3

p(Ci|x)

1 C1

C2

0.5 0 −3

−2

−1

0 x (c)

1

2

3

1 p(Ci|x)

C

C

1

2

0.5 0 −3

−2

−1

0 x

1

2

3

Figure 3.1 The boundary changes as the misclassification losses change. (a) The boundary is where the two posteriors are equal when both misclassifications are equally costly. (b) When the losses are not symmetric, the boundary shifts toward the class that incurs higher risk when misclassified. (c) When there is the option of reject, a region around the boundary is the region of reject.

5. Propose a three-level cascade where when one level rejects, the next one is used as in equation 3.10. How can we fix the λ on different levels?

Let us say we have three classifiers Mi , i = 1, . . . , 3, with increasing cost. We want to be able to classify with M1 if we can, so that we will not need to resort to costlier M2 or M3 . For M1 , we define λ1 so that we make a decision if with M1 , the highest posterior is greater than 1 − λ1 , otherwise we use M2 . If the highest posterior with M2 is also less than 1 − λ2 , we use M3 . It then makes sense to require λ2 < λ1 , as

19 we proceed we require higher confidence to justify the extra cost.

We discuss such a cascading method in more detail in section 17.11.

6. Somebody tosses a fair coin and if the result is heads, you get nothing; otherwise, you get $5. How much would you pay to play this game?

What if the win is $500 instead of $5?

With five dollars, the expected earning is (1/2) · 0 + (1/2) · 5 = 2.5 dollars and one can bet up that amount to play the game. With a reward of 500 dollars, the expected earning is 250 and one can bet up that amount. However research has shown that people are risk aversive, in the sense that though they may risk small amounts, they do not like to risk larger amounts, even though odds may be more favorable. See for example, A. Tversky, D. Kahneman. 1981. “The framing of decisions and the psychology of choice,” Science 211: 453– 458.

7. Given the following data of transactions at a shop, calculate the support and confidence values of milk → bananas, bananas → milk, milk → chocolate, and chocolate → milk.

Transaction 1 2 3 4 5 6

Items in basket milk, bananas, chocolate milk, chocolate milk, bananas chocolate chocolate milk, chocolate

The association rules and their support and confidence values are as follows: milk → bananas

:

bananas → milk

:

chocolate → milk

:

milk → chocolate

:

Support = 2/6, Confidence = 2/4

Support = 2/6, Confidence = 2/2

Support = 3/6, Confidence = 3/4 Support = 3/6, Confidence = 3/5

Though only half of the people who buy milk buy bananas too, anyone who buys bananas also buys milk.

20

3

Bayesian Decision Theory

8. Generalize the confidence and support formulas for basket analysis to calculate k-dependencies, namely, P (Y |X1 , . . . , Xk ).

We are interested in rules of the form X1 , X2 , . . . , Xk → Y :

Support: P (X1 , X2 , . . . , Xk , Y ) =

#{customers who bought X1 and . . . Xk and Y } #{customers}

Confidence: P (Y |X1 , X2 , . . . , Xk )

= =

P (X1 , X2 , . . . , Xk , Y ) P (X1 , X2 , . . . , Xk ) #{customers who bought X1 and . . . Xk and Y } #{customers who bought X1 and . . . Xk }

Lift: Lift(X1 , X2 , . . . , Xk → Y )

= =

P (X1 , X2 , . . . , Xk , Y ) P (X1 , X2 , . . . , Xk )P (Y ) P (Y |X1 , X2 , . . . , Xk ) P (Y )

Note that people who bought X1 , X2 , X3 over a certain number should have bought X1 , X2 and X1 , X3 and X2 , X3 over the same amount. So one can expand k-dependencies from (k − 1)-dependencies. This is the basic idea behind the Apriori algorithm.

9. Show that as we move an item from the consequent to the antecedent, confidence can never increase: confidence(ABC → D) ≥ confidence(AB → CD). confidence(ABC → D)

=

confidence(AB → CD)

=

P (A, B, C, D) P (D) P (A, B, C, D) P (C, D)

Since P (C, D) = P (C|D)P (D) ≤ P (D) because P (C|D) ≤ 1, we have confidence(ABC → D) ≥ confidence(AB → CD).

10. Associated with each item sold in basket analysis, if we also have a number indicating how much the customer enjoyed the product, for example, on a scale of 0 to 10, how can you use this extra information to calculate which item to propose to a customer?

21 One can come up with a weighted confidence measure, or considering that these are numeric values, one can calculate correlations and propose a new product that has the highest positive correlation with the current one, if the correlation is high enough. In later chapters of the book, we discuss many methods that can be used in such recommendation systems.

11. Show example transaction data where for the rule X → Y : (a) Both support and confidence are high. (b) Support is high and confidence is low. (c) Support is low and confidence is high. (d) Both support and confidence are low. (a) Both P (X, Y ) and P (X) are high. People who buy cornflakes buy milk. (b) P (X, Y ) is high and P (X) > P (X, Y ). People who buy matches buy cigarettes. There are a large number of people who buy matches for different reasons even though they may not be smokers. (c) P (X, Y ) is low and P (X) is also low, so that confidence is high.

People who buy Rolls Royce buy Rolex. (d) P (X, Y ) is low and P (X) is high, so that confidence is also low.

People who buy leather jackets buy Porsche.

4

Parametric Methods

1. Write the code that generates a Bernoulli sample with given parameter p, and the code that calculates p̂ from the sample.

The Matlab code is given in ex4_1.m.

2. Let us say that we toss a fair coin ten times. What is the probability that we see four to six heads? What is the probability that we see forty to sixty heads when we toss it one hundred times?

Let p0 be the probability of heads. The probability that we see x heads in n tosses is given by the binomial distribution

P (x|n, p0 ) =

! n x p (1 − p0 )n−x x 0

The probability that we see four to six heads when we toss a fair coin ten times is

P (4 ≤ x ≤ 6 10, 1/2) =

6 X

x=4

!

10 1/2x (1 − 1/2)n−x x

The probability that we see forty to sixty heads when we toss a fair coin one hundred times is

P (40 ≤ x ≤ 60 100, 1/2) = which is a larger value.

60 X

x=40

!

100 1/2x (1 − 1/2)n−x x

24

4

Parametric Methods

3. Write the log likelihood for a multinomial sample and show equation 4.6.

We add the constraint as a Lagrange term and maximize it: =

XX i

t

∂J ∂pi

=

P

t t xi

λ

=

pi λ

=

i

t

J(pi )

X i

pi

=

X

xti log pi + λ(1 −

pi )

i

−λ=0 pi P t X t xi xti ⇒ pi λ = pi t XX XX t xti xi ⇒ λ = t

i

P t P t X t xi t xi P P = xti = 1 because t N x t i i i

4. Write the code that generates a normal sample with given µ and σ , and the code that calculates m and s from the sample. Do the same using the Bayes’ estimator assuming a prior distribution for µ.

The Matlab code is given in ex4_3.m and we see an example output in figure 4.1.

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0

0

1

2

3

4

5

6

Figure 4.1 Actual Gaussian density (continuous line), a sample drawn from it (×), and a Gaussian fitted to the sample (dotted line).

25 5. If xt ∼ N (µ, σ 2 ), show that m ∼ N (µ, σ 2 /N).

We know that the linear combination of independent normally distributed random variables is also normally distributed, and we saw in section 4.3 that E[m] = µ and Var(m) = σ 2 /N. 6. Given two normal distributions p(x|C1 ) ∼ N (µ1 , σ12 ) and p(x|C2 ) ∼ N (µ2 , σ22 ) and P (C1 ) and P (C2 ), calculate the Bayes’ discriminant points analytically.

Given that p(x|C1 )

∼

N (µ1 , σ12 ) = √

p(x|C2 )

∼

N (µ2 , σ22 )

" # (x − µ1 )2 1 exp − 2π σ1 2σ12

we would like to find x that satisfy P (C1 |x) = P (C2 |x), or p(x|C1 )P (C1 ) log p(x|C1 ) + log P (C1 ) 2

(x − µ1 ) 1 + log P (C1 ) log 2π − log σ1 − 2 2σ12

1 2 2 − log σ1 − 2 x − 2xµ1 + µ1 + log P (C1 ) 2σ1 ! !

1 µ1 1 µ2 2 x + − − 2 x+ 2σ22 2σ12 σ12 σ2 !

2 2 µ2 µ σ2 P (C1 ) − 12 + log + log =0 σ1 P (C2 ) 2σ22 2σ1 −

=

p(x|C2 )P (C2 )

=

log p(x|C2 ) + log P (C2 )

=

···

=

···

This is of the form ax2 + bx + c = 0 and the two roots are x1 , x2 =

−b ±

√ b2 − 4ac 2a

Note that if the variances are equal, the quadratic terms vanishes and there is one root, that is, the two posteriors intersect at a single x value.

26

4

Parametric Methods

7. What is the likelihood ratio p(x|C1 ) p(x|C2 ) in the case of Gaussian densities? √ 1 exp 2π σ1

p(x|C1 ) = p(x|C2 ) √ 1

2π σ2

exp

(x−µ )2 − 2σ 21 1 (x−µ )2 − 2σ 22 2

If we have σ12 = σ22 = σ 2 , we can simplify p(x|C1 ) p(x|C2 )

=

# (x − µ1 )2 (x − µ2 )2 exp − + 2σ 2 2σ 2 " # (µ22 − µ12 ) (µ1 − µ2 ) exp x + σ2 2σ 2

=

exp(w x + w0 )

=

"

for w = (µ1 − µ2 )/σ 2 and w0 = (µ22 − µ12 )/2σ 2 .

8. For a two-class problem, generate normal samples for two classes with different variances, then use parametric classification to estimate the discriminant points. Compare these with the theoretical values.

The Matlab code is given in ex4_6.m, and its output plot is in figure 4.2.

Output: Real values C1:(3.0,1.0)- C2:(2.0,0.3) Estimates C1:(3.45,1.23)- C2:(2.15,0.26) Actual intersect pts: 2.49 1.31 Estimated intersect pts: 2.64 1.53

9. Let us say we have a two-class problem where both classes are normally distributed with the same mean but different variances. What will the discriminant function look like in this case?

Let us say we have two classes C1 and C2 with the same mean but C1 having smaller variance. In this case, for inputs that are close to the mean, P (C1 |x) will be larger (disregarding the effect of the prior)

27

1.6

1.4

1.2

1

0.8

0.6

0.4

0.2

0

0

1

2

3

4

5

6

Figure 4.2 Actual densities, posteriors, drawn samples and the fitted densities and posteriors for a two-class problem. The continuous lines are the actual densities and posteriors and the dotted lines are the estimations using the data points shown by ‘×’ and ‘◦’.

and when the input is sufficiently away, P (C2 |x) starts to get larger; the threshold of the decision will depend on the two variances (and the priors). So in one dimension, we will have an interval symmetric around the mean for C1 on both sides surrounded by unbounded intervals for C2 .

10. Assume a linear model and then add 0-mean Gaussian noise to generate a sample. Divide your sample into two as training and validation sets.

Use linear regression using the training half. Compute error on the validation set. Do the same for polynomials of degrees 2 and 3 as well.

The Matlab code given in ex4_7.m samples data from 2 sin(1.5x) + N (0, 1), and fits five times polynomials of order 1, 2 and 3. It also calculates their bias and variance. This is the code used to generate figure 4.5 of the book (with orders 1, 3 and 5). The plots for orders 1, 2, and 3 are given in figure 4.3 of this document: Output:

28

4

Parametric Methods

Order 1 B=1.80 V=0.29 E=2.09 Order 2 B=1.25 V=0.38 E=1.63 Order 3 B=0.28 V=0.30 E=0.58

(a) Function and data

(b) Order 1

5

5

0

0

−5

0

1

2

3

4

5

−5

0

1

(c) Order 2 5

0

0

0

1

2

3

3

4

5

4

5

(d) Order 3

5

−5

2

4

5

−5

0

1

2

3

Figure 4.3 Function, one random noisy sample and five fitted polynomials of order 1, 2 and 3.

11. When the training set is small, the contribution of variance to error may be more than that of bias and in such a case, we may prefer a simple model even though we know that it is too simple for the task. Can you give an example?

With small datasets and complex models, the fit will have high variance because there is not enough data to sufficiently constrain the model and the fit will follow noise. In such a case, it is better to use a simple model though it risks having high bias.

If we run the program of exercise 4.7 here with six data points (ex4_8.m has N = 6 on line 2), we get the result in figure 4.4 and the following output: Order 1 B=1.66 V=0.37 E=2.03 Order 2 B=1.31 V=1.59 E=2.90

29

Order 3 B=3.74 V=2.86 E=6.60 (a) Function and data

(b) Order 1

5

5

0

0

−5

0

1

2

3

4

5

−5

0

1

(c) Order 2 5

0

0

0

1

2

3

3

4

5

4

5

(d) Order 3

5

−5

2

4

5

−5

0

1

2

3

Figure 4.4 Function, one random noisy sample and five fitted polynomials of order 1, 2 and 3.

We see that a linear fit has less error though a third-degree polynomial is ideally the best.

12. Let us say, given the samples Xi = {xti , rit }, we define gi (x) = ri1 , namely, our estimate for any x is the r value of the first instance in the (unordered) dataset Xi . What can you say about its bias and variP ance, as compared with gi (x) = 2 and gi (x) = t rit /N? What if the sample is ordered, so that gi (x) = mint rit ?

Taking any instance has less bias than taking a constant but has higher variance. It has higher variance than the average and it may have higher bias. If the sample is ordered so that the instance we pick is the minimum, variance decreases (different minima tend to get more similar to each other) and bias may also increase. The Matlab code is given in ex4_9.m and running it, we get the plot of figure 4.5 with the following output: Order 0 B=1.87 V=0.10 E=1.97

30

4

Parametric Methods

First inst B=2.63 V=2.30 E=4.93 Min inst B=14.42 V=0.53 E=14.95

(a) Function and data

(b) Order 0

5

5

0

0

−5

0

1

2

3

4

5

−5

0

1

2

(c) First instance 5

0

0

0

1

2

3

4

5

3

4

5

(d) Min

5

−5

3

4

5

−5

0

1

2

Figure 4.5 Function, one random noisy sample and the fitted models are: P t 1 t t r /N, r , mint r .

13. In equation 4.40, what is the effect of changing λ on bias and variance? λ controls smoothness: If it is large, we may smooth too much and decrease variance at the expense of an increase in bias; if it is small, bias may be small but variance will be high.

5

Multivariate Methods

1. Show equation 5.11.

Given that " σ12 Σ= ρσ1 σ2

ρσ1 σ2 σ22

#

we have |Σ|

=

|Σ|1/2

=

Σ−1

=

σ12 σ22 − ρ 2 σ12 σ2 = σ12 σ2 (1 − ρ 2 ) q σ1 σ2 1 − ρ 2 # " σ22 −ρσ1 σ2 1 −ρσ1 σ2 σ12 σ12 σ2 (1 − ρ 2 )

and (x − µ)T Σ−1 (x − µ) can be expanded as 

σ22 σ12 σ2 (1−ρ 2 ) ρσ1 σ2 − σ 2 σ (1−ρ 2) 1 2

 " # ρσ1 σ2 − x1 − µ1  σ12 σ2 (1−ρ 2 )  [x1 − µ1 x2 − µ2 ]   σ12 x2 − µ2 σ12 σ2 (1−ρ 2 ) "

#

1 x2 − µ2 x2 − µ2 2 x1 − µ1 2 x1 − µ1 = + − 2ρ 1 − ρ2 σ1 σ1 σ2 σ2 2. Generate a sample from a multivariate normal density N (µ, Σ), calculate m and S, and compare them with µ and Σ. Check how your estimates change as the sample size changes.

The Matlab code is given in ex5_2.m. The result is given in figure 5.1 for different sample sizes.

32

5

Data and fitted Gaussian

4

4

3.5

3.5

3

3

2.5

2.5

2

y

y

Population

1.5 1

1 0.5 0

1

2 x

3

4

4

3.5

3.5

3

3

2.5

2.5

2

y

y

0

4

1.5

2 x

3

4

Data and fitted Gaussian

2

1

1 0.5 0

1

2 x

3

0

4

0

(b) N = 50

Population 4

4

3.5

3.5

3

3

2.5

2.5 y

2 1.5

1

2 x

3

4

Data and fitted Gaussian

2 1.5

1

1

0.5

0.5

0

1

1.5

0.5 0

0

(a) N = 10

Population

y

2 1.5

0.5 0

Multivariate Methods

0

1

2 x

3

4

0

0

1

2 x

3

4

(c) N = 500 Figure 5.1 sizes.

Gaussian population and the fitted Gaussian for different sample

33 3. Give an example where mean imputation will not work well.

Let us say we have two attributes, gender and name. There may be more males than females in the data, but the most frequent name may be “Jane.”

4. Generate samples from two multivariate normal densities N (µi , Σi ), i = 1, 2, and calculate the Bayes’ optimal discriminant for the four cases in table 5.1.

The Matlab code is given in ex5_3.m and its output is given in figure 5.7 of the book.

5. For a two-class problem, for the four cases of Gaussian densities in table 5.1, derive log

P (C1 |x) P (C2 |x)

Using Bayes’ rule, we have log

P (C1 |x) p(x|C1 ) P (C1 ) = log + log P (C2 |x) p(x|C2 ) P (C2 )

Given that p(xCi ) =

1 T −1 exp − (x − µ ) Σ (x − µ ) i i i (2π )d/2 |Σi |1/2 2 1

we have the following four cases: (a) Σ1 6= Σ2 : log

|Σ1 |−1/2 exp[−(1/2)(x − µ1 )T Σ−1 p(x|C1 ) 1 (x − µ1 )] = log −1 −1/2 T p(x|C2 ) |Σ2 | exp[−(1/2)(x − µ2 ) Σ2 (x − µ2 )]

(b) Σ1 = Σ2 = Σ: log

p(x|C1 ) p(x|C2 )

= =

1 1 T −1 − (x − µ1 )T Σ−1 1 (x − µ1 ) + (x − µ2 ) Σ2 (x − µ2 ) 2 2 1 x T Σ−1 (µ1 − µ2 ) + (µ2 − µ1 )T Σ−1 (µ2 + µ1 ) 2

We can write log

P (C1 |x) = w T x + w0 P (C2 |x)

34

5

Multivariate Methods

where w

=

w0

=

Σ−1 1 (µ1 − µ2 ) 1 P (C1 ) − (µ1 + µ2 )T Σ−1 (µ1 − µ2 ) + log 2 P (C2 )

(c) Σ diagonal with σj2 , j = 1, . . . , d: d X P (C1 |x) = log P (C2 |x) j=1

µ1j − µ2j σj

!2

d

1 X xj − 2 j=1

2 2 µ1j − µ2j

σj2

!

+ log

P (C1 ) P (C2 )

(d) Σ diagonal with σ 2 : log

d d 1 X 1 X 2 P (C1 ) P (C1 |x) 2 = 2 (µ1j − µ2j )2 xj − (µ − µ2j ) + log P (C2 |x) σ j=1 2σ 2 j=1 1j P (C2 )

6. Another possibility using Gaussian densities is to have them all diagonal but allow them to be different. Derive the discriminant for this case.

When the covariance matrices are diagonal, we get axis-aligned distributions. Again, we get a linear discriminant if the two matrices are equal, and otherwise, we get quadratic discriminants. Examples are given in figure 5.2, generated by ex5_5.m.

7. Let us say in two dimensions, we have two classes with exactly the same mean. What type of boundaries can be defined?

Examples are given in figure 5.3, generated by ex5_6.m.

8. Sometimes the data may contain outliers due to noise. How can we find them?

In Section 8.7, we discuss a method to find outliers.

9. In certain applications we can define hierarchies of classes, and this can make discrimination easier. For example, first we discriminate cats from dogs, and then we discriminate between different breeds of cat.

Discuss how this can be done. Can we learn hierarchies from data?

Given such hierarchical definitions, we can learn a hierarchy of classifiers. For example, first we train a two-class classifier where all breeds of cats together make up the positive instances and all breeds

35

(b) 4

3

3

2

2

y

y

(a) 4

1

0

1

0

1

2 x

3

0

4

0

1

4

3

3

2

2

1

0

3

4

3

4

(d)

4

y

y

(c)

2 x

1

0

1

2 x

3

4

0

0

1

2 x

Figure 5.2 Distributions and the discriminant for different cases of diagonal matrices. (a) The two matrices are equal and the discriminant is linear; in other cases, the matrices are different and the discriminant is quadratic.

of dogs together make up the negative instances and then on the negative branch, we have a three-class classifier to distinguish between the three cat breeds and on the positive branch, we can have a four-class classifier to choose between the four dog breeds.

In certain examples, for example as between cats and dogs, the hierarchy may be evident. When it is not, we can learn such hierarchies from data where we try to group classes to form superclasses in different resolutions. One can think of possible ways to do this. In the simplest case, we can just look for distances between the class means, or we can do it nonparametrically taking all instances into account; for example, the hierarchical clustering methods that we will discuss in section 7.8 can help with this.

10. Let us say we have two variables x1 and x2 and we want to make a quadratic fit using them, namely f (x1 , x2 ) = w0 + w1 x1 + w2 x2 + w3 x1 x2 + w4 (x1 )2 + w5 (x2 )2

36

5

(b)

4

4

3

3

2

2

y

y

(a)

1

0

1

0

1

2 x

3

0

4

0

1

2 x

3

4

3

4

(d) 4

3

3

2

2

y

y

(c) 4

1

0

Multivariate Methods

1

0

1

2 x

3

4

0

0

1

2 x

Figure 5.3 The means are identical but the matrices are different: Distributions contours are shown by one line for visibility. In (a), when one matrix is just a scaled version of the other, we get a circular discriminant, otherwise we get hyperboles.

How can we find wi , i = 0, . . . , 5, given a sample of X = {xt1 , xt2 , r t }?

We write the fit as f (x1 , x2 ) = w0 + w1 z1 + w2 z2 + w3 z3 + w4 z4 + w5 z5 where z1 = x1 , z2 = x2 , z3 = x1 x2 , z4 = (x1 )2 , and z5 = (x2 )2 . We can then use linear regression to learn wi , i = 0, . . . , 5. The linear fit in the five-dimensional (z1 , z2 , z3 , z4 , z5 ) corresponds to a quadratic fit in the two-dimensional (x1 , x2 ) space. We discuss such generalized linear models in more detail (and other nonlinear basis functions) in chapter 10.

11. In regression we saw that fitting a quadratic is equivalent to fitting a linear model with an extra input corresponding to the square of the input. Can we also do this in classification?

37 Yes. We can define new, auxiliary variables corresponding to powers and cross-product terms and then use a linear model. For example, just as in exercise 7, we can define z1 = x1 , z2 = x2 , z3 = x1 x2 , z4 = (x1 )2 , and z5 = (x2 )2 and then use a linear model to learn wi , i = 0, . . . , 5. The linear discriminant in the five-dimensional (z1 , z2 , z3 , z4 , z5 ) corresponds to a quadratic discriminant in the twodimensional (x1 , x2 ) space.

12. In document clustering, ambiguity of words can be decreased by taking the context into account, for example, by considering pairs of words, as in “cocktail party” vs. “party elections.” Discuss how this can be implemented.

We define extra variables that correspond to such particular combinations, for example, set a new variable to 1 if there are the two consecutive words “cocktail party” somewhere in the document.

6

Dimensionality Reduction

1. Assuming that the classes are normally distributed, in subset selection, when one variable is added or removed, how can the new discriminant be calculated quickly? For example, how can the new S−1 new be calculated from S−1 ? old When we add a new variable, we are adding a row and column to the covariance matrix. It can be shown that if we partition a symmetric nonsingular matrix A into (Rencher, 1995; p. 28)

A=

"

A11 a T12

a 12 a22

#

the inverse is given by −1

A

1 = b

"

−1 T −1 bA−1 11 + A11 a 12 a 12 A11 T −1 −a 12 A11

−A−1 11 a 12 1

#

−1 where b = a22 − a T12 A−1 11 a 12 . So given A11 and A11 , we can easily calculate A−1 when a 12 and a22 (covariances of the new variable with the existing variables, and its variance) are added. Another possibility is to break down the Mahalanobis distance calculation (if we have the previous values stored). Let A be partitioned as (T. Cormen, C. Leiserson, R. Rivest, C. Stein. 2001. Introduction to Algorithms, The MIT Press. 2nd edition, p. 761)

A=

"

Ak B

BT C

#

40

6

Dimensionality Reduction

where Ak is the leading k × k submatrix of A and x similarly broken into two parts, we can write T

x Ax

T

T

"

Ak B

BT C

#"

y z

#

=

[y z ]

=

−1 T T T (y + Ak−1 BT z)T Ak (y + A−1 k B z) + z (C − BAk B )z

In our case of adding one feature, y will be d-dimensional and z will be a scalar. If we already have the d-dimensional y T Ak y values calculated and stored, we can use the equality above to calculate the (d + 1)dimensional x T Ax values easily.

2. Using Optdigits from the UCI repository, implement PCA. For various number of eigenvectors, reconstruct the digit images and calculate the reconstruction error (equation 6.12).

The Matlab code is given in ex6_2.m (which also generates figure 6.5 in the book). The contour plot of an example ‘6’ and its reconstruction is given in figure 6.1 of this document. The original 64 dimensional data and its reconstruction are as follows: tf = 0 0 0 10 12 0 0 0

0 0 9 14 4 0 0 0

0 0 15 3 0 0 0 0

0 2 15 2 6 1 0 0

0 2 16 15 12 15 4 0

0 0 16 5 0 3 14 0

0 0 12 10 4 11 14 0

0 0 1 11 14 12 1 0

tfn = 0 0.0844 3.2523 10.8280 8.5466 -0.4493 -1.3608 -0.1859

-0.0000 1.2471 9.7849 13.9309 9.3411 1.5186 -1.5062 -0.1063

0.0023 2.9502 14.2097 10.1036 3.6087 2.1009 0.5341 0.0335

0.0010 3.6126 14.2816 7.0768 2.5096 3.1204 2.1385 0.0034

0.0034 4.2342 12.7876 7.8097 6.3869 7.5014 3.2335 0.0000

0.0852 3.2692 13.9419 8.2514 6.0329 10.8696 6.8721 0.0789

0.0572 1.3435 11.7729 10.9618 10.4288 13.2739 7.8419 0.4969

-0.0000 0.0996 2.9551 11.4303 16.4897 11.3677 4.2543 0.4434

41

8 7 6 5 4 3 2 1

1

2

3

4

5

6

7

8

1

2

3

4

5

6

7

8

8 7 6 5 4 3 2 1

Figure 6.1

A ‘6’ and its reconstruction using the top two eigenvectors.

3. PCA minimizes the reconstruction error (equation 6.12). Another way to do this is to consider a regression problem where x t is the input and x̂ t is the output. Discuss how this can be done. (In chapter 11, we will discuss a neural network model called the autoencoder that does this.)

One can use any regression method; given the d-dimensional x, we will have d regression outputs each one estimating one of the dimensions given all the dimensions. One important requirement is that the regressor model should be constrained so that it does not just learn to copy the corresponding input to its output. In the autoencoder neural network this is done by using a layer of hidden units that combines all the inputs, so the input values are not directly accessible. The advantage of using the other inputs is that when the inputs are correlated, they can be used for error correction, e.g., eliminating noise; see section 11.10.

4. Plot the map of your state/country using MDS, given the road travel distances as input.

The Matlab code that generates figure 6.9 in the book is given in ex6_3.m.

42

6

Dimensionality Reduction

It generates the map of Europe from pairwise distances between 12 cities.

5. In Sammon mapping, if the mapping is linear, namely, g(x|W) = WT x, how can W that minimizes the Sammon stress be calculated?

MDS already does this linear mapping. Another possibility is to plug in the linear model in Sammon stress (equation 6.37) and find W that minimizes it. One can for example use an iterative, gradient-descent procedure; this is especially interesting when the mapping is nonlinear. We discuss this in chapter 11 when we use multilayer perceptrons (p. 304).

6. In figure 6.11, we see a synthetic two-dimensional data where LDA does a better job than PCA. Draw a similar dataset where PCA and LDA find the same good direction. Draw another where neither PCA nor LDA find a good direction.

See figures 6.2a and b; the Matlab code is in ex6_5.m.

7. Redo exercise 4, this time using Isomap where two cities are connected only if there is a direct road between them that does not pass through any other city.

That is actually how intercity distances are calculated, namely by connecting the distances between the cities that lie on the shortest path.

8. In Isomap, instead of using Euclidean distance, we can also use Mahalanobis distance between neighboring points. What are the advantages and disadvantages of this approach, if any?

The disadvantage is the need to calculate, store and process the covariance matrix used in Mahalanobis distance calculation. There can be one global matrix or better still, for each instance, instances in its neighborhood can be used to calculate a local Mahalanobis matrix.

The advantage is that Euclidean distance assumes equal scale on all dimensions and no correlation, and hence, Mahalanobis distance would be expected to work better.

9. Multidimensional scaling can work as long as we have the pairwise distances between objects. We do not actually need to represent the objects as vectors at all as long as we have some measure of similarity. Can you give an example?

43 (a) 1

1

0.5

0.5

0

0

−0.5

−0.5

4

2 pca 0 lda −2 −2

0

2

4

−1

1

2 PCA projection

3

−1 −3

−2 LDA projection

−1

1 2 LDA projection

3

(b) 1

1

0.5

0.5

0

0

−0.5

−0.5

4

2 pca,lda 0

−2 −2

0

2

4

−1

0

1 2 PCA projection

3

−1

0

Figure 6.2 (a) Both PCA and LDA find the good direction; (b) Neither work well.

Let us say we have a database of documents. Then if dr s denotes the number of terms that documents r and s share, we can use MDS to map these documents to a lower-dimensional space, for example, to visualize them and check for structure. Note that here we can count the number of shared terms without needing to explicitly represent the documents as vectors using, for example, the bag of words representation.

10. How can we incorporate class information into Isomap and LLE such that instances of the same class are mapped to nearby locations in the new space?

We can include an additional penalty term for instances belonging to different classes; MDS will then map instances of the same class to nearby points.

44

6

Dimensionality Reduction

11. In factor analysis, how can we find the remaining ones if we already know some of the factors?

If we already know some of the factors, we can find their loadings by regression and then remove their effect from the data. We will then get the residual of what is not explained by those factors and look for additional factors that can explain this residual.

12. Discuss an application where there are hidden factors (not necessarily linear) and where factor analysis would be expected to work well.

One example is the data of student grades at a university. The grade a student gets for a set of courses depends on a number of hidden factors—for example, the student’s aptitude for the subject, the amount of time he/she can allocate to studying, the comfort of his/her lodging, and so on.

13. In t-SNE, when calculating the probabilities in the original space, instead of the Euclidean distance we can also use the Mahalanobis distance.

What are the advantages and the disadvantages?

Euclidean distance is fine if we have a large sample but when the sample is small and there is strong correlations between input attributes, calculating similarity using the Mahalanobis distance may be preferrable. This however requires that one estimates, stores, and uses a d × d covariance matrix. One can use a single global matrix for the whole data for simplicity, or estimate matrices locally, maybe on the fly, which though makes the model more flexible also makes it costlier in terms of memory and computation.

14. Just because a dimensionality reduction method can find a mapping to a lower-dimensional space does not mean that it is a good mapping.

How can we assess the quality of a mapping found?

We need to check if the neighborhood structure is preserved. For example, the KL criterion optimized by t-SNE can be used where pr s and qr s are the neighborhood probabilities calculated before and after dimensionality reduction. One can for example decide on how much to decrease dimensionality using such a criterion.

7

Clustering

1. In image compression, k-means can be used as follows: The image is divided into nonoverlapping c × c windows and these c 2 -dimensional vectors make up the sample. For a given k, which is generally a power of two, we do k-means clustering. The reference vectors and the indices for each window is sent over the communication line. At the receiving end, the image is then reconstructed by reading from the table of reference vectors using the indices. Write the computer program that does this for different values of k and c. For each case, calculate the reconstruction error and the compression rate.

This code is very straightforward, given the pseudocode of k-means in figure 7.3 (p. 167) of the book. One word of caution: When calculating the compression rate, do not forget to take into account the storage needed for the reference vectors. Given an n × n image, c × c windows, m reference vectors, the index of each window is coded using log2 m bits. There are (n/c)2 indices that should be so coded; so the image is transferred using (n/c)2 log2 m bits. There are m reference vectors each of dimensionality c 2 . If we need b bits for each dimension, we need mc 2 b bits to store the reference vectors.

2. We can do k-means clustering, partition the instances, and then calculate Si separately in each group. Why is this not a good idea?

There are basically two reasons: First, k-means does hard partitioning but it is always better to do a soft partitioning (using hti ∈ (0, 1) instead of bit ∈ {0, 1}) so that instances (in between two clusters) can contribute to the parameters (the covariance matrix in this case) of more than one cluster allowing a smooth transition between clusters.

Second, k-means proper uses the Euclidean distance and we remember

46

7

Clustering

that Euclidean distance implies features that have the same scale and are independent. Using Si implies the use of Mahalanobis distance and hence taking care of differences in scale and dependencies.

3. Derive the M-step equations for S in the case of shared arbitrary covariance matrix S (equation 7.15) and s 2 , in the case of shared diagonal covariance matrix (equation 7.16).

In the case of a shared arbitrary covariance matrix, in the E-step, we have hti = P

t πi exp[−(1/2)(x t − m i )T S−1 i (x − m i )]

j πj exp[−(1/2)(x

t − m )T S−1 (x t − m )] j j j

and in the M-step for the component parameters, we have XX min ht (x t − m i )T S−1 (x t − m i ) m i ,S t i i The update equation for m i does not change but for the common covariance matrix, we have S

l

= =

t l+1 l+1 T t t i hi (x − m i )(x − m i ) P P t t i hi P P t t l+1 l+1 T t t i hi (x − m i )(x − m i )

P P t

N

Another way we can see this is by considering that S=

X i

P (Gi )Si =

X i

P

t t hi

N

!

Si

In the case of a shared diagonal matrix, for the E-step we have πi exp −(1/2s 2 )kx t − m i k2 i h hti = P 2 t 2 j πj exp −(1/2s )kx − m j k

and in the M-step, we have X X kx t − m i k2 min ht s2 m i ,s t i i

47 The update equation for the shared variance is 2

s =

P P Pd t

t t 2 k=1 hi (xk − mik )

i

Nd

4. Define a multivariate Bernoulli mixture where inputs are binary and derive the EM equations.

When the components are multivariate Bernouilli, we have binary vectors that are d-dimensional. Assuming that the dimensions are independent, we have (see section 5.7) pi (x t |Φ) =

d Y

xtj

j=1

t

pij (1 − pij )1−xj

l l l where Φl = {pi1 , pi2 , . . . , pid }ki=1 . The E-step does not change (equation 7.9). In the M-step, for the component parameters pij , i = 1, . . . , k, j = 1, . . . , d, we maximize

Q′

=

XX

hti log pi (x t |φl )

=

XX

hti

t

t

i

i

X j

l l ) + (1 − xtj ) log(1 − pij xtj log pij

Taking the derivative with respect to pij and setting it equal to 0, we get l+1 pij =

P

t t t hi xj t t hj

P

Note that this is the same as in equation 5.31, except that estimated “soft” labels hti replace the supervised labels rit .

5. In the mixture of mixtures approach for classification, how can we finetune ki , the number of components for class Ci ?

As usual, by cross-validation. We can try different ki and choose the one that performs best on the validation data. We can start with small ki for all i and then, looking at the confusion matrix, we can pinpoint which classes are frequently confused and increase their ki to give them more flexibility.

48

7

Clustering

6. In what type of data scenarios, would clustering be a good preprocessing stage before regression?

In regression too, the input probability density p(x) may be different from uniform. For example, a plant may have operation modes where each mode corresponds to a different oombination of operational parameters. In such a case, we can first do clustering to find these different modes and then learn a different regression model for each mode, instead of trying to fit one global regression model to the whole data. In Section 13.8, we will discuss the Mixture of Experts model where we have multiple experts each localized in one region of the input space.

7. Edit distance between two strings—for example, gene sequences—is the number of character operations (insertions, deletions, substitutions) it takes to convert one string into another. List the advantages of doing spectral clustering using the edit distance as opposed to vanilla k-means using Euclidean distance on strings.

Euclidean, actually Hamming, distance on strings would not be useful, because a single insertion would cause large distances. And strings should be of same length. Edit distance takes character operations such as insertions, deletions, and substitutions and hence makes more sense. Thatis why pectral clustering with a Gaussian kernel using the edit distance would allow a much better clustering.

8. How can we do hierarchical clustering with binary input vectors, for example, for text clustering using the bag of words representation?

We can use any variant as long as we have a good distance measure.

For example, if nr s is the number of shared words, it is a similarity metric, and we can use 1/(nr s + ǫ) as the distance measure.

9. What are the similarities and differences between average-link clustering and k-means?

They both measure similarity by looking at the average of instances that fall in a cluster. Note however that in a hierarchical scheme, there are clusters at different resolutions.

10. In hierarchical clustering, how can we have locally adaptive distances?

What are the advantages and disadvantages of this?

49 When we use Euclidean distance we give equal weight to all dimensions and ignore correlations. We can calculate a local covariance matrix and use Mahalanobis distance. This is costlier but allows a better approximation of the local density.

11. How can we parallelize hierarchical clustering?

A significant portion of the processing in clustering goes to calculating distances between groups which can be efficiently parallelized.

12. How can we make k-means robust to outliers?

An outlier is an instance that is very far from all centers. We would not want outliers to affect the solution. One possibility is to not take such instances into account when calculating the parameters—for example, means and covariances. Note that to detect an outlier, we can use the Mahalanobis distance or the likelihood, but we cannot use the posterior. We discuss a nonparametric method for detecting outliers in section 8.7.

13. Having generated a dendrogram, can we “prune” it?

At any level, if one of the branches has a relatively small number of descendants when compared to others, we may ignore that branch; it will correspond to a cluster containing too few instances.

14. In some applications we have multiple views of the same data; for example, in image clustering we may have the actual image and a set of labels. How can we do clustering in such a multimodal setting?

For each view, we can calculate a distance seperately and then take a weighted sum of them to calculate the overall distance–the sum should be weighted because distances calculated over different would have different scales.

8

Nonparametric Methods

1. How can we have a smooth histogram?

We can interpolate between the two nearest bin centers. We can consider the bin centers as xt , consider the histogram values as r t , and use any interpolation scheme, linear or kernel-based.

2. Show equation 8.16.

Given that p̂(x|Ci ) =

ki Ni V k (x)

P̂ (Ci ) =

Ni N

we can write ki

P̂ (Ci |x)

= =

Ni

p̂(x|Ci )P̂ (Ci ) N V k (x ) N P = P i kj Nj j p̂(x|Cj )P̂ (Cj ) j Nj V k (x ) N k k P i = i k j kj

3. How can we have a version of equation 8.20 where instead of one global M we have localized Mj in different parts of the input space?

In the simplest case, M can be calculated over the whole input space.

One can do clustering and calculate a different Mj for each cluster, which will be similar to Mixture of PCA/FA (with M = LT L and we have a different L for each cluster). Generalizing LMNN so that it can learn different Mj will be an interesting extension; see (Gönen and Alpaydın 2008) reference from chapter 14 for a similar approach.

52

8

Nonparametric Methods

4. Parametric regression (section 5.8) assumes Gaussian noise and hence is not robust to outliers; how can we make it more robust ?

One approach would be to use a robust error function, such as the one used in support vector regression, that adds up differences linearly rather than quadratically.

Another approach—and this is related to our discussion in this chapter— would be to try to find the outliers and remove them. Towards this aim, one possibility is to use the local outlier factor of section 8.7 in a regression setting—prune instances that are far away (in the input space) from all others. Another possibility is to make an initial fit, find instances on which we have a bad fit, remove those, and refit. In both methods, there are thresholds that need to be adjusted carefully by cross-validation.

Note that on small datasets, deciding whether an instance is an outlier becomes a very difficult and critical problem.

5. How can we detect outliers after hierarchical clustering (section 7.8) ?

After hierarchical clustering, we would expect outliers to form separate leaves, far from the other leaves and containing very few instances.

6. How does condensed nearest neighbor behave if k > 1?

When k > 1, to get full accuracy without any misclassification, it may be necessary to store an instance multiple times so that the correct class gets the majority of the votes. For example, if k = 3 and x has two neighbors both belonging to a different class, we need to store x twice (i.e., it gets added in two epochs), so that if x is seen during test, the majority (two in this case) out of three neighbors belong to the correct class.

7. How can we find the minimal consistent subset (section 8.5) using algorithms that do a wider search than greedy, such as genetic algorithms or simulated annealing?

We consider this as the minimization of an augmented error function, as shown in Equation 8.17, that is the sum of error and complexity, i.e., the number of instances stored. We then define operators that modify Z such as one that adds one instance and another that removes one instance.

53 The advantage of simulated annealing would be to accept operators when temperature is high even if they do not lead to improvement; this will allow a wider search than hill climbing. In genetic algorithms, Z can be represented by a binary chromosome of length N; operators such as crossover and mutation would work on that and the augmented error can be used as the fitness function.

8. In condensed nearest neighbor, an instance previously added to Z may no longer be necessary after a later addition. How can we find such instances that are no longer necessary?

We pick randomly an instance from Z and check if we can correctly classify it using the rest of Z; if we can, this means that we would not have added it if we were seeing it now for the first time and therefore can remove it from Z; if we misclassify it, we keep it. I used to call this the “sleep” mode and alternated it with the “awake” mode that added instances from the training set to Z; see E Alpaydın (1994) "GAL: Networks that Grow when they Learn and Shrink when they Forget," International Journal of Pattern Recognition and Artificial Intelligence, 8, 391–414.

9. In a regressogram, instead of averaging in a bin and doing a constant fit, one can use the instances falling in a bin and do a linear fit (see figure 8.14). Write the code and compare this with the regressogram proper.

The Matlab code is given in ex8_3.m which is the code used to generate figure 8.14 of the book. Compared with the regressogram proper, it is more costly both in terms of space and computation: We need one more parameter (slope) for each bin; we need computation during test to calculate the output; we need more computation during training to calculate the linear fit in each bin.

10. Write the error function for loess discussed in section 8.8.3.

The output is calculated using a linear model g(x) = ax + b, where, in the running line smoother, we minimize

E(a, b|x, X) =

X t

w

x − xt h

!

[r t − (axt + b)]2

54

8

Nonparametric Methods

and w (u) =

(

1 0

if |u| < 1 otherwise

Note that we do not have one error function but rather, for each test input x, we have another error function taking into account only the data closest to x, which is minimized to fit a line in that neighborhood.

Loess is the weighted version of running line smoother where a kernel function K(·) ∈ (0, 1) replaces the w (·) ∈ {0, 1}: !

X x − xt [r t − (axt + b)]2 K E(a, b|x, X) = h t 11. Propose an incremental version of the running mean estimator, which, like the condensed nearest neighbor, stores instances only when necessary.

Let us say that we are using nearest neighbor regression, that is, given test input x, we find closest xt ∈ X and give its desired output r t as the output. In such a case, in the condensed nearest neighbor regression, we do not need to store a given pair (x, r ) if (x′ , r ′ ) is the closest and |r ′ − r | < θ where θ is some tolerance threshold.

We modify the pseudocode of figure 8.6 (on p. 195) of the book) for condensed nearest neighbor classification as shown in figure 8.1 of this document. Instead of checking whether the class labels are the same, we check if the predicted output is close enough to the desired value.

This can be generalized to any nonparametric smoother: We check if the estimate using current Z gives an acceptable output (within a certain tolerance), if not, we add the current pair to Z. Again (see exercise 8.5 above), it may be necessary to store the current pair multiple times.

12. Generalize the kernel smoother to multivariate data.

We can just use a multivariate kernel, for example, d-variate Gaussian, in equation 8.26. Again, we have the usual concerns of the suitability of using Euclidean distance vs. estimating a local covariance and using Mahalanobis distance.

13. In the running smoother, we can fit a constant, a line, or a higherdegree polynomial at a test point. How can we choose between them?

By cross-validation.

55

Z←∅ Repeat For all (x, r ) ∈ X (in random order) Find (x ′ , r ′ ) ∈ Z such that kx − x ′ k = minx j ∈Z kx − x j k If |r − r ′ | > θ add (x, r ) to Z Until Z does not change Figure 8.1 Condensed nearest neighbor regression algorithm.

14. In the running mean smoother, additional to giving an estimate, can we also calculate a confidence interval indicating the variance (uncertainty) around the estimate at that point?

We can calculate a confidence interval as follows: We sample k datasets from our data, for example, using k-fold cross-validation and fit k distinct smooths, gi (·). Then, for any x, we can calculate the average and standard deviation of gi (x), i = 1, . . . , k: g(x) =

Pk

i=1 gi (x)

k

2

and s (x) =

P

i (gi (x) − g(x))

2

k−1

The 95 percent confidence interval is then (g(x) − 1.96s(x), g(x) + 1.96s(x)).

This is a parametric confidence interval; we can also define a nonparametric confidence interval. For example, if k = 20, we train 20 smooths and then for any x, we sort the 20 estimates and the interval between the second and the nineteenth gives us the 95 percent nonparametric confidence interval.

15. We can use the nonparametric estimators as long as we can calculate pairwise distances, which we can do even if we do not have explicit vectorial representations. Give an example.

Let us say we want to classify documents; we can define a similarity measure between two documents as the number of shared words.

In such a case, we can use for example k-nn without explicitly needing to first convert the two documents to a bag-of-words vectorial representation and then use Hamming/Euclidean distance. This can work with any data structure (e.g., graphs) as long as we have a distance/similarity measure appropriate for that data structure. In chap-

56

8

Nonparametric Methods

ter 14, we will see this in more detail where we will call such applicationspecific similarity functions, kernel functions.

9

Decision Trees

1. Generalize the Gini index (equation 9.5) and the misclassification error (equation 9.6) for K > 2 classes. Generalize misclassification error to risk, taking a loss function into account.

PK P Gini index with K > 2 classes: φ(p1 , p2 , . . . , pK ) = j<i pi pj i=1

Misclassification error: φ(p1 , p2 , . . . , pK ) = 1 − maxK i=1 pi PK K Risk: φΛ (p1 , p2 , . . . , pK ) = mini=1 k=1 λik pk where Λ is the K × K loss matrix (equation 3.7).

2. When we have K > 2 classes, another possibility is to train K trees, each separating one class from all others. Discuss how such trees are trained and how the final decision can be made.

We convert the K-class problem into K two-class problems for which we train K distinct trees. Because these correspond to different problems, different thresholds and final structures will be learned. Once we have all, for any new input, we can get the output of all K and choose the class that has the highest positive value.

Note that this can also be done with a regression problem with K outputs: We can learn one tree where at each split we sum the squared error on all outputs, or learn a separate tree for each output.

3. For a numeric input, instead of a binary split, one can use a ternary split with two thresholds and three branches as xj < wma , wma ≤ xj < wmb , xj ≥ wmb Propose a modification of the tree induction method to learn the two thresholds, wma , wmb . What are the advantages and the disadvantages of such a node over a binary node?

58

9

Decision Trees

For the numeric attributes, instead of one split threshold, we need to try all possible pairs of split thresholds and choose the best. When there are two splits, there are three children, and in calculating the entropy after the splits, we need to sum up over the three sets corresponding to the instances taking the three branches.

2 The complexity of finding the best pair is O(Nm ) instead of O(Nm ) and each node stores two thresholds instead of one and has three branches instead of two. The advantage is that one ternary node splits an input into three, whereas this requires two successive binary nodes. Which one is better depends on the data at hand; if we have hypotheses that require bounded intervals (e.g., rectangles), a ternary node may be advantageous.

4. Propose a tree induction algorithm with backtracking.

The decision tree induction algorithm is greedy and stops when there is no further decrease in entropy. One possibility is to backtrack: Let us say that we have a tree where no matter now we add a node, entropy does not decrease further; we can go back, undo the last decision, choose the next best split and continue, to see whether we get to a better state. In fact, the decision tree induction does depth-first search and any other search mechanism, best-first search, breadthfirst search, etc. can be used which can search wider in the space (of possible trees) at the expense of more space and computation.

5. In generating a univariate tree, a discrete attribute with n possible values can be represented by n 0/1 dummy variables and then treated as n separate numeric attributes. What are the advantages and disadvantages of this approach?

If we do this, we get binary splits instead of n-ary splits. For example, if we have three values {red,green,blue}, a discrete node has a ternary split whereas doing this, we can have a split such as {red, not-red}, thereby grouping all the complements of a value in the other branch.

Whether this leads to simpler trees or not depends on the data.

6. Derive a learning algorithm for sphere trees (equation 9.21). Generalize to ellipsoid trees.

A sphere node uses a binary decision node (see figure 9.1): fm (x) : kx − c m k ≤ αm

59 and a simple algorithm is as follows: Calculate the centroid of instances reaching node m and take it as c m . Then sort all instances according to their (Euclidean) distance to the center and for all possible Nm − 1 values between them, check what the entropy would be and choose the best as the radius αm . In an ellipsoid tree, we calculate the covariance matrix of the instances and use the Mahalanobis distance from an instance to the center.

c a

Figure 9.1

A sphere decision node.

7. In a regression tree, we discussed that in a leaf node, instead of calculating the mean, we can do a linear regression fit and make the response at the leaf dependent on the input. Propose a similar method for classification trees.

This implies that at each leaf, we will have a linear classifier trained with instances reaching there. That linear classifier will generate posterior probabilities for the different classes, and those probabilities will be used in the entropy calculation. That is, it is not necessary for a leaf to be pure, that is, to contain instances of only one class; it is enough that the classifier in that leaf generates posterior probabilities close to 0 or 1.

8. Propose a rule induction algorithm for regression.

Rule induction is similar to tree induction except that at each split, we keep one of the branches (better one) and ignore the other, going depth-first and generating a single path, instead of a tree. Each node corresponds to one condition, the path corresponds to their conjunction and the leaf is the average of their outputs. We would like to find a subset which is as large as possible and on which we have small estimation error (MSE in regression); we can combine these two criteria

60

9

Decision Trees

of accuracy and complexity using some MDL measure. Once we get a rule, we remove the subset it covers (those instances on which we give a good enough fit) from the training set. We then run the same algorithm again to get another rule that covers another subset, until we cover all the training set.

9. In building a regression tree, instead of the mean we can use the median, and instead of minimizing the squared error we can minimize the absolute error. Why does this help in the case of noise?

The absolute error is more robust to outliers, because differences increase the error linearly, as opposed to quadratically for the case of squared error. In such a case, we need to use the median because the median is the best estimator for absolute error, just like the mean is the best estimator for squared error.

10. In regression trees, how can we get rid of discountinuities at the leaf boundaries?

There is the concept of a “soft decision tree,” where at a decision node, instead of taking one of the splits, we take all splits but with a probability. This means that for any input, we take all the paths and arrive to all the leaves, but all with a probability (which should sum up to 1 over all the leaves). We can then output a weighted average of all the leaf values weighted by these probabilities. This will generate a smooth fit. See section 13.9 for more detail.

11. Let us say that for a classification problem, we already have a trained decision tree. How can we use it in addition to the training set in constructing a k-nearest neighbor classifier?

The decision tree does feature selection, and we can only use the features used by the tree.

12. In a multivariate tree, very probably, at each internal node, we will not be needing all the input variables. How can we decrease dimensionality at a node?

Each subtree handles a local region in the input space that can be explained by a small number of features. We can do feature selection or extraction using only the subset of the instances reaching that node.

Ideally we would expect to need fewer features as we go down the tree.

61 13. Propose an online learning method for trees where we see one training instance at a time and update the tree structure and/or parameters accordingly.

This would be a very complex algorithm because as we see new instances, we do not just update the thresholds but previously learned instances need to be moved to other subtrees which in turn needs updating their parameters and structures.

A better approach is to use a soft decision tree where splits are soft and the parameters of the whole tree can be learned together using gradient-descent; see section 13.9.

10

Linear Discrimination

1. For each of the following basis function, describe where it is nonzero: a. sin(x1 ) This is used when a trigonometric mapping is necessary, for example, in robot arm kinematics, or in recurring phenomena, for example, seasonal repeating behavior in time-series. b. exp(−(x1 − a)2 /c)

This is a bell-shaped function with a as its center and c its spread. √ √ Roughly, speaking it is nonzero between (a − 2 c, a + 2 c).

c. exp(−kx − ak2 /c)

This is a d-dimensional bell-shaped function with a as its center and c its spread in d dimensions.

d. log(x2 ) This is useful when x2 has a wide scale and a log transformation is useful. e. 1(x1 > c) This is similar to a univariate split of a decision tree node. f. 1(ax1 + bx2 > c)

This defines a multivariate oblique split.

2. For the two-dimensional case of figure 10.2, show equations 10.4 and 10.5.

Given figure 10.1, first let us take input x 0 on the hyperplane. The angle between x 0 and w is a0 and because it is on the hyperplane

64

x2

10 Linear Discrimination

g(x)=0 g(x)>0

g(x)<0 d 0=|w0|/||w|| xo

x a0

w

a d=|g(x)|/||w|| x1

Figure 10.1 The geometric interpretation of the linear discriminant.

g(x 0 ) = 0. Then g(x 0 )

=

d0

=

w T x 0 + w0 = kwkkx 0 k cos a0 + w0 = 0 |w0 | kx 0 k cos a0 = kwk

For any x with angle a to w, similarly we have g(x)

=

d

=

w T x + w0 = kwkkxk cos a + w0 w0 g(x) − w0 + w0 g(x) kxk cos a − = = kwk kwk kwk

P 3. Show that the derivative of the softmax, yi = exp(ai )/ j exp(aj ), is ∂yi /∂aj = yi (δij − yj ), where δij is 1 if i = j and 0 otherwise.

Given that exp ai yi = P j exp aj

65 for i = j, we have ∂yi ∂ai

= = =

exp ai

P

exp ai P j exp aj

yi (1 − yi )

− exp ai exp aj 2 j exp aj !

P j exp aj − exp ai P j exp aj

j exp aj

P

and for i 6= j, we have ∂yi ∂aj

= = =

− exp ai exp aj P 2 j exp aj ! exp ai − P j exp aj yi (0 − yj )

P P

j exp aj j exp aj

!

which we can combine in one equation as ∂yi = yi (δij − yj ) ∂aj 4. With K = 2, show that using two softmax outputs is equal to using one sigmoid output.

y1

= =

exp o1 1 1 = = exp o1 + exp o2 1 + exp(o2 − o1 ) 1 + exp(−(o1 − o2 )) sigmoid(o1 − o2 )

For example, if we have o1 = w T1 x, we have y1 =

exp w T1 x exp w T1 x + exp w T2 x

= sigmoid(w T1 x − w T2 x) = sigmoid(w T x)

where w ≡ w 1 − w 2 and y2 = 1 − y1 .

66

10 Linear Discrimination

5. How can we learn Wi in equation 10.36?

For example, if we have two inputs x1 and x2 , we have log

p(x1 , x2 |Ci ) p(x1 , x2 |CK )

=

Wi11 x21 + Wi12 x1 x2 + Wi21 x2 x1 + Wi22 x22

+

wi1 x1 + wi2 x2 + wi0

Then we can use gradient descent and take derivative with respect to any Wjkl to calculate an update rule: ∆Wjkl = η

X (rjt − yjt )xtk xtl t

6. In using quadratic (or higher-order) discriminants as in equation 10.36, how can we keep variance under control?

For a quadratic discriminant, Wi of equation 10.34 has O(d 2 ) terms. If we know that certain pairs of input variables xj , xk are not correlated, we can have the corresponding off-diagonal Wijk set to 0. We can have different discriminants share a common W, or, we can write a low-rank approximation to Wi .

7. What is the implication of the use of a single η for all xj in gradient descent?

Using a single η for all xj implies doing updates in the same scale, which in turn implies that all xj are in the same scale. If they are not, it is a good idea to normalize all xj , for example, by z-normalization, before training. Note that we need to save the scaling parameters for all inputs, so that the same scaling can also be done later to the test instances.

8. In the univariate case for classification as in figure 10.7, what do w and w0 correspond to?

The slope and the intercept of the line, which is then fed to the sigmoid.

9. Let us say for univariate x, x ∈ (2, 4) belong to C1 and x < 2 or x > 4 belong to C2 . How can we separate the two classes using a linear discriminant? We define an extra variable z ≡ x2 and use the linear discriminant w2 z + w1 x + w0 in the (z, x) space, which corresponds to

67 a quadratic discriminant in the x space. For example, we can manually write ( C1 if (x − 3)2 − 1 ≤ 0 Choose C2 otherwise or rewrite it using a sigmoid (see figure 10.2): Choose

(

C1 C2

if sigmoid((x − 3)2 − 1) ≤ 0.5 otherwise

Or, we can use two linear discriminants in the x space, one separating at 2 and the other separating at 4, and then we can OR them. Such layered linear discriminants are discussed in chapter 11.

8 6 4 2

quadratic

sigmoid

0

C

↓ boundaries ↓ C

C

−2 0

1

2

5

2

1

3

4

2

6

Figure 10.2 The quadratic discriminant, before and after the sigmoid. The boundaries are where the discriminant is 0 or where the sigmoid is 0.5.

10. For the sample data in figure 10.11, define ranks such that a linear model would not be able to learn them. Explain how the model can be generalized so that they can be learned.

Let us take the case in figure 10.3a where we see a dataset that cannot be ranked by a linear model. We can generalize the linear model as usual by adding nonlinear, for example higher order, terms. For example, if we add a new dimension as the square of the original dimension,

68

10 Linear Discrimination

in this new space, as we see in figure 10.3b, the linear model can learn the good ranking. (a)

(b)

2

2

1.5

1.5

1

1

0.5

0.5

1

4 3

1

x

2

0

2

2

−0.5

−0.5

−1

−1

−1.5

−1.5

−2 −2

−1

0 x

1

2

4

0

−2 −2

−1

0 x

3

1

2

Figure 10.3 (a) One dimensional data that cannot be ranked by a linear ranker. (b) A second dimension is added as x2 and in this new two dimensional space a linear ranker works well.

11. In classification, it does not make sense to minimize the sum of the squared errors after a sigmoid. Why not?

We know that minimizing the sum of the squared errors corresponds to maximizing the log likelihood with Gaussian outputs. In classification, the desired outputs are 0 or 1 and clearly not Gaussian. They are Bernoulli, and hence we minimize the cross-entropy, which corresponds to maximizing the log likelihood with Bernoulli outputs.

12. In multilabel classification, we can also have instances with no labels.

Will they be useful for training?

69 Yes. They will act as negative instances for all labels.

13. In ranking, we can also have cases of “don’t care,” where we do not want to favor one over the other. How can these be handled during training?

We will simply ignore them in the error (Eq. 10.44).

11

Multilayer Perceptrons

1. Show the perceptron that calculates NOT of its input. y = s(−x + 0.5) 2. Show the perceptron that calculates NAND of its two inputs. y = s(−x1 − x2 + 1.5) 3. Show the perceptron that calculates the parity of its three inputs.

h1 h2 h3 h4 y

=

=

s(−x1 − x2 + 2x3 − 1.5) 001

s(−x1 + 2x2 − x3 − 1.5) 010

=

s(2x1 − x2 − x3 − 1.5) 100

=

s(h1 + h2 + h3 + h4 − 0.5)

=

s(x1 + x2 + x3 − 2.5) 111

The four hidden units corresponding to the four cases of (x1 , x2 , x3 ) values where the parity is 1, namely, 001, 010, 100, and 111. They are then OR’d to calculate the overall output. Note that another possibility is to calculate the three bit parity is in terms of two-bit parity (XOR) as: (x1 XOR x2 ) XOR x3 .

4. Derive the update equations when the hidden units use tanh, instead of the sigmoid. Use the fact that tanh′ = (1 − tanh2 ).

The only difference from the case where hidden units use sigmoid is that the derivative of the hidden unit activations will change. In

72

11 Multilayer Perceptrons

updating the first layer weights (equation 11.16), instead of zh (1 − zh ), we will have (1 − zh2 ).

5. Consider a MLP architecture with one hidden layer where there are also direct weights from the inputs directly to the output units. Explain when such a structure would be helpful and how it can be trained.

It can be trained as usual by backpropagation. The direct weights from the input to the outputs is a perceptron and the rest is a MLP, all being trained together as usual. The perceptron that contains the direct weights from the inputs to the outputs is a linear model and can be thought of as implemening the linear component of whatever we are trying to learn; the MLP then learns to correct this “default” linear model. In section 12.6.2, we will talk in more detail about such “skip” connections.

6. In section 11.6, we discuss how an MLP with two hidden layers can implement piecewise constant approximation. Show that if the weight in the last layer is not a constant but a linear function of the input, we can implement piecewise linear approximation.

We have basically the same architecture where each hidden units delimits one small region in the input space, the weight from that unit to output, which is a linear function of the input, defines the line in that region. In section 13.8, we will discuss the Mixture of Experts which works like that.

7. In gradient descent, we use a constant η that is the same for all weights and for all epochs. How can we make it adaptive?

We will discuss this in section 12.3.2.

8. In the autoencoder network, how can we decide on the number of hidden units?

We can do cross-validation on the reconstruction error.

For an autoencoder with one hidden layer, we can also use the number of eigenvectors that explain, for example 95 percent of the variance.

9. Word2vec learns a vectorial representation for words. In an application where inputs are sentences, such as sentiment classification, we need to get a representation for the whole sentence. Typically this is done by averaging the word2vec vectors for the words in the sentence. How

73 does this representation compare with bag of words? Can you think of a better way?

If the sentence is short, averaging may not be too bad but averaging too many words would lose information. In such a case, bag of words may even be better because we will know which words are there. If we want to use word2vec with long sentences, it is better to treat it as a sequence learning task where we see one word at a time; in section 12.7, we will discuss neural network models that can learn sequences.

10. How can we train sentence2vec?

This is related to the exercise above; we need to have a neural network that sees the whole sequence to generate a code; see section 12.7.

11. FastText works at the subword level on short sequences of characters.

What are its advantages and disadvantages with respect to word2vec?

In certain languages, syllables may be meaningful, for example, prefixes or suffixes, and longer words may be composite of short words and such suffixes. In such a case, starting at the level of characters and learning and coding character sequences and then learning how to combine them to form word-level codes would be preferrable to starting directly at the word level.

12. In section 10.8, we discussed ranking and derived update equations for a linear model. Derive the update equations for an MLP with one hidden layer of H hidden units.

Given the error function of Equation 10.42, and g(x|θ) defined as an MLP, we use the chain rule to derive second- and first-layer weights.

Note that we do not need a bias weight in the second layer, but we need them in the first layer. g(x u ) u

zh (x )

= =

H X

vh zh (x u )

h=1



sigmoid 

d X

j=1



whj xuj 

The second-layer weight update has the same form as Equation 10.45: ∆vh = −η(zh (x v ) − zh (x u ))

74

11 Multilayer Perceptrons

The first-layer weight, whj , updates can similarly be derived using chain rule.

12

Deep Learning

1. Derive the update equations for an MLP that uses ReLU in its hidden units. Assume one hidden layer of H units and one output trained for regression.

These will be the same as Equations 11.19 and 11.20, except that in Equatiob 11.20, instead of zht (1 − zht ), we will have the derivative of ReLU. 2. Some researchers say that in batch normalization calculating new statistics for each minibatch and normalizing accordingly is equivalent to adding some noise to hidden unit outputs and that this has a regularizing effect. Comment.

How batch normalization helps learning is an ongoing discussion; one explanation is this, namely that, it causes a small random perturbance that helps in getting out of saddle points or local minima; because it decreases reliance on the particular training data it has a regularizing effect. The shape of the error function of a neural network, especially when there are many nonlinear deep layers, and how gradient-descent works on such error surfaces are not yet well understood and there is significant amount of research on heuristics that can help learning.

3. Propose a hint that helps in image recognition and explain how it can be incorporated into the learning process.

Let us say we want to get invariance to small translations. We can (i) introduce additional virtual examples, (ii) use a pooling layer after convolutions or (iii) add an additional term that does not penalize translations but other distortions.

76

12 Deep Learning

4. In weight decay, does it make sense to use different λ in different layers?

This may help, especially if different layers have largely varying weights in them. This is not done in practice though, because of the additional effort needed to finetune those different λ.

5. Derive the update equations for soft weight sharing.

Assume a one-layer model for two-class classification for simplicity, X y t = sigmoid( wi xti ) i

the augmented error is E ′ = log

X t

r t log y t + λ

X

log

i

M X

αj pj (wi )

j=1

where pj (wi ) ∼ N (mj , sj2 ). In the general case of a multilayer perceptron, {wi }i includes all the weights in all layers including the bias weights. When we use gradient-descent, we get ∆wit = η(r t − y t )xti − ηλ

X

πj (wi )

j

(wi − mj ) sj2

where αj pj (wi ) πj (wi ) = P l αl pl (wi )

is the posterior probability. The weight is updated both to decrease the cross-entropy and to move it closer to the mean of the nearest Gaussian. Using such a scheme, we can also update the mixture parameters, for example ∆mj = ηλ

X i

πj (wi )

(wi − mj ) sj2

πj (wi ) is close to 1 if it is highly probable that wi comes from component j; in such a case, the mean mj is updated to be closer to the weight wi it represents. This is an iterative clustering procedure, and we will discuss such methods in more detail in chapter 12, see for example, equation 12.5.

77 6. In some studies, different dropout rates are used for input and hidden units. Discuss when this will be appropriate.

The dropout rate depends on how many inputs there are and how much redundancy we expect them to have. If we have a 100×100 pixel image, we can afford dropping out a large percentage of the pixels, but if we have only few features and if we believe that there is not much correlation between them, we would better not drop many of them out.

For hidden units, the dropout rate depends on their number and the connectivity. If we have a large layer of fully-connected units, again we can tolarate larger dropouts than for example when we have a convolutional layer where the task of each unit is clearly defined.

7. In designing a convolutional layer for an image recognition data, what are the factors that affect the filter size, number of filters, and stride?

The filter size depends on the size of details that we need to capture for correct decision making; if the input is a 24 × 24 handwritten digit, 5 × 5 features may be good but if the image is a 1000 × 1000 face, they may be too small. The stride also depends on the filter size, that is, it depends on how much the neighboring coýnvolutions overlap.

The number of filters depend on how many different features we would expect to see to define our image at that level; it is sort of the size of our alphabet. This depends on the level of abstraction; in lower levels we may have fewer than at higher levels where features tend to get more abstract.

8. Propose a one-dimensional convolution layer for sentence classification where the basic inputs are characters.

At the lowest level, we will have characters and then we can have a layer of 1d convolutions combining a few characters, and we expect them to detect important character sequences, such as syllables, and suffixes. A later layer can connect them to define longer sequences and so on, until we get to the whole sentence. Note that this requires that all sentences be of the same length; if this is not the case, we need to have a recurrent structure, e.g., LSTM, at a high layer.

9. Incremental learning of the structure of a MLP can be viewed as a state space search. What are the operators? What is the goodness function?

78

12 Deep Learning

What type of search strategies are appropriate? Define these in such a way that dynamic node creation and cascade-correlation are special instantiations.

We need to define an error function in terms of misclassification error or regression. We need to have a training set and a separate validation set on which we calculate the error to make sure that we do not overfit.

We can do a forward search starting from an initial state which may be a linear model. We can have operators to add/delete hidden layers and add/delete hidden units. We can do depth-first search and try one operator at a time and accept if it leads to an improvement, or we can, at any time, try all possible operators, and pick the one that leads to most improvement. We stop when no operator leads to any improvement.

In dynamic node creation, there is a single operator that adds a hidden unit to an existing layer. In cascade correlation, again, there is a single operator that adds a new layer with a single hidden unit.

Once we define structure learning as a state space search, all sorts of search algorithms, depth-first, breadth-first, using a lookahead (add three hidden units then check for improvement), are possible. See O. Aran, O. T. Yıldız, E. Alpaydın (2009) “An Incremental Framework Based on Cross-Validation for Estimating the Architecture of a Multilayer Perceptron,” International Journal of Pattern Recognition and Artificial Intelligence, 23(2), 159–190.

10. For the MLP given in figure 12.12, derive the update equations for the unfolded network.

The update equations are as follows (see figure 12.1): ∆Vi δh3i 2 ∆Wik 2 ∆Rij

δh2i

=

η(r − y)h3i

=

ηδh3i x3k

=

=

=

.. .

0 δWik

=

(r − y)Vi This is backpropagated error

ηδh3i h2j X δh3l Rli l

ηδh0l xk

79

Figure 12.1 MLP and its unfolded version.

11. Derive the update equations for the parameters of the gating units in highway networks.

For a skip connection, as given in Equation 12.24:   X X zlh = f  wl,h,i zl−1,i + vl,h,j zl−L,j  i

j

the direct and skip connection weights are updated as ∆wl,h,i

=

∆vl,h,j

=

∂E ′ f (·)zl−1,i ∂zlh ∂E ′ −η f (·)zl−L,j ∂zlh

−η

where ∂E/∂zlh is the error backpropagated until zlh , and f ′ (·) is the derivative of the activation function there.

For a gated connection, as given in Equations 12.25 and 26:   X X zlh = f glh wlhi zl−1,i + (1 − glh ) vlhj zl−L,j  i

j

80

12 Deep Learning

glh

=

sigmoid a Tlh z l−L + alh0

the direct, skip, and gating connection weights are updated as ∆wl,h,i

=

∆vl,h,j

=

∆alhk

=

∂E ′ f (·)zl−1,i glh ∂zlh ∂E ′ −η f (·)zl−L,j (1 − glh ) ∂zlh   X X ∂E ′  f (·) wl,h,i zl−1,i − −η vl,h,j  glh (1 − glh )zl−L,k ∂zlh i j −η

12. Compare LSTM and GRU units and propose a new unit mechanism distinct from those, justifying it by an application.

I would consider a model between GRU and LSTM, where we have a forget gate in Equation 12.35, instead of using 1− input gate: zht = fht · zht−1 + iht · cht with

fht = sigmoid w Tr x̃ t This is simpler than LSTM because there is no output gate and no extra sht .

There is no known direct relationship between application properties and whether LSTM or GRU is appropriate; it is basically a selection done by trial and error. LSTM is more widely used because it is a more flexible model, but GRU is also powerful in many applications.

My personal experience has shown that these gates rarely saturate at 0 or 1; most of the time, they blend the two sources or attenuate or amplify an input. That is why I tend to view them more as hidden units extracting some useful features rather than gates that turn some contribution on or off.

13. Assuming a GAN composed of a linear G and a linear D, derive the update equations for training them for (a) vanilla GAN and (b) Wasserstein GAN.

81 Let us assume for simplcity that both x and z are scalars. Such a GAN will not be useful, but it makes the derivation easy. (a) We have a linear generator: G(z) = w z + w0 and a linear discriminator: D(x) = sigmoid(vx + v0 ) The log-likelihood criterion is

X X log 1 − D(G(z t )) log D(xt ) + L= t

t

which we maximize for D and minimize for G.

∆v

=

∆v0

=

∆w

=

∆w0

=

  X X ∂L D(G(z t ))G(z t ) = η  (1 − D(xt ))xt − η ∂v t t   X X ∂L η D(G(z t )) = η  (1 − D(xt )) − ∂v0 t t X ∂L = η D(G(z t ))vz t ∂w t X ∂L = η D(G(z t ))v −η ∂w0 t

−η

(b) For the Wasserstein loss, we have a linear model, D(x) = vx; there is no need for v0 : L=

X t

D(xt ) −

X

D(G(z t ))

t

which we maximize for D and minimize for G.

∆v

=

  X X ∂L η G(z t ) = η  xt − ∂v t t

82

12 Deep Learning

∆w

=

∆w0

=

X ∂L = η vz t ∂w t X ∂L −η =η v ∂w0 t −η

13

Local Models

1. Show an RBF network that implements XOR. There are two possibilities: (a) We can have two circular Gaussians centered on the two positive instances and the second layer ORs them, (b) We can have one elliptic Gaussian centered on (0.5, 0.5) with negative correlation to cover the two positive instances. z2

z1

z1

(a)

(b)

Figure 13.1 Two ways of implementing XOR with RBF.

2. Write down the RBF network that uses elliptic units instead of radial units as in equation 13.13.

1 t pht = exp − (x t − m h )T S−1 (x − m ) h h 2 where Sh is the local covariance matrix.

3. Derive the update equations for the RBF network for classification (equations 13.20 and 13.21).

84

13 Local Models

Because of the use of cross-entropy and softmax, the update equations will be the same with equations 13.17, 13.18, and 13.19 (see equation 10.33 for a similar derivation).

4. Show how the system given in equation 13.22 can be trained.

There are two sets of parameters: v, v0 of the default model and wh , m h , sh of the exceptions. Using gradient-descent and starting from random values, we can update both iteratively. We update v, v0 as if we are training a linear model and wh , m h , sh as if we are training a RBF network.

Another possibility is to separate their training: First, we train the linear default model and then once it converges, we freeze its weights and calculate the residuals, that is, differences not explained by the default. We train the RBF on these residuals, so that the RBF learns the exceptions, that is, instances not covered by the default “rule.” See E. Alpaydın. 1997. REx: Learning A Rule and Exceptions, International Computer Science Institute TR-97-040, Berkeley CA. 5. Compare the number of parameters of a mixture of experts architecture with an RBF network.

With d inputs, K classes and H Gaussians, an RBF network needs H · d parameters for the centers, H parameters for the spreads and (H +1)K parameters for the second-layer weights. For the case of the MoE, for each second-layer weight, we need a d + 1 dimensional vector of the linear model, but there is no bias; hence we have HK(d + 1) parameters.

Note that the number of parameters in the first layer is the same with RBF and it is the same whether we have Gaussian or softmax gating: For each hidden unit, in the case of Gaussian gating, we need d parameters for the center and 1 for the spread; in the case of softmax gating, the linear model has d + 1 parameters (d inputs and a bias).

6. Formalize a mixture of experts architecture where the experts and the gating network are multilayer perceptrons. Derive the update equations for regression and classification.

The output is yit =

H X

h=1

t t wih gh

85 The experts are MLP with M hidden units: t wih

bkt

= =

M X

k=1

vihk bkt + vih0 

sigmoid 

d X

j=1



ukj xtj + uk0 

The gating networks are MLP with G hidden units: ght

=

oth

=

akt

=

exp oth P t l exp ol G X

k=1

mhk akt + mh0 

sigmoid 

d X

j=1



nkj xtj + nk0 

As usual, we use gradient-descent: E({v, u, m, n}|X)

=

∆vihk

=

∆ukj ∆mhk ∆nkj

1 XX t (r − yit )2 2 t i i X η (rit − yit )ght bkt t

=

XXX (rit − yit )ght vihk bkt (1 − bkt )xtj η

=

XX t (rit − yit )(wih − yit )ght akt η

=

XXX t (rit − yit )(wih − yit )ght mhk akt (1 − akt )xtj η

t

t

t

h

i

i

h

i

In classification, we have yit

=

E({v, u, m, n}|X)

=

PH t t gh exp h=1 wih P PH t t k exp h=1 wkh gh XX rit log yit − t

i

Update equations turn out to be the same.

86

13 Local Models

7. Derive the update equations for the cooperative mixture of experts for classification.

XX

rit log yit

=

−

yit

=

t wih

=

PH t t gh exp h=1 wih P PH t t k exp h=1 wkh gh

ght

=

E

t

i

T t vih x

exp m Th x t P T t l exp m l x

We use gradient-descent: ∆vih ∆m h

X (rit − yit )ght bkt

=

η

=

−η

= = =

t

X X X ∂E ∂y t ∂g t ∂pt i

k

h

∂yit ∂gkt ∂pht ∂m h k XX X η (rit − yit ) wik gkt (δkh − ght )x t t

t

i

i

k

  X X XX t t t t t t  wik g g x wik g δkh − (r − y ) η i

t

i

i

k h

k

k

k

XX (rit − yit )(wih − yit )ght x t η t

i

P remembering that δkh = 0 for k 6= h and k wik gkt = yit . This is for softmax gating; for the case of Gaussian gating, the last term changes to: ∂pht (x t − m h ) = ∂m h 2sh2 8. Derive the update equations for the competitive mixture of experts for classification.

Let us see regression first:   X X X 1 t 2 (rit − yih ) log ght exp − L = 2 t i h

87 t yih

=

ght

=

v Tih x t exp m Th x P T l exp m l x

We use gradient-ascent to maximize the log likelihood:

=

∆v ih

=

i h P t 2 X ght exp −(1/2) i (rit − yih ) t h i (rit − yih η )x t P t P t t 2 t l gl exp −(1/2) i (ri − yil ) X t t t t η (ri − yih )fh x t

where h

i

t 2 ) ght exp −(1/2) i (rit − yih h i fht = P P t t t 2 l gl exp −(1/2) i (ri − yil )

P

Similarly,

=

∆m h

= =

i h   P t 2 X X ) exp −(1/2) i (rit − yih t t i gl (δlh − gh ) x t h η  P P t t 2 t t i (ri − yij ) j gl exp −(1/2) l   X X η  flt (δlh − ght ) x t t

 X X X t t t f g  xt η  f δlh − t

η

given that

P

X

X

t

log

l

l

h

l

l

X (fht − ght )x t

=

L=

l



t

t l fl = 1. For the case of classification, the log likelihood is

h

  X g t exp  r t log y t  i

h

ih

i

and we get the same update equations where hP i t ght exp rit log yih i hP i fht = P t t t i ri log yil l gl exp

88

13 Local Models

9. Formalize the hierarchical mixture of experts architecture with two levels. Derive the update equations using gradient descent for regression and classification.

The following is from Jordan and Jacobs 1994; notation is slightly changed to match the notation of the book.

Let us see the case of regression with a single output: y is the overall output, yi are the outputs on the first level, and yij are the outputs on the second level, which are the leaves in a model with two levels.

Similarly, gi are the gating outputs on the first level and gj|i are the outputs on the second level, that is, the gating value of expert j on the second level given that we have chosen the branch i on the first level: X gi yi y = i

yi

yij

=

X

=

v Tij x and

gj|i yij

and

j

exp m Ti x gi = P T k exp m k x

exp m Tij x gj|i = P T l exp m il x

In regression, the error to be minimized is as follows (note that we are using a competitive version here):

X X X 1 t t 2 log git E= gj|i exp − (r t − yij ) 2 t i j and using gradient descent, we get the following update equations: X t (r t − y t )x t ∆v ij = η fit fj|i t

∆m i

=

X η (fit − git )x t

∆m ij

=

η

t

X t

t t fit (fj|i − gj|i )x t

where we make use of the following posteriors: P t t 2 exp[−(1/2)(r t − yij ) ] git j gj|i t fi = P t P t t 2 t j gj|k exp[−(1/2)(r − ykj ) ] k gk

t fj|i

=

t t 2 gj|i exp[−(1/2)(r t − yij ) ] P t t 2 t l gl|i exp[−(1/2)(r − yil ) ]

89

fijt

=

P

t t 2 git gj|i exp[−(1/2)(r t − yij ) ] P t t t 2 t k gk l gl|k exp[−(1/2)(r − ykl ) ]

Note how we multiply the gating values on the path starting from the root to a leaf expert.

For the case of classification with K > 2 classes, one possibility is to have K separate HMEs as above (having single output experts), whose outputs we softmax to maximize the log likelihood: # " X X X X t t t t log gi L = rc log pc gj|i exp t

pct

i

j

c

exp yct

=

t k exp yk

P

where each yct denotes the output of one single-output HME. The more interesting case of a single multiclass HME where experts have K softmax outputs is discussed in Waterhouse and Robinson 1994.

10. For the soft decision tree we discussed in section 13.9, derive the update equations for K > 2 classes.

In each leaf we have K linear models, one for each class. For example: t yLLi = v TLLi x t , i = 1, . . . , K

which we combine as we go up the tree separately for each class, but sharing the same gating models: t yLi

Oit yit

t t + (1 − g t )yLRi gLt yLLi

=

t t g t yLi + (1 − g t )yRi

=

exp Oit P t k exp Ok

=

Given the cross-entropy XX E=− rit log yit t

i

we find the update rules using gradient descent, for example: XX t t (rit − yit )(yLi − yRi )g t (1 − g t )x t ∆m = η t

∆v LLi

=

i

X η (rit − yit )g t gLt x t t

90

13 Local Models

11. In the mixture of experts, because different experts specialize in different parts of the input space, they may need to focus on different inputs.

Discuss how dimensionality can be locally reduced in the experts.

If experts are multilayer perceptrons with few hidden units, then each performs dimensionality reduction before doing the actual task of classification or regression. If the gating model is a multilayer perceptron, it does dimensionality reduction before partitioning the input space among experts.

You should not do the following: Train a MoE as usual with linear experts, find the set of data that falls in each expert’s scope (by taking the max of gating values), train a separate PCA in each, and then use the PCA’d data once more to train a MoE with reduced number of inputs in each expert. There are two reasons why this is bad: (1) The dimensionality reduction should be coupled with the training of the overall model. (2) This performs a hard clustering of the data.

14

Kernel Machines

1. Propose a filtering algorithm to find training instances that are very unlikely to be support vectors.

Support vectors are those instances that are close to the boundaries.

So if there is an instance surrounded by a large number of instances all of the same class, it will very probably not be chosen as a support vector. So, for example, we can do an 11-nearest neighbor search for all instances and if all its 11 neighbors are of the same class, we can prune that instance from the training set.

2. In equation 14.31, how can we estimate S? We can calculate the covariance matrix of the data and use that as S. Another possibility is to have a local St for each support vector, and we can use a number of neighborhood data points to estimate it; we may need to take measures in such a case to make sure that S is not singular or decrease dimensionality is some way.

3. In the empirical kernel map, how can we choose the templates?

The easiest and most frequently used approach is to use all the training instances, and in such a case φ(·) is N-dimensional. We can decrease complexity and make the model more efficient by choosing a subset; we can use a randomly chosen subset, do some clustering, and use the cluster centers as templates (as in vector quantization), or use a subset that covers the input space well using as few instances as possible.

4. In the localized multiple kernel of equation 14.40, propose a suitable model for ηi (x|θi ) and discuss how it can be trained.

92

14 Kernel Machines

The easiest is to use a linear model: exp V Ti x ηi (x|{V i }i ) = P T k exp V k x

It is also possible to use a kernel function here as well, to have nonlinear gating.

The main problem here is that with these new parameters V i , we have a problem which is no longer convex. It becomes convex once we fix V i . So we use an alternating procedure where we start with random V i , solve for SVM parameters, then having fixed them, we to gradientdescent on the dual to optimize V i , fix them and solve for SVM again and so on. See (Gönen and Alpaydın 2008) for details.

5. In kernel regression, what is the relation, if any, between ǫ and noise variance?

The advantage of having ǫ is that if for an instance, the prediction error is less than ǫ, we can ignore it. Noise is what we have by chance and is not due to the underlying function, so we need to have ǫ just large enough to cover what can be due to noise. For example, if we know that noise is Gaussian with variance σ 2 and if we know σ 2 , then a good ǫ value is 1.96σ .

6. In kernel regression, what is the effect of using different ǫ on bias and variance? ǫ is a smoothing parameter. When it is too large, we smooth too much which reduces variance but risks increasing bias. If it is too small, the variance may be large and bias would be small.

7. Derive the kernelized version of the primal, dual, and the score functions for ranking.

The primal is Lp =

X 1 ξt kwk2 + C 2 t

subject to w T φ(x u − x v ) ξt

≥

≥

1 − ξt

0

93 The dual is Ld =

X t

αt −

1 XX t s α α K(x u − x v , x k − x l ) 2 t s

where K(x u −x v , x k −x l ) = φ(x u −x v )T φ(x k −x l ). For new test instance x, the score is calculated as g(x) =

X t

αt K(x u − x v , x)

8. How can we use one-class SVM for classification?

We can use a separate one-class SVM for each class and then combine them to make a decision. For example, for each class Ci , we fit a oneclass SVM to find parameters αit : X

αit KG (x, x t )

t

and this then can be taken as an estimator for p(x|Ci ). If the priors are more or less equal, we can simply choose the class having the largest value; otherwise we can use Bayes’ rule for classification.

9. In a setting such as that in figure 14.12, use kernel PCA with a Gaussian kernel.

See figure 14.1.

10. Let us say we have two representations for the same object and associated with each, we have a different kernel. How can we use both to implement a joint dimensionality reduction using kernel PCA? There are two possibilities: (1) For each kernel, we can do a separate kernel PCA and get a lower-dimensional vector and then at the end, concatenate these vectors, or (2) We can take a sum of the two kernel functions and then use kernel PCA once on the sum. When taking the sum, we need to make sure that kernel matrices contain comparable values.

94

14 Kernel Machines

(a) Gaussian kernel in the x space 1 1 0

−1

−1 −1

−0.5

0

0.5

1

(b) Linear kernel in the z space 2

0 −1 −2 0.9

0.95

Figure 14.1

1

1.05

1.1

1.15

Kernel PCA applied to a Gaussian kernel.

15

Graphical Models

1. With two independent inputs in a classification problem, that is, p(x1 , x2 |C) = p(x1 |C)p(x2 |C), how can we calculate p(x1 |x2 , C)? Derive the formula for p(xj |Ci ) ∼ N (µij , σij2 ).

This is a trick question. If x1 and x2 are independent, p(x1 |x2 , C) = p(x1 |C). If the two are dependent, p(x1 , x2 |C) is a bivariate density and we have p(x1 |x2 , C) =

p(x1 , x2 |C) p(x2 |C)

When the two are independent, the bivariate is a product of two univariates of which one cancels with the denominator.

2. For a head-to-head node, show that equation 15.10 implies P (X, Y ) = P (X) · P (Y ).

We know that P (X, Y , Z) = P (Z|X, Y )P (X, Y ), and if we also know that P (X, Y , Z) = P (X)P (Y )P (Z|X, Y ), we see that P (X, Y ) = P (X)P (Y ).

3. In figure 15.4, calculate P (R|W ), P (R|W , S), and P (R|W , ∼S).

P (R|W )

= =

P (R|W , S)

=

P (R|W , ∼S)

=

P P (R, W ) S P (R, W , S) P P = P (W ) R S P (R, W , S) P S P (R)P (S)P (W |R, S) P P R S P (R)P (S)P (W |R, S) P (R)P (S)P (W |R, S) P (R, W , S) = P P (W , S) R P (R)P (S)P (W |R, S) P (R, W , ∼S) P (R)P (∼S)P (W |R, ∼S) = P P (W , ∼S) R P (R)P (∼S)P (W |R, ∼S)

96

15 Graphical Models

4. In equation 15.30, X is binary. How do we need to modify it if X can take one of K discrete values?

Let us say there are j = 1, . . . , K states. Then, keeping the model linear, we need to parameterize each by a separate w j and use softmax to map to probabilities: Pk

i=1 wji Ui + wj0 Pk l=1 exp i=1 wli Ui + wl0

exp

P (X = j|U1 , . . . , Uk , {wji }) = PK

5. Show that in a directed graph where the joint distribution is written as P equation 15.12, x p(x) = 1.

The terms cancel when we sum up over all possible values because these are probabilities. Let us, for example, take figure 14.3a: P (X, Y , Z)

XXX X

Y

Z

P (X, Y , Z)

= =

P (X)P (Y |X)P (Z|X) XXX P (X)P (Y |X)P (Z|X)

=

XX

P (X)P (Y |X)

=

XX

P (X)P (Y |X)

=

XX

P (X)P (Y |X)

=

XX X

Y

P (X)P (Y |X)

=

X

P (X)

X

X

X

X

X

Y

Y

Y

Y

Z

X Y

X

P (Z|X)

Z

X P (Z, X) Z

P (X)

P (X) P (X)

P (Y |X) =

X X

P (X) = 1

6. Draw the Necker cube as a graphical model defining links to indicate mutually reinforcing or inhibiting relations between different corner interpretations.

We are going to have nodes corresponding to corners and they take values depending on the interpretation; there will be positive, enforcing, excitatory connections between corners that are part of the same interpretation, and negative, inhibitory connections between corners that are part of different interpretations (see figure 15.1).

97

Figure 15.1 Two different interpretations of the Necker cube.

7. Write down the graphical model for linear logistic regression for two classes in the manner of figure 15.7.

It is shown in figure 15.2 where q corresponds to priors, and µ and Σ parameterize p(x|C). We can add priors for these if we like to get Bayesian logistic classification, as we discuss in chapter 16.

Figure 15.2

Graphical model for classification.

8. Propose a suitable goodness measure that can be used in learning graph structure as a state-space search. What are suitable operators?

We need a score function that is the sum of two parts, one quantifying a goodness of fit, that is, how likely is the data given the model, and one quantifying the complexity of the graph, to alleviate overfitting.

In measuring complexity, we must take into account the total number of nodes and the number of parameters needed to represent the conditional probability distributions. For example, we should try to have nodes with as few parents as possible. Possible operators are there to add/remove an edge and add/remove a hidden node.

98

15 Graphical Models

9. Generally, in a newspaper, a reporter writes a series of articles on successive days related to the same topic as the story develops. How can we model this using a graphical model?

We talk about modeling temporal dependencies in chapter 15 on hidden Markov models. Basically for each day, we have a graphical model where a hidden node or nodes for the topic causes a certain word distribution for the article of that day. The topic nodes of successive days will be linked and this will create a dependency between them. So the topic today causes the words of today’s article and also has an effect on tomorrow’s topic.

16

Hidden Markov Models

1. Given the observable Markov model with three states, S1 , S2 , S3 , initial probabilities Π = [0.5, 0.2, 0.3]T and transition probabilities 

0.4  A =  0.2 0.1

0.3 0.6 0.1

 0.3  0.2  0.8

generate 100 sequences of 1,000 states.

The Matlab code is given in ex15_1.m.

2. Using the data generated by the previous exercise, estimate Π, A and compare with the parameters used to generate the data.

The Matlab code is given in ex15_2.m. Its output is Π̂ Â

= =

[0.54, 0.20, 0.26]T  0.3987 0.3005   0.2057 0.5937 0.0998 0.1001

 0.3008  0.2005  0.8001

3. Formalize a second-order Markov model. What are the parameters?

How can we calculate the probability of a given state sequence? How can the parameters be learned for the case of a observable model?

100

16 Hidden Markov Models

In a second-order MM, the current state depends on the two previous states: aijk ≡ P (qt+2 = Sk |qt+1 = Sj , qt = Si ) Initial state probability defines the probability of the first state: πi ≡ P (q1 = Si ) We also need parameters to define the probability of the second state given the first state: θij ≡ P (q2 = Sj |q1 = Si ) Given a second-order observable MM with parameters λ = (Π, Θ, A), the probability of an observed state sequence is

P (O = Q|λ)

T Y

=

P (q1 )P (q2 |q1 )

=

πq1 θq2 q1 aq3 q2 q1 aq4 q3 q2 · · · aqT qT −1 qT −2

t=3

P (qt |qt−1 , qt−2 )

The probabilities are estimated as proportions: =

P

θ̂ij

=

P

âijk

=

π̂i

k k 1(q1 = Si )

K

k k k 1(q2 = Sj and q1 = Si ) P k k 1(q1 = Si ) P PT k k k k t=3 1(qt = Sk and qt−1 = Sj and qt−2 = Si ) P PT k k k t=3 1(qt−1 = Sj and qt−2 = Si )

4. Show that any second- (or higher-order) Markov model can be converted to a first-order Markov model.

In a second-order model, each state depends on the two previous states. We can define a new set of states corresponding to the Cartesian product of the original set of states with itself. A first-order model on this new N 2 states corrresponds to a second-order model on the original N states.

101 5. Some researchers define a Markov model as generating an observation while traversing an arc, instead of on arrival to a state. Is this model any more powerful than what we have discussed?

Similar to the case of the previous exercise, if the output depends not only on the current state but also on the next state, we can define new states corresponding to this pair and have the output generated by this (joint) state.

6. Generate training and validation sequences from an HMM of your choosing. Then train different HMMs by varying the number of hidden states on the same training set and calculate the validation likelihoods. Observe how the validation likelihood changes as the number of states increases. ex15_6a.m samples from an HMM, and ex15_6b.m learns the parameters given a sample.

7. If in equation 16.38 we have multivariate observations, what will be the M-step equations?

If we have d-dimensional Ot ∈ ℜd , drawn from d-variate Gaussians with their mean vectors and covariance matrices p(Ot |qt = Sj , λ) ∼ N (µj , Σj ) the M-step equations are P t γt (j)Ot P µ̂j = t γt (j) P T t γt (j)(Ot − µ̂j )(Ot − µ̂j ) P Σ̂j = t γt (j) 8. Consider the urn-and-ball example where we draw without replacement. How will it be different?

If we draw without replacement, then at each iteration, the number of balls change, which means that the observation probabilities, B, change. We will no longer have a homogenous model.

9. Let us say at any time we have two observations from two different alphabets; for example, let us say we are observing the values of two currencies every day. How can we implement this using HMM?

102

16 Hidden Markov Models

In such a case, what we have is a hidden state generating two different observations. That is, we have two B, each trained with its own observation sequence. These two observations then need to be combined to estimate A and π .

10. How can we have an incremental HMM where we add new hidden states when necessary?

Again, this is a state space search. Our aim may be to maximize validation log likelihood and an operator allows us to add a hidden state.

We do then a forward search. There are structure learning algorithms for the more general case of graphical models, as we will discuss in chapter 16.

17

Bayesian Estimation

1. For the setting of figure 17.3, observe how the posterior changes as we change N, σ 2 , and σ02 .

Matlab code that generates this is ex14_3.m. See figures 17.1, 17.2, 17.3, and 17.4.

2. Let us denote by x the number of spam emails I receive in a random sample of n. Assume that the prior for q, the proportion of spam emails is uniform in [0, 1]. Find the posterior distribution for p(q|x). x is binomial and unless n is small, it can be approximated by a Gaussian. Uniform prior is a flat prior and the posterior will have the same shape as the likelihood, and peaks at the same point.

3. As above, except that assume that p(q) ∼ N (µ0 , σ02 ). Also assume n is large so that you can use central limit theorem and approximate binomial by a Gaussian. Derive p(q|x).

In this case both the likelihood and prior are Gaussian and we can use equations 17.6 and 17.7.

4. What is Var(r ′ ) when the maximum likelihood estimator is used? Compare it with equation 17.25.

We have maximum likelihood when α goes to 0 in equation 17.23, which we can then plug in equation 17.25.

5. In figure 17.10, how does the fit change when we change s 2 ?

As usual, s is the smoothing parameter and we get smoother fits as we increase s.

104

17

Bayesian Estimation

1.4 1.2 1 ← p(µ|X)

0.8 0.6 0.4

p(µ)→

0.2 0 0

Figure 17.1 N (4, 0.82 ).

← p(x|µ) 2

4

6

8

10

20 data points are drawn from p(x) ∼ N (6, 1.52 ), prior is p(µ) ∼

4 3.5 3 2.5 2 1.5 1 0.5 0 0

← p(µ|X) p(µ)→ 2

4

6

← p(x|µ) 8 10

Figure 17.2 200 data points are drawn from p(x) ∼ N (6, 1.52 ), prior is p(µ) ∼ N (4, 0.82 ).

0.7 0.6 0.5 0.4

p(µ)→

0.3 0.2 ← p(x|µ) 0.1 0 0

Figure 17.3 N (4, 0.82 ).

2

4

6

8

10

20 data points are drawn from p(x) ∼ N (6, 4.52 ), prior is p(µ) ∼

105 6 5 4 3 2 1

← p(µ|X) p(µ)→

0 0

2

4

6

← p(x|µ) 8 10

Figure 17.4 20 data points are drawn from p(x) ∼ N (6, 4.52 ), prior is p(µ) ∼ N (4, 0.082 ).

6. Propose a filtering algorithm to choose a subset of the training set in Gaussian processes.

One nice thing we have with Gaussian processes is that we can also calculate the variance at a certain point. What we can do is the following: For any instance from the training set, we can calculate the leave-one-out estimate there and check if the actual output is in, for example, the 95 percent prediction interval. If it is, this means that we do not need that instance and it can be left out. Those at the end that cannot be pruned will be just like the support vectors in a kernel machine, namely, those instances that are stored and needed, to bound the total error of the fit.

7. Active learning is when the learner is able to generate x itself and ask a supervisor to provide the corresponding r value during learning one by one, instead of passively being given a training set. How can we implement active learning using Gaussian processes? (Hint: Where do we have the largest uncertainty?)

This is just like the previous exercise, except that we add instead of prune. Using the same logic, we can see that we need instances where the prediction interval is large. Given the variance as a function of x, we search for its local maxima. In the case of a Gaussian kernel, we expect points that are distant from training data to have high variance, but this need not be the case for all kernels. While searching, we need to make sure that we do not go out of the valid input bounds.

8. Let us say we have a set of documents where for each document, we

106

17

Bayesian Estimation

have one copy in English and one in French. How can we extend latent Dirichlet allocation for this case?

In the two documents, the actual words (and their number) and the topic to word mappings may be different but the topics and topic distributions are the same. So we have the α node and π shared, but from the same π , we have two different blocks of word plates and their different associated w.

18

Combining Multiple Learners

1. If each base-learner is iid and correct with probability p > 1/2, what is the probability that a majority vote over L classifiers gives the correct answer?

It is given by a binomial distribution (see figure 18.1). The code is in ex17_1.m.

P (X ≥ ⌊L/2⌋ + 1) =

L X

i=⌊L/2⌋+1

L i

!

pi (1 − p)L−i

2. In bagging, to generate the L training sets, what would be the effect of using L-fold cross-validation instead of bootstrap?

With L-fold cross-validation, each training set has L − 1 parts of the original set and different sets share L − 2 parts. For example, for L = 10, each training set contains 90 percent of the dataset whereas this percentage is 63.2 percent for bagging. Different training sets share 80 percent of the data which is higher than the percentage of data shared by two bootstrap samples.

3. Propose an incremental algorithm for learning error-correcting output codes where new two-class problems are added as they are needed to better solve the multiclass problem.

At each iteration, we look at the confusion matrix and find the pair of classes that are most confused. We then generate a new dichotomizer, that is, add a new column to the ecoc matrix where one of the most confused classes is labeled as +1, the other is labeled as −1 and all other classes are labeled as 0. We continue adding until we overfit over a validation set.

108

18 Combining Multiple Learners

1 p=0.9

p=0.8

0.9

p=0.7

p=0.6

0.8

prob that majority is correct

0.7

0.6 p=0.5 0.5

0.4

0.3

0.2 p=0.4 0.1

0

p=0.2

0

p=0.3 10

20

30 No of base classifiers

40

50

60

Figure 18.1 Probability that a majority vote is correct as a function of the number of base-learners for different p. The probability increases only for p > 0.5.

4. In the mixture of experts architecture, we can have different experts use different input representations. How can we design the gating network in such a case?

There are different possibilities: The gating network can see a concatenation of all input representations, or a subset. In any case, the input dimensionality may be high and one may need to either reduce dimensionality beforehand, or train a multilayer perceptron for gating. Note that the task of the gating model is easier, so a single representation may suffice to decide between experts.

5. Propose a dynamic regressor selection algorithm.

It can work just as dynamic classifier selection except that in calculating competence, we use mean square error instead of misclassification error.

6. What is the difference between voting and stacking using a linear perceptron as the combiner function?

109 If the voting system is also trained, the only difference would be that with stacking, the weights need not be positive or sum up to 1, and there is also a bias term. Of course, the main advantage of stacking is when the combiner is nonlinear.

7. In cascading, why do we require θj+1 ≥ θj ?

Instances on which the confidence is less than θj have already been filtered out by dj ; we require the threshold to increase so that we can have higher confidences.

8. To be able to use cascading for regression, during test, a regressor should be able to say if it is confident of its output. How can we implement this?

Just as in a dynamic classifier/regressor selection algorithm, we need to be able to say if for a given input, a regressor in the cascade can be trusted. During validation (not training), we then find for which instances the regressor has error less than a certain threshold and where it makes too much error. We can then use this data to train another binary classifier (for example, k-nn) as the confidence estimator for that regressor.

9. How can we combine the results of multiple clustering solutions?

The easiest is the following: Let us take any two training instances.

Each clustering solution either places them in the same cluster or not; denote it as 1 and 0. The average of these counts over all clustering solutions is the overall probability that those two are in the same cluster (Kuncheva 2004).

10. In section 18.10, we discussed that if we use a decision tree as a combiner in stacking, it works both as a selector and a combiner. What are the other advantages and disadvantages?

A tree uses only a subset of the classifiers and not the whole. Using the tree is fast, and we need to evaluate only the nodes on our path, which may be short. See Ulaş et al. 2009 for more detail. The disadvantage is that the combiner cannot look at combinations of classifier decisions (assuming that the tree is univariate). Using a subset may also be harmful; we do not get the redundancy we need if some classifiers are faulty.

19

Reinforcement Learning

1. Given the grid world in figure 19.12, if the reward on reaching on the goal is 100 and γ = 0.9, calculate manually Q∗ (s, a), V ∗ (S), and the actions of optimal policy.

The Matlab code given in ex18_1.m can be used.

2. With the same configuration given in exercise 1, use Q learning to learn the optimal policy.

The code is given in ex18_1.m.

3. In exercise 1, how does the optimal policy change if another goal state is added to the lower-right corner? What happens if a state of reward −100 (a very bad state) is defined in the lower-right corner?

The Matlab code given in ex18_3.m has both a goal state and a bad state with negative reward. We see that the optimal policy navigates around the bad state.

4. Instead of having γ < 1, we can have γ = 1 but with a negative reward of −c for all intermediate (nongoal) states. What is the difference?

The Matlab code is given in ex18_4.m where γ = 1 and c = −1. The Q values converge to reward minus the number of steps to the goal.

5. In exercise 1, assume that the reward on arrival to the goal state is normal distributed with mean 100 and variance 40. Assume also that the actions are also stochastic in that when the robot advances in a direction, it moves in the intended direction with probability 0.5 and there is a 0.25 probability that it moves in one of the lateral directions.

Learn Q(s, a) in this case.

The Matlab code is given in ex18_5.m.

112

19 Reinforcement Learning

6. Assume we are estimating the value function for states V (s) and that we want to use TD(λ) algorithm. Derive the tabular value iteration update.

The temporal error at time t is δt = rt+1 + γV (st+1 ) − V (st ) All state values are updated as V (s) ← V (s) + ηδt et (s), ∀s where the eligibility of states decay in time: et (s) =

(

1 γλet−1 (s)

if s = st otherwise

7. Using equation 19.22, derive the weight update equations when a multilayer perceptron is used to estimate Q. Let us say for simplicity we have one-dimensional state value st and one-dimensional action value at and let us assume a linear model: Q(s, a) = w1 s + w2 a + w3 We can update the three parameters w1 , w2 , w3 using gradient-descent (equation 16.21): =

∆w1

=

∆w2

=

∆w3

η rt+1 + γQ(st+1 , at+1 ) − Q(st , at ) st

η rt+1 + γQ(st+1 , at+1 ) − Q(st , at ) at

η rt+1 + γQ(st+1 , at+1 ) − Q(st , at )

In the case of a multilayer perceptron, only the last term will differ to update the weights on all layers.

In the case of Sarsa(λ), e is three-dimensional: e1 for w1 , e2 for w2 , and e3 for w0 . We update the eligibilities (equation 16.23): et1 et2 et3

=

=

=

1 γλet−1 + st

2 γλet−1 + at

3 γλet−1

113 and we update the weights using the eligibilities (equation 16.22): ∆w1 ∆w2 ∆w3

=

=

=

η rt+1 + γQ(st+1 , at+1 ) − Q(st , at ) et1

η rt+1 + γQ(st+1 , at+1 ) − Q(st , at ) et2

η rt+1 + γQ(st+1 , at+1 ) − Q(st , at ) et3

8. Give an example of a reinforcement learning application that can be modeled by a POMDP. Define the states, actions, observations, and reward.

An example is a soccer playing robot. A robot has sensors, typically a camera, but cannot fully observe its state, that is, its exact position in the field. Using the camera, the robot can see only a part of the field, maybe the goals, the boundaries or beacons, but this is not enough to pinpoint its location accurately. It has actuators for action, to move in the field. The goal is to win the game.

9. For the grid world of figure 19.12, explain how the policy gradient method can be used.

We will have a policy network with four outputs corresponding to the four actions with state coordinates as its input implemented as a multilayer neural network; we can also have a value network using the same input. The two networks can then be trained using the actorcritic algorithm of Equations 19.32 and 19.33.

10. In AlphaGo, the value and policy networks are separate; in AlphaGo Zero, there is one network with two sets of outputs. Why do you think one is better than two?

The two networks learn different aspects of the same problem from the same input, so it makes sense to think that they will learn similar features especially in early layers; a single network will be equivalent to pooling data. The single network is deeper, so I’m not sure but it may also be cheaper and faster than two networks.

11. In Tesauro’s original TD-Gammon, hidden units are fully connected to all inputs. Design a convolutional neural network for backgammon like the one used in AlphaGo.

114

19 Reinforcement Learning

In backgammon, the positions though organized as a horse shoe are basically linearly ordered. We can for example have a one-dimensional convolution filters that check for neighboring positions, where for each position we have two values for white and black pieces. There cannot be any stride or pooling. We can have layers of convolutions followed by fully connected layer(s) to learn a value or policy network, or both. TD-Gammon uses specialized input features, it will be interesting to see if a convolutional neural network trained from raw features can be as good, or even better.

20

Design and Analysis of Machine Learning Experiments

1. In a two-class problem, let us say we have the loss matrix where λ11 = λ22 = 0, λ21 = 1 and λ12 = α. Determine the threshold of decision as a function of α.

The risk of choosing the first class is 0 · P (C1 |x) + α · P (C2 |x) and the risk of choosing the second class is 1 · P (C1 |x) + 0 · P (C2 |x) (section 3.3). We choose C1 if the former is less than the latter and given that P (C2 |x) = 1 − P (C1 |x), we choose C1 if P (C1 |x) >

α 1+α

That is, varying the threshold decision corresponds to varying the relative cost of false positives and false negatives.

2. We can simulate a classifier with error probability p by drawing samples from a Bernoulli distribution. Doing this, implement the binomial, approximate, and t tests for p0 ∈ (0, 1). Repeat these tests at least 1,000 times for several values of p and calculate the probability of rejecting the null hypothesis. What do you expect the probability of reject to be when p0 = p?

The Matlab code is given in ex19_2.m. When p0 = p, we expect the probability of reject to be equal to α, maximum allowed type I probability (see figure 20.1).

3. Assume xt ∼ N (µ, σ 2 ) where σ 2 is known. How can we test for H0 : µ ≥ µ0 vs. H1 : µ < µ0 ?

Under H0 √ N(m − µ0 ) ∼Z z= σ

116

20 Design and Analysis of Machine Learning Experiments

1

0.9

0.8

Prob of accepting p0<=0.7

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0 0.1

0.2

0.3

0.4

0.5 p0

0.6

0.7

0.8

0.9

Figure 20.1 Probability of acceptance of the three tests of H0 : p ≤ 0..7 for different values of p (‘+’: Binomial test, ‘x’: Approximate normal test, ‘o’: t test; the three tests agree for N = 250 and 500 iterations). Note that for p = 0.7, the probability of acceptance is 0.95, which is expected because α = 0.05.

We accept H0 if z ∈ (−zα , ∞).

4. The K-fold cross-validated t test only tests for the equality of error rates.

If the test rejects, we do not know which classification algorithm has the lower error rate. How can we test whether the first classification algorithm does not have higher error rate than the second one? Hint: We have to test H0 : µ ≤ 0 vs. H1 : µ > 0.

The test statistic is √ k(m − 0) t= ∼ tK−1 s

We accept H0 if t ∈ (−∞, tα,K−1 ). If we accept, we cannot say anything but if we reject, we can say that the second algorithm has less error rate.

5. Show that the total sum of squares can be split into between-group sum of squares and within-group sum of squares as SST = SSb + SSw .

117 Start with SST = continue.

P P j

i (Xij − m)

2

=

P P j

i (Xij − mj + mj − m)

2

and

6. Use the normal approximation to the binomial for the sign test.

Under the null hypothesis that the two are equally good, we have p = 1/2 and over N datasets, we expect the number of wins X to be approximately Gaussian with µ = pN = N/2 and σ 2 = p(1 − p)N = N/4. e−N/2 ) > α.

If there are e wins, we reject if P (X < e) > α, or if P (Z < √ N/4

7. Let us say we have three classification algorithms. How can we order these three from best to worst?

Given three algorithms, if we have a linear ordering, such as 2 has significantly less error than 1 and 3, 1 has significantly less error than 3, then we have a perfect order: 2 < 1 < 3. Unfortunately, this is rarely the case; frequently we have one algorithm whose error rate is significantly different from the other two whereas there is no significant difference between the two. This implies that to be able to define an order, we need an additional criterion, and the best candidate for this is complexity (time and/or space). That is we order algorithms in terms of error rates by looking at the results of pairwise tests and two algorithms whose error rates are not statistically significantly different are ordered in terms of complexity. So let us say 1 has significantly less error than 2 and 3, there is no difference between 2 and 3 and 3 is simpler than 2, then we have the ordering: 1 < 3 < 2.

For a generalization of this idea to an arbitrary number of algorithms, see O. T. Yıldız, E. Alpaydın. 2006. “Ordering and Finding the Best of K > 2 Supervised Learning Algorithms.” IEEE Transactions on Pattern Analysis and Machine Inteligence 28: 392–402.

8. If we have two variants of algorithm A and three variants of algorithm B, how can we compare the overall accuracies of A and B taking all their variants into account?

We can use contrasts (Montgomery 2005). Basically, what we would be doing is comparing the average of the two variants of A with the average of the three variants of B. 9. Propose a suitable test to compare the errors of two regression algorithms.

118

20 Design and Analysis of Machine Learning Experiments

In regression, we minimize the sum of squares which is a measure of variance, which we know is chi-squared distributed. Since we use the F test to compare variances (as we did in ANOVA), we can also use it to compare the squared errors of two regressors.

10. Propose a suitable test to compare the expected rewards of two reinforcement learning algorithms.

I am not sure whether we can derive a distribution for the expected reward but we can always use a nonparametric test. We run both algorithms a number of times where due to randomness in exploration, they will return different rewards in different runs, and then we can use Sign test or the Wilcoxon rank test to check if the difference between the rewards of two algorithms is small enough to be due to chance or statistically significant.

The title documents are available in full on the following links Title: SOLUTIONS Manual for Introduction to Machine Learning 4th Edition;Adaptive Computation & Machine Learning series by Ethem Alpaydin - Complete Elaborated & Latest. ALL Chapters (1-20) Included & Updated Link I: https://drive.google.com/file/d/16F5EY4aoiPU9P9UFzt3JLrwVcEyCH0Ae/view?usp=sharing

Title: EBOOK for Introduction to Machine Learning 4th Edition;Adaptive Computation & Machine Learning series by Ethem Alpaydin - Complete Elaborated & Latest. ALL Chapters (1-20) Included & Updated Link II: https://drive.google.com/file/d/1CwGlY3iAwWGa3HWTdDjARR7LdotgvRnV/view?usp=sharing