CS 412 Introduction To Data Mining - University of Illinois-2020 Midterm Exam Q&A

downloaded from

Scholarfriends

https://scholarfriends.com Downloaded by: writerforacademic@gmail.com | https://scholarfriends.com/singlePaper/307199/cs-412-introduction-to-data-mining-universityof-illinois-2020-midterm-exam-qa Distribution of this document is illegal.

Midterm Exam • There are 5 problems in this exam, worth 70 points in total.

• You must not communicate with other students during this test.

• Books or notes are not allowed. You can use: one sheet of reference (double sided) • Electronic devices except calculators are not allowed.

• You cannot use your mobile phone as calculator.

• The duration of this exam is 75 minutes.

• For each question, please show your work for full credit. Necessary calculation steps and reasoning are required.

• For the questions that need standard deviation/variance, calculate these metrics for population.

• Wait for instructions before turning this page.

• Please leave the table below empty

1. Fill in your information: Full Name: NetID:

Questions

Points

Q1 Q2 Q3 Q4 Q5 Total

1

Question 1 (20 points): Get to Know your Data A. [5] The least and greatest number in a list of 7 integers are 2 and 20 respectively. The median and mode of list are 6 and 3 respectively. Find out which of the following options can be mean of the list. (a) 4 (b) 7 (c) 6.85 (d) 6.71 Note there may be more than one correct option.

B. [5] Consider the following data on age distribution of a population of 3594 people. age 1-5 6-15 16-20 21-50 51-80 81-110

frequency 200 450 300 1500 700 444

(a) Compute the approximate median age for the given population. (b) What special case will result in maximum error in the approximate median you calculated above? Assume that the frequency for each interval remains unchanged.

C. [5] Consider the following data on two persons Jack and Jill.

Height and Weight are given in ft and lbs respectively.

Name Jack Jill

Age 24 14

Height 7 4

Weight 210 140

Age , Height and Weight are ordinal attributes with following interval states. (a) Age : [3-18], [19-40], [41-80] (b) Height : [3-5], [6-9] (c) Weight : [70-110], [110-150], [150-190], [190-230] Compute the Manhattan distance between two persons. Show the steps of your calculation.

D. [5] The following table contains the medical record of two patients (John and Jane) on six lab tests. Calculate the Dissimilarity between John and Jane based on these records (P & N mean positive and negative test results, respectively). Write down the steps.

John Jane

test-1 P P

test-2 N N

test-3 P P

Solution.

2

test-4 N N

test-5 N P

test-6 N N

A. 2, 3, 3, 6, x, y, 20 . x and y has to be at least 7 and 8 respectively. B is the correct answer. 7 can be the mean of the list B. (a) Approximate median = 21 + 1797−950 ∗ (50 − 21) = 37.38 1500 (b) When all 1500 people in the median interval have age 21. 37.38 - 21 = 16.38 C. Jack : (2 , 2, 4) , Jill : (1, 1, 2) [replacing ordinal attribute by rank] Each ordinal attributes has different number of states. So they need to be normalized. Jack : ( 12 , 1, 1) Jill : (0, 0, 31 ) Manhattan Distance : 13 6 D. these are asymmetric binary attributes, so we need to draw the contingency table and calculate q, r, and s.

2 = 0.66 2+0+1

3

Question 2 (15 points): Data Preprocessing A. [3] Consider the following data for the attribute price: 8, 9, 15, 16, 21, 21, 24, 26, 27, 30, 30, 34 Use smoothing by bin means to smooth this data using a equi-depth bins and a bin depth of 4.

B. [12] Consider the following data for two attributes A and B: A 21 25 42 43 57 59

B 65 79 75 99 87 81

(a) [5] Normalize attribute ‘A’ based on z-score normalization. (b) [5] Calculate the correlation coefficient. Are these two attributes positively correlated or negatively correlated? (c) [2] Compute the covariance of the two attributes.

Solution.

A. Bin means: Bin 1: 8+9+15+16 = 12 4 Bin 2: 21+21+24+26 = 23 4 Bin 3: 27+30+30+34 = 30 4 Smoothed data: Bin 1 = 12, 12, 12, 12 Bin 2 = 23, 23, 23, 23 Bin 2 = 30, 30, 30, 30 Pn

A

i B. (a) Mean µ = i=1 = 247 = 41.167 n Pn6 Pn 2 1 1 2 (11409 − 247 ) = 248.167 Variance s = n−1 ( i=1 A2i − n1 ( i=1 Ai )2 ) = 6−1 6 √ Standard deviation s = 248.167 = 15.753 z-score = Ais−µ

A

z-score

21

21−41.167 = −1.280 15.753

25

25−41.167 = −1.026 15.753

42

42−41.167 = 0.053 15.753

43

43−41.167 = 0.116 15.753

57

57−41.167 = 1.005 15.753

59

59−41.167 = 1.132 15.753

4

(b)

A 21 25 42 43 57 59

B 65 79 75 99 87 81

AB 1365 1975 3150 4257 4959 4779

A2 441 625 1764 1849 3249 3481

B2 4225 6241 5625 9801 7569 6561

Correlation coefficient P P P n AB − A B p r= P P P P (n A2 − ( A)2 )(n B 2 − ( B)2 ) 2868 6(20, 485)–(247x486) = =p 5413.27 ((6(11, 409)–(2472))(6(40, 022)–4862)) = 0.5298 The correlation coefficient is 0.5298 or 52.98%, which means the variables have a moderate positive correlation.

Pn

(c) Mean/expectation of A = P i=1 n

Ai

= 41.167

n i=1 Bi

Mean/expectation of B = = 81 n Covariance - P n cov(A, B) = n1 i=1 (Ai − E(A))(Bi − E(B)) = 79.667

5

Question 3 (10 points): Apriori algorithm Consider the following transaction database: TransactionID T1 T2 T3 T4 T5

Items I, P, A, D, B, C D, A, E, F C, D, B, E B, A, C, K, D A, G, T, C

Suppose that minimum support is set to 0.6 and minimum confidence to 0.8.

A. [3] List all frequent 1-itemsets together with their support.

B. [2 + 2] List all candidate frequent 2-itemsets together with their support. What will be the pruned frequent 2-itemsets after applying Apriori?.

C. [3] List all frequent 3-itemsets based on the frequent 2-itemsets from the previous question using Apriori. Also, list the support for each pattern.

Solution.

A. Frequent 1-itemset = {A:4, B:3, C:4, D:4} B. Candidate frequent 2-itemset = {AB:2, AC:3, AD:3, BC:3, BD:3, CD:3} Pruned frequent itemsets = {AC:3, AD:3, BC:3, BD:3, CD:3} C. Candidate frequent 2-itemset = {BCD:3}, {ACD:2} Frequent 3-itemset = {BCD:3}

6

Question 4 (10 points): Frequent Pattern Mining Shown below is a FP-tree from a transactional database. Let min sup =2. {} f:4

c:1

c:3

b:1

a:3

b:1 p:1

m:2

b:1

p:2

m:1

d:2 g:2 A. [3] Find all the frequent 1-itemsets with their supports.

B. [7] Find the Conditional Databases of each frequent 1-itemset.

Solution.

A. f:4, c:4, a:3, b:3, p:3, m:3, d:2, g:2

B.

c:3 a:3 b:3 p:3 m:3 d:2 g:2

f:3 fc:3 f:1, c:1, fca:1 fcam:2, cb:1 fca:2, fcab:1 fcamp:2 fcampd:2

7

Question 5 (15 points): Pattern Evaluation Consider the following frequent patterns and their corresponding support.

Itemset {bread, milk, egg} {bread, milk} {milk, egg} {bread, egg} {bread} {milk} {egg}

Support 2 3 3 4 6 5 6

Additionally, consider the following contingency table derived from the same transaction database.

bread ¬ bread sum(col.)

egg 4 2 6

¬ egg 2 1 3

sum(row) 6 3 9

A. [4] What is the lift of (bread, egg)?

B. [4] What is the χ2 of (bread, egg)?

C. [4] What is the Kulczynski of (bread, egg)?

D. [2] Which of the above measures are null-invariant?

E. [1] Are bread and egg correlated (positively or negatively)?

Solution.

A.

4 9 6 6 × 9 9

= 1.0 2

2

2

2

) + (2−2) + (2−2) + (1−1) = 0.0 B. (4−4) 4 2 2 1 4

4

C. 12 × ( 96 + 96 ) = 0.67 9

4 9 6 9

9

4 9 6 9

D. max( , ) = 0.67 E. Kulczynski and MaxConf are null-invariant F. As the lift of (bread, egg) is 1.0, bread and egg are not correlated

8