Foundations of Algorithms, Fifth Edition [5th Ed] Richard Neapolitan Instructor Solution Manual

SOLUTION MANUAL

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

Section 1.1

1) Write an algorithm that finds the largest number in a list (an array) of n numbers.

Input: positive integer n, list of numbers S indexed from 1 to n Output: the maximum element in the list S number findMax(int n, const keytype S[ ]) { index i; number max = S[1]; for(i =2; i<=n; i++) if(S[i] > max) max = S[i]; return max; }

2) Write an algorithm that finds the m smallest numbers in a list of n numbers.

Input: positive integer n, list of numbers S indexed from 1 to n, empty list of numbers Small indexed from 1 to m.

Output: the list Small with the smallest m numbers in it. void find_m_Smallest(int n, const keytype S[ ], keytype Small[]) { index i, j, k; Small[1] = S[1]; for(i =2; i<=n; i++) { if (j <=m) find index j where S[i] needs to be inserted in Small, such that\ Small stays sorted in non-decreasing order; for (k = m-1; k>=j, k--) Small[k+1) = Small[k]; //shift right Small[j] = Small[i]; //insert S[i] } }

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

3) Write an algorithm that prints out all the subsets of three elements of a set of n elements. The elements of this set are stored in a list that is the input to the algorithm.

Input: positive integer n, list of numbers S indexed from 1 to n Output: none (the results are simply printed) void nC3(int n, const keytype S[ ]) { index i, j, k; for (i =1; i<=n; i++) for (j=i+1; j<=n;, j++) for (k=j+1; k<=n; k++) cout <<S[i] <<S[j] <<S[k] <<endl;

4) Write an Insertion Sort algorithm (Insertion Sort is discussed in Section 7.2) that uses Binary Search to find the position where the next insertion should take place. void insertionsort (int n, keytype S[]) { index i , j ; index &location keytype x; for (i = 2; i <= n; i++) { x = S[ i ]; binsearch(i, S, x, location); for (j=i-1; j>=location; j--) S[ j + 1] = S[ j ]; //shift right S[ j ] = x; //insert S[i] in final position } } The function binsearch is the one in Algorithm 1.5, but modified so that, if the search is unsuccessful, location is the index of the first element greater than x: void binsearch (int n , const keytype S [ ] , keytype x , index& location) { index low , high , mid ; bool found = false low = 1; hi gh = n; location = 0; while ( low <= hi gh && !found) { mid = (low + high)/2 ; if (x == S [mid ]) location = mid ; elseif (x < S [mid ])

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

hi gh = mid 􀀀 1; else low = mid + 1 ; } }

5) Write an algorithm that finds the greatest common divisor of two integers.

Input: two integers a and b Output: the greatest common divisor of a and b int gcd(int a, int b) { if(b == 0) return a; return gcd(b, a%b) ; }

6) Write an algorithm that finds both the smallest and largest numbers in a list of n numbers. Try to find a method that does at most 1.5n comparisons of array items.

If we simply write one loop to find the minimum and one to find the maximum, there will be 2(n-1) comparisons. We can reduce this number by comparing pairs of numbers; this finds max(a,b) and min(a,b) in one comparison. Then we need only 2 more comparisons to update the overall min and max, for a total of 3 comparisons for each pair of numbers. The total nr. of comparisons  3n/2 = 1.5n.

7) Write an algorithm that determines whether or not an almost complete binary tree is a heap.

Input: a pointer tree to an almost complete binary tree Output: true if the input tree is a heap, false otherwise bool isHeap(tree_pointer tree) {

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

if(tree->left == NULL && tree-> right == NULL) return true; if(tree->key < tree->left->key) return false; if(tree->right != NULL && tree->key < tree->right->key) return false; if(tree->right == null) return isHeap(tree->left); return isHeap(tree->left) && isHeap(tree->right); }

Section 1.2

8) Under what circumstances, when a searching operation is needed, would Sequential Search (Algorithm 1.1) not be appropriate?

If the data is already sorted and  

there are a very large number of elements, or the access to the elements is slow (e.g. they are on magnetic tape)

9) Give a practical example in which you would not use Exchange Sort (Algorithm 1.3) to do a sorting task.

Is we had to sort an array of 10 million elements, the number of comparisons in Exchange Sort would be  107 x 107 = 1014. Even performing one comparison every nanosecond, the program would still run for 105 seconds, which is more than one day.

A more efficient algorithm like Mergesort would run in less than half a second.

If we tried Exchange Sort algorithm for 100M elements, it would take almost 4 months, whereas Mergesort would finish in about 5 seconds!

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

Section 1.3

10) Define basic operations for your algorithms in Exercises 1-7, and study the performance of these algorithms. If a given algorithm has an every-case time complexity, determine it. Otherwise, determine the worst-case time complexity.

Exercise 1: Basic operation is comparison. Every-time count T(n)=n-1 .

Exercise 2: Basic operation is the assignment needed in the shift phase. Worst-case count W(n,m) = n x m, if array is initially sorted in reverse order.

Exercise 3: Basic operation is accessing an array element. Every-time count T(n)=n(n1)(n-2).

Exercise 4: Basic operation is assigning an array element. Worst-case count W(n) = 3 + 4 + … + n = n(n+1)/2 -3, if array is initially sorted in reverse order.

Exercise 5: Basic operation is a%b, which is performed once per recursive call. Lamé proved in 1844 that, when the initial numbers are Fibonacci numbers a = F n+2, b = Fn+1, then the algorithm will perform exactly n steps, so the worst case does not have a closed-form for every number n.

Exercise 6: Basic operation is comparison. Every-time count T(n) = 3n/2.

Exercise 7: Basic operation is comparison. Worst-time count is 2 for every internal node of the heap, which is  2n/2 = n.

11) Determine the worst-case, average-case, and best-case time complexities for the basic Insertion Sort and for the version given in Exercise 4, which uses Binary Search.

In this solution, we only count comparisons: Basic Insertion Sort: B(n) = n (array already sorted), A(n) = n2/4, W(n) = n2/2 (array reverse sorted). Proofs can be found in Section 7.2.

Insertion Sort with Binary Search: B(n) = 2lg n (each new key is always inserted in the middle of the array, so only 2 comparisons are needed to find its location … if repeated keys are allowed, than an array of identical elements only requires 1 comparison per key!), A(n) = nlg n, W(n) = nlg n.

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

12) Write a (n) algorithm that sorts n distinct integers, ranging in size between 1 and kn inclusive, where k is a constant positive integer. (Hint: Use a kn-element array.)

Input: positive integer n, positive integer k, array of numbers S indexed from 1 to n Output: sorted array S void sort(int n, int S[]) { int temp[] = int[k*n]; //initially all zeros index i, j; for(i=1; i<=n; i++) { temp[S[i]] = x; } for(i=1, j=0; i<=k*n; i++) { if(temp[i] != 0) { a[j] = temp[i]; j++; } } }

13) Algorithm A performs 10n2 basic operations, and algorithm B performs 300ln n basic operations. For what value of n does algorithm B start to show its better performance?

For n = 7, we have 490 operations in A, and 584 in B, so A is better, but for for n = 8, we have 640 operations in A and 624 in B, so B is better (and stays that way for all n >= 8).

14) There are two algorithms called Alg1 and Alg2 for a problem of size n. Alg1 runs in n2 microseconds and Alg2 runs in 100n log n microseconds. Alg1 can be implemented using 4 hours of programmer time and needs 2 minutes of CPU time. On the other hand, Alg2 requires 15 hours of programmer time and 6 minutes of CPU time. If programmers are paid 20 dollars per hour and CPU time costs 50 dollars per minute, how many times must a problem instance of size 500 be solved using Alg2 in order to justify its development cost?

For instance size n = 500, the problem is solved in 500 5 x 10-6 = 25 x 104 x 10-6 = 0.25 s with Alg1, and in 100 x 500 x log 500 x 10-6 = 0.1349 s with Alg2.

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

The break-even point is found from this linear equation: 4*20 + 2*50 + y*0.25*50/60 = 15*20 + 6*50 + y*0.1349*50/60.

The solution is y =4379 times.

Section 1.4

15) Show directly that f(n) = n2 + 3n3  (n3). That is, use the definitions of O and  to show that f(n) is in both O(n3) and (n3).

We show that n2 + 3n3  O(n3) because for n0, n2 + 3n3  4n3 , we can take c = 4, N = 0 to obtain our result.

We show that n2 + 3n3  (n3) because for n0, n2 + 3n3  3n3 , we can take c = 3, N = 0 to obtain our result.

Thus, since n2 + 3n3  O(n3) and n2 + 3n3  (n3), n2 + 3n3  (n3).

16) Using the definitions of O and , show that 6n2 + 20n  O(n3), but 6n2 + 20n  n3).

Starting at N = 9, 6n2 + 20n < n3, so we can take c = 1, N = 9 in the definition of O. On the other hand, no matter how large c were chosen, the limit c(6n 2 + 20n)/n3 is zero, so c(6n2 + 20n) cannot stay > n3which contradicts the definition of .

17) Using the Properties of Order in Section 1.4.2, show that 5n5 + 4n4 + 6n3 + 2n2 + n + 7  (n5).

Apply Property 7, with g(n) = n4 + 6n3/4 + 2n2/4 + n/4 + 7/4, h(n) = n5 , c = 4, and d = 5.

A direct proof is also possible: take f(n)= 5n5 + 4n4 + 6n3 + 2n2 + n + 7, and we have 5n5 + 4n4 + 6n3 + 2n2 + n + 7 <=5n5 + 4n4 + 6n3 + 2n2 + n + 7 <= 5n5 + 4 n5 + 6 n5 + 2 n5 + n5 + 7 n 5

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

5n5<=5n5 + 4n4 + 6n3 + 2n2 + n + 7 <=25n5 Therefore the order will be Θ(n5), with c1=5 and c2=25, where n>=1.

18) Let p(n) = aknk + ak-1nk-1 + … + a1n + a0, where ak > 0. Using the Properties of Order in Section 1.4.2, show that p(n)  (n5).

As in Example 1.23, we repeatedly apply Properties 6 and 7 to show: aknk  (nk) aknk + ak-1nk-1  (nk) … aknk + ak-1nk-1 + … + a1n + a0 (nk) As a technicality, if some of the lower-order coefficients ak-1, … a0 are ≤ 0, but in this case we simply have a negative function aini, which is obviously in O(nk).

19) The function f(x) = 3n2+10n log n+1000n+4log n+9999 belongs in which of the following complexity categories:

f(n)  (n2), by repeatedly applying Properties 6 and 7 (or “throwing out” all the lower-order terms).

20) The function f(x) = (log n)2 + 2n + 4n + log n + 50 belongs in which of the following complexity categories:

f(n)  (n), by repeatedly applying Properties 6 and 7 (or “throwing out” all the lowerorder terms).

Note: (log n)2 is not listed in Property 6, but we can apply Theorem 1.4 (L’Hȏpital’s Rule) twice to show that lim[(log n)2/n] = lim2[(log n)/n] = lim[2/n] = 0, when n → ∞.

Now Theorem 1.3 ensures that (log n)2  (n).

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

21) The function f(x) = n + n2 + 2n + n4 belongs in which of the following complexity categories:

Solution: None of these; it is actually (2n), according to Property 6.

22) Group the following functions by complexity category.

Solution:

From “small” to “large”, the groups are:

G1 G2

= { (lg n)2 } = { 5lg n }

G3

={

G4 G5 G6 G7 G8 G9 G10 G11 G12 G13

= { 8n +12 } = { nln n, lg(n!) } = { 5n2 + 7n } = { n5/2 } = { (lg n)! } = { en } = { 4n } = { 10n + n20 } = { nn , nn + ln(n)} = { 2n! }

n }

As examples, consider some of the more difficult functions:  f(n) = 5lg n → With algebraic manipulations we have: f(n) = 5log5(n)/log5(2) = (5log5(n))1/log5(2) ≈ n0.43 ∊ o(n0.5 = n ), which explains G2.  f(n) = (lg n)! → We first change the logarithm into a natural logarithm: f(n) = (ln n)/ln2)!, and for simplicity we ignore the constant factor ln2 ≈ 0.69, so we consider n n f(n) ≈ (ln n)! Then we apply Stirling’s formula (n! ≈ 2n   , where e is the base e of the natural logarithm), substituting ln n for n. Leaving out the constant factors,

Chapter 1: Algorithms: Efficiency, Analysis, and Order



Solutions

we have f(n) ≈ sqrt(ln n)·(ln n)ln n/n (*). Consider only (ln n)ln n; if we write it in the form ng(n), where g(n) is an unknown function, we can take logs on both sides to find g(n) = ln(ln n). Formula (*) above thus becomes f(n) ≈ sqrt(ln n)· n ln(ln n)-1 .

This is obviously larger than any polynomial, so let’s compare it with the exponential en: taking logs on both sides we find that ln(f(n)) ∊ O(en), which explains why G8 comes before G9. f(n) = lg(n!) → We first change the base of the logarithm: f(n) = ln(n!) / ln2. Then we apply Stirling’s formula to ln(n!): f(n) ≈ [ ln 2 + 0.5ln(n) + n(ln(n) – 1) ]/ln2 ∊(nln(n) = (nlg n), which explains G5.

23) Establish Properties 1, 2, 6, and 7 of the Properties of Order in Section 1.4.2.

Property 1: g(n)∈O(f(n)) if and only if f(n)∈ Ω(g(n)).

Proof: “If and only if” means double implication.

We prove first that g(n)∈O(f(n)) implies f(n)∈ Ω(g(n)). According to the definition of O, there exist c, N > 0 such that, for all n ≥ N, g(n) ≤ cf(n). This means f(n) ≥ c-1g(n), which means f(n)∈ Ω(g(n)).

The opposite implication is proved in the same manner.

Property 2: g(n) ∈ Θ(f(n)) if and only if f(n)∈ Θ(g(n)).

Proof: We prove first that g(n)∈(f(n)) implies f(n)∈ (g(n)). According to the definition of , there exist c, d, N > 0 such that, for all n ≥ N, cf(n) ≤ g(n) ≤ df(n) ⇨ ⇨ d-1g(n) ≤ f(n) ≤ c-1g(n) ⇨ f(n)∈ (g(n)).

The opposite implication is proved in the same manner.

Property 6: Here we have to compare each category with the next in the sequence.

As an example, we prove bn ∈ o(n!): Proof: According to the definition of “little-oh”, we have to show that, for every positive c, there exists N such than bn ≤ cn!, for all n ≥ N. Let n0 be the first integer with the property n0 > 2b. We rewrite the desired inequality thus:

b b n 0 n0 (n0  1) ... n  bn0∙bn-n0 ≤ c(1∙2∙…∙n0∙ … ∙n) ⇔  , where the left-hand side c n0 ! b  b ... b is a constant C, and each ratio on the right-hand-side is > 2. Therefore we simply make N large enough so that C < 2N-n0+1, or N =

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

Property 7: If c>=0, d>0, g(n) ∈ O(f(n)), and h(n) ∈ (f(n)), then c*g(n)+d*h(n) ∈ Θ(f(n)) Proof: According to the definitions of O and , there are positive constants c1, N1, c2, d2, N2 such that g(n) ≤ c1∙f(n) for all n ≥ N1 c2∙f(n) ≤ h(n) ≤ d2∙f(n) for all n ≥ N2 We multiply the first inequality by c and the second one by d, and denote by N the maximum of N1 and N2: cg(n) ≤ cc1∙f(n) (*) dc2∙f(n) ≤ dh(n) ≤ dd2∙f(n) (**) for all n ≥ N Adding (*) and the right inequality in (**), we obtain cg(n) + dh(n) ≤ (cc1 + dd2)∙f(n) for all n ≥ N, which is the right-hand-side inequality in the definition of .

From the left-hand-side inequality of (**), we have dc2∙f(n) ≤ dh(n), and we add cg(n) to obtain dc2∙f(n) ≤ cg(n) + dh(n) for all n ≥ N, which is the left-hand-side inequality in the definition of .

24) Discuss the reflexive, symmetric, and transitive properties for asymptotic comparisons (O, , , o).

Reflexivity: O, , and are reflexive, which is shown by taking N = 1 and c = 1 in their respective definitions. o is not reflexive, since, by contradiction, it would lead to the conclusion that every positive real constant c has the property 1 ≤ c.

Symmetry: Only is symmetrical, as implied by property 2. For all the other it’s easy to give simple counterexamples, e.g. n and n2.

Transitivity: They are all transitive. For example, we prove that o is: Assume f is in o(g) and g is in o(h). Let c>0, and we need to find an N such that f(n) ≤ ch(n) for all n ≥N. Because g is in o(h), let N1 be that integer with the property that g(n) ≤ ch(n) for all n ≥N1. Because f is in o(g), let N2 be that integer with the property that f(n) ≤ g(n) for all n ≥N2. For all n ≥max(N1, N2), we have f(n) ≤ g(n) ≤ ch(n), so f(n) ≤ ch(n). Q.e.d.

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

25) Suppose you have a computer that requires 1 minute to solve problem instances of size n =1,000. Suppose you buy a new computer that runs 1,000 times faster than the old one. What instance sizes can be run in 1 minute, assuming the following time complexities T(n) for our algorithm? a) T(n) = n Answer: 106 b) T(n) = n3 Answer: 104 n c) T(n) = 10 Answer: 1003

26) Derive the proof of Theorem 1.3.

If the limit is c > 0, then for any  > 0 there is N > 0 such that g(n)/f(n) is in between c- and c+ for all n ≥ N ⇨ (c-)f(n) ≤ g(n) ≤ (c+)f(n) for all n ≥ N, which is the definition of g(n)  (f(n)).

If the limit is 0, then for any  > 0 there is N > 0 such that g(n)/f(n) <  for all n ≥ N ⇨ g(n) ≤ f(n) for all n ≥ N, which is the definition of g(n)  (f(n)).

If the limit is ∞, we can apply the definition of the limit as above, or simply notice that the reciprocal limit f(n)/g(n) is zero, and use the previous case.

27) Show the correctness of the following statements.

For all these results, we use Theorem 1.3 and Theorem 1.4 ( L’Hôpital’s rule): a) lim[(lg n)/n] = lim[(lg n)’/n’] = lim[1/n]/ln2 = 0, so lg n is in o(n), which, according to Theorem 1.2, is included in O(n), so lg n is in O(n). b) lim[n/(nlg n)] = lim[1/(nlg’n + lg n)] = 0, and continue as in (a) . c) lim[nlg n/n2] = lim[(lg n)/n] same as (a).

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

d) With algebraic manipulations we have 5lg n = 5log5(n)/log5(2) = (5log5(n))1/log5(2) ≈ n0.43 , ao lim[5lg n/2n] = lim[n0.43/2n] = 0.43∙ln2∙lim[1/n0.572n] = 0, so as above we have that 5lgn  O(2n), which, by Property 1, implies 2n  (5lgn). e) lim[lg3n/n0.5] = lim[(3lg2n/(nln2)) / (0.5/n0.5)] = C1lim[lg2n/n0.5] = … = C2lim[lg n/n0.5]= … = C3lim[1/n0.5] = 0, and from Theorem 1.3 this implies that lg 3n  o(n0.5).

Additional Exercises 28) Presently we can solve problem instances of size 30 in 1 minute using algorithm A, which is a (2n) algorithm. On the other hand, we will soon have to solve problem instances twice this large in 1 minute. Do you think it would help to buy a faster (and more expensive) computer?

230 operations in 60 s means that the machine performs 17,895,697.0(6) op. per second, or that one operation takes 55.87 nanoseconds.

A problem instance twice as large requires 22n = 260 op., which, on the same machine, would take 64.4 billion years (!). The new computer would have to be at least 10 billion times faster, so a realistic approach is to find a more efficient algorithm.

29) Consider the following algorithm: for ( i = 1 ; i <= 1 . 5 n ; i++) cout << i ; for ( i = n ; i >= 1 ; i - - ) cout << i ; (a) What is the output when n = 2, n = 4, and n = 6? (b) What is the time complexity T(n)? You may assume that the input n is divisible by 2. a) When n=2, the output is: 12321 When n=4, the output is: 1234564321 When n=6, the output is: 123456789654321

Chapter 1: Algorithms: Efficiency, Analysis, and Order b) T(n) = 1.5n + n = O(n) 30) Consider the following algorithm: j=1; while ( j <= n/2) { i=1; while ( i <= j ) { cout << j << i ; i++; } j++; } (a) What is the output when n = 6, n = 8, and n = 10? (b) What is the time complexity T(n)? You may assume that the input n is divisible by 2. a) When n=6, the output is: 112122313233 When n=8, the output is: 11212231323341424344 When n=10, the output is: 112122313233414243445152535455 b) T(n) = (n2/8) – (n/4) = O(n2)

31) Consider the following algorithm: for ( i = 2 ; i <= n ; i++) { for ( j = 0 ; j <= n) { cout << i << j ; j = j + floor(n/4) ; } } (a) What is the output when n = 4, n = 16, n = 32?

Solutions

Chapter 1: Algorithms: Efficiency, Analysis, and Order Solution:

n = 4:

Solutions

202122232430313233344041424344

n = 16: 2 0 2 4 2 8 2 12 2 16 3 0 3 4 3 8 3 12 3 16 4 0 4 4 4 8 4 12 4 16 5 0 5 4 5 8 5 12 5 16 6 0 6 4 6 8 6 12 6 16 7 0 7 4 7 8 7 12 7 16 8 0 8 4 8 8 8 12 8 16 9 0 9 4 9 8 9 12 9 16 10 0 10 4 10 8 10 12 10 16 11 0 11 4 11 8 11 12 11 16 12 0 12 4 12 8 12 12 12 16 13 0 13 4 13 8 13 12 13 16 14 0 14 4 14 8 14 12 14 16 15 0 15 4 15 8 15 12 15 16 16 0 16 4 16 8 16 12 16 16 n = 32:

too long to show here.

(b) What is the time complexity T(n). You may assume that n is divisible 4.

Solution: T(n) = (n-1)(1+n/4)  O(n2)

32) What is the time complexity T(n) of the nested loops below? For simplicity, you may assume that n is a power of 2. That is, n = 2k for some positive integer k. for ( i = 1 ; i <= n ; i++) { j=n; while ( j >= 1) { < body o f the while loop> //Needs ( 1 ) . j = floor(j/2) ; } } Solution: T(n) = n*log n = O(n*log n)

33) Give an algorithm for the following problem and determine its time complexity.

Given a list of n distinct positive integers, partition the list into two sublists, each of size n/2, such that the difference between the sums of the integers in the two sublists is maximized. You may assume that n is a multiple of 2.

Solution: The first list should have the larger half of the elements, and the second list should have the smaller half, so we have to find the median. The easiest way to do it is to sort the list, which is (n2) with Exchange Sort – Algorithm 1.3.

The median can, however, be found very efficiently with the “Median of Medians” algorithm, whose complexity is only (n), after which the list can be partitioned using

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

the median as a pivot in another (n) operations, which makes the entire algorithm (n).

34) What is the time complexity T(n) of the nested loops below? For simplicity, you may assume that n is a power of 2. That is, n = 2k for some positive integer k. i=n; while ( i >= 1) { j=i; while ( j <= n) { < body o f the while loop> //Needs ( 1 ) . j = 2 * j; } i = floor(i /2) ; } Solution: T(n) = (log2n + log n)/2 = O(log2n)

35) Consider the following algorithm, where the array A is indexed 1 through n: int add_them ( int n , int A[ ] ) { index i , j , k ; j=0; for ( i = 1 ; i <= n ; i++) j = j + A[i] ; k=1; for ( i = 1 ; i <= n ; i++) k=k+k; return j + k ; } (a) If n = 5 and the array A contains 2, 5, 3, 7, and 8, what is returned?

Solution: 57 (b) What is the time complexity T(n) of the algorithm?

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

Solution: T(n) = 2n (c) Try to improve the efficiency of the algorithm.

Solution: Since the second loop simply turns k into 2n, we can calculate 2n directly, e.g. with the library function “power”: k = pow(2, n). As explained in Ch.3, there are algorithms more efficient than linear to calculate powers, based on Dynamic Programming.

36) Consider the following algorithm: int any_equal ( int n , int A[ ] [ ] ) { index i , j , k ,m; for ( i = 1 ; i <= n ; i++) for ( j = 1 ; j <= n ; j++) for ( k = 1 ; k <= n ; k++) for (m = 1 ; m <= n ; m++) if (A[ i ] [ j ]==A[ k ] [m] && ! ( i==k && j==m)) return 1 ; return 0 ; } (a) What is the best case time complexity of the algorithm (assuming n > 1)?

Solution: The best-case occurs when A[1][1] is equal to A[1][2]; the algorithm returns 1 after only the second iteration of the innermost loop (i = j = k = 1, m=2). Time complexity is B(n) = 2 (counting comparisons of array elements).

(b) What is the worst case time complexity of the algorithm?

Solution: The worst -case occurs when all the elements of A are unique, in which case there are W(n) = n4 comparisons.

(c) Try to improve the efficiency of the algorithm.

Solution: Each pair of different elements is compared twice, so we can improve the efficiency by comparing only “ahead”. Also, we can avid comparing elements with themselves, because in that case ! ( i==k && j==m) is false, so the if statement is redundant. The improved pseudocode is:

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

int any_equal ( int n , int A[ ] [ ] ) { index i , j , k ,m, index_first, index_second; for ( i = 1 ; i <= n ; i++) for ( j = 1 ; j <= n ; j++) { index_first = n*i + j; for ( k = 1 ; k <= n ; k++) for (m = 1 ; m <= n ; m++) { index_second = n*k + m; if (index_second > index_first) if (A[ i ] [ j ]==A[ k ] [m]) return 1 ; { } return 0 ; } Now the worst case is W’(n) = (n2-1) + (n2-2) + (n2-3) + … + (n2-(n2-1)) = n2(n2-1) -

n 2 1

i i 1

and the sum can be calculated as in Example A.1, yielding n2(n2-1) - n2(n2-1)/2 = n2(n21)/2. (d) What property holds for the array A if the algorithm returns 0?

Solution: The array has no duplicates. (e) What property holds for the array A if the algorithm returns 1?

Solution: The array has at least one pair of duplicates.

37) Give a (lg n) algorithm that computes the remainder when xn is divided by p. For simplicity, you may assume that n is a power of 2. That is, n = 2 k for some positive integer k.

Solution: If we have an integer, m , we divide it by p to obtain the remainder: m = q·p + r. The square n = m2 = m·m is (q·p + r)(q·p + r) = (q2p + 2qr)p + r2, which shows that only r2 must be considered when calculating the remainder of m2 modulo p: m2 % p = r2 % p.

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

Back to our problem, let m = xn/2 . We split the problem the problem into two identical subproblems: xn = xn/2·xn/2. All we need to do now is to find r = xn/2 % p, so we can write T(n) = T(n/2) + C, where C is the constant cost of squaring r and taking r 2 % p.

Since n = 2k, n/2 is always an integer, until n = 1, so we have recursively T(n) = T(n/2) + C = T(n/4) + 2C = T(n/8) + 3C = … = T(n/2 j) +jC. It’s clear that when n/2j = 1, j is lg n, so the solutions is T(n) = (lg n)C  (lg n).

38) Explain in English what functions are in the following sets. (a) nO(1) Solution: All functions f(n) bounded above by a polynomial of the form CnD, with C, D > 0: f(n) ≤ CnD. (b) O(nO(1)) Solution: All functions g(n) bounded above by Ef(n), whereE > 0, and f(n) is as above.

We have g(n) ≤ Ef(n) ≤ ECnD = FnD, with F, D > 0, therefore O(nO(1)) = nO(1), meaning that the set of functions at (b) is the same as that at (a). (c) O(O(nO(1))) Solution: As above, we have that O(nO(1)) = O(nO(1)) = nO(1).

39) Show that the function f (n) =n2sin n is in neither O(n) nor (n).

Solution: For n  2k + /2 , For n  2k ,

f(n)  n2sin(/2) = n2, so f is not in O(n). f(n)  n2sin(0) = 0, so f is not in Ω(n).

Technical note: the approximations above can be given a clear mathematical meaning, based on the irrationality of , which implies that multiples of  will approach integers an infinite number of times; integers from that series can be chosen for n. A detailed proof, however, is outside the scope of this course.

For a more elementary (if a bit contrived) example, consider the function g(n) defined to be 0 for n even and n2 for n odd.

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

40) Justify the correctness of the following statements assuming that f(n) and g(n) are asymptotically positive functions. (a) f(n) + g(n)  O (max (f(n)) ; g(n)) Solution: f(n) + g(n) ≤ 2max(f(n) + g(n)) (b) f2(n)  (f(n)) Solution: Assuming that, for n > N, f(n) > 1, we have f2(n) ≥ f(n) for n > N. (c) f(n) + o(f(n))  (f(n)), where o(f(n)) means any function g(n)  o(f(n)) Solution: f(n) + o(f(n)) ≥ f(n) + cf(n) = (1+c)f(n) ⇨ f(n) + o(f(n))  (f(n)) On the other hand, f(n) + o(f(n)) ≤ f(n) ⇨ f(n) + o(f(n))  (f(n))

41) Give an algorithm for the following problem. Given a list of n distinct positive integers, partition the list into two sublists, each of size n/2, such that the difference between the sums of the integers in the two sublists is minimized. Determine the time complexity of your algorithm. You may assume that n is a multiple of 2.

Solution: Let the set of all list nodes be denoted by A. The algorithm needs a temporary set of n/2 elements, TempSet. int min_diff = LARGE_NUMBER; int diff; for (each subset S of size n/2 of A) { diff = abs(sum(S) – sum(A-S)); if (diff < min_diff) { min_diff = diff; TempSet = S; } } print min_diff, TempSet; Each iteration of the for loop has a constant worst-case cost (one comparison, an integer assignment, a set assignment), and the loop executes n-choose-n/2 times, so   n! . the algorithm is in   2   (( n / 2)!) 

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

42) Algorithm 1.7 (nth Fibonacci Term, Iterative) is clearly linear in n, but is it a lineartime algorithm? In Section 1.3.1 we defined the input size as the size of the input. In the case of the nth Fibonacci term, n is the input, and the number of bits it takes to encode n could be used as the input size. Using this measure, the size of 64 is lg64= 6, and the size of 1,024 is lg1,024= 10. Show that Algorithm 1.7 is exponential-time in terms of its input size. Show further that any algorithm for computing the nth Fibonacci term must be an exponential-time algorithm because the size of the output is exponential in the input size. (See Section 9.2 for a related discussion of the input size.)

Solution: Proof of exponential output size: We can find a closed-form formula for the nth Fibonacci number by solving the recursion F(n) = F(n-1) + F(n-2) directly, with the tools in Appendix B. The associated homogeneous equation x2 = x + 1 has two roots: r1 1 5 1 5 = , r2 = , so the solution has the form F(n) = C1r1n+ C2r2n. The constants 2 2 are determined from the values of F(0) and F(1), and they turn out to be both equal to 1/√5, so we have n

n

1  5  1 5    + F(n) = (    2  ) / √5 2     Given that the absolute value of r2 is < 1, the second power in the expression tends to 0 as n →∞, so we have n

1  5   / √5  1.618n/ 2.236 F(n)    2   The size of the output is lg(F(n))  nlg(1.618)  0.69n = 0.69∙2lgn, therefore exponential in terms of the size of the input.

Proof of exponential time: the lgn bits used to represent the input n are being processed to generate about 0.69∙2lgn bits of the output F(n). We define our fundamental operation as being the generation of one bit, since any binary computer has this type of operation at the hardware level.

43) Determine the time complexity of Algorithm 1.6 (nth Fibonacci Term, Recursive) in terms of its input size (see Exercise 42).

Chapter 1: Algorithms: Efficiency, Analysis, and Order

Solutions

Solution: According to Exercise 42, any algorithm for computing the nth Fibonacci term must be an exponential-time algorithm, so in particular Algorithm 1.6 is exponential, too.

44) Can you verify the correctness of your algorithms for Exercises 1 to 7?

Solution: We prove the correctness of the algorithm from Exercise 1, which finds the maximum in an array of n numbers.

Input: positive integer n, list of numbers S indexed from 1 to n Output: the maximum element in the list S number findMax(int n, const keytype S[ ]) { index i; number max = S[1]; for(i =2; i<=n; i++) if(S[i] > max) max = S[i]; return max; } Proof: Induction on n: If n = 1, the algorithm returns max = S[1] which is the only number in the array. This is correct.

Induction Hypothesis: The algorithm is correct for n.

Induction step: We show that the algorithm is correct for n+1: When j = n, max contains the true maximum of the sub-array S[1…n]. When j = n+1, there are two cases:  

if S[n+1] > max, it means that the last element is largest than any among the first n, so the algorithm correctly changes max to the last element.

If S[n+1] ≤ max, it means that among the first n elements there is (at least) one that is ≥ S[n+1], so the algorithm correctly leaves max unchanged.

Chapter 2: Divide-and-Conquer

Solutions

Section 2.1

1) Use Binary Search, Recursive (Algorithm 2.1) to search for the integer 120 in the following list (array) of integers. Show the actions step by step.

12 34 37 45 57 82 99 120 134 Solution: 1. Compute the middle index from initial low (1) and high (9) indices: mid = 5 2. Check key at position mid: 120 > S[mid] = 57 3. Compute new middle index from new low (mid=5) and previous high (9) indices: mid =7 4. Check key at position mid: 120 > S[mid] = 99 5. Compute new middle index from new low (mid=5) and previous high (9) indices: mid =8 6. Check key at position mid: 120 == S[mid] 7. Key found, return index mid=8

2) Suppose that, even unrealistically, we are to search a list of 700 million items using Binary Search, Recursive (Algorithm 2.1). What is the maximum number of comparisons that this algorithm must perform before finding a given item or concluding that it is not in the list?

Solution: ceil(log2700,000,000) = 30

Chapter 2: Divide-and-Conquer

Solutions

3) Let us assume that we always perform a successful search. That is, in Algorithm 2.1 the item x can always be found in the list S. Improve Algorithm 2.1 by removing all unnecessary operations.

Solution: index location (index low, index high){ index mid; mid = (low + high)/2; if (x = S[mid]) return mid else if (x < s[mid]) return location (low, mid-1); else return location (mid+1, high); } where Prob: Determine whether x is in the Sorted array ‘s’ of size ‘n’ Input: Positive integer n, sorted array of key ‘s’, indexed from 1 to n a key x.

Output: Location, the location of x is s

4) Show that the worst-case time complexity for Binary Search (Algorithm 2.1) is given by W (n) = lg n + 1 when n is not restricted to being a power of 2.

Solution: As hinted, we first show that the recurrence equation for W (n) is given by

n W (n) = 1 + W     for n > 1, and W (1) = 1. To do this, we consider even and odd values 2 of n separately:

Chapter 2: Divide-and-Conquer

Solutions

case 1: n is even → n = 2m → the array is split into two equal subarrays of size m = n/2 = n  2  each, and mid is (low + high)/2, which is the right-most element of the left array. To decide which way to continue the search, we need one comparison, and in the worst n case we go right, so we have   elements for the next level of the recursion, for a total 2 n of   + 1. 2 case 2: n is odd → n = 2m + 1 → mid is (low + high)/2, which is the central element of the array. To decide which way to continue the search, we need one comparison, and n n either way we have   for the next level of the recursion, for a total of   + 1. 2 2 If n = 1, we only have the comparison (x == S(mid)), so W(1) = 1.

n We have proved that W (n) = 1 + W     for n > 1, and W (1) = 1. 2

Now we use induction to solve the recurrence equation: Base case: n = 1 → lg n = 0 → W(n) = 0 + 1 = 1 correct.

Induction step: We assume W (k) = lg k  + 1 for all k ≤ n, and prove it for n+1.

  n  1   n  1  n  1  . Is  According to the result proven above, W(n+1) = 1 + W   ≤n?  ≤    2   2   2  (n+1)/2, and (n+1)/2 ≤n means n+1 ≤ 2n, or 1 ≤ n, which is true. According to the induction

  n  1    n  1   lg ( hypothesis, W    ) + 1,   =    2    2  



 n  1  ) .  2  

so we have that W(n+1) = 2 + lg ( 



On the other hand, we want to show that W(n+1) = lg (n  1) + 1, so it remains to show that



 n  1  ) = lg (n  1)  2  

1+ lg ( 



(*)

Chapter 2: Divide-and-Conquer

Solutions

Taking 2 for the base of the logarithm, and letting n = 2m – p , where 0 ≤ p < 2m-1. We have three cases:   

If p = 0, eq. (*) above becomes 1 + m – 1 = m, which is true.

If p = 1, eq. (*) becomes 1 + m – 1 = m, also true.

If p > 1, eq. (*) becomes 1 + m – 2 = m-1, also true. Q.e.d.

5) Suppose that, in Algorithm 2.1 (line 4), the splitting function is changed to mid = low;.

Explain the new search strategy. Analyze the performance of this strategy and show the results using order notation.

Solution: The algorithm now is: index location (index low, index high) { index mid; if (low > high) return 0; else { mid = low; if (x = = s [mid]) return mid else if (x < s[mid]) return location (low, mid-1); else return location (mid+1, high) } }

Complexity analysis: The worst case for this algorithm is when x is either the largest array element, or larger than the largest. In either case, n comparisons are needed, since always the branch location (mid+1, high) is taken. This is a Linear Search → (n).

Chapter 2: Divide-and-Conquer

Solutions

6) Write an algorithm that searches a sorted list of n items by dividing it into three sublists of almost n=3 items. This algorithm finds the sublist that might contain the given item and divides it into three smaller sublists of almost equal size. The algorithm repeats this process until it finds the item or concludes that the item is not in the list. Analyze your algorithm and give the results using order notation.

Solution: Input: Positive integer ‘n’ Sorted array of Keys ‘A’ indexed from 1 to n a key ‘x’ Output: Location, the Location of x in ‘A’ (0 if x is no in ‘A’) index location3 (int A[ ], index L, index H){ index m1, m2; if (L > H) return 0; else { m1 = L + (H – L)/3;

//one third

m2 = L + 2*(H – L)/3;

//two thirds

if (x == A[m1])

return m1;

elseif (x < A[m1])

return location3 (A, L, m1 −1);

elseif (x < A[m2])

return location3 (A, m1 +1, m2 – 1);

elseif (x == A[m2])

return m2;

else

return location3 (A, m2 + 1, H);

} }

Complexity analysis: Assuming n = 3k, and counting only the operations of addressing the array, the worst case is when the right third is chosen at each call, in which case the recursion is T(n) = 2 + T(n/3). As in Example B.18, this is found to be (lg n), which can be extended to any n with the tools in Section B.4.

Chapter 2: Divide-and-Conquer

Solutions

7) Use the divide-and-conquer approach to write an algorithm that finds the largest item in a list of n items. Analyze your algorithm, and show the results in order notation.

Solution: int maximum (int low, int high) { if (low == high) return a[Low]; int mid = (low + high)/2;

//integer division

int L1 = maximum (low, mid); int L2 = maximum (mid + 1, high); if A[L1] > A[L2] then return L1 else return L2 } Example: Let A be this array: 7 1

3 2

8 3

13 4

9 5

14 6

Level 1 : 1 ≠ 6 → mid = (1+6)/2 = 3 → A[mid] = 8 Recursive calls: Level 2: 7 1

3 2

8 3

13 4

9

14 5

↑

6

↑

4/2 10/2 = 5

7 1

3 2

8 ↑

13

9

14

Chapter 2: Divide-and-Conquer

Solutions

max 7

3

7 9

8

13

9

14

6

(0 + 2)/2 = 1

0, 1 7

9

6 2 1/2 0

Complexity analysis: T(n) = 2T(n/2) + 1, which yields (n).

Section 2.2

8) Starting with array S[] = { 123 34 89 56 150 12 9 240 } indexed from 1 to 8, sort S[] using Mergesort. Using algorithm mergesort2 on page62: 1. Begin with low = 1, high = 8; mid = (1+8)/2 = 4 (recursion level 0) 2. Recur with low = 1, high = 4; mid = (1+4)/2 = 2 (recursion level 1) 3. Recur with low = 1, high = 2; mid = (1+2)/2 = 1 (recursion level 2) 4. Both recursive calls to recursion level 3 with (low = 1, high = 1), and (low = 2, high = 2) return since low==high 5. Call merge2 with low = 1, mid = 1, high = 2 6. merge2 produces partially sorted array S[] = { 34 123 89 56 150 12 9 240 } 7. Return from recursion level 2 8. Recur with low = 3, high = 4; mid = (3+4)/2 = 3 (recursion level 2)

Chapter 2: Divide-and-Conquer

Solutions

9. Both recursive calls to recursion level 3 with (low = 3, high = 3), and (low = 4, high = 4) return since low==high 10. Call merge2 with low = 3, mid = 3, high = 4 11. merge2 produces partially sorted array S[] = { 34 123 56 89 150 12 9 240 } 12. Return from recursion level 2 13. Call merge2 with low = 1, mid = 2, high = 4 14. merge2 produces partially sorted array S[] = { 34 56 89 123 150 12 9 240 } 15. Return from recursion level 1 16. Recur with low = 5, high = 8; mid = (5+8)/2 = 6 (recursion level 1) 17. Recur with low = 5, high = 6; mid = (1+2)/2 = 5 (recursion level 2) 18. Both recursive calls to recursion level 3 with (low = 5, high = 5), and (low = 6, high = 6) return since low==high 19. Call merge2 with low = 5, mid = 5, high = 6 20. merge2 produces partially sorted array S[] = { 34 56 89 123 12 150 9 240 } 21. Return from recursion level 2 22. Recur with low = 7, high = 8; mid = (1+2)/2 = 7 (recursion level 2) 23. Both recursive calls to recursion level 3 with (low = 7, high = 7), and (low = 8, high = 8) return since low==high 24. Call merge2 with low = 7, mid = 7, high = 8 25. merge2 produces partially sorted array S[] = { 34 56 89 123 12 150 9 240 } 26. Return from recursion level 2 27. Call merge2 with low = 5, mid = 6, high = 8 28. merge2 produces partially sorted array S[] = { 34 56 89 123 9 12 150 240 } 29. Return from recursion level 1 30. Call merge2 with low = 1, mid = 4, high = 8 31. merge2 produces sorted array S[] = { 9 12 34 56 89 123 150 240 }

Chapter 2: Divide-and-Conquer

Solutions

9) Give the tree of recursive calls in Exercise 8.

Solution:

10) Write for the following problem a recursive algorithm whose worst-case time complexity is not worse than Θ(n∙lgn). Given a list of n distinct positive integers, partition the list into two sublists, each of size n/2, such that the difference between the sums of the integers in the two sublists is maximized. You may assume that n is a multiple of 2.

Solution: The difference is maximized if one list contains all the elements less than the median, and the other list contains all the elements larger.

A simple solution is to sort the list using a Θ(n∙lgn) algorithm (e.g. Mergesort, Quicksort), and then split it into two halves.

One could object that sorting is overkill for this problem: we don’t need the two halves to be sorted, after all. If we could find the median in another, easier way, we could then partition in (n) by using the Partition algorithm from Quicksort (Algorithm 2.7). Indeed, the Median of Medians algorithm allows us to find the k-th largest element of an unsorted sequence in linear time Θ(n). The algorithm was invented by Blum, Floyd, Pratt, Rivest, and Tarjan in their 1973 paper “Time Bounds for Selection”, and thus is sometimes referred to as the BFPRT algorithm.

The original paper and many good explanations of BFPRT, including pseudocode, are freely available on the Web.

Chapter 2: Divide-and-Conquer

Solutions

11) void nonrecursiveMergesort(int n, keytype S[]) { index i, j; for(i=1; i<=n; i = i*2) for(j=1; j<=n; j = j + 2*i) merge2(j, j+k, j+2*k, S[]); } The merge2 method is specified on page 63. We make the small modification of passing S[] as a parameter.

12) Show that the recurrence equation for the worst-case time complexity for Mergesort (Algorithms 2.2 and 2.4) is given by

when n is not restricted to being a power of 2.

Solution: In Algorithm 2.4, we have two cases: A. If n = (high – low + 1) is even, low + high is odd, so (low + high)/2 = (low + high)/2.0 – 0.5. a. The array [low, mid] has length (low + high)/2.0 – 0.5 – low + 1 = (high-low)/2.0 + 0.5 = n (high – low + 1)/2.0 = (high – low + 1)/2 = n/2 =   2 n b. Similarly, the array [mid+1, high] has length   2 B. If n = (high – low + 1) is odd, low + high is even, so (low + high)/2 = (low + high)/2.0 . a. The array [low, mid] has length (low + high)/2.0 – low + 1 = (high-low)/2.0 + 1 = (high – n low + 1)/2.0 – 0.5 =   2 n b. Similarly, the array [mid+1, high] has length   2

The same formula was proved for both cases, q.e.d.

Chapter 2: Divide-and-Conquer

Solutions

13) Write an algorithm that sorts a list of n items by dividing it into three sublists of about n=3 items, sorting each sublist recursively and merging the three sorted sublists. Analyze your algorithm, and give the results under order notation.

Solution: Input : Positive integer n, array of keys S, indexed from 1 to n Output :The array S containing the keys in nondecreasing order. void mergesort3 (int n, keytype S[ ]) { if (n>1) { int h = floor(n/3), i = 2*h; int m = i - h , t = n – i; int U[1…h], V[1…m], W[1…t] Copy S[1] through S[h]

to U[1] through U[m]

Copy S[h+1] through S[i]

to V[1] through V[t]

Copy S[i+1] through S[n]

to W[1] through W[t]

mergesort3 (h, U); mergesort3 (m, V); mergesort3 (t, W); merge (h, m, t, U, V, W, S); } } void merge3 (int h, int m, int t, const int U[], const int V[], const int W[], int S[]) { //Similar to Algorithm 2.3 }

Chapter 2: Divide-and-Conquer

Solutions

Section 2.3

14) Given the recurrence relation

find T(625).

Solution: T(625) = 7T (

625 ) + 10(625) = 7T(125) + 6,250 5

T(125) = 7T (

125 ) + 10(125) = 7T(25) + 1,250 5

T(25) = 7T (

25 ) + 10(25) 5

5 T(5) = 7T ( ) + 10(5) 5

= 7T(5) + 250 = 7T(1) + 50, and T(1) = 1.

Substituting back, we have: T(5) = 57, T(25) = 649, T(125) = 5,793, T(625) = 46,801

Chapter 2: Divide-and-Conquer

Solutions

15)

Assume g(n) basic operations for partitioning and combining, and none for an instance of size 1. (a) Write a recurrence equation T(n) for the number of basic operations needed to solve P when the input size is n.

Solution: T(n) =5T(n/3) + g(n)

(b) What is the solution to this recurrence equation if g(n)∈ Θ(n)? (Proof is not required.)

Solution: We assume a linear function g(n) = a∙n+b, and n a power of 3: n = 3k. The recursion is T(3k) = 5T(3k-1) + a∙3k + b We denote T(3k) = xk, and the recursion becomes xk – 5xk-1 = a∙3k + b Which is solved as a non-homogeneous recursion as explained in Theorem B.3: The solution to the homogeneous part is r5 = 5, and the two non-homogeneous terms each contribute a root: r1 = 1, and r3 = 3. The general form of the solution is: xk = C11k + C33k + C55k If need be, the constants are found by making use of the first 3 terms of the recursion: x 0 = 0, x1 = 3a +b, x2 = 24a + 6b. We solve the linear system of 3 eqns. with 3 unknowns to find C3 = – 3a/2, etc.

(c) Assuming that g(n) =n2, solve the recurrence equation exactly for n =27.

Chapter 2: Divide-and-Conquer

Solutions

Solution: By direct substitution, we find T(1) = 0, T(3) = 9, t(9) = 126, T(27) = 1359. (d) Find the general solution for n a power of 3.

Solution: We still assume g(n) = n2, as above. The recursion is T(n) = 5T(n/3) + n2, so we substitute n = 3k, and rename T(3k) = xk to obtain a new recursion xk – 5xk-1 = 9k. This is solved as a non-homogeneous recursion (Theorem B.3), with the solution x k = C55k + C99k.

The constants are determined by solving the linear system x0 = 0, x1 = 9, and we have C5 = 9/4, C9 = 9/4.

The final solution is T(3k) = xk = -9/4∙5k + 9/4∙9k

16) Suppose that, in a divide-and-conquer algorithm, we always divide an instance of size n of a problem into 10 sub-instances of size n=3, and the dividing and combining steps take a time in (n2). Write a recurrence equation for the running time T (n), and solve the equation for T (n).

Solution: T(n) = 10T(n/3) + Cn2, where C is a constant. Assuming that n is a power of 3, we have n = 3k.

T(3k) = 10T(3k-1) + C9k, and we change the recursion by denoting xk = T(3k) → xk = 10xk-1 + C9k. This is a non-homogeneous recursion, solved as in Theorem B.3: there are two roots: 9 and 10, so the solution has the form C99k + C1010k. The constants are determined by plugging in the first two values of k: k = 0 and k = 1. We assume x0 = 0, meaning that the instance n=1 needs no operations to solve. This means x1 = 9C. Solving for C9 and C10 we have C9 = -9C, C10 = 9C. The solution is xk = 9C(10k – 9k).

17) Divide-and-conquer algorithm for Towers of Hanoi with n disks.

Solution: void ToH(int n, char x, char y, char z) { if (n > 0) { ToH (n−1, x, y, z); cout << x << " moves to " << y << endl; ToH (n−1, z, y, x); } }

Chapter 2: Divide-and-Conquer

Solutions

(a) Show for your algorithm that S (n) = 2n-1. (Here S (n) denotes the number of steps (moves), given an input of n disks.)

Solution: The recursion equation is T(n) = 2T(n-1) + 1 for n > 1, T(1) = 1. By direct substitution, we have T(n) = 2·(2·(…) + 1) + 1 = 2 n-1 +2n-2 + … + 2 + 1, which, using the sum of the geometric series from Example A.4, is 2n – 1.

(b) Prove that any other algorithm takes at least as many moves as given in part (a).

Solution: In order to move the largest disk (call it Dn), any other algorithm A’ must first place disks 1 through n-1 on the intermediate peg, then perform at least one move to place Dn on the destination peg, and then move disks 1 through n-1 on the destination peg, so we have that T’(n) ≥ 2T’(n-1) + 1, for n > 1.

The boundary condition is T’(1) > 1.

If we now write the sum from the solution to part (a), it’s clear that T’(n) ≥ T(n).

18) When a divide-and-conquer algorithm divides an instance of size n of a problem into sub-instances each of size n=c, the recurrence relation is typically given by

where g (n) is the cost of the dividing and combining processes, and d is a constant.

Let n = ck.

(a) Show that

Solution: Induction on k: Base case: When k = 0, we have T(1) = dx1 + 0 = d → correct.

Induction step: T(ck+1) = aT(ck) + g(ck+1), and we substitute the formula for T(ck):

Chapter 2: Divide-and-Conquer



= a  d  a 

T(ck+1)

k



k

a j 1

k

= d x ak+1 +

a

Solutions

k j

  g (c j )  + g(ck+1) = 

k 1 j

 g (c j ) + ak+1-(k+1) ·g(ck+1) =

j 1

k 1

k 1 k ! j  g (c j ) = d a  a

Q.e.d.

j 1

(b) Solve the recurrence relation given that g(n) ∊ (n).

Solution: Assume g(n) = E·n. Denoting T(ck) by xk, we have the new recursion xk = axk-1 + Eck , which is solved as a non-homogeneous recursion (Theorem B.3). The roots are a and c, and we consider two cases:  

If a ≠c, the solution has the form xk = C1ak + C2ck, and the constants are determined by using the eqns. x0 = d and x1 = ad + Ec. We find C1 = d + cE/(a-c), C2 = -cE/(a-c).

If a = c, we can either apply the non-homogeneous case with the double root a, or expand the recursion directly to find xk = (d+kE)ak.

Section 2.4

19) Use Quicksort (Algorithm 2.6) to sort the following list. Show the actions step by step.

123 34 189 56 150 12 9 240 Solution: i − 1 2 3 5 −

j − 7 6 5 4 −

S[0] 123 123 123 123 123 123

S[1] 34 34 34 34 34 34

S[2] 189 189 9 9 9 9

S[3] 56 56 56 56 56 56

S[4] 150 150 150 150 150 150

S[5] 12 12 12 12 12 12

S[6] 9 9 189 189 189 189

S[7] 240 240 240 240 240 240

Chapter 2: Divide-and-Conquer

I 0 1 2

j 4 2 1

I 5 6 9

Solutions

S[0] 123 12 9

S[1] 34 34 34

J 7 6 34

S[5] 150 150 12

56

S[3] 56 56 56

S[6] 189 189 123

20) Give the tree of recursive calls in Exercise 19.

Solution:

S[2] 9 9 12

66

S[4] 12 123 123 S[7] 240 240

150

189

240

Chapter 2: Divide-and-Conquer

21) Show that if

Solutions

then

This result is used in the discussion of the worst-case time complexity analysis of Algorithm 2.6 (Quicksort).

Solution: w(n) 



(p  1)(p  2) (n  p)(n  p  1)   (n  1) 2 2

(n  1)(n  2) 2(n  1) n 2  3n  2  2n 2 n 2  n n(n  1) 0    2 2 2 2 2

Second solution:

w(n)  

(n  1)(n  2) (n  p)(n  p  1) 2(n  1)   = 2 2 2

n2  3n  2  n 2  np  n  pn  p 2  p  2n  2 2

= 2n2 – 2np −2n + p2 p/2 = 2n2 – 2n (p + 1) + p (p + 1)/2 = 2n2 + (p – 2n) (p + 1)/2 = 2n2 – n (n + 1)/2 = 2n2 − n2 – n = n2 – n/2



n(n  1) 2

22) Verify the following identity

This result is used in the discussion of the average-case time complexity analysis of Algorithm 2.6 (Quicksort).

Chapter 2: Divide-and-Conquer

Solutions

Solution: Induction on n.

Base case: n = 1 → A(0) + A(0) = 2A(0) → correct.

Induction step: We split the sum for n+1 so that we isolate a sum for n: n 1

[ A( p  1)  A(n  ( p  1))]  p 1

n

n

p 1

p 1

 A( p  1)  A(n)  A(n)   A(n  p) 

n

 [ A( p  1)  A(n  p)]  2 A(n) p 1

n

By the induction hypothesis, the first sum is 2 A( p  1) , and, when we add 2A(n), the p 1

sum is complete up to n+1.

23) Write a non-recursive algorithm for Quicksort (Algorithm 2.6). Analyze your algorithm, and give the results using order notation.

Solution: Non-Recursive Quick sort Algorithm by using a stack. struct Stack { int low, high: }; void quick-sort (int a[], int n) { //n elements in the array struct Stack S[100]; int top, i, j, pivot, lo, hi; top = 0; S[top].low = 0; //first array index S[top].high = n − 1; //last array index top++; //pushing while (top > 0) { //stack is not empty lo = s[top].low; hi = s[top].high; top--; //pulling sub-problem off stack if (lo >= hi) continue; //empty problem pivot = a[lo]; //pivot is leftmost element i = lo; j = hi; while (i < j) { while (i < hi && a[i] <= k) i++; while (j < lo && a[j] >= k)

Chapter 2: Divide-and-Conquer

Solutions

j−−; if (i < j) swap(i,j); } if (lo! = j) swap(lo, j); } S[top].low = j + 1; S[top].high = hi; top++; S[top].low = lo; S[top].high = j − 1; top++; }// end of while

//place pivot in the “middle”

//pushing right sub-array //pushing left sub-array

}

The operation is identical to that of the recursive Quicksort, so it has the same complexity: (n2) in the worst case and (n∙lg n) on average.

24) Assuming that Quicksort uses the first item in the list as the pivot item: (a) Give a list of n items (for example, an array of 10 integers) representing the worst-case scenario. (b) Give a list of n items (for example, an array of 10 integers) representing the best-case scenario.

Solution: (a) { 1 2 3 4 5 6 7 8 9 10 } (b) { 6 3 8 2 7 5 1 9 4 10 }

Section 2.5

25) Show that the number of additions performed by Algorithm 1.4 (MatrixMult) can be reduced to n3 − n2 after a slight modification of this algorithm.

Solution:

Chapter 2: Divide-and-Conquer

Solutions

The loops Algorithm 1.4 are modified thus: for (i=1; i<=n; i++) for (j=1; j<=n; j++) C[i][j] = C[i][1] * C[1][j]; for (k=2; k<=n; k++) C[i][j] = C[i][j] + C[i][k] * C[k][j];

This way, one less addition is performed per innermost for loop, and, since that loop executes n2 times, the total number of additions is n2 less: n3 − n2.

26) In Example 2.4, we gave Strassen’s product of two 2×2 matrices. Verify the correctness of this product.

Solution: We verify c11, which from regular matrix multiplication is a11b11 + a12b21.

We substitute in Strassen’s formula: c11 = m1 + m4 – m5 + m7 = = [a11 + a22] [b11 + b22] + a22 [b21 – b11] – [a11 + a12] b22 + [a12 − a22] [b21 + b22] = = a11b11 + a11b22 + a22b11 + a22b22 + a22b21 – a22b11 – a11b22 – a12b22 + a12b21 + a12b22 – a22b21– – a22 b22 All term simplify our except for a11b11 + a12b21, which is the correct expression.

The positions c12, c21, c22 are verified similarly by substitution.

27) When n=64, n3 = 262,144

28) When n=64, nlog27 = 117,649

29) Write a recurrence equation for the modified Strassen’s algorithm developed by Shmuel Winograd that uses 15 additions/subtractions instead of 18. Solve the

Chapter 2: Divide-and-Conquer

Solutions

recurrence equation, and verify your answer using the time complexity shown at the end of Section 2.5.

Solution: The recursion is the one on p.75, with 15 instead of 18: T(n) = 7T(n/2) + 15(n/2)2, if n>1 (with n = 2k) T(1) = 0 The solution follows closely that in Example B.20: By substituting xk instead of T(2k), we obtain the recursion xk = 7xk-1 + 15∙4k-1, which is nonhomogeneous, with roots 7 and 4. The solution has the form x k = c17k + c24k.

We find the constants by plugging in the values for x0 T(1) = 0, and x1 = T(2) = 15. We obtain c1 = 5, c2 = -5, so xk = 5∙7k - c2∙4k.

Going back to the variable n, we have 4k = n2, and 7k = (2log2(7))k = (2k)log2(7) = n2.8073… , so T(n) = 5n2.8073 – 5n2, which corresponds to the formula on p.76 of the text.

Section 2.6 30) Use Algorithm 2.10 (Large Integer Multiplication 2) to find the product of 1253 and 23,103.

Solution: u = 1253, v = 23103, and we calculate u × v.

No. of digits in u = 4 Level 1:

No. of digits in v = 5

n = max (4, 5) = 5 x = 12

We choose threshold = 2 digits.

m = floor(n/2) = 5/2 = 2 → 10m = 100

y = 53

w = 231

z=3

r = prod2(65, 234) → recursive call → 15,210 (see Level 2(a) below) p = prod2(12, 231) → recursive call → 2,772 (see Level 2(b) below) q = prod2(53,

3) → calculated directly → 53 x 3 = 159

return 2,772 x 10,000 + (15,210 – 2,772 – 159) x 100 + 159 = 28,948,059 Level 2:

(a) prod2(65, 234) → n = 3 > threshold, m = floor(n/2) = 1 → 10m = 10 x = 6, y = 5, w = 23, z = 4 r = prod2 (11, 27)

→ directly 11 x 18

= 297

p = prod2 ( 6, 23)

→ directly 6 x 23

= 138

Chapter 2: Divide-and-Conquer q = prod2 ( 5, 4)

Solutions → directly 5 x 4

= 20

return 138 x 100 + (297 – 138 – 20) x 10 + 20 = 15,210 (b) prod2(12, 231) → n = 3 > threshold, m = floor(n/2) = 1 → 10m = 10 x = 1, y = 2, w = 23, z = 1 r = prod2(3, 24)

→ directly 3 x 24

= 72

p = prod2(1, 23)

→ directly 1 x 23

= 23

q = prod2(2, 1)

→ directly 2 x 1

= 2

return 23 x 100 + (72 – 23 – 2) x 10 + 2 = 2,772

31) How many multiplications are needed to find the product of the two integers in Exercise 30?

Solution: We count 7 multiplications in the solution of Ex.30 (only the ones labeled directly, not counting the multiplications with powers of 10, which are linear time – see Ex. 32)

32) Write algorithms that perform the operations u x 10m; u divide 10m; u rem 10m, where u represents a large integer, m is a nonnegative integer, divide returns the quotient in integer division, and rem returns the remainder. Analyze your algorithms, and show that these operations can be done in linear time.

Solution: These are some possible algorithms. Remember that the large numbers are represented as arrays of integers (digits), as explained in Section 2.6.1. i)  

For u × 10m: Find the end of the array u; //constant time if we maintain the index of the end Insert m zero digits; //(m)

(ii)  

For u divide 10m: Find the end of the array u; // constant time if we maintain the index of the end Delete the last m digits; //(m)

(iii)  

For u rem 10m:

Find the end of the array u; // constant time if we maintain the index of the end Copy the last m digits in new array; //(m)

Chapter 2: Divide-and-Conquer

Solutions

33) Modify Algorithm 2.9 (Large Integer Multiplication) so that it divides each n-digit integer into (a) three smaller integers of n=3 digits (you may assume that n = 3k). (b) four smaller integers of n=4 digits (you may assume that n = 4k).

Analyze your algorithms, and show their time complexities in order notation.

Solution: (a) large_integer prod3 (large_integer u, large_integer v) { large_integer x1, y1, z1, x2, y2, z2, temp; int n, m; n = max (nr. of digits in u, nr. of digits in v) if (u==0 || v==0)

return 0;

else if (n <= threshold) return (u × v); else { m = floor(n/3); x1 = u divide 102m ; temp = u rem 102m; y1 = temp divide10m; z1 = temp rem 10m; x2 = v divide 102m ; temp = u rem 102m; y2 = temp divide10m; z2 = temp rem 10m; return prod3 (x1, x2) × 104m + (prod3 (x1, y2) + prod3 (x2, y1)) × 103m + (prod3 (x1, z2) + (prod3 (x2, z1) + (prod3 (y1, y2)) × 102m + (prod3 (y1, z2) + (prod3 (z1, y2)) × 10m + (prod3 (z1, z2); } } Complexity analysis: W(n) = 9W(n/3) + c·n for n > 1, n = 3k, and W(1) = 1.

Solving it as a non-homogeneous equation with the substitution T(3 k) = xk, we have the roots 9 and 3, so the dominant term in the solution is 9 k = (3k)2 = n2. The algorithm is (n2). (b) The function

large_integer prod4 (large_integer u, large_integer v)

is written in a manner similar to prod3() above. Each argument is now broken into four parts: w1, x1, y1, z1, and w2, x2, y2, z2.

Complexity analysis: The recurrence is

Chapter 2: Divide-and-Conquer

Solutions

W(n) = 16W(n/4) + c·n for n > 1, n = 4k, and W(1) = 1.

Solving it as a non-homogeneous equation with the substitution T(4 k) = xk, we have the roots 16 and 4, so the dominant term in the solution is 16 k = (4k)2 = n2. The algorithm is still (n2).

Section 2.7 34) Implement both Exchange Sort and Quicksort algorithms on your computer to sort a list of n elements. Find the lower bound for n that justifies application of the Quicksort algorithm with its overhead.

Solutions and bounds will vary.

35) Implement both the standard algorithm and Strassen's algorithm on your computer to multiply two n x n matrices (n = 2k). Find the lower bound for n that justifies application of Strassen's algorithm with its overhead.

Solutions and bounds will vary.

36) Suppose that on a particular computer it takes 12n2 s to decompose and recombine an instance of size n in the case of Algorithm 2.8 (Strassen). Note that this time includes the time it takes to do all the additions and subtractions. If it takes n 3 s to multiply two n x n matrices using the standard algorithm, determine thresholds at which we should call the standard algorithm instead of dividing the instance further. Is there a unique optimal threshold?

Solution: At the end of Section 2.3, there are mentioned three possible methods to deal with the cases where n is not a power of 2; this solution uses the second one, i.e. if at some step of the algorithm n is odd, we pad the matrix with one row and one column of zeroes to make the size even. The recursion is:

n T(n) = 7 T      12n 2 , 2

if n is even

n 7 T      12(n  1) 2 , if n is odd 2

Chapter 2: Divide-and-Conquer

Solutions

t  We find the threshold t separately for n even and odd. In either case,   < t, so the 2 3

t  t  recursion at   is   . 2 2

For n even, we have 7/8∙t3 + 12t2 = t3, with solution t = 96.

For n odd, we have 7((t+1)/2)3 + 12(t+1)2 = t3, which leads to a cubic equation. Solving numerically, we find t = 118.8, so for n ≥ 119 we split, but for n ≤ 117 we take the direct approach.

Since the thresholds are different, there is no unique optimal threshold: in between 96 and 118 we take different decisions based on whether n is odd or even.

Section 2.8

37) Use the divide-and-conquer approach to write a recursive algorithm that computes n!.

Define the input size (see Exercise 36 in Chapter 1), and answer the following questions.

Does your function have an exponential time complexity?

Does this violate the statement of case 1 given in Section 2.8?

Solution:

Input: n and m (m = 1 for first call) Output: n! integer factorial (n, m) { if (n==m) return n; else { if (m > n) return 1; else n  m n  m return (factorial (n,  ) × factorial (  + 1, m));   2   2  } } Complexity analysis: In the above algorithm, an instance of size n – m + 1 is divided into two instances, each of almost size (n – m + 1)/2. The recurrence is T(n) = 2T(n/2) + 1, which is easily solved as a non-homogeneous equation (Theorem B.3) or by direct expansion to yield T(n) = n – 1 ∊ (n).

Chapter 2: Divide-and-Conquer

Solutions

If, however, we define the input size as the number of bits required to represent the input n (as in Exercise 36 in Chapter 1), then:   



The input size is log2n.

The output size is log2(n!) = log2(2) + log2(3) + … + log2(n/2 -1) + log2(n/2) + … + log2(n).

Let n = 2k o The input size is k o The output size is log2(2) + log2(3) + … + log2(n/2 -1) + (k – 1) + … + k. o Since the sequence is strictly increasing, the sum above is less than o (k – 1) + (k – 1) + … + (k – 1) = (k – 1)·n/2 = (k – 1)2k-1.

Therefore, the relationship between input and output is k → (k·2k), which means a super-exponential time complexity.

The statement of case 1 in Section 2.8 is not violated, since the two instances are not each of size almost equal to the input size; as seen in the last sum above, the second instance is (k – 1)2k-1, that is super-exponential in the input k.

38) Suppose that, in a divide-and-conquer algorithm, we always divide an instance of size n of a problem into n sub-instances of size n/3, and the dividing and combining steps take linear time. Write a recurrence equation for the running time T (n), and solve this recurrence equation for T (n). Show your solution in order notation.

Solution: T(n) = n(T(n/3) + C), for n > 1, and T(1) = 0.

We consider n = 3k and substitute T(3k) by xk to obtain the recurrence xk = 3k(xk-1 + C). This hasn’t any of the general forms covered in Appendix B, so we perform direct substitution: After a few steps, a pattern emerges. Below is the expression after m substitutions: xk = 3k+(k-1)+…+(k-m+1)xk-m + [3k+(k-1)+…+(k-(m-1)) + 3k+(k-1)+…+(k-(m-2)) + … + 3k+(k-1) + 3k ]·C, and so, after k-1 substitutions, we have: xk = 3k+(k-1)+…+2+1x0 + [3k+(k-1)+…+2+1 + 3k+(k-1)+…+2 + … + 3k+(k-1) + 3k ]·C = =

0

+ [3k+(k-1)+…+2+1 + 3k+(k-1)+…+2 + … + 3k+(k-1) + 3k ]·C = = 3k ·[3(k-1)+…+2+1 + 3(k-1)+…+2 + … + 3(k-1) + 1]·C

Chapter 2: Divide-and-Conquer

Solutions

Complexity analysis: The first exponent in the bracket above is k(k-1)/2, as shown in Example A.1. We replace all the terms in the bracket with the first one, obtaining the inequality xk < 3k(k-1)3k(k-1)/2C ≈ n(lg n)n(lg n)/2, so the algorithm is O( n1+(lg n)/2 lg n).

Additional Exercises

39) Implement both algorithms for the Fibonacci Sequence (Algorithms 1.6 and 1.7). Test each algorithm to verify that it is correct. Determine the largest number that the recursive algorithm can accept as its argument and still compute the answer within 60 seconds. See how long it takes the iterative algorithm to compute this answer.

/*Program that times Fibonacci function calls*/ #include <time.h> #include <iostream> using namespace std; //Recursive fibonacci int fibo_rec(int n){ if ( n == 0 || n == 1 ) /* base case */ return n; else /* recursive step */ return fibo_rec(n - 1) + fibo_rec(n - 2); } //Iterative Fibonacci int fibo_iter(int n){ if ( n == 0 || n == 1 ) /* base case */ return n; int minus_two = 0; int minus_one = 1; int fi; for (int i=2; i<=n; i++){ fi = minus_two + minus_one; minus_two = minus_one; minus_one = fi; } return fi; } int main(){ time_t begin, end; int N; cout << ("Enter positive integer: "); cin >> N; begin = time(NULL); cout << "Fibonacci recursive: " << fibo_rec(N) << endl;

Chapter 2: Divide-and-Conquer

Solutions

end = time(NULL); cout << "\ttime = " << difftime(end, begin) << " s" << endl; begin = time(NULL); cout << "Fibonacci iterative: " << fibo_iter(N) << endl; end = time(NULL); cout << "\ttime = " << difftime(end, begin) << " s" << endl <<endl; return 0; }

As seen from the following output, on the particular machine on which the program was run, N=44 was the largest Fibonacci number that required less than 60 s recursively.

The iterative solution took less than 1 s (and probably actually less than 1 microsecond!).

40) Problem: Search for a key x in a sorted two-dimensional array of m  n values. The array is sorted by row and column.

Input: positive integers n and m, two-dimensional sorted array of keys S[][] indexed by pairs (1, 1) through (n,m), address of two integer variables x and y to store result.

Output: index pair of key if found, (0,0) otherwise. void find(int n, int m, keytype S[][], int &x, int &y) { index low = 1, high = n, mid = (low+high)/2; while(low <= high) { if(S[mid][1] <= key && key <= S[mid][m-1]) { break; } if(S[mid][m-1] < key) { low = mid + 1; } else { high = mid – 1; } mid = (low+high)/2; } x = mid; low = 1; high = m-1; mid = (low + high)/2; while(low <= high) {

Chapter 2: Divide-and-Conquer

Solutions

if(S[x][mid] == key) { y = mid; return; } if(S[x][mid] < key) { low = mid + 1; } else { high = mid – 1; } mid = (low+high)/2; } x = 0; y = 0; } Time complexity is O(log2n + log2m).

41) Suppose that there are n = 2k teams in an elimination tournament, in which there are n=2 games in the first round, with the n/2 = 2k-1 winners playing in the second round, and so on. (a) Develop a recurrence equation for the number of rounds in the tournament. (b) How many rounds are there in the tournament when there are 64 teams? (c) Solve the recurrence equation of part (a).

Solution: (a) Obviously, with 1 team, there are no rounds needed, so T(1) = 0. With 2 teams, one round is needed, so T(2) = 1. In general, T(2k) = 1 + T(2k-1). (b) T(64) = T(26) = 1 + 1 + 1 + 1 + 1 + 1 + T(1) = 6. (c) Use simple induction on k to show that T(2k) = k , so in terms of n we have T(n) = log2n.

42) Problem: Tile an n  n board (where n is a power of 2) with trominoes so that only one given square x on the board is left uncovered and no two trominoes overlap.

Input: integer n, location x of square to be left uncovered Output: a board tiled according to the problem specifications void tile(int n, location x) {

Chapter 2: Divide-and-Conquer

Solutions

if(n==2) { tile with one tromino so that all rules are obeyed (only one possibility); return; } divide board in to four n/2  n/2 sub-boards; place one tromino so that each of its three squares covers one square in each of the three sub-boards that do not contain x; let x2, x3, and x4 be the locations of the squares in each sub-board that have been covered by the placed tromino; tile(n/2, x); tile(n/2, x2); tile(n/2, x3); tile(n/2, x4); }

43) a) Divide the nine coins into 3 groups of 3.

Compare the weights of two of the groups (first weighing).

If either of the groups is heavier, choose the heavier group.

If both groups are equal, choose the third group that wasn’t weighed.

Compare the weights of two coins in the heavier group (second weighing).

If one is heavier, that is the counterfeit coin.

If both are equal, the un-weighed coin is the counterfeit coin. b) Problem: Given n coins indexed from 1 to n, one of which is counterfeit and heavier than all the others, find the index of the counterfeit coin.

Input: integer n, n coins indexed from 1 to n Output: index of the counterfeit coin int findCounterfeit(int n) { index i = n, j = 1; while(i>=3) { int w1 = weight(j, j + i/3 - 1); int w2 = weight(j+i/3, j+2*i/3 - 1);

Chapter 2: Divide-and-Conquer

Solutions

if(w1 < w2) j = j + i/3; else if(w1 == w2) j = j + 2*i/3 + 1; i = i/3; } return j; } Here weight(x, y) returns the total weight of the coins of index x through index y.

The complexity of this algorithm is O(log3 n).

44) Write a recursive (n∙lg n) algorithm whose parameters are three integers x, n, and p, and which computes the remainder when xn is divided by p. For simplicity, you may assume that n is a power of 2, that is, that n = 2k for some positive integer k.

Solution: If we have two integers, m and n, we can divide each by p to obtain their remainders: m = q1·p + r1, n = q2·p + r2.

The product m·n is (q1·p + r1)(q2·p + r2) = (q1·q2p + q1·r2 + q2·r1)p + r1·r2, which shows that only r1·r2 must be considered when calculating the remainder of m·n modulo p: m·n % p = r1·r2 % p.

Back to our problem, we use the divide and conquer approach by splitting the problem thus: xn = xn/2·xn/2. n 2  

n 2  

In general n is not even, we have to use floor and ceiling: xn = x  x , and then we recursively solve the two smaller instances until we reach the base case n = 1. n n For complexity analysis, we have the recursion T(n) = T(   ) + T(   ) + C, where C is the 2 2 (constant) cost of multiplying two integers and finding their remainder modulo p. If we now make the simplifying assumption n = 2k, then the instances are always even, and we have the recursion T(n) = 2T(n/2) +C, which is solved as a non-homogeneous recursion (Theorem B.3): T(n) = (2C-1)n – C ∊ (n). Using the techniques in Section B.4, we can prove that the algorithm is (n) for all n.

If we know in advance that n = 2k , the algorithm becomes (lg n).

Chapter 2: Divide-and-Conquer

Solutions

45) Use the divide-and-conquer approach to write a recursive algorithm that finds the maximum sum in any contiguous sublist of a given list of n real values.

Analyze your algorithm, and show the results in order notation.

Solution: This is known as the Maximum Subarray problem, and there are several algorithms that solve it. Here we present the divide-and-conquer algorithm published by John Bentley in the journal article Programming Pearls (Communications of the ACM, vol.27, 1984): 

Split the array in half, as suggested by the vertical line in the figure:

ML  

MC

MR

Recursively calculate the maximum subarrays in the left and right halves, M L, and MR. Combine the solutions:  We have to take into account the possibility that the maximum subarray straddles the boundary, so we have to find MC.  Return max(ML, MC, MR).

Complexity analysis: Finding MC is done in n comparisons (worst case), since all we need to do is move left and right from the boundary until we hit the first negative element on each side. The maximum of 3 elements is found with 2 comparisons.

We have the recursion T(n) = 2T(n/2) + (n+2), which, as in the case of Mergesort, is proven to be (n·lg n).

Chapter 3: Dynamic Programming

Solutions

Section 3.1

1) Establish Equality 3.1 given in this section.

If k = 0, the definition of the binomial coefficient

gives n!/(1∙n!) = 1. If k=n, the same definition yields n!/(n!∙1) = 1.

 n  1  n  1 (n  1)! (n  1)!       We are left with the case 0 < k < n:  =  k  1  k  (k  1)!(n  k )! k!(n  k  1)! n n(n  1)! (n  1)! n! (n  1)! (n  1)! = (k  n  k ) = = =   k  (n  k ) k!(n  k )! k!(n  k )!  k  k!(n  k )! k!(n  k )! k!(n  k )!

2) Use induction on n to show that the divide-and-conquer algorithm for the Binomial Coefficient problem (Algorithm 3.1), based on Equality 3.1, computes

n n 2   – 1 terms to determine   . k  k 

Induction base: If n = 1, there are two possibilities for 0 ≤ k ≤ n: 

If k == 0, Algorithm 3.1 returns the term directly, so it calculates 1 term. The formula 2 n   – 1 also gives 2∙1 – 1 = 1. k   If k = 1 we have k == n, and Algorithm 3.1 again returns the term directly, so it calculates 1 n term. The formula 2   – 1 also gives 2∙n – 1 = 2 – 1 = 1. k  Induction hypothesis: The result holds up to n.

Induction step: We prove that the result holds for n + 1: The cases k == 0 and k == n are easy, so consider general k: Algorithm 3.1 returns the sum bin(n+1-1, k-1) + bin(n+1-1, k) = bin(n,

Chapter 3: Dynamic Programming

Solutions

 n  n  – 1 + 2   – 1 + 1 (the k-1) + bin(n,k), which, according to the hypothesis, requires 2   k 1 k   n  n  +   ) – 1. According to formula 3.1, the sum of last 1 counts the addition)= 2(   k 1  k   n  1  n  1  , so we have 2   – 1. Q.e.d. binomials is   k   k 

3) Implement both algorithms for the Binomial Coefficient problem (Algorithms 3.1 and 3.2) on your system and study their performances using different problem instances.

/*Program that times Binomial function calls*/ #include <time.h> #include <iostream> using namespace std; //Recursive binomial int bino_rec(int n, int k){ if ( k == 0 || k == n ) /* base case */ return 1; else /* recursive step */ return bino_rec(n-1, k-1) + bino_rec(n-1, k); } int B[101][101];

//for algorithm simplicity only!

//Dynamic programming binomial int bino_dyn(int n, int k){ int min; for (int i=0; i<=n; i++){ if (i<k) min = i; else min = k; for(int j=0; j<=min; j++) if (j==0 || j==i) B[i][j] = 1; else B[i][j] = B[i-1][j-1] + B[i-1][j]; } return B[n][k]; } int main(){ time_t begin, end; int N, k; cout << ("Enter positive integer N(<=100): "); cin >> N;

Chapter 3: Dynamic Programming

Solutions

cout << ("Enter positive integer k(<=100): "); cin >> k; begin = time(NULL); cout << "Binomial recursive : " << bino_rec(N,k) << endl; end = time(NULL); cout << "\ttime = " << difftime(end, begin) << " s" << endl; begin = time(NULL); cout << "Fibonacci dynamic prog.: " << bino_dyn(N,k) << endl; end = time(NULL); cout << "\ttime = " << difftime(end, begin) << " s" << endl <<endl; return 0; }

As expected, increasing k makes the recursive algorithm “go exponential”.

4) Modify Algorithm 3.2 (Binomial Coefficient Using Dynamic Programming) so that it uses only a one-dimensional array indexed from 0 to k.

int bin3 (int n, int k) { index i ; int B[0..k ] = {0};

//initialize with zeroes

int saved_old, saved_new;

//two temporary variables

for (i = 0; i <= n; i++) { saved_old = saved_new = B[0]; for (j = 0; j <= minimum(i ,k); j++) if (j == 0 || j == i) B[ j ] = 1; else { saved_new = B[j]; B[ j ] = saved_old + B[j ];

Chapter 3: Dynamic Programming

Solutions

saved_old = saved_new; } //end else } //end for return B[k ]; }

Section 3.2

5) Use Floyd’s algorithm for the Shortest Paths problem 2 (Algorithm 3.4) to construct the matrixD, which contains the lengths of the shortest paths, and the matrixP, which contains the highest indices of the intermediate vertices on the shortest paths, for the following graph. Show the actions step by step.

Constructing D: D0: 0 3 INF INF INF INF INF

4 0 6 5 INF INF INF

INF INF 0 15 12 INF INF

INF 18 INF 0 1 INF 8

INF INF INF 2 0 INF INF

10 INF INF 19 INF 0 INF

INF INF INF 5 INF 10 0

D1: 0

4

INF

INF

INF

10

INF

Chapter 3: Dynamic Programming

Solutions

3 INF INF INF INF INF

0 6 5 INF INF INF

INF 0 15 12 INF INF

18 INF 0 1 INF 8

INF INF 2 0 INF INF

13 INF 19 INF 0 INF

INF INF 5 INF 10 0

D2: 0 3 9 8 INF INF INF

4 0 6 5 INF INF INF

INF INF 0 15 12 INF INF

22 18 24 0 1 INF 8

INF INF INF 2 0 INF INF

10 13 19 18 INF 0 INF

INF INF INF 5 INF 10 0

D3: 0 3 9 8 21 INF INF

4 0 6 5 18 INF INF

INF INF 0 15 12 INF INF

22 18 24 0 1 INF 8

INF INF INF 2 0 INF INF

10 13 19 18 31 0 INF

INF INF INF 5 INF 10 0

D4: 0 3 9 8 9 INF 16

4 0 6 5 6 INF 13

37 33 0 15 12 INF 23

22 18 24 0 1 INF 8

24 20 26 2 0 INF 10

10 13 19 18 19 0 26

27 23 29 5 6 10 0

D5: 0 3 9 8 9 INF 16

4 0 6 5 6 INF 13

36 32 0 14 12 INF 22

22 18 24 0 1 INF 8

24 20 26 2 0 INF 10

10 13 19 18 19 0 26

27 23 29 5 6 10 0

D6: 0 3

4 0

36 32

22 18

24 20

10 13

20 23

Chapter 3: Dynamic Programming

Solutions

9 8 9 INF 16

6 5 6 INF 13

0 14 12 INF 22

24 0 1 INF 8

26 2 0 INF 10

19 18 19 0 26

29 5 6 10 0

D7: 0 3 9 8 9 26 16

4 0 6 5 6 23 13

36 32 0 14 12 32 22

22 18 24 0 1 18 8

24 20 26 2 0 20 10

10 13 19 18 19 0 26

20 23 29 5 6 10 0

Constructing P: P0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0

0 0 0 0 0 0 0

0 0 0 0 0 0 0

0 0 0 0 0 0 0

P1: 0 0 0 0 0 0 0

0 0 0 0 0 0 0

0 0 0 0 0 0 0

0 0 0 0 0 0 0

0 0 0 0 0 0 0

0 0 0 0 0 0 0

0 0 0 0 0 0 0

P2: 0 0 2 2 0 0 0

0 0 0 0 0 0 0

0 0 0 0 0 0 0

2 0 2 0 0 0 0

0 0 0 0 0 0 0

0 0 2 2 0 0 0

0 0 0 0 0 0 0

P3: 0 0

0 0

0 0

2 0

0 0

0 0

0 0

Chapter 3: Dynamic Programming

Solutions

2 2 3 0 0

0 0 3 0 0

0 0 0 0 0

2 0 0 0 0

0 0 0 0 0

2 2 3 0 0

0 0 0 0 0

P4: 0 0 2 2 4 0 4

0 0 0 0 4 0 4

4 4 0 0 0 0 4

2 0 2 0 0 0 0

4 4 4 0 0 0 4

0 0 2 2 4 0 4

4 4 4 0 4 0 0

P5: 0 0 2 2 4 0 4

0 0 0 0 4 0 4

5 5 0 5 0 0 5

2 0 2 0 0 0 0

4 4 4 0 0 0 4

0 0 2 2 4 0 4

4 4 4 0 4 0 0

P6: 0 0 2 2 4 0 4

0 0 0 0 4 0 4

5 5 0 5 0 0 5

2 0 2 0 0 0 0

4 4 4 0 0 0 4

0 0 2 2 4 0 4

6 4 4 0 4 0 0

P7: 0 0 2 2 4 7 4

0 0 0 0 4 7 4

5 5 0 5 0 7 5

2 0 2 0 0 7 0

4 4 4 0 0 7 4

0 0 2 2 4 0 4

6 4 4 0 4 0 0

Chapter 3: Dynamic Programming

Solutions

6) Use the Print Shortest Path algorithm (Algorithm 3.5) to find the shortest path from vertex v7 to vertex v3, in the graph of Exercise 5, using the matrix P found in that exercise. Show the actions step by step.

The shortest path from v7 to v3 is: v7 -> v4 -> v5 -> v3 Steps: 1) Given P7 from Exercise 5 P7: 0 0 5 2 4 0 0 5 0 4 2 0 0 2 4 2 0 5 0 0 4 4 0 0 0 7 7 7 7 7 4 4 5 0 4 and q=7, r=3, call path(7, 3).

0 0 2 2 4 0 4

6 4 4 0 4 0 0

2) Call path(7, 5).

3) Call path(7, 4) -> P[q][r]==0, so return.

4) Print v4.

5) Call path(4, 5) -> P[q][r]==0 so return.

6) Return from call to path(7, 5).

7) Print v5.

8) Call path(4 ,5) -> P[q][r] ==0, so return.

9) Return from call to path(7, 3).

10) Done. Intermediate vertices v4 and v5 on shortest path from v7 to v3 have been printed in order.

7) Analyze the Print Shortest Path algorithm (Algorithm 3.5) and show that it has a lineartime complexity.

To evaluate complexity, we count the number of array accesses P[i][j].

The path from any vertex q to any vertex r has in the worst case n-1 edges (n is the nr. of vertices in the graph). For each of the n-1 edges, Algorithm 3.4 evaluates P[i][j] four times, so the worst-case complexity is 4(n-1) (n).

Chapter 3: Dynamic Programming

Solutions

8) Implement Floyd’s algorithm for the Shortest Paths problem 2 (Algorithm 3.4) on your system, and study its performance using different graphs.

Implementations and performance will vary.

9) Can Floyd’s algorithm for the Shortest Paths problem 2 (Algorithm 3.4) be modified to give just the shortest path from a given vertex to another specified vertex in a graph? Justify your answer.

Working backwards from D(n)(i,j), we note that it only depends on 3 values from the previous table D(n-1). Based on this it is possible to reduce the number of operations by only calculating one value from D(n), three values from D(n-1), etc. As we regress towards D(0), however, the number of values grows and ends up being n2 (the entire table), and the asymptotic complexity remains (n3).

10) Can Floyd’s algorithm for the Shortest Paths problem 2 (Algorithm 3.4) be used to find the shortest paths in a graph with some negative weights? Justify your answer.

Yes, as long as there are no cycles whose total length is negative (in which case the path can be made arbitrarily small by repeating such negative cycles).

Section 3.3 11) Find an optimization problem in which the principle of optimality does not apply and therefore the optimal solution cannot be obtained using dynamic programming. Justify your answer.

The text example: Finding the longest path between two vertices A → C of a graph (without loops) does not have optimal substructure: if the longest path is A → … → B → … → C, this doesn’t mean that:  the subpath B → … → C is longest; for instance, in the B→C problem we may find that the longest path contains A, which would create a cycle in the original problem A → C, so it’s not allowed there.  the subpath A → … → B is longest; for instance, in the A→B problem we may find that the longest path contains C. Another example: Raising a number to an integer power n with the fewest number of multiplications (a.k.a. the Addition Chain Exponentiation problem). When n is a power of 2, the best way is successive squaring, e.g. x8 =((x∙x)2)2 requires only 3 multiplications. If n is not a power of 2, however, there is no optimal substructure, because a subproblem is not always solved optimally by itself; we can prefer a suboptimal solution to a subproblem because parts

Chapter 3: Dynamic Programming

Solutions

of the solution can be reused to solve other subproblems. This problem was proved to be NPcomplete (see Ch.9).

Section 3.4 12) List all of the different orders in which we can multiply five matrices A, B, C, D, and E. There are 14 possible orderings: A(B((CD)E)) A(B(C(DE))) A((BC)(DE)) A(((BC)D)E) A((B(CD))E) (AB)((CD)E) (AB)(C(DE)) (A(BC))(DE) ((AB)C)(DE) (A(B(CD)))E (A((BC)D))E ((AB)(CD))E ((A(BC))D)E (((AB)C)D)E

13) Find the optimal order, and its cost, for evaluating the product A1×A2× A3×A4×A5, where A1 is (10×4) A2 is (4×5) A3 is (5×20) A4 is (20×2) A5 is (2×50) Show the final matrices M and P produced by Algorithm 3.6.

M:

Chapter 3: Dynamic Programming 0

Solutions

200 0

1200 400 0

320 240 200 0

1320 640 240 2000 0

1

1 2

1 2 3

4 4 4 4

P:

Optimal order = (A1(A2(A3∙A4)))A5

Optimal cost: 1320 multiplications

14) Implement the Minimum Multiplications algorithm (Algorithm 3.6) and the Print Optimal Order algorithm (Algorithm 3.7) on your system, and study their performances using different problem instances.

Implementations and performances will vary.

15) Show that a divide-and-conquer algorithm based on Equality 3.5 has an exponential-time complexity.

For illustration, we use the M table from Exercise 13: 0

200 0

1200 400 0

320 240 200 0

1320 640 240 2000 0

As in the analysis of the Minimum Multiplications algorithm, we take the basic operation to be one evaluation of k. A problem of size n (obtaining the 1320 in example above) requires evaluation of more than two problems of size n-1 (320 and 640 above), so we have the inequality T(n) ≥ 2T(n-1) ≥ 2∙2T(n-2) ≥ … ≥ 2n.

Chapter 3: Dynamic Programming

Solutions

16) a) Problem: Determine the number of different orders in which n matrices can be multiplied.

Input: An integer n representing the number of matrices to be multiplied.

Output: Number of different orders in which n matrices can be multiplied. int numOrderings(int n) { rec(1, n); } int recNumOrderings(int j, int k) { if(k-j<=1) return 1; int sum = 0; for(int i=j; i<k; i++) { sum += rec(j,i)*rec(i+1,k); } return sum; } b) n=2: 1 n=3: 2 n=4: 5 n=5: 14 n=6: 42 n=7: 132 n=8: 429 n=9: 1430 n=10: 4862

Chapter 3: Dynamic Programming

Solutions

17) Establish the equality

n 1

n 1

d 1

d 1

By multiplying out each term inside the sum, we have: n d   d 2 . The first sum is solved in Example A.1, and the second in Example A.2; we only have to replace n with n-1: (n  1)n (n  1)n [2(n  1)  1] n(n  1)( n  1) n  = .

2 6 6

18) Show that to fully parenthesize an expression having n matrices we need n−1 pairs of parentheses.

Induction on n: Base case: for n = 1, we have only one matrix, which needs 0 pairs. n – 1 = 1 – 1 = 0.

Hypothesis: Result holds up to n.

Induction step: We show that the result holds for n + 1: Number the matrices from 1 to n, and let k be the “breaking point” of the top-level product: (A1…Ak)∙(Ak+1…An+1) The left factor has k matrices, and the right has n-k+1. Both numbers are ≤ n, so we can apply to them the hypothesis: the left needs k – 1 pairs, and the right n – k, for a total of n – 1. We now apply the outer parenthesis to obtain n pairs: ((A1…Ak)∙(Ak+1…An+1)).

19) Analyze Algorithm 3.7 and show that it has a linear-time complexity.

Every call to order prints one pair of parentheses, and according to Exercise 18, a problem of size n needs only n-1 pairs, which is (n).

20) Write an efficient algorithm that will find an optimal order for multiplying n matrices A1 ×A2×···×An, where the dimension of each matrix is 1×1, 1×d, d×1, or d×d for some positive integer d. Analyze your algorithm and show the results using order notation.

We use the array of (n+1) dimensions d from Algorithm 3.6, but here we call it Dim to distinguish it from the integer d. The elements of Dim are either 1 and d.

1. We scan Dim and make each 1 the boundary of a sub-product. ((n)) 2. For each sub-product Ai … Aj:

Chapter 3: Dynamic Programming   

Solutions

If both Ai … Aj: are 1x1, then we can multiply all the intervening dxd matrices in any order. We arbitrarily choose Ai+1(Ai+2( … Aj-1) …) ((j-i)) If Ai is 1xd, then we multiply from left to right Ai(Ai+1( … Aj) …) ((j-i)) If Aj is dx1, then we multiply from right to left (…(Ai( … Aj-1)Aj) ((j-i))

All the branches of Step 2 are linear, with the same multiplicative constant (the numbers of operations needed to insert a pair of parentheses), so the entire Step 2 is (n), which makes the entire algorithm (n).

Section 3.5 21) How many different binary search trees can be constructed using six distinct keys?

By the formula in Exercise 36: (1/7)*(12 choose 6) = (1/7)*924 = 132

22) Create the optimal binary search tree for the following items, where the probability occurrence of each word is given in parentheses: CASE (.05), ELSE (.15), END (.05), IF (.35), OF (.05), THEN (.35).

For simplicity, number the words from CASE = 1 to THEN = 6 A: 1 2 3 4 5 6 7

0

1

2

3

4

5

6

0

.05

.25

.35

.95

1.05

1.8

0

.15

.25

.8

.9

1.65

0

.05

.45

.55

1.3

0

.35

.45

1.2

0

.05

.45

0

.35 0

Chapter 3: Dynamic Programming

Solutions

R: 1

0

1

2

3

4

5

6

0

1

2

2

4

4

4

0

2

2

4

4

4

0

3

4

4

4

0

4

4

4

0

5

6

0

6

2 3 4 5 6 7

0

Optimal tree:

4 2

6

1

3

5

j

23) Find an efficient way to compute  pm which is used in the Optimal Binary Search Tree m i

Algorithm (Algorithm 3.9). j

The partial sums can be stored in an n x n table B, whose position B[i][j] stores  pm . We m i

have the recursion B[i][j+1] = B[i][j] + pj+1.

Below is a partial example with the probabilities from Exercise 22: 1 2 3 4 5 6

1

2

3

.05

.20

.25

4

5

6

.15 .05 .35 .05 .35

Chapter 3: Dynamic Programming

Solutions

24) Implement the Optimal Binary Search Tree algorithm (Algorithm 3.9) and the Build Optimal Binary Search Tree algorithm (Algorithm 3.10) on your system, and study their performances using different problem instances.

Implementations and performances will vary.

25) Analyze Algorithm 3.10, and show its time complexity using order notation.

We define the basic operation to be the access of an element of the array R; this is done once per each call to the function tree, and each such call inserts in the tree one node. Since there are n nodes total, the number of calls and that of basic operations is n, so the complexity is linear, (n).

26) Generalize the Optimal Binary Search Tree algorithm (Algorithm 3.9) to the case in which the search key may not be in the tree. That is, you should let qi, in which i =0, 1, 2, . . . , n, be the probability that a missing search key can be situated between Keyi and Keyi+1. Analyze your generalized algorithm and show the results using order notation.

The solution is very similar, only, aside from the keys, we now also have the intervals I0 = [-∞…Key1], I1=[Key1…Key2], … , In-1 = [Keyn-1…Keyn], In = [Keyn…+∞] The intervals can be thought of as leaves of the tree, 40 20

I0

60

I1

I5

50

30 I2

I3

I4

however, they are not really there, i.e. there are no pointers pointing to them. Their j

probabilities simply add up into the sum of probabilities of the keys: instead of  pm , we now m i

j

j

m i

m  i 1

have  pm +  qm . The search function can either stop at an internal node (key), or “at a

Chapter 3: Dynamic Programming

Solutions

leaf” … but the leaves are not accessed through an extra pointer operation from the key, so the depth of the interval represented by the leaf is the same as its parent key.

Complexity is identical: (n3).

27) Show that a divide-and-conquer algorithm based on Equality 3.6 has an exponential time complexity.

For illustration, we use the A table from Exercise 22:

1 2 3 4 5 6 7

0

1

2

3

4

5

6

0

.05

.25

.35

.95

1.05

1.8

0

.15

.25

.8

.9

1.65

0

.05

.45

.55

1.3

0

.35

.45

1.2

0

.05

.45

0

.35 0

As in the analysis of the Minimum Multiplications algorithm, we take the basic operation to be one evaluation of k. A problem of size n (obtaining the 1.8 in example above) requires evaluation of more than two sub-problems of size n-1 (1.05 and 1.65 above), so we have the inequality T(n) ≥ 2T(n-1) ≥ 2∙2T(n-2) ≥ … ≥ 2n.

Chapter 3: Dynamic Programming

Solutions

Section 3.6 28) Find an optimal circuit for the weighted, direct graph represented by the following matrix W. Show the actions step by step.

D[v2][∅] = 3 D[v3][∅] = 4 D[v4][∅] = 6 D[v5][∅] = 10 D[v2][v3] = 11

D[v2][v4] = 14

D[v2][v5] = 20

D[v3][v2] = 14

D[v3][v4] = 16

D[v3][v5] = 17

D[v4][v2] = 9

D[v4][v3] = 11

D[v4][v5] = 21

D[v5][v2] = 9

D[v5][v3] = 6

D[v5][v4] = 7

D[v2][v3,v4] = max(7+16, 8+11) = 19, j = 3

D[2][3,5] = 16, j = 5 D[2][4,5] = 16, j = 5

D[3][2,4] = 19, j = 3 D[3][2,5] = 16, j = 5 D[3][4,5] = 14, j = 5 D[4][2,3] = 17, j = 2 D[4][2,5] = 20, j = 5 D[4][3,5] = 17, j = 5 D[5][2,3] = 16, j = 3 D[5][2,4] = 10, j = 4 D[5][3,4] = 12, j = 4 D[2][3,4,5] = 22, j=5 D[3][2,4,5] = 17, j=5 D[4][2,3,5] = 22, j=2 D[5][2,3,4] = 18, j=4 D[1][2,3,4,5] = min(30, 30, 40, 38), which shows that there are two minimal tours of cost 30, one with j = 2, the other with j = 3: Minimal Tour A: 1→2→5→4→3→1

Minimal Tour B: 1→3→5→4→2→1

29) Write a more detailed version of the Dynamic Programming algorithm for the TSP (Algorithm 3.11).

One design decision is how to store the subsets of vertices. This can be done, for example, with an array of n linked lists, each of them holding the set of vertices (from zero – empty set – to (n – 2) sets) and the respective optimal cost and j vertex associated.

Below is an example of matrix D for n = 5 (from Exercise 28). The “ground” symbol means a null pointer. Only the line for v2 is shown:

Chapter 3: Dynamic Programming

Solutions

The sets themselves can be stored as strings, arrays of boolean (used below), or even the data structure described in Appendix C. For homogeneity, the list node for the empty set can be made the same as the others (although technically it does not need a j index or a next pointer): struct NodeType{ bool set[n-1]; int min; int jay; struct NodeType* next; };

When we generate the subsets A ⊆ V− {v1} containing k vertices, we need to store them in an array: bool arrayOfSubsets[1..maxSubsets][n-1]

where maxSubsets is the maximum over k of Binomial(n-2, k), so it can store the most subsets that can be generated. Alternatively, the subsets can also be stored in a linked list.

The more detailed pseudocode is: void travel (int n, const number W[][], index P[][], number& minlength) { index i , j , k, temp_j, minIndex; number value, temp_min; struct nodeType* D[2..n][ 0..n-2]; for (i = 2; i <= n; i++) D[ i ][0]->min = W[ i ][1]; for (k = 1; k <= n−2; k++) {

Chapter 3: Dynamic Programming

Solutions

Generate all subsets A ⊆ V−{v1} containing k vertices, store them in arrayOfSubsets; for ( all subsets A in arrayOfSubsets) for (i such that i != 1 and vi is not in A) { temp_min = INFINITY; for (each j:vj∈A) { find in the list D[ j ][k-1] the node x containing the set A− {vj}; value = W[ i ][ j ] + x.min; if (value < temp_min) { temp_min = value; temp_j = j ; } } Navigate to end of list D[i][k]; Append a node y with: y.min = temp_min; y.jay = temp_j; } //end for i } } //end for k minlength = minimum (W[1][ j ] + D[ j ][V− {v1,vj}]); 2≤j≤n minIndex = value of j that gave the minimum; }

30) Implement your detailed version of Algorithm 3.11 from Exercise 27 on your system and study its performance using several problem instances.

Implementations and performances will vary.

Section 3.7 31) Analyze the time complexity of Algorithm opt, which appears in Section 3.7.

The complexity of the recursive version (Algorithm 3.12) is worse than exponential: Denote by T(n,m) the complexity of calculating opt(0,0) for the original two sequences.

Without loss of generality, let n be the smaller length. Clearly, T(n,m) ≤ T(n,n), which in turn we denote S(n). The recursive call on the last line of Algorithm 3.12 shows that each instance of opt calls three sub-instances with sizes n-1 or larger, so we have S(n) ≤ 3S(n-1), which by repeated substitution leads to S(n) ≤ 3n.

Chapter 3: Dynamic Programming

Solutions

If we use the Dynamic Programming version, however, the table used in the solution has dimensions [0..n+1] and [0..m+1], therefore there are (n+1)(m+2) locations i, j at which opt must be called. Each call reads at most three locations that have already been filled in, so the complexity is ≤ 3(n+2)(m+2)  (nm).

32) Problem: Determine an optimal alignment of two homologous DNA sequences.

Input: A DNA sequence x of length m and a DNA sequence y of length n represented as arrays.

Output: The cost of an optimal alignment of the two sequences. int align(int X[], int Y[]) { index i, j; int opt[0…m][0…n]; for(i=0; i<=n; i++) opt[i][n] = 2*(m-i); for(j=0; j<=m; j++) opt[m][j] = 2*(n-j); for(i=m-1; i>=0; i--) { for(j=n-1; j>=0; j--) { if(X[i] == Y[j]) penalty = 0; else penalty = 1; opt[i][j] = min(opt[i+1][j+1] + penalty, opt[i+1][j] + 2, opt[i][j+1] + 2); } } return opt[0][0]; } 33) 0G

1G

2A

3G

4T

5T

6C

7A

8-

0C

8

9

11

12

14

16

18

20

22

1C

6

7

9

10

12

14

16

18

20

2G

5

5

7

8

10

12

14

16

18

3G

4

5

5

6

8

10

12

14

16

Chapter 3: Dynamic Programming

Solutions

4G

6

4

5

4

6

8

10

12

14

5T

7

6

4

4

4

6

8

10

12

6T

8

6

5

3

3

4

6

8

10

7A

9

7

5

4

2

3

4

6

8

8C

11

9

7

5

3

1

2

4

6

9C

12

10

8

6

4

2

0

2

4

10 A

14

12

10

8

6

4

2

0

2

11 -

16

14

12

10

8

6

4

2

0

Optimal alignment is CCGGGTTACCA with –GGAGTT-C-A, which has cost 8.

34) Like algorithms for computing the nth Fibonacci term (see Exercise 42 in Chapter 1), the input size in Algorithm 3.2 (Binomial Coefficient Using Dynamic Programming) is the number of symbols it takes to encode the numbers n and k. Analyze the algorithm in terms of its input size.

The complexity of Algorithm 3.2 in terms of n and k is (nk), and the maximum k is n, so in the worst case (n2). If the input is encoded in binary, we have lg n symbols for n and maximum lg n symbols for k = n, for a total of 2lg n = lg n2. Denote n2 by m → the input is lg m, and the ouput is (m). Now denote lg m by p → The complexity is m = 2p, so the complexity is exponential in terms of the nr. of symbols (bits) in the input.

35) Determine the number of possible orders for multiplying n matrices A1, A2, . . . , An. The number of ways of fully parenthesizing n factors is given by the (n-1)st Catalan number Cn1 = (1/n)*((2*(n-1)) choose (n-1)).

36) Show that the number of binary search trees with n keys is given by the formula

Denote by B(n) the number of BSTs with n keys. Any of the keys can be the root. If key j is the root, then keys 1 … j-1 must be in the left subtree, and keys j+1…n in the right subtree. The left

Chapter 3: Dynamic Programming

Solutions

and right subtrees are in turn BSTs, with (j-1) and (n-j) keys, respectively, so we have the recursion: n

B(n) =  B(j-1)B(n-j), where B(0) = 1

(*)

j 1

For instance:  

for B(1) the recursion yields B(0)B(0) = 1∙1 = 1, which is correct. for B(2): B(0)B(1) + B(1)B(0) = 1 + 1 = 2, also correct.

It can be proved that the number given recursion (*).

, a.k.a. the nth Catalan number satisfies the

37) Can you develop a quadratic-time algorithm for the Optimal Binary Search Tree problem (Algorithm 3.9)?

Inside the innermost for loop of Algorithm 3.9 we have the calculation of A[i][j], which requires a minimum for values of k ranging from i to j:

In the paper by Donald E. Knuth, "Optimum binary search trees", Acta Informatica 1 (1): 14–25, (1971), it is proved that R[i][j] is always in between R[i][j-1] and R[i+1][j]. Below is an example that uses the second diagonal of matrix R from Exercise 22: 1 2

0

1

2

3

4

5

6

0

1

2

2

4

4

4

0

2

2

4

4

4

0

3

4

4

4

0

4

4

4

0

5

6

0

6

3 4 5 6 We replace in the algorithm

m inimu m m inimu m with .

R[i ][ j  1]  k  R[i  1][ j ] ik j

The values of k we now need to examine for an entire diagonal are at most twice the range of the previous diagonal, which is at most 1 through n. There are n diagonals, so the complexity is T(n) ≤ 2n2 O(n2).

Chapter 3: Dynamic Programming

Solutions

38) Problem: Find the maximum sum in any contiguous sub-list in a list of n numbers.

Input: A list S[] of n numbers indexed from 1 to n.

Output: The maximum sum in any contiguous sub-list of S[]. number maxSublist(int n, number S[]) { index i=1; number minTotal = min(0, S[1]); number total = S[1]; number val = S[1]; for(i=2; i<=n; i++) { total = total + S[i]; if(total – minTotal > val) val = total – minTotal; if(total < minTotal) minTotal = total; } return val; } The algorithm performs exactly n iterations for a list of n numbers, each of which contains a constant number of operations, and so its complexity is O(n).

39) Problem: Find the longest common subsequence of two sequences of characters S1 and S2.

Input: Two sequences of characters S1 and S2 and integers m and n representing their respective lengths.

Output: An array C[0…m][0…n] whose element C[m][n] contains the length of the longest common subsequence. int[][] LCS(char S1[], int m, char S2,[] int n) { int C[0…m][0…n]; index i, j; for(i=0; i<=m; i++) C[i][0] = 0;

Chapter 3: Dynamic Programming

Solutions

for(j=0; j<=n; j++) C[0][j] = 0; for(i=1; i<=m; i++) { for(j=0; j<=n; j++) { if(S1[i]==S2[j]) C[i][j] = C[i-1][j-1]; else C[i][j] = max(C[i][j-1], C[i-1][j]); } } return C; } function LCSLength(X[1..m], Y[1..n]) C = array(0..m, 0..n) for i := 0..m C[i,0] = 0 for j := 0..n C[0,j] = 0 for i := 1..m for j := 1..n if X[i] = Y[j] C[i,j] := C[i-1,j-1] + 1 Else: C[i,j] := max(C[i,j-1], C[i-1,j]) return C[m,n] The longest common subsequence can be printed out by calling the following function using the C array resulting from running the LCS function above, the two original sequences S1 and S2, and the lengths of the sequences m and n. void printLCS(int C[][], char S1[], char S2[], int i, int j) { if(i==0 or j==0) return “”; if(S1[i] == S2[i])

Chapter 3: Dynamic Programming return printLCS(C, S1, S2, i-1, j-1) + S1[i]; if(C[i, j-1] > C[i-1, j]) return printLCS(C, S1, S2, i, j-1); return printLCS(C, S1, S2, i-1, j); }

Solutions

Chapter 4: The Greedy Approach

Solutions

1) Show that the greedy approach always finds an optimal solution for the Change problem when the coins are in the denominations D0, D1, D2,... , Di for some integers i > 0 and D > 1.

Let Sg be the greedy solution, written in polynomial order S g = giDi + gi-1Di-1 + … + g1D + g0, and let Sh be a non-greedy solution Sh = hiDi + hi-1Di-1 + … + h1D + h0. (All the numbers involved are positive integers.) i

i

 g j   hj

We are going to show that j  0

j 0

, which means that Sg is the optimal solution.

Being solutions, both sums are equal to the required number of units of change: S = Sg = Sh. Since Sg is greedy, all the coefficients gj are in the interval [0…D-1].

Since Sh is not greedy, let m be the first index (from left to right) where Sh does not make the greedy choice, i.e. gm > hm. By subtraction we get: Sg – Sh = (gm – hm)Dm + (gm-1 – hm-1)Dm-1 + … + (g1 – h1)D + (g0 – h0) = 0 Since all the coefficients greater than m are identical, it remains to prove that (gm – hm) + (gm-1 – hm-1) + … + (g1 – h1) + (g0 – h0) < 0 For concision, we denote (gj – hj) by pj. Remember that gm > hm, so pm > 0. (But we don’t know the signs of the other differences.) We now have pmDm + pm-1Dm-1 + … + p1D + p0 = 0, and we need to show that pm + pm-1 + … + p1 + p0 < 0.

Apply induction on m: Base case: When m = 1, we have p1D + p0 = n0 → p1(D-1) + p1 + p0= n0 → p1 + p0 < n0 General case: We assume that the result holds for m-1, and we prove it for m. We start from pmDm + pm-1Dm-1 + … + p1D + p0 = 0, and divide by D to get pmDm-1 + pm-1Dm-2 + … + p1 + p0/D = 0.

According to the induction hypothesis, we have pm + pm-1 + … + p1 + p0/D < 0

(*)

Consider the 2 possible cases for p0: 1. If p0 < 0, since D > 1, (*) implies pm + pm-1 + … + p1 + p0 < 0 2. If p0 ≥ 0, it is time to remember that p0 = g0 – h0 , and, as noted at the beginning, all greedy coefficients gj, including g0, are in the interval [0…D-1]. Since h0 ≥ 0, we have that also p0 = [0…D-1]. This makes it impossible for p0/D in eq. (*) to be an integer unless p0 = 0, in which case (*) trivially implies pm + pm-1 + … + p1 + p0 < 0 The induction conclusion was proven, q.e.d.

1

Chapter 4: The Greedy Approach

Solutions

2) 1. Begin with Y = { v1 }, F = { }.

2. Nearest vertex to Y is v4 (distance 14), so add v4 to Y and (v1, v4) to F. 3. Nearest vertex to Y is v8 (distance 3), so add v8 to Y and (v4, v8) to F. 4. Nearest vertex to Y is v9 (distance 4), so add v9 to Y and (v8, v9) to F. 5. Nearest vertex to Y is v5 (distance 10), so add v5 to Y and (v4, v5) to F. 6. Nearest vertex to Y is v10 (distance 12), so add v10 to Y and (v9, v10) to F. 7. Nearest vertex to Y is v6 (distance 6), so add v6 to Y and (v6, v10) to F. 8. Nearest vertex to Y is v3 (distance 18), so add v3 to Y and (v3, v4) to F. 9. Nearest vertex to Y is v7 (distance 5), so add v7 to Y and (v3, v7) to F. 10. Nearest vertex to Y is v2 (distance 32), so add v2 to Y and (v1, v2) to F. 11. Now Y == V, so instance is solved. Spanning tree is given by edges in F = { (v1, v4), (v4, v8), (v8, v9), (v4, v5), (v9, v10), (v6, v10), (v3, v4), (v3, v7), (v1, v2) }

3) a) 1. Beginning with vertex v4, each table shows the set of edges over which we search for the min at each step (indicated by shaded squares). Initially, Y = { v4 } and F = { }.

1

2

3

4

5

6

1

0

INF

72

50

90

35

2

INF

0

71

70

73

75

3

72

71

0

INF

77

90

4

50

70

INF

0

60

40

5

90

73

77

60

0

80

6

35

75

90

40

80

0

2. Min edge is 40 from v4 to v6, so we add v6 to Y and (v4, v6) to F.

3. Pick greedily again with Y = { v4, v6 }.

2

Chapter 4: The Greedy Approach

Solutions

1

2

3

4

5

6

1

0

INF

72

50

90

35

2

INF

0

71

70

73

75

3

72

71

0

INF

77

90

4

50

70

INF

0

60

40

5

90

73

77

60

0

80

6

35

75

90

40

80

0

4. Min edge is 35 from v1 to v6, so we add v1 to Y and (v1, v6) to F. 5. Pick greedily again with Y = { v1, v4, v6 } and F = { (v4, v6), (v1, v6) }.

1

2

3

4

5

6

1

0

INF

72

50

90

35

2

INF

0

71

70

73

75

3

72

71

0

INF

77

90

4

50

70

INF

0

60

40

5

90

73

77

60

0

80

6

35

75

90

40

80

0

6. Min edge is 60 from v4 to v5, so we add v5 to Y and (v4, v5) to F. 7. Pick greedily again with Y = { v1, v4, v5, v6 } and F = { (v4, v6), (v1, v6), (v4, v5) }.

1

2

3

4

5

6

1

0

INF

72

50

90

35

2

INF

0

71

70

73

75

3

72

71

0

INF

77

90

4

50

70

INF

0

60

40

5

90

73

77

60

0

80

6

35

75

90

40

80

0

8. Min edge is 70 from v2 to v4, so we add v2 to Y and (v2, v4) to F. 3

Chapter 4: The Greedy Approach

Solutions

9. Pick greedily again with Y = { v1, v2, v4, v5, v6 } and F = { (v4, v6), (v1, v6), (v4, v5), (v2, v4) }.

1

2

3

4

5

6

1

0

INF

72

50

90

35

2

INF

0

71

70

73

75

3

72

71

0

INF

77

90

4

50

70

INF

0

60

40

5

90

73

77

60

0

80

6

35

75

90

40

80

0

10. Min edge is 71 from v2 to v3, so we add v3 to Y and (v2, v3) to F. 11. Now Y == V, so problem is solved. b) Minimum spanning tree is given by F = { { (v4, v6), (v1, v6), (v4, v5), (v2, v4), (v2, v3) }. c) Cost of spanning tree is 40 + 35 + 60 + 70 + 71 = 276.

4) Graph G containing more than one spanning tree is given by adjacency matrix: 0

5

9

INF

5

0

6

6

9

6

0

7

INF

6

7

0

Two minimum spanning trees are given by F = { (v1, v2), (v2, v4), (v3, v4) } and F = { (v1, v2), (v2, v3), (v3, v4) }, each with cost 18.

5) Solutions will vary.

6) Modify Prim’s algorithm (Algorithm 4.1) to check if an undirected, weighted graph is connected. Analyze your algorithm and show the results using order notation.

4

Chapter 4: The Greedy Approach

Solutions

Boolean connected (int n, const number w[ ] [ ]) { index i, vnear; number min; edge e; index nearest [2, … n] number distance [2, … n]; boolean connected = true; for (i = 2; i < = n; i++){ nearest [i] = 1 distance [i] = w [1] [i]; } repeat (n − 1 times){ min = infinity; for (i = 2; i < = n; i + +) if (0 < = distance [i] < min) { min = distance [i]; vnear = i; } if (min == infinity) { connected = false; break; } distance [vnear] = −1; for (i = 2; i < = n; i++) if (w [i] [vnear] < distance [i]) { distance [i] = w [i] [vnear]; nearest [i] = vnear; } } return connected; } 5

Chapter 4: The Greedy Approach

Solutions

Complexity analysis: As in the case of the original Prim algorithm, T(n) = (n-1)*2*(n-1) → (n2).

7) Use Kruskal’s algorithm (Algorithm 4.2) to find a minimum spanning tree for the graph in Exercise 2. Show the actions step by step. (v4, v8) → 3 → Accepted (v8, v9) → 4 → Accepted (v3, v7) → 5 → Accepted (v6, v10) →6 → Accepted (v4, v5) → 10 → Accepted (v9, v10) → 12 → Accepted (v1, v4) → 17 → Accepted (v3, v4) → 18 → Accepted (v5, v9) → 25 → Rejected (v5, v6) → 28 → Rejected (v1, v2) → 32 → Accepted (v2, v5) → 45 → Rejected (v7, v8) → 59 → Rejected

8) Implement Kruskal’s algorithm (Algorithm 4.2) on your system, and study its performance using different graphs.

6

Chapter 4: The Greedy Approach

Solutions

Solutions and performance will vary.

9) It is not possible for a minimum spanning tree to have a cycle since all trees are acyclic by definition.

10) Assume that in a network of computers any two computers can be linked. Given a cost estimate for each possible link, should Algorithm 4.1(Prim’s algorithm) or Algorithm 4.2 (Kruskal’s algorithm) be used? Justify your answer.

Prim’s algorithm is(n2), and Kruskal’s algorithm is(m∙lgm), where m is the number of edges in the graph. Since our graph is completely connected, m = n(n-1)/2. Kruskal’s algorithm in terms of n is therefore(n2∙lgn), slightly less efficient than Prim’s.

11) Apply Lemma 4.2 to complete the proof of Theorem 4.2.

Let G = (V, E) be a graph with a set of nodes V and a set of edges E. Let n be the number of nodes.

We use induction by n:  Base case: If n = 1, the graph has only one node and no edges (no self-edges), so the while loop in Kruskal’s algorithm does not execute at all. The minimal spanning tree (MST) has just a node and no edges, which is correct.  Induction step: We assume that Kruskal’s algorithm produces a MST for any graph of n-1 nodes. In a graph with n nodes, we apply Kruskal’s algorithm and focus our attention on the set of edges Fn-2 that is obtained in the next-to-last iteration. According to the hypothesis, Fn-2 is a MST for the sub-graph with n-1 nodes. Is Fn-2 also promising? Yes! (Reasoning by contradiction, if it were not promising, there would be no edge from the n-1node subgraph to node n that would create a MST for the n-node graph. But such an edge e must be part of any MST. If we remove e from the MST, we get a smaller MST for the n-1node subgraph – contradiction.) We can apply the lemma, which says that Fn-1 is promising in G, therefore it is MST for G. q.e.d.

4.2 7

Chapter 4: The Greedy Approach

Solutions

12) Use Dijkstra’s algorithm to find the shortest path from vertex v5 to all other vertices for the graph represented by the array shown in the table. Show the actions step by step.

1

2

3

4

5

6

1

0



72

50

90

35

2



0

71

70

73

75

3

72

71

0



77

90

4

50

70



0

60

40

5

90

73

77

60

0

80

6

35

75

90

40

80

0

Solution: From the above table, form the graph:

Use Dijkstra’s algorithm to find the shortest path from vertex v5: Unvisited

Visited

{1, 2, 3, 4, 5, 6}

5

1

2

3

4

6

(0, −)

(, −)

(, −)

(, −)

(, −)

(, −)

 {1, 2, 3, 4, 5, 6}

{5}

( )

(90, 5)

(73, 5)

(77, 5)

(60, 5)

(80, 5)

 {1, 2, 3, 6}

{4, 5}

(90, 5)

(73, 5)

(77, 5)

(80, 5)

 {1, 3, 6}

{2, 4, 5}

(90, 5)

(77, 5)

(80, 5) 

{1, 6}

{2, 3, 4, 5}

(90, 5)

(80, 5)

 {1}

{1, 2, 3, 4, 5, 6}

{}

{1, 2, 3, 4, 5, 6}

(90, 5)

8

Chapter 4: The Greedy Approach

Solutions



Unvisited nodes indicate that the node has not yet processed/reached.



5 being the initialized vertex, it is chosen first as the visited node.



Using the algorithm, the cost from v5 to all searchable nodes is calculated.

 Then, the neighbor vertex in the lowest path is chosen as the next vertex [Here, vertex 4 with cost 60 is chosen]. 

Then, using the conditional statements,

From the table, the following graph is drawn:

13) Use Dijkstra’s algorithm to find the shortest paths from vertex v4 to all other vertices of the graph shown in the figure. Show the actions step by step. Assume that each undirected edge represents two directed edges with the same weight.

9

Chapter 4: The Greedy Approach

Solutions

Unvisited

Visited

V4

V1

V2

V3

V5

V6

V7

V8

{1, 2, 3, 5, 6, 7, 8, 9, 10}

{4}

(0, )

(17, 4)

(, −)

(18, 4)

(10, 4)

(, −) (, −) (3, 4)

V9

V10

(, −) (, −)

 {1, 2, 3, 5, 6, 7, 9, 10}

{4, 8}

(17, 4)

(, −)

(18, 4)

(10, 4)

(, −) (62, 8)

(7, 8)

(, −)

 {1, 2, 3, 5, 6, 7, 10}

{4, 8, 9}

(17, 4)

(, −)

(18, 4)

(55, 5)

(18, 4)

(10, 4)

(, −) (62, 8)

(, −)

(38, 5)

(62, 8)

(19, 9)

(38, 5)

(23, 3)

(19, 9)

 {1, 2, 3, 6, 7, 10}

{4, 5, 8, 9}

(17, 4)

 {2, 3, 6, 7, 10}

{1, 4, 5, 8, 9}

(17, 4)

(49, 1)

(18, 4)

 {2, 6, 7, 10}

{1, 3, 4, 5, 8, 9}

(49, 1)

(38, 5)

(23, 3)

(19, 9)

 {2, 6, 7}

{1, 3, 4, 5, 8, 9, 10}

(49, 1)

(25, 10)

(23, 3)

 {2, 6,}

{1, 3, 4, 5, 7, 8, 9, 10}

(49, 1)

(25, 10)

 {2}

{1, 3, 4, 5, 6, 7, 8, 9, 10}

{}

{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

(49, 1)

10

Chapter 4: The Greedy Approach

Solutions

Solution graph:

14) Solutions and performance will vary.

15) Modify Dijkstra's algorithm (Algorithm 4.3) so that it computes the lengths of the shortest paths. Analyze the modified algorithm and show the results using order notation.

The solution requires only a slight change in Algorithm 4.3, based on the observation that, until vnear is added to Y by setting its length to -1 (last line), the array length does store the length of the shortest path. All we need to do is to save that length in another array, say report which the function returns. The function will have one more argument: void dijkstra(int n, const number W[][], number report[], set_of_edges& F)

16) Modify Dijkstra’s algorithm (Algorithm 4.3) so that it checks if a directed graph has a cycle. Analyze your algorithm and show the results using order notation.

The existence of a cycle has nothing to do with the metric properties of the graph, so for simplicity we can replace all the weights with the value 1.

For each vertex v: Run Dijkstra (Algorithm 4.3) on the new graph

//(n2)

Check for “back edge” from all other nodes to v

//(n)

If back edge exists cycle was found exit 11

Chapter 4: The Greedy Approach

Solutions

Complexity analysis: The outer for loop executes n times → (n3)

17) Can Dijkstra’s algorithm be used to find the shortest paths in a graph with some negative weights? Justify your answer.

Dijkstra’s algorithm cannot in general handle negative weights. Here’s a counterexample:

First, source A is visited, and the distances to B and C are calculated

B has the shortest distance, so it’s added to the set of visited nodes. The distances are updated:

D has the shortest distance, so we visit it and update distances:

12

Chapter 4: The Greedy Approach

Solutions

The only vertex now left in the queue is C, so we visit it, with a distance of 5, and the algorithm terminates. The path A → C → B → D has total length 5 + (–4) + 1 = 2 < 4; thus, the length of the shortest path from source A to D was not correctly evaluated.

End of example The best-known algorithm to find single source shortest path in a directed graph with negative weights is Bellman–Ford algorithm. It is possible to adapt Dijkstra’s algorithm to handle negative weights by combining it with Bellman–Ford (Johnson’s algorithm).

18) Use induction to prove the correctness of Dijkstra’s algorithm.

Theorem (the correctness of Dijkstra’s algorithm 4.3): When a node joins the set of edges F, the distance in its label is the length of the shortest path.

Proof: At each iteration of the repeat loop, the nodes are partitioned into subsets F and V – F. We prove by induction on the size of F that the label for each node i  F is the shortest distance from node 1.

Base case: when |F| = 1, the only node in F is 1. It is correctly labeled with a distance 0.

Inductive step: Assume for |F| = k and prove for |F| = k + 1. Suppose that node i with d(i) = min{d(j): j  V – F} is added to F, and assume by contradiction that its label d(i) is not the shortest path label. Then, there should be some nodes in V – F along the shortest path from 1 to i.

Let j be the first node in V – F on the shortest path from 1 to i. The length of shortest path from 1 to i is d(i) + Cji. Since d(i) is not the shortest label, d(j) + Cji < d(i), and since Cji is positive, we have d(j)<d(i). This contradicts that d(i) is the minimum of d(j)  V – F.

13

Chapter 4: The Greedy Approach

Solutions

4.3 19) Consider the following jobs and service times. Use the algorithm in Section 4.3.1 to minimize the total amount of tunes spent in the system.

Job

Service Time

1

7

2

3

3

10

4

5

Solution: Step1: Arrange and sort the jobs by service time in nondecreasing order. t1  7

t2  3

t3  10

t4  5

Schedule

Total Time in the System

[t2, t4, t1, t3]

3 + (3 + 5) + (3 + 5 + 7) + (3 + 5 + 7 + 10) = 51

Schedule [2, 4, 1, 3] is optional with a total time of 32.

14

Chapter 4: The Greedy Approach

Solutions

20) Implement the algorithm in Section 4.3.1 on your system, and run it on the instance in Exercise 19.

21) Write an algorithm for the generalization of the single-server scheduling problem to the multiple-server scheduling problem in Section 4.3.1. Analyze your algorithm and show the results using order notation.

Solution: The algorithm is based on the explanations at the end of Section 4.3.2.

Let m be the number of servers, and n the number of jobs. The servers are identical for our purposes, so their order does not matter.

Algorithm: Sort the n jobs by service time in nondecreasing order Set variable job_counter to 0 For i from 0 to infinity: For j from 1 to m: Schedule job j + i*m on server j Increment job_counter If (job_counter == n) exit

//All jobs are scheduled 15

Chapter 4: The Greedy Approach

Solutions

As an example, consider m = 2 servers and the following jobs: Job

Service Time

1

7

2

3

3

10

4

5

Sort the jobs in nondecreasing order, t2 = 3, t4 = 5, t1 = 7, t3 = 10.

Server 1

Server 2

t2, t1

t4, t3

= 3 + (3 + 7)

= (5) + (5 + 10)

= 13

= 20

Total time spent in the system = 33 [t2, t4, t1, t3].

16

Chapter 4: The Greedy Approach

Solutions

22) Consider the following jobs, deadlines, and profits, and use the scheduling with deadlines algorithm (Algorithm 4.4) to maximize the total profit.

Job

Deadline

Profit

1

2

40

2

4

15

3

3

60

4

2

20

5

3

10

6

1

45

7

1

55

Algorithm 4.4 does the following: 0)

Sort the jobs in non-increasing order of profits: Job

Deadline

Profit

3

3

60

7

1

55

6

1

45

1

2

40

4

2

20

2

4

15

5

3

10

1)

J is set to [3].

2)

K is set to [3,7] and is determined to be feasible, so J is set to [3,7].

3)

K is set to [3,7,6] and is determined to be not feasible, so J remains [3, 7].

4)

K is set to [3,7,1] and is determined to be feasible, so J is set to [3,7,1].

5)

K is set to [3,7,1,4] and is determined to be not feasible, so J remains [3,7,1].

6)

K is set to [3,7,1,2] and is determined to be feasible, so J is set to [3,7,1,2].

7)

K is set to [3,7,1,2,5] and is determined to be not feasible, so J remains [3,7,1,2].

The maximum profit for schedule [3,7,1,2] is 170.

17

Chapter 4: The Greedy Approach

Solutions

23) Consider procedure schedule in the Scheduling with Deadlines algorithm (Algorithm 4.4). Let d be the maximum of the deadlines for n jobs. Modify the procedure so that it adds a job as late as possible to the schedule being built, but no later than its deadline. Do this by initializing d+1 disjoint sets, containing the integers 0, 1, ..., d. Let small(S) be the smallest member of set S. When a job is scheduled, find the set S containing the minimum of its deadline and n. If small(S) = 0, reject the job. Otherwise, schedule it at time small(S), and merge S with the set containing small(S)−1. Assuming we use Disjoint Set Data Structure III in Appendix C, show that this version is θ(nlgm), where m is the minimum of d and n.

The modified scheduling procedure is: void schedule_just_in_time ( int n, int d, const int deadline[], sequence & J) { index i, minim, set_ptr; int smalley; J = [];

//empty sequence

universe U;

//see Disjoint Set Data Structure III in App.C

for (i=0; i<=d; i++) makeset(i); for (i=1; i <= n; i++){ minim = min(n, deadline[i]); set_ptr = find(minim); smalley = small(set_ptr); if (smalley > 0){ add the pair (i, smalley) to J; merge(set_ptr, find(smalley-1)); } } } Complexity analysis: The main for loop above executes n times, and in it the dominant function is find(), whose complexity in the worst case is the maximum height of the trees used to represent the sets in Disjoint Set Data Structure III. The maximum number of elements in any of those sets is indeed bounded by both n and d: 18

Chapter 4: The Greedy Approach

 

Solutions

If n >= d, the total number of elements in all sets is d+1, so the largest tree to be searched will have at most d+1-1 = d elements (at the previous step).

If n < d, remember that the algorithm always calculates min(n, deadline[i]), and the scheduled time smalley is possibly even less than that minimum, so the size of the largest tree is also bounded by n.

Denote m = min(d, n). The height of a balanced tree is logarithmic (lgm), so overall we have (n∙lgm) q.e.d.

As an example, let’s apply the algorithm to the jobs from Ex.22, sorted by profit: 3

3

60

7

1

55

6

1

45

1

2

40

4

2

20

2

4

15

5

3

10

We have d = 4 and n = 7. Create these 5 sets: {0}

{1}

{2}

{3}

{4}

Scheduling job 3: min(3,7) = 3 → go to set {3}, whose small is 3 → schedule at t=3, merge {3} and {2} into {2,3}. The sets are: {0}

{1}

{2, 3}

{4}

Scheduling job 7: min(1,7) = 1 → go to set {1}, whose small is 1 → schedule at t=1, merge {1} and {0} into {0,1}. The sets are: {0, 1}

{2, 3}

{4}

Scheduling job 6: min(1,7) = 1 → go to set {0,1}, whose small is 0 → cannot schedule Scheduling job 1: min(2,7) = 2 → go to set {2,3}, whose small is 2 → schedule at t=2, merge {2,3} and {0,1} into {0,1,2,3}. The sets are: {0, 1, 2, 3}

{4}

Scheduling job 4: cannot schedule Scheduling job 2: schedule at t=4 and merge everything in to {0,1,2,3,4} Scheduling job 5: cannot schedule

19

Chapter 4: The Greedy Approach

Solutions

24) Implement the algorithm developed in Exercise 21.

25)

The constant C represents parameters such as the speed of the drive and the recording density.

a) In what order should a greedy approach store these files to guarantee minimum average access time?

20

Chapter 4: The Greedy Approach

Solutions

Solution: Consider the following example: l

p

f1

5

0.4

f2

4

0.5

f3

3

0.1

Sort the files from smallest ratio of length over access probability to largest ratio of length over access probability: 4 5 3   0.5 0.4 0.1 8  12.5  15

The files are stored in the tape in the following order: f2, f1, f3, hence the average access time is minimized. b) Write the algorithm to store the files, analyze your algorithm, and show the results using order notation.

Solution: The dominant term is sorting n ratios, which can be done in (n·lgn), e.g. with Mergesort or Quicksort.

4.4 26) A: 00 B: 011 I: 10 M: 111 S: 110 X: 0101 Z: 0100

21

Chapter 4: The Greedy Approach

Solutions

27) c: 001 e: 01 i: 111 r: 100 s: 110 t: 000 x: 101

28) Decode each bit string using the binary code in Exercise 26.

From the answer to Exercise 26 we have: A − 00 B − 011 I − 10 M − 111 S − 110 X − 0101 Z − 0100 a) 011

00

0101

0101

0

B

A

X

X

10

00

10

00

0101

I

A

I

A

X

111

00

10

011

110

M

A

I

B

S

b) 0

c) 1

22

Chapter 4: The Greedy Approach

Solutions

d) 10

00

0100

111

00

I

A

Z

M

A

Note: When there are unused bits at the end of a string, they are part of the next character, which has not completely arrived yet.

29) Using the code generated in Exercise 27: a) rise -> 10011111001 b) exit -> 01101111000 c) text -> 00001101000 d) exercise -> 011010110000111111001

30) a: 00, b: 01, c: 101, d: 110, e: 111, so x = 1, y = 1, and z = 1.

31) Implement Huffman's algorithm, and run it on the problem instances of Exercises 26 and 27.

Solutions will vary.

23

Chapter 4: The Greedy Approach

Solutions

32) Show that a binary tree corresponding to an optimal binary prefix code must be full. A full binary tree is a binary tree such that every node is either a leaf or it has two children.

Proof by contradiction: Assuming that a node in an optimal tree T has only 1 child, the entire sub-tree rooted at the child can be shifted up by collapsing the parent and the child:

The resulting tree T’ has lower cost, because all the depths in the shifted sub-tree have decreased by 1. This contradicts the optimality of T.

33) Prove that for an optimal binary prefix code, if the characters are ordered so that their frequencies are non-increasing, then their codeword lengths are non-decreasing.

Proof by contradiction: Let the characters c1, c2, … cn be ordered in non-increasing order of frequencies: f1 ≥ f2 ≥ … ≥ fn. Assume the codeword lengths L1, L2, …, Ln are not non-decreasing, meaning that there exists (at least) an index i such that L i > Li+1. This means that ci is placed deeper in the tree than ci+1. Since fi ≥ fi+1, we have two cases:  fi > fi+1 →By exchanging ci and ci+1,we obtain a tree whose cost is (Li - Li+1)( fi - fi+1) smaller, which contradicts optimality.  fi = fi+1 →We can reasonably assume that in the priority queue used in the Huffman algorithm, fi+1 is placed closer to the front than fi, so this case cannot occur.

Example: A = { e/45, a/20, b/15, d/15, c/5 } is the non-increasing frequency order.

24

Chapter 4: The Greedy Approach

Solutions

e → 1, a → 000, b → 001, d → 011, c → 010, so the codewords are non-decreasing.

34) Given the binary tree corresponding to a binary prefix code, write an algorithm that determines the codewords for all the characters. Determine the time complexity of your algorithm.

We perform a tree traversal (any traversal will do, e.g. BFS, DFS), while keeping track of the sequence of 0s and 1s that leads to the current node. The sequence is stored in a list LZO (List of Zeroes and Ones) Pseudocode for recursive infix traversal, i.e. Center – Left – Right: void codewords(node, LZO){ add the node’s zero or one to LZO if node is leaf print node_symbol, LZO else codeword(left_child, LZO) codeword(roght_child, LZO) }

25

Chapter 4: The Greedy Approach

Solutions

Complexity analysis: (n), because at each node in the tree we only have to update the list of 0s and 1s twice: once when we enter the node from above, and once when we leave upwards.

4.5 35) Problem: Given a set S of n items each with a weight w and a value v, select a subset of the items such that the total value of the items in the subset is maximized and the total weight of the items in the subset does not exceed a given threshold W. Input: A set S of items indexed from 1 to n, each of which has value vi and weight wi, and an integer total weight capacity W. Output: The value of the maximum-valued set of items in S whose total weight does not exceed W. int knapsack(int w[1…n], int v[1…n], int W) { index i, j; int V[0…n][0…W]; for(i=0; i<=n; i++) V[i][0] = 0; for(j=0; j<=W; j++) V[0][j] = 0; for(i=1; i<=n; i++) { for(j=1; j<=W; j++) { if(w[i] > j) V[i][j] = V[i-1][j]; else V[i][j] = max(V[i-1][j], v[i] + V[i-1][j – w[i]); } } return V[n][W]; }

26

Chapter 4: The Greedy Approach

Solutions

36) Use a greedy approach to construct an optimal binary tree by considering the most probable key, Keyk, for the root, and constructing the left and right subtrees for Key 1, Key2, … , Keyk-1 and Keyk+1, Keyk+2, … , Keyn recursively in the same way.

Consider the following example: Bat

Cat

Dog

Hog

Rat

.19

.205

.19

.195

.22

Sort in descending order of probability: Rat

Cat

Hog

Bat

Dog

.22

.205

.195

.19

.19

Scan the sorted array from left to right and insert into an initially empty binary tree:

Expected symbol length = .22x1 + .205x2 + .19x3 + .195x3 + .19x4 = 2.545 Complexity analysis: while sorting is done in (n·lgn), in the worst case (when the keys are already sorted), the tree is a chain, and the insertions require 1 + 2 + … + (n-1) = n(n-1) node accesses → (n2) Note that this tree is not optimal; a better one is

Expected search = .19 + .85 + .155 = 2.195

27

Chapter 4: The Greedy Approach

Solutions

37) Suppose we assign n persons to n jobs. Let Cij be the cost of assigning the i th person to the jth job. Use a greedy approach to write an algorithm that finds an assignment that minimizes the total cost of assigning all n persons to all n jobs. Analyze your algorithm and show the results using order notation.

For illustration, consider the following matrix of costs: J1

J2

J3

P1

10

5

5

P2

2

4

10

P3

5

1

7

This assignment problem can be solved with the following greedy algorithm: 1.

2. a. b. c.

Create a set F of assigned jobs, initially empty.

For i from 1 to n: //for each row in the matrix Find the minimum cost Cij, where j is not in F. Assign job j to person i.

Include j in the set F In the example above, the algorithm assigns P1 → J2, P2 → J1, P3 → J3, with a total cost of 14.

Note that this is not in general the optimum, e.g. assigns P1 → J3, P2 → J1, P3 → J2, has a total cost of only 8.

Complexity analysis: the loop 2 executes n times, each iteration involving finding the minimum of at most n elements, which can be done in n comparisons, yielding (n2).

38) Use the dynamic programming approach to write an algorithm for the problem of Exercise 37. Analyze your algorithm and show the results using order notation.

For illustration, consider the following matrix of costs: 28

Chapter 4: The Greedy Approach

Solutions

J1

J2

J3

P1

10

5

5

P2

2

4

10

P3

5

6

7

This optimal assignment problem can be solved with the so-called Hungarian Algorithm: 1.

Row reduction: Find the minimum element in each row and subtract it from all the

elements on that row. For our example, this results in the matrix:

2.

J1

J2

J3

P1

5

0

0

P2

0

2

8

P3

0

1

2

Column reduction: Find the minimum element in each column and subtract it from

all the elements on that column. In our example, each column already has a zero, so there are no changes.

3.

Cover the minimum number of row and columns such that all zeroes are covered. In

our example, this is column 1 and row 1: J1

J2

J3

P1

5

0

0

P2

0

2

8

P3

0

1

2

If the number equals n, and assignment exists, if not go to step 4. In our example 2 < 3, so we go to step 4.

29

Chapter 4: The Greedy Approach

4.

Solutions

Let x be the smallest uncovered number. Subtract x from all uncovered numbers and

add it to all numbers covered twice. (Do nothing to the numbers covered once.) Then go back to step 3. In our example, x = 1, and the matrix becomes: J1

J2

J3

P1

6

0

0

P2

0

1

7

P3

0

0

1

The complexity of the Hungarian Algorithm: n2 for step 1, n2 for step 2, and the loop at steps 3-4 can execute for a maximum of n-1 times, each time having to find and subtract the minimum of at most (n-1)2 elements. The loop dominates, yielding (n3).

Note: For clarity, the details about how to find the covering lines in step 3 were omitted.

The procedure is somewhat technical, involving a sequence of “stars” and “primes”, as described below. Since only (n2) operations are required, the result of (n3) still stands .

A. Find a non-covered zero and prime it. If there is no starred zero in the row containing this primed zero, go to step 6. Otherwise, cover this row and uncover the column containing the starred zero. Continue in this manner until there are no uncovered zeros left. Save the smallest uncovered value and go to step 4.

B. Construct a series of alternating primed and starred zeros as follows. Let Z 0 represent the uncovered primed zero found at A. Let Z1 denote the starred zero in the column of Z0 (if any). Let Z2 denote the primed zero in the row of Z1 (there will always be one). Continue until the series terminates at a primed zero that has no starred zero in its column. Unstar each starred zero of the series, star each primed zero of the series, erase all primes, and uncover every line in the matrix.

30

Chapter 4: The Greedy Approach

Solutions

39) Use a greedy approach to write an algorithm that minimizes the number of record moves in the problem of merging n files. Use a two-way merge pattern.

(Two files are merged during each merge step.) Analyze your algorithm, and show the results using order notation.

Let the files be F1, F2, …, Fn, each containing r1, r2, …, rn records, respectively.

1.

For i from 1 to n-1

1.

Pick the two shortest files

//This is greedy!

2.

Merge them

//See Algorithm 2.3 - Merge

3.

Remove the two files

4.

Insert the merged file

The exercise does not require to prove that the algorithm is optimal, but a simple proof exists.

Complexity analysis: The for loop executes (n-1) times. If we use a priority queue (see Section 4.4.2) to store the files, picking and removing the two shortest is done in constant time, and inserting the merged file in the queue is (lgn). The total is (n·lgn).

Note: The total above does not include the cost of actually merging the files, rather it describes the complexity of finding only the sequence of merges. Accordingly, the Merge step in the pseudocode above should be understood as simply adding up the numbers of records in the two files.

31

Chapter 4: The Greedy Approach

Solutions

40) Use the dynamic programming approach to write an algorithm for Exercise 39. Analyze your algorithm and show the results using order notation.

The merging process is best visualized as a tree having the initial files as leaves. Every two-way merge creates a parent node whose number of records is the sum of the two children:

In order to apply dynamic programming, we first make sure that the problem has optimal sub-structure. Indeed, if we consider any subtree T' of an optimal tree T, the merging order in T' must be optimal, because otherwise we could replace T' with a T'' of smaller cost, which would result in a smaller cost of T! Let the files be F1, F2, …, Fn (random order), each containing r1, r2, …, rn records, respectively.

We construct partial solutions bottom-up, and place the results in an (n-1)x(n-1) matrix M. Element M[k,i] contains the minimum cost of merging k files, where the left subtree is a merge of i files (and the right is a merge of k-i files): 2 files → 3 files → 4 files → …. k files → …. n files →

1,1 1,2

2,1

1,3

2,2

3,1

1, k-1

2, k-2

3, k-3

….

i, k-i

….

1, n-1

2, n-2

3, n-3

….

i, n-i

…

n-1, 1 32

Chapter 4: The Greedy Approach

Solutions

 n k files can be chosen in   ways among the total n, so the third dimension of the matrix k  consists of all the possible such subsets. The general element M[k,i] has therefore a third index, denoting the particular set of k chosen; let’s call this index S (uppercase). Finally, a k  subset of i out of the k can in turn be chosen in   ways, and let’s call this index s i (lowercase) → M[k,S,i,s], k = 2…n, i = 1,k-1.

The recursion formula is M[k,S,i,s] = min[M(i,Si,j,sj) + M(k-i,Sk-i,t,st)] , taken over all possible partitions of S into Si, Sk-i, and all pairs sj, st. k  We calculate the number of sums over which the min above is taken: For each of the   i k  i i  subsets Si of S, the sub-subset sj can be chosen in   ways, and the sub-subset st in   t   j ways, so there are  i  k i 1  k  i   terms. Making use of a well-known combinatorial identity      j  t 1  t  k  k  (Example A.10), we get   (2i-2)(2k-i-2)  2k   terms. Since finding the maximum is (n), i i we have the same number of comparisons for one entry in “row” k.  k  i 1     i  j 1

k 1 k  For the entire “slice” k, we have about 2k    , which as above is  2k2k-1 = 22k/2 = 4k/2 i 1 i comparisons.

1 n 16 n  2 k 8 n 1 k= 4   4 = 3 4 , using the geometric series 2 k 2 2 k 0 identity (Example A.4). So the divide-and-conquer algorithm described is (4n).

Finally, for the entire table we have

41) Prove that the greedy approach to the Fractional Knapsack problem yields an optimal solution.

Proof by contradiction: Assume there is an optimal solution S’ that does not coincide with the solution S obtained by the algorithm in Section 4.5.2. Let itemi be the first item in S 33

Chapter 4: The Greedy Approach

Solutions

where the corresponding item in S’ is different. Let itemk be the item that has replaced it in S’. Because the items are sorted by density, k > i, so it is possible to replace the quantity of itemk in S’ by the corresponding quantity of itemi which has not been used yet in S’. (If there is not enough of itemi, replace only wi.) With this modification, the value of S’ has increased, which contradicts the optimality of S’. Q.e.d.

42) Show that the worst-case number of entries computed by the refined dynamic programming algorithm for the 0-1 Knapsack problem is in (2n). Do this by considering the instance in which W = 2n – 2 and wi = 2i–1 for 1 ≤ i ≤ n.

Consider first an example: wi = [0 2 4 8]

1i4

w = 14 if n = 4 The total number of array entries computed is 1 + 1 + 2 + 22 Generalizing for n we have 1 + (1 + 2 + 22 ++2n)

= 1 + |(2n + 1 – 1) =2n + 1 = 2·2n = (2n)

43) Show that in the refined dynamic programming algorithm for the 0-1 Knapsack problem, the total number of entries computed is about (W + 1) x (n + 1) / 2, when n = W + 1 and wi = 1 for all i.

Let’s examine first the example n = 4, W = 3, w = [1 1 1 1], 1  i  4 (W + 1) x (n + 1) / 2 = 10, which exactly matches the number of entries:

Generalizing, the total number of 34

Chapter 4: The Greedy Approach

Solutions

array entries computed is 1 2  3   n 

n(n  1) 2

When n = w + 1, the total array entries computed is therefore

(w  1)(n  1) .

2

Additional Exercises 44) Show with a counterexample that the greedy approach does not always yield an optimal solution for the Change problem when the coins are U.S. coins and we do not have at least one of each type of coin.

If we do not have any nickels, but have any number of quarters, dimes, and pennies, and we use greedy approach to find change $0.40, the greedy approach gives .25 + .10 + .1 + .1 + .1 + .1 + .1 = .40, using 7 coins.

However, we could non-greedily find .10 + .10 + .10 + .10 = .40, which uses only 4 coins.

35

Chapter 4: The Greedy Approach

Solutions

45. Prove that a complete graph (a graph in which there is an edge between every pair of vertices) has nn−2 spanning trees. Here n is the number of vertices in the graph.

Solution: This result was first found by Carl Wilhelm Borchardt in 1860, so here we shall denote the number by B(n). While there are many proofs (and generalizations), the most intuitve is probably Jim Pitman’s (as explained in Aigner, Ziegler’s “Proofs from THE BOOK”, Springer-Verlag, 1998): In the complete graph with n nodes, we count the ways in which we can add edges to build a spanning tree, starting from an arbitrary node. We can do this in two ways: A. First we choose any of the B(n) spanning trees. Then we can pick the root of the tree in n possible ways. For each of the roots, we can add the (n-1) edges of the tree in (n-1)! Ways, for a total of n(n-1)!B(n) = n!B(n) ways.

B. We put together he tree by picking random edges at each step. The first edge can start in any of the n vertices of the graph, and it can end in any of the remaining (n-1) vertices, so we have n(n-1). The second edge can also start in any of the n vertices, but it can only end in the remaining (n-2), etc. Since we have (n-1) edges to add, the product is nn-1(n-1)∙(n2)∙…∙1 = n!nn-2.

Comparing the results from A and B above, we can simplify n! to obtain B(n) = n n-2.

As an example, consider this complete graph with n = 3:

It should have nn – 2 spanning tree = 33 – 2 = 3. Indeed, below are shown all the possible spanning trees, and there are 3 of them:

36

Chapter 4: The Greedy Approach

Solutions

46) Use a greedy approach to write an algorithm for the traveling salesman problem. Show that your algorithm does not always find a minimum-length tour.

Nearest-neighbor algorithm: Initialize: set of unused vertices is all vertices in graph: U = V list of nodes in the Hamiltonian circuit is empty: H = [] Start with a randomly chosen vertex vi → add vi to H Repeat (n-1) times: Choose the edge of minimum weight that starts in vi: e = (vi, vj) Add vj to H Remove vi from U This is obviously Q(n2) in the worst case, since at any of the (n-1) vertices we may have to examine (n-1) edges. In a general graph, it can fail to find a circuit, even if such a circuit exists, but then the problem of finding a circuit is NP-complete anyway (see ch.9), so it cannot be solved greedily.

The Nearest-neighbor algorithm is, however, guaranteed to find a cycle in a complete graph, so this is what we shall use for the example:

Here the optimal cycle starting at v1 is obviously v1 → v2 → v3 → v4, of cost 6, but the algorithm finds v1 → v3 → v2 → v4, of cost 14.

37

Chapter 4: The Greedy Approach

Solutions

47) Prove that the algorithm for the Multiple-Server Scheduling problem of Exercise 19 always finds an optimal schedule.

As described at the end of Section 4.3.1, the schedule S is: Server 1 serves jobs 1,

(1+m), (1+2m),

(1+3m),...

Server 2 serves jobs 2,

(2+m), (2+2m),

(2+3m),...

…

Server i serves jobs i, (i+m), (i+2m),

(i+3m),..., (i+km), …

…

Server m serves jobs m,

(m+m),

(m+2m),(m+3m),...

Assume by contradiction that S is not optimal, and let S’ be an optimal schedule. S and S’ are different, so let (i,k) be the first matrix index where they differ, i.e. instead of job i+km, server i has in S’ another job, whose index in S is j+lm, and Time(i+km) < Time(j+lm). If we swap in S’ the jobs i+km and j+lm, we can shift left all jobs scheduled after that. Let’s call the resulting schedule S’’.

On server i, the total time for S’’ is shorter. On the other hand, the server where j+lm went still has the same jobs before the end of j+lm, so that end (and everything after it) is still scheduled at the same time. This shows that the net change is that Time(S’’) < Time(S’), which contradicts the optimality of S’.

38

Chapter 4: The Greedy Approach

Solutions

48) Without constructing a Huffman tree, generate Huffman code for a given set of characters.

At each step, we merge the two symbols of minimal probability into a new symbol with probability equal to the sum, and re-sort the list of symbols.

For example, let A = {a1, a2, a3, a4, a5} and P (ai) = {0.2, 0.4, 0.2, 0.1, 0.1} Symbol

Step 1

Step 2

Step 3

Step 4

Code word

Reverse code

a2

0.4

0.4

0.4

0.6

0

1

1

a1

0.2

0.2

0.4 0

0.4

1

10

01

a3

0.2

0.2 0

0.2 1

000

000

a4

0.1 0

0.2 1

0100

0010

a5

0.1 1

1100

0011

49) Generalize Huffman’s algorithm to ternary codewords and prove that it yields optimal ternary codes.

Ternary Huffman tree: 1.Sort the symbols by probability.

2.If nr. of symbols is odd, add a “filler” symbol with probability 0. This way we always have an even nr. of symbols from now on.

2. Combine the three symbols with the lowest 3 probabilities.

3. Assign the sum of the 3 probabilities to the combined symbol.

4. Go back to 1.

Example: X=

x1

x2

x3

x4

x5

x6

x7

0.50

0.26

0.11

0.04

0.04

0.03

0.02

39

Chapter 4: The Greedy Approach

Solutions

codewords x1

0.50

0.50

0.5

1

0

x2

0.26

0.26

0.26

1

x3

0.11

0.11

0.24

20

x4

0.04

0.04

21

x5

0.04

0.9

220

x6

0.03

221

x7

0.02

222

The average nr. of trits for this code is 0.50 x 1 + 0.50 x 2 = 1.33 trit/symbol.

The proof closely parallels the one in the text for the binary Huffman code: First we have the equivalent of Lemma 4.4 (solved in Exercise 32), which says that an optimal ternary tree is always full. This is proved with the same type of argument: if a node has fewer than 3 children, there are 2 possibilities:  

1 child → As in Exercise 32, we move up the entire subtree rooted at the child, thus obtaining a tree of smaller cost.

2 children → We move up one of the grandchildren, thus obtaining a tree of smaller cost, as illustrated below:

If there are no grandchildren, then, because the total nr. of nodes is odd, somewhere higher in the tree must be another node with only 2 children. If we move the node in the higher position, we have again a tree of smaller cost:

40

Chapter 4: The Greedy Approach

Solutions

This proves the lemma.

Now we have the equivalent of Theorem 4.5: The ternary Huffman algorithm produces an optimal tree.

The proof is by induction, just like the proof in the text.

50) If the characters are sorted according to their frequencies, then the priority queue used in Huffman’s algorithm on page 180 can be replaced with two standard queues to keep tracking of the two highest priority elements. Initially, the first queue contains all of the characters in their frequency-sorted order. At each iteration of the loop, the front of both queues is examined and the elements with the lowest frequencies are removed from the head of either one or both queues. When two nodes are combined, they are inserted into the rear of the second queue with a frequency that is the sum of their individual frequencies. The process is repeated n  1 times as in the original algorithm.

Since each call to the remove or insert function of either queue takes constant time, instead of log time, each iteration of the loop executes seven constant-time operations, and thus the entire loop executes 7*(n  1) constant-time operations. The algorithm is thus O(n).

41

Chapter 5: Backtracking

Solutions

Sections 5.1, 5.2 1) n=6: Q

Q Q

Q Q

Q

Q

Q Q

Q Q

Q

n=7: Q

Q

Q

Q

Q

Q Q

Q

Q

Q Q

Q Q

Q

1

Chapter 5: Backtracking

Solutions

2) Apply the Backtracking algorithm for then-Queens problem (Algorithm 5.1) to the problem instance in which n = 8, and show the actions step by step. Draw the pruned state space tree produced by this algorithm up to the point where the first solution is found.

Solution:

2

Chapter 5: Backtracking

Solutions

Solution {1, 3, 5, 7, 2, 4, 6, 8}

3) Problem: Position n queens on an n  n chessboard so that no two queens threaten each other.

Input: positive integer n Output: All possible ways in which n queens can be placed on an nn chessboard so that no two queens threaten each other. Each output consists of an array of integers col indexed from 1 to n, where col[i] is the column in where the queen in the ith row is placed. void queens(index i) { index j; if(i==n) cout << col[1] through col[n]; else for (j=1; j <= n; j++) { col[i+1] = j; if (promising(i+1)) queens(i+1); } } Initial call is queens(0). The promising function is identical to the one on page 212.

3

Chapter 5: Backtracking

Solutions

4) Problem: Find the number of ways in which n queens can be placed on an nn chessboard so that no two queens threaten each other.

Input: positive integer n Output: Number of ways in which n queens can be placed on an nn chessboard so that no two queens threaten each other. void queens(index i, int& count) { index j; if(i==n) count++; else for(j=1; j <= n; j++) { col[i+1] = j; if(promising(i+1)) queens(i+1); } } With an integer variable count declared and initialized to 0, initial call is queens(0, &count). The promising function is identical to the one on page 212.

5) Show that, without backtracking, 155 nodes must be checked before the first solution to the n =4 instance of then-Queens problem is found (in contrast to the 27 nodes in Figure 5.4).

Solution: Without backtracking, we have to also visit all the subtrees of the “x”-ed nodes from Fig.5.4: The 5 nodes on level 2 have 5 x 4 x 4 =80 nodes.

The 7 nodes on level 3 have 7 x 4 = 28 nodes.

Total = 80 + 28 = 108 extra nodes.

Including the 27 from backtracking, we have 135 total nodes.

4

Chapter 5: Backtracking

Solutions

6) Implement the Backtracking algorithm for the n-Queens problem (Algorithm 5.1) on your system, and run it on problem instances in which n = 4, 8, 10, and 12.

Solution: The code is the one in Algorithm 5.1. n=4

:

2 solutions

n=8

:

92 solutions

n = 10

:

724 solutions

n = 12

:

14, 200 solutions

7) Improve the Backtracking algorithm for the n-Queens problem (Algorithm 5.1) by having the promising function keep track of the set of columns, of left diagonals, and of right diagonals controlled by the queens already placed.

Solution: The set of columns can be an array col_used[n+1], with zeroes for the columns in use and ones for the columns not in use. Similarly, since there are 2n-1 left diagonals and 2n-1 right diagonals (anti-diagonals), these can be stored, respectively, in arrays left_diag_used[2n] and right_diag_used[2n].

8) Modify the Backtracking algorithm for then-Queens problem (Algorithm 5.1) so that, instead of generating all possible solutions, it finds only a single solution.

Solution: Insert a return statement in the queens() function after the solution is displayed: if (promising(i)) if (i == n) { cout << col [1] through col [n]; return; } else ….

5

Chapter 5: Backtracking

Solutions

6

Chapter 5: Backtracking

Solutions

9) Suppose we have a solution to the n-Queens problem instance in which n = 4. Can we extend this solution to find a solution to the problem instance in which n = 5? Can we then use the solutions for n = 4 and n = 5 to construct a solution to the instance in which n=6 and continue this dynamic programming approach to find a solution to any instance in which n> 4?

Justify your answer.

Solution: for n = 4 queens, there are only two solutions (for each column from left to right, the row where the queen is placed is shown): 1. (2, 4, 1, 3)

2. (3, 1, 4, 2)

For n= 5 queens, there are 10 solutions, and two of them contain as sub-solutions the solutions from n = 4: 1. (1, 3, 5, 2, 4)

2. (1, 4, 2, 5,3)

3. (2, 4, 1, 3, 5)

4. (2, 5, 3, 1, 4)

5. (3, 1, 4, 2, 5)

6. (3, 5, 2, 4, 1)

7. (4, 1, 3, 5, 2)

8. (4, 2, 5, 3, 1)

9. (5, 2, 4, 1, 3)

10. (5, 3, 1, 4, 2)

For n= 6 queens, there are only 4 solutions, and none of them contains as sub-solutions the solutions from n = 4 or n = 5: 1. (2, 4, 6, 1, 3, 5)

2. (3, 6, 2, 5, 1, 4)

3. (4, 1, 5, 2, 6, 3)

4. (5, 3, 1, 6, 4, 2)

In conclusion, this approach does not work for general n.

The only known dynamic programming solution to the n-Queens problem employs more advanced concepts. It was published as I. Rivin and R. Zabih, A dynamic programming solution to the n-queens problem, Information Processing Letters 41, pp.253-256, 1992.

7

Chapter 5: Backtracking

Solutions

10) There are no solutions to the n queens problem when n=2 or n=3.

Section 5.3 11) Implement algorithm 5.3 (Monte Carlo estimate for the Backtracking algorithm for then-Queens problem) on your system, run it 20 times on the problem instance in whichn = 8, and find the average of the 20 estimates.

Solution: Promising nodes can be found using 1 + n + n (n − 1) + n (n − 1) (n − 2) +  + n!

Total number of nodes checked using 1 + 1 × t0 + m0 t1 + m0 m1 t2 + m0 m1 m2 t3 +m0 m1 … mi−1 ti where ti = total number of children at level i mi = no of promising children for a randomly selected node at level; The code is an implementation of Algorithm 5.3. Being a probabilistic algorithm, the estimates will vary.

12) Modify the Backtracking algorithm for the n-Queens problem (Algorithm 5.1) so that it finds the number of nodes checked for an instance of a problem, run it on the problem instance in which n = 8, and compare the result against the average of Exercise 11.

Solution: We create a global variable int node_counter = 0;, which we increment in the for loop of Algorithm 5.1 thus: for (j = 1; j <= n; j++) { col [ i + 1] = j ; queens(i + 1); node_counter++; } The number of nodes checked is 15,721 when n = 8, as listed in Table 5.1.

8

Chapter 5: Backtracking

Solutions

Section 5.4 13) Use the Backtracking algorithm for the Sum-of-Subsets problem (Algorithm 5.4) to find all combinations of the following numbers that sum to W =52: w1 =2

w2 =10

w3 =13

w4 =17

w5 =22

w6 =42

Show the actions step by step.

Solution: As specified in the text, we assume that the indices range 1 through n. The weights are already sorted. The following presents the first 4 levels of recursion (out of the total of 6). Each innermost node below has to be further split twice (unless a node evaluates not promising): 0. Start from the non-existent w0, weight=0 promising → include[1] = yes a. weight=2, w1 promising → include[2] = yes i. weight=12, w2 promising → include[3] = yes 1. weight=25, w3 promising → include[4] = yes 2. weight=25, w3 promising → include[4] = no ii. weight=12, w2 is promising → include[3] = no 1. weight=12, w3 promising → include[4] = yes 2. weight=12, w3 promising → include[4] = no b. weight=2, w1 promising → include[2] = no i. weight=2, w2 promising → include[3] = yes 1. weight=15, w3 promising → include[4] = yes 2. weight=15, w3 promising → include[4] = no ii. weight=2, w2 promising → include[3] = no 1. weight=2, w3 promising → include[4] = yes 2. weight=2, w3 promising → include[4] = no 1. Back to the non-existent w[0], weight=0, promising → include[1] = no a. weight=0, w1 promising → include[2] = yes i. weight=10, w2 promising → include[3] = yes 1. weight=23, w3 promising → include[4] = yes 2. weight=23, w3 promising → include[4] = no ii. weight=10, w2 promising → include[3] = no 1. weight=10, w3 promising → include[4] = yes 2. weight=10, w3 promising → include[4] = no b. weight=0, w1 promising → include[2] = no i. weight=0, w2 promising → include[3] = yes 1. weight=13, w3 promising → include[4] = yes 2. weight=13, w3 promising → include[4] = no ii. weight=0, w2 promising → include[3] = no 1. weight=0, w3 promising → include[4] = yes 2. weight=0, w3 promising → include[4] = no There are two solutions: {10, 42} and {13, 17, 22}.

9

Chapter 5: Backtracking

Solutions

14) Implement the Backtracking algorithm for the Sum-of-Subsets problem (Algorithm 5.4) on your system, and run it on the problem instance of Exercise 13.

Solution:

10

Chapter 5: Backtracking

Solutions

15) Write a backtracking algorithm for the Sum-of-Subsets problem that does not sort the weights in advance. Compare the performance of this algorithm with that of Algorithm 5.4.

Solution: The only difference from Algorithm 5.4 is that the promising() function cannot use the condition weight + w[ i + 1 ] <= W; the array not being sorted, it is possible for the sum to exceed W for i+1, but still be ≤ W for a subsequent index. The new code for is: 11

Chapter 5: Backtracking

Solutions

bool promising ( index i ) { return (weight + total >= W) && (weight == W) ; } This algorithm is slower than Algorithm 5.4, because the absence of the extra condition in promising() causes it to explore more “dead-end” subtrees.

12

Chapter 5: Backtracking

Solutions

16) void sum_of_subsets(index i, int weight, int total) { if(promising(i)) { if (weight==W) { cout << include[1] through include[i]; return; } else { include[i+1] = “yes”; sum_of_subsets(i+1, weight + w[i+1], total – w[i+1]); include[i+1] = “no”; sum_of_subsets(i+1, weight, total – w[i+1]); } } } Promising function is the same as the one on page 224.

The worst-case complexity of this algorithm is the same as that of Algorithm 5.4, since in the event of no solution the same number of nodes must be checked. Even if there is a solution, problem instances may be constructed that have only one solution but still require the visitation of exponentially many nodes in the state-space tree.

17) Use the Monte Carlo technique to estimate the efficiency of the Backtracking algorithm for the Sum-of-Subsets problem (Algorithm 5.4).

Solution: Use Algorithm 5.3 (Monte Carlo for n-Queens) as template, and adapt it to implement the data types and data structures from Algorithm 5.4 (Sum-of-Subsets). Only the minimal changes (indicated in boldface) were made in the code below:

13

Chapter 5: Backtracking

Solutions

int estimate_sum_subsets ( int n) { index i bool j; bool include[1…n]; //solution is an array of yes/no int m, mprod , numnodes ; set_of_index prom_children ; i = 0; //root numnodes = 1; m = 1; //we assume the root is promising mprod = 1; while (m != 0 && i !=n) { mprod = mprod * m; numnodes = numnodes + mprod * n ; i++; //next level in the tree m=0; //assume no promising children prom_children = ⊘; //initialize with empty set for ( j in {True, False}) { include[ i ] = j; i f ( promising ( i ) ) { m++; prom_children = prom_children ∪ { j}; } } i f (m != 0) { j = random selection from prom_children ; include[ i ] = j ; } } return numnodes ; } The promising function is the same as in Algorithm 5.4.

14

Chapter 5: Backtracking

Solutions

Section 5.5 18) Use the Backtracking algorithm for the m-Coloring problem (Algorithm 5.5) to find all possible colorings of the graph below using the three colors red, green, and white. Show the actions step by step.

Solution: v1 red → promising v2 red → not promising v2 gre → promising v3 red → promising v4 red → not promising v4 gre → promising v5 red → promising v6 red → not promising v6 gre → promising v6 whi → promising v5 gre → not promising v5 whi → promising v6 red → not promising v6 gre → promising v6 whi → not promising v4 whi → promising v5 red → promising v6 red → not promising v6 gre → promising v6 whi → promising v5 gre → not promising v5 whi → not promising v3 gre → not promising v3 whi → promising v4 red → not promising v4 gre → promising v5 red → promising v6 red → not promising v6 gre → promising v6 whi → not promising 15

Chapter 5: Backtracking

Solutions

v5 gre → not promising v5 whi → promising v6 red → promising v6 gre → promising v6 whi → not promising v4 whi → promising v5 red → promising v6 red → not promising v6 gre → promising v6 whi → not promising v5 gre → not promising v5 whi → not promising v2 whi → promising v3 red → promising v4 red → not promising v4 gre → promising v5 red → promising v6 red → not promising v6 gre → promising v6 whi → promising v5 gre → not promising v5 whi → not promising v4 whi → promising v5 red → promising v6 red → not promising v6 gre → promising v6 whi → promising v5 gre → promising v6 red → not promising v6 gre → not promising v6 whi → promising v5 whi → not promising v3 gre → promising v4 red → not promising v4 gre → promising v5 red → promising v6 red → not promising v6 gre → not promising v6 whi → promising v5 gre → not promising v5 whi → not promising v4 whi → promising v5 red → promising v6 red → not promising v6 gre → not promising v6 whi → promising 16

Chapter 5: Backtracking

Solutions

v5 gre → promising v6 red → not promising v6 gre → promising v6 whi → promising v5 whi → not promising v3 whi → not promising v1 gre → promising v2 red → promising v3 red → not promising v3 gre → promising v4 red → promising v5 red → not promising v5 gre → promising v6 red → promising v6 gre → not promising v6 whi → promising v5 whi → promising v6 red → promising v6 gre → not promising v6 whi → not promising v4 gre → not promising v4 whi → promising v5 red → not promising v5 gre → promising v6 red → promising v6 gre → not promising v6 whi → promising v5 whi → not promising v3 whi → promising v4 red → promising v5 red → not promising v5 gre → promising v6 red → promising v6 gre → not promising v6 whi → not promising v5 whi → promising v6 red → promising v6 gre → promising v6 whi → not promising v4 gre → not promising v4 whi → promising v5 red → not promising v5 gre → promising v6 red → promising v6 gre → not promising v6 whi → not promising 17

Chapter 5: Backtracking

Solutions

v5 whi → not promising v2 gre → not promising v2 whi → promising v3 red → promising v4 red → promising v5 red → not promising v5 gre → promising v6 red → not promising v6 gre → not promising v6 whi → promising v5 whi → not promising v4 gre → not promising v4 whi → promising v5 red → promising v6 red → not promising v6 gre → promising v6 whi → promising v5 gre → promising v6 red → not promising v6 gre → not promising v6 whi → promising v5 whi → not promising v3 gre → promising v4 red → promising v5 red → not promising v5 gre → promising v6 red → promising v6 gre → not promising v6 whi → promising v5 whi → not promising v4 gre → not promising v4 whi → promising v5 red → promising v6 red → not promising v6 gre → not promising v6 whi → promising v5 gre → promising v6 red → promising v6 gre → not promising v6 whi → promising v5 whi → not promising v3 whi → not promising v1 whi → promising v2 red → promising v3 red → not promising v3 gre → promising 18

Chapter 5: Backtracking

Solutions

v4 red → promising v5 red → not promising v5 gre → promising v6 red → promising v6 gre → not promising v6 whi → promising v5 whi → promising v6 red → promising v6 gre → not promising v6 whi → not promising v4 gre → promising v5 red → not promising v5 gre → not promising v5 whi → promising v6 red → promising v6 gre → not promising v6 whi → not promising v4 whi → not promising v3 whi → promising v4 red → promising v5 red → not promising v5 gre → promising v6 red → promising v6 gre → not promising v6 whi → not promising v5 whi → promising v6 red → promising v6 gre → promising v6 whi → not promising v4 gre → promising v5 red → not promising v5 gre → not promising v5 whi → promising v6 red → promising v6 gre → promising v6 whi → promising v4 whi → not promising v2 gre → promising v3 red → promising v4 red → promising v5 red → not promising v5 gre → not promising v5 whi → promising v6 red → not promising v6 gre → promising v6 whi → not promising 19

Chapter 5: Backtracking

Solutions

v4 gre → promising v5 red → promising v6 red → not promising v6 gre → promising v6 whi → promising v5 gre → not promising v5 whi → promising v6 red → not promising v6 gre → promising v6 whi → not promising v4 whi → not promising v3 gre → not promising v3 whi → promising v4 red → promising v5 red → not promising v5 gre → not promising v5 whi → promising v6 red → promising v6 gre → promising v6 whi → not promising v4 gre → promising v5 red → promising v6 red → not promising v6 gre → promising v6 whi → not promising v5 gre → not promising v5 whi → promising v6 red → promising v6 gre → promising v6 whi → not promising v4 whi → not promising v2 whi → not promising There are 54 solutions. We can read them off starting from a bold leaf and working our way up the tree; for example, the last bold leaf above represents the solution (underlined): Vertex

Color

v1

white

v2

green

v3

white

v4

green

v5

white

v6

green 20

Chapter 5: Backtracking

Solutions

19) Suppose that to color a graph properly we choose a starting vertex and a color to color as many vertices as possible. Then we select a new color and a new uncolored vertex to color as many more vertices as possible. We repeat this process until all the vertices of the graph are colored or all the colors are exhausted. Write an algorithm for this greedy approach to color a graph of n vertices. Analyze this algorithm and show the results using order notation.

Solution: Inputs: positive integers n and m, and an undirected graph containing n vertices. The graph is represented by a two-dimensional array W, which has both its rows and columns indexed from 1 to n, where W[i]j] is true if there is an edge between i th vertex and the jth vertex and false otherwise.

Outputs: An m-coloring of the graph, as an array vcolor indexed from 1 to n (vcolor [i] is the color - integer between 1 and m - assigned to vertex i). In case the algorithm fails to find a solution, no array vcolor is printed. As in Algorithm 5.5, we assume that vcolor is a global array. void m_coloring_greedy (int n, int m) { set_of_colored_nodes = ⊘; //no nodes are colored yet for (int color = 1; color <= m, color++) { //cycle through all colors for (int i not in set_of_colored_nodes) //cycle through all nodes of graph if(promising (i, color)) { set_of_colored_nodes = set_of_colored_nodes ∪ {i}; vcolor(i) = color; if(set_of_colored_nodes is full) { cout << vcolor[1] through vcolor[n]; //found solution return; //exit function } } } } bool promising ( int i, int color) { bool switch = true; for (int j in set_of_colored_nodes){ //Check if adjacent vertex is this color if (W[i][j] && vcolor[i]==vcolor[j]) switch=false; } return switch; } Complexity analysis: promising() is (n), and it is called inside the double loop of m_coloring_greedy, which executes n x m times → (n2m) overall.

21

Chapter 5: Backtracking

Solutions

20) m-Coloring using the above algorithm. The order of the colors is red, green, white.

Solution: Step 1 → color = red → nodes 1, 3, 5 are colored red Step 2 → color = green → nodes 2, 4, 6 are colored green

21) Suppose we are interested in minimizing the number of colors used in coloring a graph. Does the greedy approach of Exercise 17 guarantee an optimal solution? Justify your answer.

Solution: No, the algorithm does not always use the minimum number of colors, because it is sensitive to the ordering of the vertices. For instance, the graph on the left is colored with two colors, but the one on the right requires three colors:

22) Any graph in which node n has an edge to every other node in the graph, and nodes n2, n-1, and n form a clique. The graph given by the following adjacency matrix is one example.

0

0

0

0

0

1

0

0

0

0

0

1

0

0

0

0

0

1

0

0

0

0

1

1

0

0

0

1

0

1

1

1

1

1

1

0 22

Chapter 5: Backtracking

Solutions

23) Compare the performance of the Backtracking algorithm for the m-Coloring problem (Algorithm 5.5) and the greedy algorithm of Exercise 19. Considering the result(s) of the comparison and your answer to Exercise 19, why might one be interested in using an algorithm based on the greedy approach?

Solution: The greedy algorithm is more efficient ((n2m)) than the Backtracking algorithm ((mn)), so for large n it may be worth finding a non-optimal solution fast.

Besides, there are simple heuristics that can greatly improve the performance of the greedy algorithm at little extra cost, like the Welsh-Powell algorithm (in which the vertices are sorted in decreasing (non-increasing) order of their degrees, so the added cost is only (n2) or (n∙lg n).

24) Problem: Determine all ways in which all n vertices of an undirected graph can be colored using only two colors so that adjacent vertices are not the same color.

Input: positive integer n and an undirected graph containing n vertices represented as a two-dimensional boolean array W[1…n][1…n] , where W[i][j] is true if there is an edge between i and j.

Output: all possible colorings of the graph using at most 2 colors so that no two adjacent vertices are the same color. The output for each coloring is an array vcolor[1…n], where vcolor[i] is the color (1 or 2) assigned to the ith vertex. void twoColor() { bool visited[1…n]; //initialized to all false Queue q; //standard queue index i; for(i=1; i<=n; i++) { if(!visited[i]) { insert(q, i); visited[i] = true; vcolor[i] = 1; } while(q is not empty) { remove(q, current); for(j=current+1; j<=n; j++) if(W[i][j] && !visited[j]) { insert(q, j); visited[j] = true; vcolor[j] = (vcolor[current] % 2) +1; //opposite color } } } //end while cout >> vcolor[1] through vcolor[n]; //print coloring 23

Chapter 5: Backtracking

}

Solutions

for(i=1; i<=n; i++) vcolor[i] = (vcolor[i] % 2) +1; cout >> vcolor[1] through vcolor[n]; //print inverse coloring

Complexity is O(n2).

25) Job scheduling with conflicts can be formulated as an m-coloring problem. Jobs must be scheduled into certain time slots, but some jobs conflict with each other (e.g., because they share the same resource). Each node represents a job and each edge represents a conflict between two jobs. A valid coloring represents a valid assignment schedule. The minimum number of colors that can color the graph is the optimal time to finish all of the jobs without conflicts.

Assignment of radio frequencies to radio stations can be formulated as an mcoloring problem. Nodes are radio stations and edges represent conflicts, which are based on the minimum distance between stations that doesn’t cause interference. Any two stations closer than this distance have an edge between them. A valid coloring represents a valid assignment of frequencies to radio stations. The minimum number of colors that can color the graph is the minimum number of frequencies needed.

The game Sudoku can be formulated as a 9-coloring of a graph with 81 nodes and as specific set of edges. A valid coloring is a solution to the puzzle. Partial colorings are given as puzzle instances.

Section 5.6 26) Use the Backtracking algorithm for the Hamiltonian Circuits problem (Algorithm 5.6) to find all possible Hamiltonian Circuits of the following graph.

Show the actions step by step.

24

Chapter 5: Backtracking

Solutions

Solution:

Hence there is no Hamiltonian circuit in the above graph, because, if there were one, any vertex can be made into the starting vertex.

27) Implement the Backtracking algorithm for the Hamiltonian Circuits problem (Algorithm 5.6) on your system, and run it on the problem instance of Exercise 26.

Solution: hamiltonian(index i) { index j; if (promising (i)) if (i==n−1) cout << vindex [0] through index [n−1]; else for (j=1; j<n; j++) { vindex [i+1]=j; hamiltonian(i+1); } } 25

Chapter 5: Backtracking

Solutions

bool promising (index i) { index j; bool switch; if (i==n−1 !w [index [n−1]][vindex [0]]) switch = false; else if (i >0 && !w [vindex[i−1]][vindex[i]]) switch = false; else { switch true; j=1; while (j<i && switch) { if (vindex [i] == vindex [j]) switch = false; j++; } return switch; } } void main() { int i[]; hamiltonian[0]; }

28) Change the starting vertex in the Backtracking algorithm for the Hamiltonian Circuits problem (Algorithm 5.6) in Exercise27 and compare its performance with that of Algorithm 5.6.

Solution: This is the state space tree for the graph given above starting vertex = v12:

26

Chapter 5: Backtracking

Solutions

When we take v12 as a starting vertex, there is still no Hamiltonian circuit. This is as expected from Exercise 26, because, if there were one, any vertex could be made into the starting vertex.

Performance comparison: Only 33 nodes searched, compared to 60 at Ex.26.

29) void hamiltonian(index i) { index j; if(promising(i)) { if(i==n-1) { cout << vindex[0] through vindex[n-1]; return; } for(j=1; j<n; j++) { vindex[i+1] = j; hamiltonian(i+1); } } } Promising function is identical to the one on page 233.

27

Chapter 5: Backtracking

Solutions

This algorithm has the same worst-case complexity as Algorithm 5.6 since it must check the same number of nodes in the case where there is no Hamiltonian circuit.

30) Analyze the Backtracking algorithm for the Hamiltonian Circuits problem (Algorithm 5.6) and show the worst-case complexity using order notation.

Solution: For the worst case time complexity, the number of nodes in the state space tree searched is given on p.233 as 1 + (n−1) + (n−1)2 + (n−1)3 + … +(n−1)n =

(n  1) n 1  1 n2

With order notation we have O(nn+1).

For a problem with n = 12, we have



(12  1)121  1 12  2

(11)13  1  10



34522712143931  1 34522712143930  10 10

= 3,452,271,214,393 nodes in the worst case.

31) Use the Monte Carlo Technique to estimate the efficiency of the Backtracking algorithm for the Hamiltonian Circuits problem (Algorithm 5.6).

Solution: Use Algorithm 5.3 (Monte Carlo for n-Queens) as template, and adapt it to implement the data types and data structures from Algorithm 5.6 (Hamiltonian Circuits).

Only the minimal changes (indicated in boldface) were made in the code below:

28

Chapter 5: Backtracking int estimate_Hamiltonian ( int n) { index i, j; index vindex[n]; int m, mprod , numnodes ; set_of_index prom_children ;

}

Solutions

//solution is an array of indices in the cycle

i = 0; //root numnodes = 1; m = 1; //we assume the root is promising mprod = 1; while (m != 0 && i !=n) { mprod = mprod * m; numnodes = numnodes + mprod * n ; i++; //next level in the tree m=0; //assume no promising children prom_children = ⊘; //Initialize with empty set for ( j=0; j<n; j++) { vindex[ i ] = j; i f ( promising ( i ) ) { m++; prom_children = prom_children ∪ { j}; } } i f (m != 0) { j = random selection from prom_children ; vindex[ i ] = j ; } } return numnodes ;

The promising function is the same as in Algorithm 5.6.

29

Chapter 5: Backtracking

Solutions

Section 5.7 32) Compute the remaining values and bounds after visiting node (4, 1) in Example 5.6 (Section 5.7.1).

Solution: (4, 2): Item 4 was not chosen, so the profit and total weight are the same as in the parent $70, 7. There are no further items, so the bound is the current profit $70.

(2, 2): Item 2 was not chosen, so the profit and total weight are the same as in the parent $40, 2. Bound is 40 + 50 +10 x 2/5 = $98.

(3, 3): Profit 40 + 50 = 90. Weight 2 + 10 = 12. Since the item was chosen, the bound stays equal to that of the parent $98.

(4, 3):

Profit 40 + 50 + 10 = $100. Weight 12 + 5 = 17 > 16, so no bound needed.

(4, 4): Item 4 was not chosen, so the profit and total weight are the same as in the parent $90, 12. There are no further items, so the bound is the current profit $90.

(3, 4): Item 3 was not chosen, so the profit and total weight are the same as in the parent $40, 2. Bound 40 + 10 = $50.

(1, 2): Item 1 was not chosen, so the profit and total weight are the same as in the parent $0, 0. Bound 30 + 50 + 10 x 1/5 = $82.

33) Use the Backtracking algorithm for the 0-1 Knapsack problem (Algorithm 5.7) to maximize the profit for the following problem instance. Show the actions step by step.

Solution: maxprofit goes through the values $0, $20, $50, $55. The tree is below:

30

Chapter 5: Backtracking

Solutions

Profit: $0 Weight: 0 Bound: $60

Profit: $20 Weight: 2 Bound: $60

Profit: $0 Weight: 0 Bound: $50

Profit: $50 Weight: 7 Bound: $60

Profit: $85 Weight: 14 Bound:

Profit: $20 Weight: 2 Bound: $55

Profit: $50 Weight: 7 Bound: $58

Profit: $55 Weight: 9 Bound: $55

X

Profit: $30 Weight: 5 Bound: $50

Profit: $0 Weight: 0 Bound: $43

X

X

Profit: $20 Weight: 2 Bound: $35

X Profit: $62 Weight: 10 Bound:

Profit: $50 Weight: 7 Bound: $53

X

X

34) Implement the Backtracking algorithm for the 0-1 Knapsack problem (Algorithm 5.7) on your system, and run it on the problem instance of Exercise 33.

Solutions will vary.

35) Implement the dynamic programming algorithm for the 0-1 Knapsack problem (see Section 4.5.3) and compare the performance of this algorithm with the Backtracking algorithm for the 0-1 Knapsack problem (Algorithm 5.7) using large instances of the problem.

Solutions and performance will vary.

31

Chapter 5: Backtracking

Solutions

36) Improve the Backtracking algorithm for the 0-1 Knapsack problem (Algorithm 5.7) by calling the promising function after only a move to the right.

Solution: The idea is to pass to the function knapsack in Algorithm 5.7 a fourth argument, eval_promising, which tells it whether it should evaluate promising or not. The changes are shown in boldface below: void knapsack (index i, int profit, int weight, bool eval_promising) { if (weight <= W && profit > maxprofit) { maxprofit = profit; numbest = i; bestset = include; } if ( (eval_promising && promising(i)) || !eval_promising ) { include [ i + 1] = “yes”;

// Include w[ i +1]

knapsack(i + 1, profit + p[ i + 1] , weight + w[ i + 1], false); include [ i + 1] = “no”;

// Do not include

knapsack(i + 1, profit , weight, true); // w[ i + 1]. } } The function promising is the same as in Algorithm 5.7.

37) Use the Monte Carlo technique to estimate the efficiency of the Backtracking algorithm for the 0-1 Knapsack problem (Algorithm 5.7).

Solution: Use Algorithm 5.3 (Monte Carlo for n-Queens) as template, and adapt it to implement the data types and data structures from Algorithm 5.7. Only the minimal changes (indicated in boldface) were made in the code below:

32

Chapter 5: Backtracking

Solutions

int estimate_backpack ( int n) { index i ; bool j; bool include[0…n-1]; //solution is an array of yes/no int m, mprod , numnodes , maxprofit; set_of_index prom_children ; i = 0; maxprofit = 0; //root numnodes = 1; m = 1; //we assume the root is promising mprod = 1; while (m != 0 && i !=n) { mprod = mprod * m; numnodes = numnodes + mprod * n ; i++; //next level in the tree m=0; //assume no promising children prom_children = ⊘; //initialize with empty set for ( j in {True, False}) { include[ i ] = j; int maxprof = maxprofit; if (j) //node i was included maxprof = maxprof + p[i]; i f ( promising ( i ) ) { m++; prom_children = prom_children ∪ { j}; } } i f (m != 0) { j = random selection from prom_children ; include[ i ] = j ; } } return numnodes ; } The promising function is the same as in Algorithm 5.7.

33

Chapter 5: Backtracking

Solutions

Additional Exercises 38) 1. Searching for optimal moves in a two-player game (e.g., tic-tac-toe or chess).

2. Discretized path-finding (e.g., optimal routes via waypoints for non-player characters in video games).

3. Anagram solving/generation given an input string and a dictionary.

39) Modify the Backtracking algorithm for the n-Queens problem (Algorithm 5.1) so that it produces only the solutions that are invariant under reflections or rotations.

Solution: The function queens of Alg. 5.1, stores each new solution in a global set of solutions, set_of_solutions. Before accepting and printing a candidate for a new solution, a check is performed to see if the candidate matches an existing solution through reflection or rotation: set_of_solutions = ⊘;

//initialize with empty set

void queens(index i) { ... ... ... if (promising(i)) if (i == n) if ( !matching_existing(col) ) { set_of_solutions = set_of_solutions ∪ {col}; cout << col [1] through col [n]; } ... ... ...

The function matching_existing has the following pseudocode: bool matching_existing(int col[ ]){ bool match = false; for (sol in set_of_solutions) { if ( match1(col, sol) ) match = true; elseif ( match2(col, sol) ) match = true; ... ... ... elseif ( match42(col, sol) ) match = true; } return match; }

34

Chapter 5: Backtracking

Solutions

The functions match1, match2, … implement checks for the various symmetries that we want to check: horizontal, vertical, with respect to the main diagonal or anti-diagonal, rotation of 90/180/270 deg., etc.

40) Given an n × n × n cube containing n3 cells, we are to place n queens in the cube so that no two queens challenge each other (so that no two queens are in the same row, column, or diagonal). Can then-Queens algorithm (Algorithm 5.1) be extended to solve this problem? If so, write the algorithm and implement it on your system to solve problem instances in which n = 4 and n = 8.

Solution: This Algorithm can be extended to solve n × n × n cube n-queens, as described below.

We represent the positions in the cube by a bi-dimensional array of integers base[][], which is analogous to col[]. We can imagine base as being the base of the cube, and the number stored at base[i][j] as being the height of that queen in the cube. The base has n2 positions, but only n of them are occupied by queens in a solution. To be consistent with the notation in Section 5.2, we use indices 1 through n to store base positions, and integers 1 though n for the height. A height of 0 means no queen in that base position.

In Algorithm 5.1, the condition for two queens being on the same diagonal is col[i] –col[k] = ± (i−k) Here, the diagonal conditions are base[i1][j1] – base[i2][j2] = ± (i1 – i2) AND base[i1][j1] – base[i2][j2] = ± (j1−j2) The horizontal line condition is base[i1][j1] = base[i2][j2] AND (i1 = i2 OR j2= j2) For simplicity, we replace the index type with int, and we define an iterator on the base, which, for any base position (i, j), returns a “next” one: (i1, j1) = next(i, j). The order is arbitrary, as long as, starting from (1, 1), all base positions are reached.

The pseudocode follows closely the one for Algorithm 5.1: void queens3D (int i, int j, int nr_queens) { int i1, j1 ; if (promising(i, j)) if (nr_queens == n) print base; else for ( k=0; k<=n; k++) { //k=0 means no queen int new_nr_queens = nr_queens; 35

Chapter 5: Backtracking

Solutions if (k>0) new_nr_queens++; (i1, j1) = next(i, j); base[ i1, j1 ] = k ; queens3D(i1, j1, new_nr_queens);

}

}

For the promising function, we assume the existence of a function before, which, when called on a pair of indices i, j, returns a set of all the “previous” pairs (according to the function next mentioned above). bool promising (int i, int j) { int i1, j1; bool switch ; i1 = 1; j1 = 1; switch = true ; while ( (i1, j1) is in before(i, j) && switch) { if ( ( abs(base[i1][j1] – base[i][j]) == abs(i1 – i2) && abs(base[i1][j1] – base[i][j]) == abs(i1 – i2) )

||

( base[i1][j1] == base[i2][j2] && (i1 == i2) )

||

( base[i1][j1] == base[i2][j2] && (j2 == j2) ) )

switch = false ; (i1, j1) = next(i1, j1); } return switch ; } In the main program we call Queens3D(1, 1, 0);

41) Modify the Backtracking algorithm for the Sum-of-Subsets (Algorithm 5.4) to produce the solutions in a variable-length list.

Solution: Instead of using the array include, we use a list or set data structure of the same name, which supports the append operation.

Instead of include[i+1] = “yes” we say include.append(i+1) Instead of include[i+1] = “no” we do nothing 36

Chapter 5: Backtracking

Solutions

42) Explain how we can use the Backtracking algorithm for the m-Coloring problem (Algorithm 5.5) to color the edges of the graph of Exercise 18 using the same three colors so that edges with a common end receive different colors.

Solution: For any graph G, we construct the “edge-to-vertex dual graph”, a.k.a. the “line graph” G’, which has a vertex for every edge in G. Two vertices in G are connected by an edge, if and only if their corresponding edges in are incident on the same node.

For G in exercise 18, the dual G’ has 7 vertices, corresponding to the 7 edges of G: v2

1 v2

2 v2

4

3

6

5

7

It is easy to see that edges with a common end in G (e.g. the ones incident on v2) correspond to a completely connected subgraph in G’ (vertices 1, 2, 4), so those vertices in G’ do receive different colors, meaning that the edges in G have different colors.

43) Modify the Backtracking algorithm for the Hamiltonian Circuits problem (Algorithm 5.6) so that it finds a Hamiltonian Circuit with minimum cost for a weighted graph. How does your algorithm perform?

Solution: Hamiltonian circuit problem with weighted graph is also known as the Traveling Salesperson (TSP) problem, first encountered in Section 3.6.

The simplest solutions is to add a function evaluate that takes a solution vindex as argument and returns its cost, i.e. the sum of the weights of all edges in that solution. We initialize a global variable minimum with a value larger than the cost of any cycle (e.g. the sum of all weights in G), and we store the best solution in another global best_vindex: minimum = INFINITY; best_vindex[n] = {-1}; void hamiltonian(index i) { 37

Chapter 5: Backtracking

Solutions

index j; if(promising(i)) { if( i == n – 1) { if (evaluate(vindex) < minimum) { minimum = evaluate(vindex); cout << vindex[0] through vindex[n-1]; best_vindex ← vindex; //use a loop! } } else for (j=2; j<n; j++) { vindex[i+1] = j; hamiltonian(i+1); } } } The promising function is identical to the one on page 233.

To increase efficiency, the promising function can also check evaluate(vindex) < minimum, so avoids exploration of candidate solutions that are already longer than the current minimum.

44) Modify the Backtracking algorithm for the 0-1Knapsack problem (Algorithm 5.7) to produce a solution in a variable-length list.

Solution: Instead of using the array include, we use a list or set data structure of the same name, which supports the append operation.

Instead of include[i+1] = “yes” we say include.append(i+1) Instead of include[i+1] = “no” we do nothing

38

Chapter 6: Branch-and-Bound

Solutions

Section 6.1

1) Use Algorithm 6.1 (The Breadth-First Search with Branch-and-Bound Pruning algorithm for the 0-1 Knapsack problem) to maximize the profit for the following problem instance. Show the actions step by step.

1. Visit node (0,0). Set its profit and weight to $0 and 0. Compute its bound to be $80. maxprofit = 0.

2. Compute profits, weights, and bounds for children of (0,0): (1,1) and (1,2).

3. (1,1) has profit of $20, weight of 2, and bound of $80. Enqueue node.

4. (1,2) has profit of $0, weight of 0, and bound of $69. Enqueue node.

5. Dequeue/visit node (1,1). maxprofit = 20.

6. Compute profits, weights, and bounds for children of (1,1): (2,1) and (2,2).

7. (2,1) has profit of $50, weight of 7, and bound of $80. Enqueue node.

8. (2,2) has profit of $20, weight of 2, and bound of $70. Enqueue node.

9. Dequeue/visit node (1,2).

10. Compute profits, weights, and bounds for children of (1,2): (2,3) and (2,4).

11. (2,3) has profit of $30, weight of 5, and bound of $69. Enqueue node.

12. (2,4) has profit of $0, weight of 0, and bound of $50. Enqueue node.

13. Dequeue/visit node (2,1). maxprofit = 50.

14. Compute profits, weights, and bounds for children of (2,1): (3,1) and (3,2).

15. (3,1) has profit of $85, weight of 14, and bound of $0. Since weight is exceeded, we do not enqueue.

16. (3,2) has profit of $50, weight of 7, and bound of $65. Enqueue node.

17. Dequeue/visit node (2,2).

18. Compute profits, weights, and bounds for children of (2,2): (3,3) and (3,4).

19. (3,3) has profit $55, weight 9, and bound $70. Enqueue node.

Chapter 6: Branch-and-Bound

Solutions

20. (3,4) has profit $20, weight 2, and bound $35. Since bound is less than maxprofit, we do not enqueue.

21. Dequeue/visit, node (2,3).

22. Compute profits, weights, and bounds for children of (2,3): (3,5) and (3,6).

23. (3,5) has profit $65, weight 12, and bound $69. Enqueue node.

24. (3,6) has profit $30, weight 5, and bound $45. Since bound is less than maxprofit, we do not enqueue.

25. Dequeue/visit node (3,2).

26. Compute profits, weights, and bounds for children of (3,2): (4,1) and (4,2).

23. (4,1) has profit $62, weight 10, and bound $65. Enqueue node.

24. (4,2) has profit $50, weight 7, and bound $53. Enqueue node.

25. Dequeue/visit node (3,3). maxprofit = 55.

26. Compute profits, weights, and bounds for children of (3,3): (4,3) and (4,4).

23. (4,3) has profit $67, weight 12, and bound $70. Enqueue node.

24. (4,4) has profit $55, weight 9, and bound $58. Enqueue node.

25. Dequeue/visit node (3,5). maxprofit = 65.

26. Compute profits, weights, and bounds for children of (3,5): (4,5) and (4,6).

23. (4,5) has profit $77, weight 15, and bound $0. Since weight exceeds limit, we do not enqueue.

24. (4,6) has profit $65, weight 12, and bound $68. Enqueue node.

25. Dequeue/visit node (4,1).

26. Bound for (4,1) is $65, which is less than maxprofit, so we do not check its children.

27. Dequeue/visit node (4,2).

28. Bound for (4,2) is $53, which is less than maxprofit, so we do not check its children.

29. Dequeue/visit node (4,3). maxprofit = 67.

30. Compute profits, weights, and bounds for children of (4,3): (5,1) and (5,2).

23. (5,1) has profit $70, weight 13, and bound $70. Enqueue node.

24. (5,2) has profit $67, weight 12, and bound $67. Since bound does not exceed maxprofit, we do not enqueue.

25. Dequeue/visit node (4,4).

26. Bound for (4,4) is $58, which is less than maxprofit, so we do not check its children.

27. Dequeue/visit node (4,6).

28. Compute profits, weights, and bounds for children of (4,6): (5,3) and (5,4).

23. (5,3) has profit $68, weight 13, and bound $68. Enqueue node.

Chapter 6: Branch-and-Bound

Solutions

24. (5,4) has profit $65, weight 12, and bound $65. Since bound does not exceed maxprofit, we do not enqueue.

25. Dequeue/visit node (5,1). maxprofit = 70.

26. Bound for (5,1) is $70, which does not exceed maxprofit, so we do not check its children.

27. Dequeue/visit node (5,3).

28. Bound for (5,3) is $68, which is less than maxprofit, so we do not check its children.

29. Queue is now empty (no promising nodes left) so problem is solved. Solution is set of items given by node (5,1): { 1, 3, 4, 5} which has profit $70.

2) Implement Algorithm 6.1 on your system and run it on the problem instance of Exercise 1.

/*Solves 0-1 knapsack with Algorithm 6.1 (breadth-first search with Branch-and-Bound pruning) Tested with MSVS 2012 professional */ #include <iostream> #include <queue> #include <list> using namespace std; #define W 13 #define n 5 //As in text pseudocode, we start arrays at index 1 int p[] = {0, 20, 30, 35, 12, 3}; //Must be sorted descending by pi/wi! int w[] = {0, 2, 5, 7, 3, 1}; int node_counter = 0; //Used to keep track of how many nodes are accessed struct node{ int level; int profit; int weight; list<int> items; };

//items included on the path to this node

void print_list(list<int> x){ list<int>::iterator i; for (i = x.begin(); i != x.end(); i++) cout <<*i <<" "; cout <<endl; } void print_node(struct node u){ cout <<u.level <<" " <<u.profit <<" " <<u.weight <<" "; print_list(u.items); } float bound(struct node u){

Chapter 6: Branch-and-Bound

Solutions

int j, k, totweight; float result; if (u.weight > W) return 0; //bound is 0 if limit is exceeded else{ result = u.profit; j = u.level + 1; //start adding next items totweight = u.weight; while (j <= n && totweight + w[j] <= W){ totweight += w[j]; result += p[j]; j++; } k = j; if (k <= n) //add a fraction of the remaining item result += ((float)(W - totweight))*p[k]/w[k]; return result; } } int main(){ struct node u, v; queue<struct node> Q; list<int> bestitems; list<int> empty; int maxprofit = 0;

//stores best set of included items so far

v.level = 0; v.profit = 0; v.weight = 0; v.items = empty; //root has no items Q.push(v); while (!Q.empty()){ v = Q.front(); Q.pop(); node_counter++; u.level = v.level + 1; //Left child includes the next item u.weight = v.weight + w[u.level]; u.profit = v.profit + p[u.level]; u.items = v.items; u.items.push_back(u.level); if (u.weight <= W && u.profit > maxprofit){ maxprofit = u.profit; bestitems = u.items; } if (bound(u) > maxprofit) Q.push(u); //Right child does not include the next item u.weight = v.weight; u.profit = v.profit; u.items = v.items; if (bound(u) > maxprofit) Q.push(u); } cout <<"maxprofit = " <<maxprofit <<endl; cout <<"list of best items (their indices): "; print_list(bestitems); cout <<"\n\nnr. of nodes examined (popped) = " <<node_counter <<endl <<endl; }

Chapter 6: Branch-and-Bound

Solutions

3) Problem: Let n items be given, where each item has a weight and a profit. The weights and profits are positive integers. Furthermore, let a positive integer W be given. Determine a set of items with maximum total profit, under the constraint that the sum of their weights cannot exceed W. Inputs: positive integers n and W, arrays of positive integers w and p, each indexed from 1 to n, and each of which is sorted in nonincreasing order according to the values of p[i]/w[i].

Outputs: an optimal set of items whose total weight does not exceed W. void knapsack2(int n, const int p[], const int w[], int W, set_of_integers& maxset) { queue_of_node Q; node u, v; initialize(Q); v.level = 0;

v.profit = 0;

v.weight = 0; v.items = ;

int maxprofit = 0; maxset = ; enqueue(Q, v); while(!empty(Q)) { dequeue(Q, v); u.level = v.level + 1; u.weight = v.weight + w[u.level]; u.profit = v.profit + p[u.level]; u.items = v.items; add u.level to u.items; if(u.weight <= W && u.profit > maxprofit) maxprofit = u.profit; maxset = u.items; } if(bound(u) > maxprofit) enqueue(Q,u); u.weight = v.weight; u.profit = v.profit; u.items = v.items; if(bound > maxprofit) enqueue(Q, u);

Chapter 6: Branch-and-Bound

Solutions

} } The bound() function is the identical to the one on page 250.

4) Use Algorithm 6.2 (The Best-First Search with Branch-and-Bound Pruning algorithm for the 0-1 Knapsack problem) to maximize the profit for the problem instance of Exercise 1. Show the actions step by step.

1. Visit node (0,0). Set its profit and weight to $0 and 0. Compute its bound to be $80. maxprofit = 0.

2. Compute profits, weights, and bounds for children of (0,0): (1,1) and (1,2).

3. (1,1) has profit of $20, weight of 2, and bound of $80. Enqueue with key 80.

4. (1,2) has profit of $0, weight of 0, and bound of $69. Enqueue with key 69.

5. Dequeue/visit highest priority node: (1,1). maxprofit = 20.

6. Compute profit, weights, and bounds for children of (1,1): (2,1) and (2,2).

7. (2,1) has profit of $50, weight of 7, and bound of $80. Enqueue with key 80.

8. (2,2) has profit of $20, weight of 2, and bound of $70. Enqueue with key 70.

9. Dequeue/visit highest priority node: (2,1). maxprofit = 50.

10. Compute profit, weights, and bounds for children of (2,1): (3,1) and (3,2).

11. (3,1) has profit of $85, weight of 14, and bound of $0. Since weight is over limit 13, we do not enqueue.

12. (3,2) has profit of $50, weight of 7, and bound of $65. Enqueue with key 65.

13. Dequeue/visit highest priority node: (2,2).

14. Compute profit, weights, and bounds for children of (2,2): (3,3) and (3,3).

15. (3,3) has profit of $55, weight of 9, and bound of $70. Enqueue with key 70.

16. (3,4) has profit of $20, weight of 2, and bound of $35. Since bound is less than maxprofit, we do not enqueue.

17. Dequeue/visit highest priority node: (3,3). maxprofit = $55.

18. Compute profit, weights, and bounds for children of (3,3): (4,1) and (4,2).

19. (4,1) has profit of $67, weight of 12, and bound of $70. Enqueue with key 70.

20. (4,2) has profit of $55, weight of 9, and bound of $58. Enqueue with key 58.

21. Dequeue/visit highest priority node: (4,1). maxprofit = $67.

22. Compute profit, weights, and bounds for children of (4,1): (5,1) and (5,2).

23. (5,1) has profit of $70, weight of 13, and bound of $70. Enqueue with key 70.

Chapter 6: Branch-and-Bound

Solutions

24. (5,2) has profit of $67, weight of 12, and bound of $67. Since bound does not exceed maxprofit, we do not enqueue.

25. Dequeue/visit highest priority node: (5,1). maxprofit = $70.

26. Since bounds of all remaining nodes are less than or equal to maxprofit, problem is solved. Solution is set of items given by node (5,1): { 1, 3, 4, 5} which has profit $70.

5) Implement Algorithm 6.2 on your system and run it on the problem instance of Exercise 1. /*Solves 0-1 knapsack with Algorithm 6.2 (best-first search with Branch-and-Bound pruning) Tested with MSVS 2012 professional #include <iostream> //#include <queue> #include <list> using namespace std;

*/

#define W 13 #define n 5 int p[] = {0, 20, 30, 35, 12, 3}; //Must be sorted descending by pi/wi! int w[] = {0, 2, 5, 7, 3, 1}; int node_counter = 0; //Used to keep track of how many nodes are accessed struct node{ int level; int profit; int weight; float boundie; list<int> items; };

//items included on the path to this node

void print_list(list<int> x){ list<int>::iterator i; for (i = x.begin(); i != x.end(); i++) cout <<*i <<" "; cout <<endl; } void print_node(struct node u){ cout <<u.level <<" " <<u.profit <<" " <<u.weight <<" "; print_list(u.items); } bool compare_nodes(const struct node a, const struct node b){ if (a.boundie < b.boundie) return true; else return false; } float bound(struct node u){ int j, k, totweight;

Chapter 6: Branch-and-Bound

Solutions

float result; if (u.weight > W) return 0; //bound is 0 if limit is exceeded else{ result = u.profit; j = u.level + 1; //start adding next items totweight = u.weight; while (j <= n && totweight + w[j] <= W){ totweight += w[j]; result += p[j]; j++; } k = j; if (k <= n) //add a fraction of the remaining item result += ((float)(W - totweight))*p[k]/w[k]; return result; } } int main(){ struct node u, v; list<struct node> PQ; list<int> bestitems; list<int> empty; int maxprofit = 0;

//Priority Queue implemented with std::list //push at front, pop at back //stores best set of included items so far

v.level = 0; v.profit = 0; v.weight = 0; v.boundie = bound(v); v.items = empty; //root has no items PQ.push_front(v); while (!PQ.empty()){ v = PQ.back(); PQ.pop_back(); node_counter++; if (v.boundie > maxprofit){ u.level = v.level + 1; //Left child includes the next item u.weight = v.weight + w[u.level]; u.profit = v.profit + p[u.level]; u.boundie = bound(u); u.items = v.items; u.items.push_back(u.level); if (u.weight <= W && u.profit > maxprofit){ maxprofit = u.profit; bestitems = u.items; } if (u.boundie > maxprofit){ PQ.push_front(u); //push_back also works, since we're sorting it PQ.sort(compare_nodes); //keep PQ sorted } //Right child does not include the next item u.weight = v.weight; u.profit = v.profit; u.items = v.items; u.boundie = bound(u); if (u.boundie > maxprofit){ PQ.push_front(u); //push_back also works, since we're sorting it PQ.sort(compare_nodes); //keep PQ sorted }

Chapter 6: Branch-and-Bound

Solutions

} } cout <<"Knapsack with BEST-FIRST Branch-and-bound" <<endl <<endl; cout <<"maxprofit = " <<maxprofit <<endl; cout <<"list of best items (their indices): "; print_list(bestitems); cout <<"nr. of nodes examined (popped) : " <<node_counter <<endl <<endl;

}

Note that, even for this small problem, the Best-First algorithm is better than the BreadthFirst: it found the solution after examining only 8 nodes instead of the 15 from Exercise 2.

6) Compare the performance of Algorithm 6.1 with that of Algorithm 6.2 for large instances of the problem.

Below are the solutions for this Knapsack problem with 15 items (already sorted descending by pi/wi) #define W 750 #define n 15 int p[] = {0,240,192,184,229,149,135,173,221,210,139,156,201,214,150,163}; int w[] = {0,120, 98, 94,118, 77, 70, 90,115,110, 73, 82,106,113, 80, 87};

The Best-First algorithm is much better than Breadth-First: it finds the solution after examining only 106 nodes instead of 909.

Chapter 6: Branch-and-Bound

Solutions

Section 6.2

7) 1. Visit node [1]. Compute bound to be 28. minlength = .

2. Visit node [1,2]. Compute bound to be 28.

3. Visit node [1,3]. Compute bound to be 31.

4. Determine next most promising node: [1,2].

5. Visit node [1,2,3]. Compute bound to be 28.

6. Visit node [1,2,5]. Compute bound to be  (there are no edges from v5 to vertices not already in the path).

7. Determine next most promising node:: [1,2,3].

8. Visit node [1,2,3,4]. Compute bound to be 28.

9. Visit node [1,2,3,7]. Compute bound to be 31.

10. Determine next most promising node:: [1,2,3,4].

11. Visit node [1,2,3,4,8]. Compute bound to be 28.

12. Determine next most promising node: [1,2,3,4,8].

13. Visit node [1,2,3,4,8,7]. Compute bound to be 28.

14. Visit node [1,2,3,4,8,6]. Compute bound to be 29.

15. Determine next most promising node: [1,2,3,4,8,7].

16. Visit node [1,2,3,4,8,7,6], which completes tour [1,2,3,4,8,7,6,5,1] with length 33. Set minlength to 33.

17. Determine next most promising node: [1,2,3,4,8,6].

18. Visit node [1,2,3,4,8,6,5]. Compute bound to be  (there are no edges from v6 to vertices not already in the path).

19. Determine next most promising node: [1,2,3,7] (ties broken arbitrarily).

20. Visit node [1,2,3,7,4]. Compute bound to be 31.

21. Visit node [1,2,3,7,6]. Compute bound to be 36. Since bound is greater than minlength, this node is not promising.

22. Determine next most promising node: [1,3] (ties broken arbitrarily).

23. Visit node [1,3,4]. Compute bound to be 31.

Chapter 6: Branch-and-Bound

Solutions

24. Visit node [1,3,7]. Compute bound to be 34. Since bound is greater than minlength, this node is not promising.

25. Determine next most promising node: [1,3,4] (ties broken arbitrarily).

26. Visit node [1,3,4,8]. Compute bound to be 31.

27. Determine next most promising node: [1,3,4,8] (ties broken arbitrarily).

28. Visit node [1,3,4,8,7]. Compute bound to be 31.

29. Visit node [1,3,4,8,6]. Compute bound to be 32.

30. Determine next most promising node: [1,3,4,8,7].

31. Visit node [1,3,4,8,7,6]. Compute bound to be 36. Since bound is greater than minlength, this node is not promising.

32. Determine next most promising node: [1,2,3,7,4].

33. Visit node [1,2,3,7,4,8]. Compute bound to be 31.

34. Determine next most promising node: [1,2,3,7,4,8].

35. Visit node [1,2,3,7,4,8,6], which completes tour [1,2,3,7,4,8,6,5,1] with length 32. Set minlength to 32.

36. Node [1,3,4,8,6] becomes not promising.

37. There are no more promising nodes, so problem is solved. Solution is given by node [1,2,3,7,4,8,6,5,1], which has cost 32.

8) 1. Visit node [1]. Compute bound to be 27. minlength = .

2. Visit node [1,2]. Compute bound to be 27.

3. Visit node [1,3]. Compute bound to be 27.

4. Visit node [1,4]. Compute bound to be 31.

5. Visit node [1,5]. Compute bound to be 29.

6. Determine next most promising node: [1,2] (tie broken arbitrarily).

7. Visit node [1,2,3]. Compute bound to be 36.

8. Visit node [1,2,4]. Compute bound to be 31.

9. Visit node [1,2,5]. Compute bound to be 30.

10. Determine next most promising node: [1,3].

Chapter 6: Branch-and-Bound

Solutions

11. Visit node [1,3,2]. Compute bound to be 27.

12. Visit node [1,3,4]. Compute bound to be 34.

13. Visit node [1,3,5]. Compute bound to be 40.

14. Determine next most promising node: [1,3,2].

15. Visit node [1,3,2,4], which completes tour [1,3,2,4,5,1] with length 31. Set minlength to 31. Nodes [1,4], [1,2,3], [1,2,4], [1,3,4] and [1,3,5] are no longer promising.

16. Visit node [1,3,2,5], which completes tour [1,3,2,5,4,1] with length 30. Set minlength to 30. Node [1,2,5] is no longer promising.

17. Determine next most promising node: [1,5].

18. Visit node [1,5,2]. Compute bound to be 31. This node is not promising.

19. Visit node [1,5,3]. Compute bound to be 33. This node is not promising.

20. Visit node [1,5,4]. Compute bound to be 29.

21. Determine next most promising node: [1,5,4].

22. Visit node [1,5,4,2], which completes tour [1,5,4,2,3,1] with cost 46.

23. Visit node [1,5,4,3], which completes tour [1,5,4,3,2,1] with cost 33.

24. There are no more promising nodes, so problem is solved. Minimum length tour is given by node [1,3,2,5,4,1] and has cost 30.

9) Write functions length and bound used in Algorithm 6.3.

Both functions assume the ordered set given in u.path has a function next() that, starting with the first element of the set, gives the next element in the set, and a size() function, which gives the number of elements in the set. int length(node u) { int sum = 0; index i; index current, next = u.path.next(); for(i=1; i<u.path.size(); i++) { current = next;

Chapter 6: Branch-and-Bound next = u.path.next(); sum += W[current][next]; } return sum; } int bound(node u) { index i, j; index sum = 0; index first = u.path.next(); index current, next = first; //compute current path cost for(i=1; i<=u.path.size(); i++) { current = next; next = u.path.next(); sum += W[current][next]; } //compute minimum of remaining length int min = ; //compute min weight out of last vertex in path for(all j such that 2 <= j <= n && j is not in u.path) { if(W[next][j] < min) min = W[next][j]; sum += min; //compute min weight into first vertex in path for(all j such that 2 <= j <= n && j is not in u.path) { if(W[j][first] < min) min = W[next][j]; sum += min; //compute min weight out of all remaining vertices for(all i such that 2 <= i <= n && i is not in u.path) { min = ; for(all j such that 2 <= j <= n && j is not in u.path)

Solutions

Chapter 6: Branch-and-Bound

Solutions

if(W[i][j] < min) min = W[i][j]; sum += min; } return sum; }

10) Consider the Traveling Salesperson problem. (a) Write the brute-force algorithm for this problem that considers all possible tours.

Solution: This was solved using backtracking in Exercise 43 of Ch.5: minimum = INFINITY; best_vindex[n] = {-1}; void hamiltonian(index i) { index j; if(promising(i)) { if( i == n – 1

&& evaluate(vindex) < minimum) {

minimum = evaluate(vindex); cout << vindex[0] through vindex[n-1]; } for(j=2; j<n; j++) { vindex[i+1] = j; hamiltonian(i+1); } } }

The promising function is identical to the one on page 233.

The function evaluate takes a solution vindex as argument and returns its cost, i.e. the sum of the weights of all edges in that solution. (b) Implement the algorithm and use it to solve instances of size 6, 7, 8, 9, 10, 15, and 20.

Solution: #include <iostream> #include <time.h> using namespace std;

Chapter 6: Branch-and-Bound

Solutions

#define n 9 int W[n+1][n+1] = { {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 7, 37, 34, 13, 48, 50, 26, 45}, {0, 36, 0, 2, 21, 16, 3, 17, 2, 5}, {0, 14, 14, 0, 44, 10, 29, 22, 2, 6}, {0, 22, 6, 5, 0, 4, 11, 15, 8, 43}, {0, 17, 14, 27, 28, 0, 22, 20, 5, 21}, {0, 38, 33, 34, 41, 49, 0, 33, 27, 32}, {0, 19, 15, 45, 37, 34, 28, 0, 2, 25}, {0, 40, 28, 36, 18, 45, 23, 27, 0, 28}, {0, 17, 48, 37, 5, 4, 34, 36, 12, 0} }; int best_vindex[n] = {-1}; int vindex[n] = {-1}; int minimum = 1000000; bool promising(int i){ int j; bool switchie; if (i==n-1 && !W[vindex[n-1]][vindex[0]]) //No edge to close circuit switchie = false; else if (i>0 && !W[vindex[i-1]][vindex[i]]) //No edge to current node switchie = false; else{ //Check if vertex has already been selected switchie = true; j = 1; while (j<i && switchie){ if (vindex[i] == vindex[j]) switchie = false; j++; } } return switchie; } int evaluate(int vindex[]){ int sum = 0; for (int i=0; i<n-1 ;i++) sum += W[vindex[i]][vindex[i+1]]; sum += W[vindex[n-1]][vindex[0]]; return sum; }

//closing the circuit

void hamiltonian(int i) { int j; if(promising(i)) { if (i == n-1){ int eva = evaluate(vindex); if (eva < minimum) { minimum = eva; cout <<"Found new minimum tour = " <<minimum <<" : "; for (int k=0; k<n; k++){ cout << vindex[k] <<" "; best_vindex[k] = vindex[k]; //update minimal circuit } cout <<endl;

Chapter 6: Branch-and-Bound

Solutions

} } else for(j=2; j<=n; j++) { vindex[i+1] = j; hamiltonian(i+1); } } } int main(){ time_t begin, end; begin = time(NULL); vindex[0] = 1; hamiltonian(0); end = time(NULL); cout << "\ttime = " << difftime(end, begin) << " s" << endl; return 0; }

Below are the minimal tours obtained for problems with n = 9, 10, 12, and 15 cities:

For n = 9 (and below), the runtime is under 1 second. For n = 15, it is 2,362 seconds. n =20 was not attempted, due to excessive runtime. (c) Compare the performance of this algorithm to that of Algorithm 6.3 using the instances developed in (b).

Solution: Performances will vary.

11) Implement Algorithm 6.3 on your system, and run it on the problem instance of Exercise 7. Use different bounding functions and study the results.

Chapter 6: Branch-and-Bound

Solutions

12) Compare the performance of your dynamic programming algorithm (see Section 3.6, Exercise 27) for the Traveling Salesperson problem with that of Algorithm 6.3 using large instances of the problem.

Performances will vary.

Section 6.3 13) Problem: Determine the m most probably sets of diseases (explanations) given a set of n diseases and a set S of symptoms. It is assumed that if a set of diseases D is a subset of diseases D’, then p(D’) <= p(D).

Inputs: positive integer n, positive integer m, a Bayesian network BN representing the probabilistic relationships among n diseases and their symptoms, and set of symptoms S. Outputs: a priority queue best containing m sets of indices of diseases representing the m most probable sets of diseases, with lower probability diseases having higher priority.

The keys of best represent the probabilities of these sets, and the data elements in best are the sets themselves. void cooper2(int n, int m, Bayesian_network_of_n_diseases BN, set_of_symptoms S, priority_queue_of_sets_of_indices best) { priority_queue_of_node PQ; node u, v; v.level = 0; v.D = ; insert(best, p(|S); v.bound = bound(v); insert(PQ, v); while(!empty(PQ)) { remove(PQ); if(v.bound > best.peek().key()) { u.level = v.level + 1; u.D = v.D; put u.level in u.D;

Chapter 6: Branch-and-Bound

Solutions

if(p(u.D|S) > best.peek().key()) { if(size(best)==m) remove(best); insert(best, u.D, p(u.D|S)); } u.bound = bound(u); if(u.bound > best.peek().key()) insert(PQ, u); u.D = v.D; u.bound = bound(u); if(u.bound > best.peek().key()) insert(PQ, u); } } } It is assumed that the priority queue best is a min-queue (the minimum-valued key is at the head of the queue) and that the function peek() is available to look at the head of the queue without removing it. Also, the key value of the element at the head of the queue can be obtained by calling peek().key(). The insert function for this queue takes in the element to be enqueued as well as the key with which it is inserted (the conditional probability) to be explicit. Finally, the number of elements in the queue can be obtained by calling the size() function. The bound function is identical to the one on page 274.

14) Show that if the diseases in Example 6.4 were sorted in nonincreasing order according to their conditional probabilities, the number of nodes checked would be 23 instead of 15.

Assume that p(d4)= 0.008 and p(d4, d1)= 0.007.

The sorted diseases are: p(d4|S) = 0.6, p(d1|S) = 0.4, p(d2|S) = 0.15, p(d3|S) = 0.1, so we try them in the order d4, d1, d2, d3.

(0, 0) p(∅|S) = 0.1 Bound = 90

(4, 1) p(d4|S) = 0.6 Bound = 0.8 best = {d4} p(best|S) = 0.6

(4, 2) p(∅|S) = 0.1 Bound = 90

Chapter 6: Branch-and-Bound

Solutions

The solution continues with the subtree rooted at (4, 2), b/c the largest bound in a leaf is 90: (4, 2) p(∅|S) = 0.1 Bound = 90

(1, 2) p(∅|S) = 0.1 Bound = 90

(1, 1) p(d1|S) = 0.4 Bound = 0.9

(2, 3) p(d1,d2|S) = 0.15 Bound = 0.3

(2, 4) p(d1|S) = 0.4 Bound = 0.9

(3, 3) p(d1,d3|S) = 0.5 Bound = 0

(2, 1) p(d2|S) = 0.15 Bound = 0.5

(3, 4) p(d1|S) = 0.4 Bound = 0

(2, 2) p(∅|S) = 0.1 Bound = 90

(3, 1) p(d3|S) = 0.1 Bound = 0

(3, 1) p(∅|S) = 0.1 Bound = 0

At this point, the largest bound in a leaf is in node (4, 1), so we start exploring that subtree: (4, 1) p(d4|S) = 0.6 Bound = 0.8 best = {d4} p(best|S) = 0.6

(1, 3) p(d4,d1|S) = 0.65 Bound = 0.7 best = {d4,d1} p(best|S) = 0.65

(1, 4) p(d4|S) = 0.6 Bound = 0.8

(2, 5) p(d4,d2|S) = 0.6 Bound = ???

Not finished. Waiting for feedback from text author.

(2, 6) p(d4|S) = 0.6 Bound = 0.8

Chapter 6: Branch-and-Bound

Solutions

15) A set of explanations satisfies a comfort measure p if the sum of the probabilities of the explanations is greater than or equal to p. Revise Algorithm 6.4 to produce a set of explanations that satisfies p, where 0≤p≤ 1. Do this with as few explanations as possible.

If the goal is to find the solution with minimum number of diseases, the priority queue should use the nodes’ number of diseases to assign priorities (rather than the bound). Higher priority is given to fewer diseases. This is made easier by simply storing the number of diseases in the node structure when the node is created: struct node{ int level ; // the node’s level in the tree set of indices D; int nr_indices; float bound; } Since this is not an optimization problem anymore, best and pbest are not needed.

We still use bound to cull off unpromising nodes by comparing it to the level of comfort p. void comfort (

int n, float p, Bayesian network of n diseases BN, set of symptoms S

){

priority queue of node PQ; node u, v; v.level = 0;

v.D = ∅;

v.nr_indices = 0;

v.bound = bound(v); insert(PQ, v); while (!empty(PQ)){ remove(PQ, v);

// Remove node with lowest nr_indices.

if (p(u.D|S) > p){

//Level of comfort reached!

print D; return; } else { u.level = v.level +1; // Set u to a child of v.

Chapter 6: Branch-and-Bound

Solutions

u.D = v.D; // Set u to the child that put u.level in u.D; // includes the next disease . u.nr_indices = v.nr_indices + 1; //one more disease u.bound = bound(u); if (u.bound > p) //node could reach level of comfort insert(PQ, u); u.D = v.D;

// Set u to the child that does

u.bound = bound(u);

// not include the next disease

u.nr_indices = v.nr_indices; if (u.bound > p) //node could reach level of comfort insert(PQ, u); } }//end while }

The function bound is the same as the one in Algorithm 6.4

16) Implement Algorithm 6.4 on your system. The user should be able to enter an integer m, as described in Exercise 11, or a comfort measure p, as described in Exercise 13.

Waiting for feedback from text author.

17) Can the branch-and-bound design strategy be used to solve the problem discussed in Exercise 38 in Chapter 3? Justify your answer.

The problem is: Find the maximum sum in any contiguous sublist in a list of n numbers.

Since the contiguous sublist is a sequence of objects chosen from a specified set (the n numbers), it can be solved through backtracking (brute-force solution): We explore the solution space with a tree similar to any of the ones used in this chapter: the left child designates the solution in which the number was chosen, and the right child – the solution in which it was not.

For branch-and bound, we need a bounding function. Since this is a maximum problem, we need an upper bound on the current candidate solution: we add up all the consecutive nonnegative numbers following the current one. We also store in the node the index of the last non-negative number that was added, in order to be able to recover the actual subsequence.

Chapter 6: Branch-and-Bound

Solutions

18) Problem: Determine the schedule with the maximum total profit given that each job has a profit that will be obtained only if the job is scheduled by its deadline.

Inputs: positive integer n (the number of jobs), an array of integers deadline indexed from 1 to n, where deadline[i] is the deadline for job i, and an array of integers profits, where profit[i] is the profit for finishing job i by its deadline. Both deadline and profits have been sorted in nonincreasing order according to the profits of the jobs.

Outputs: an optimal sequence J for the jobs. void schedule2(int n, const int deadline[], const int profit[], sequence_of_integer& J) { priority_queue_of_node PQ; node u, v; initialize(PQ); v.profit = 0; v.sequence = []; v.bound = bound(v.sequence); insert(PQ, v); maxprofit = 0; while(!empty(PQ)) { remove(PQ, v); if(v.bound > maxprofit) { index i; for(each i not in v.sequence) { K = J with i added; if(K is feasible) { if(profit(K) > maxprofit) maxprofit = profit(K); u.sequence = K; u.bound(u.sequence); if(u.bound > maxprofit) insert(PQ, u); } } } }

Chapter 6: Branch-and-Bound

Solutions

} int profit(const sequence_of_integer& J) { int sum = 0; for(each element x in K) sum += profit[x]; return sum; } int bound(const sequence_of_integer& J) { int sum = profit(J); sequence_of_integer K = J; index i; for(i=1; i<=n; i++) { if(K does not contain i) { add i to K; sum += profit[i]; } } return sum; }

19) Can the branch-and-bound design strategy be used to solve the problem discussed in Exercise 26 in Chapter 4? Justify your answer.

Waiting for feedback from text author.

20) Can the branch-and-bound design strategy be used to solve the Chained Matrix Multiplication problem discussed in Section 3.4? Justify your answer.

Chapter 6: Branch-and-Bound

Solutions

Branch-and-bound is only applicable to backtracking algorithms, which, in turn, are only applicable to problems in which a sequence of objects is chosen from a specified set (see Section 5.1). The Chained Matrix Multiplication problem is not described by a sequence, but rather by a binary tree (each pair of parentheses is a node in the tree). Note that in the CMM problem we’re not choosing the matrices themselves, but rather the pairs of matrices to multiply. Neither backtracking, nor, consequently, branch-and-bound are applicable.

21) 1. Game-tree search in video games for opponent AI or path-planning. For example, picking the highest scoring move in a chess game according to some heuristic scoring mechanism.

2. Nearest neighbor search in high-dimensional spaces. For example, a kd-tree iteratively splits a set of data points in Euclidian space into a tree of sub-regions, with each region having about half of the number of data points is its parent region. Search can then be performed quickly by checking which region the point would map at each branch of the tree.

3. Generalized integer programming problems (of which the 0-1 Knapsack problem is a specific instance), in which a function must be maximized (or minimized) according to some set of linear constraints, but the variables of the function are constrained to take on integer values.

Ch7: Introduction to Computational Complexity: Sorting

Solutions

Sections 7.1 and 7.2

1) Implement the Insertion Sort algorithm (Algorithm 7.1), run it on your system, and study its best-case, average-case, and worst-case time complexities using several problem instances.

Solution: Best case: Consider the following input 1, 2, 3, 4, 5 (completely sorted) → The for loop in Alg. 7.1 runs 4 times (n-1), and each time the comparison S[j] > x is evaluated only once → 4 comparisons. For n elements, the best-case needs n-1 comparisons.

Average-case: Consider the following input 3, 5, 6, 4, 7 for i = 2  j = 1  3, 5, 6, 4, 7 (1 comparison) for i = 3  j = 2  3, 5, 6, 4, 7 (1 comparison) for i = 4  j = 3  3, 4, 5, 6, 7 (3 comparisons) for i = 5 → j = 4  3, 4, 5, 6, 7 (1 comparison) Total: 6 comparisons, close to the asymptotical one listed in Table 7.1: A(n) = n2/4= 52/4 = 6.75.

Here it would be instructive to use a program to sort several larger arrays (say, with n = 100) of random integers. The pseudocode in Alg. 5.1 can be slightly modified to produce the exact number of comparisons: void insertionsort (int n, keytype S[]) { index i , j ; keytype x; counter = 0; for (i = 2; i <= n; i++) { x = S[ i ]; j = i− 1; while (j > 0 && S[ j ] > x) { S[ j + 1] = S[ j ]; j−−; counter++; } if (j > 0) //no comparison if j == 0 counter++; S[ j + 1] = x; } cout << counter; } When the experiment is repeated many times, the average of the number of comparisons should approach A(n) = n2/4 = 10,000/4 = 2,500.

1

Ch7: Introduction to Computational Complexity: Sorting

Solutions

Worst-case: Consider the following input 6, 5, 4, 3, 2 (sorted in reverse order) for i = 2 → j = 1  5, 6, 4, 3, 2 (1 comparison) for i = 3 → j = 2  4, 5, 6, 3, 2 (2 comparisons) for i = 4 → j = 3  3, 4, 5, 6, 2 (3 comparisons) i = 5 → j = 4  2, 3, 4, 5, 6 (4 comparisons) Total: 10 comparisons, close to the asymptotical one listed in Table 7.1: A(n) = n 2/2= 52/2 = 12.5.

2) Show that the maximum number of comparisons performed by the Insertion Sort algorithm (Algorithm 7.1) is achieved when the keys are input in nonincreasing order.

Solution: On page 289, it is shown that the maximum number of comparisons of Insertion Sort is given by n(n-1)/2. When the keys are in nonincreasing order (i.e., [n, n-1, n-2, …, 2, 1]), then each iteration of the algorithm, starting with i=2, must compare the ith element with each of the elements in the indices less than i (all the way down to index 1). This is because each element at index i will always need to be inserted at the beginning of the array, since the elements to its left, while sorted, are all greater than it by construction of the input. The total number of comparisons done is thus 1 on the first iteration, and increases by 1 on each of the subsequent n-2 iterations, increasing to n-1. This is the sum of the integers from 1 to n-1, which is given by the expression n(n-1)/2, the maximum given on page 289.

3) Show that the worst-case and average-case time complexities for the number of assignments of records performed by the Insertion Sort algorithm (Algorithm 7.1) are given by

W ( n) 

(n  4) (n  1) n 2  2 2

and

A(n) 

n ( n  7) n2 2 4 4

Solution: The worst case is when the array is sorted in reverse order, so each insertion is done in the first position of the array. For i = 2, 1 + 1+1 = 3 record assignments are needed, for i = 3, 1 + 2 + 1 = 4, and so on until i = n, where 1 + n – 1 + 1 = n + 1 assignments are needed.

Total: 3 + 4 + … + n + (n+1) = [1 + 2 + 3 + 4 + … + n + (n+1) ] – (1 + 2) = (n+1)(n+2)/2 – 3 = (n-1)(n+4)/2  n2/2.

In the average case: 2

Ch7: Introduction to Computational Complexity: Sorting    

Solutions

Each S[i] has equal probabilities 1/i to be inserted in any of the positions 1 through i.

Insertion in position k requires: one assignment for x = S[i], (i – k) assignments to shift the elements, and one more for S[k] = x, for a total of (i – j + 2).

If k = i, no assignments are really needed, since no insertion is performed, but Algorithm 5.1 still performs two useless assignments.

The average is [2 + 3 + 4 + … + (i – 1) + i + (i+1)] ∙1/i = [(i – 1)∙i/2 – 1 + i +( i +1)] / i = (i – 1)/2 + 2.



Summing the above from 2 to n, we get A(n) = n(n+7)/4 – 2  n2/4 for large n.

A quick check can be performed for n = 2: The formula gives A(2) = 2(2+7)/4 – 2 = 2.5.

There is a 0.5 chance for the array to be already sorted, in which case two assignments are performed, and a 0.5 chance for the array not to be sorted, in which case there are three assignments. The average is 0.5 x 2 + 0.5 x 3 = 2.5.

4) Show that the worst-case and average-case time complexities for the number of assignments of records performed by the Exchange Sort algorithm (Algorithm 1.3) are given by W ( n) 

3n (n  1) 2

and

A(n) 

3n (n  1) 4

Solution: The worst-case occurs when the array is sorted in reverse order, because every comparison results in an exchange. The total number of exchanges is (n-1) + (n-2) + … + 2 + 3n (n  1) 1 = n(n-1)/2. Each exchange requires three assignments, so we obtain W (n)  .

2 In the average case, only half of the comparisons result in exchanges, so we divide the 3n (n  1) previous expression by 2 to obtain A(n)  .

4

3

Ch7: Introduction to Computational Complexity: Sorting

Solutions

5) Compare the best-case time complexities of Exchange Sort (Algorithm 1.3) and Insertion Sort (Algorithm 7.1).

Solution: In either algorithm, the best case is when the array is already sorted.

Best case time complexity of comparisons Best case time complexity of assignment of records

Exchange sort n(n-1)/2 → O(n 2 )

Insertion sort n-1 → O(n)

Zero → O(1)

2(n-1) → O(n)

6) Is Exchange Sort (Algorithm 1.3) or Insertion Sort (Algorithm 7.1) more appropriate when we need to find in nonincreasing order the k largest (or in nondecreasing order the k smallest) keys in a list of n keys? Justify your answer.

Solution: Exchange Sort is more appropriate in this situation as after k iterations of Exchange Sort, the array will contain the k smallest (largest) elements in the first k indices of the array, since each iteration of the algorithm inserts the next smallest (largest) element into the appropriate spot in the array. This is not the case for Insertion Sort, which could take n iterations to find the smallest (largest) k elements.

7) Rewrite the Insertion Sort algorithm (Algorithm 7.1) as follows. Include an extra array slot S[0] that has a value smaller than any key. This eliminates the need to compare j with 0 at the top of the while loop. Determine the exact time complexity of this version of the algorithm. Is it better or worse than the time complexity of Algorithm 7.1? Which version should be more efficient? Justify your answer.

Solution:

void insertion sort (int n, key type S [ ]) { index i, j; key type x; S [0] = MINUS_INFINITY; //sufficiently small value for (i = 2; i < n; i++) x = S [i]; j = i – 1; while [S [j] > x] { S [j + 1] = S [ j ]; j––; } S [j + 1] = x; } } 4

Ch7: Introduction to Computational Complexity: Sorting

Solutions

Worst-case complexity analysis: o For comparisons: This algorithm performs in each iteration of the for loop an extra array comparison S[0] > x, which is always false. We add (n-1) to the W(n) for insertion sort →(n-1)(n+2)/2. o The number of assignments is the same.

Hence Algorithm 7.1 is better.

8) An algorithm called Shell Sort is inspired by Insertion Sort’s ability to take advantage of the order of the elements in the list. In Shell Sort, the entire list is divided into noncontiguous sublists whose elements are a distance h apart for some number h. Each sublist is then sorted using Insertion Sort. During the next pass, the value of h is reduced, increasing the size of each sublist. Usually the value of each h is chosen to be relatively prime to its previous value. The final pass uses the value 1 forh to sort the list. Write an algorithm for Shell Sort, study its performance, and compare the result with the performance of Insertion Sort.

Solution: Shells’s original publication (Shell, D.L., "A High-Speed Sorting Procedure", Communications of the ACM 2 (7): 30–32, 1959) used the sequence of h-values (a.k.a. gaps) equal to N/2, N/4, … , 1 (taking floors if necessary). With these gaps, the algorithm is int i, j, h, temp; for (h = N/2; h > 0; h = h/2) for (i = h; i < N ; i++){ temp = a[i]; for (j = i; j >= h && temp < a[j-h]; j = j-h) a[j] = a[j-h]; a[j] = temp; } }

Many better sequences have been found, e.g.  

1, 3, 5, … 2k-1, …, proven to be Q(n3/2) , and 1, 2, 3, 4, 6, …, 2a3b, …, proven to be (n∙lg2n).

Because of this, Shell Sort is theoretically important: by choosing different sequences, we can “bridge” the gap between (n2) and (n∙lg n).

Practically, it has the advantage of having a very simple code implementation (inherited from Insertion Sort), and it was found to outperform the (n∙lg n) algorithms on array sizes up to a few tens of thousands of elements.

5

Ch7: Introduction to Computational Complexity: Sorting

Solutions

Section 7.3

9) Show that the permutation [n,n−1, . . . , 2, 1] has n(n−1)/2 inversions.

Solution: Since, in the permutation [n, n-1, n-2, …, 2, 1], any element at index i is greater than the elements at exactly all indices j>i, each element at position i has n-i inversions.

Thus, the total number of inversions is given by the sum of this difference for i=1 to n, which is the sum of the integers from n-1 down to 1, which is equal to n(n-1)/2.

10) Give the transpose of the permutation [2, 5, 1, 6, 3, 4], and find the number of inversions in both permutations. What is the total number of inversions?

Solution: The permutation [2,5,1,6,3,4] has 6 inversions. Its transpose [4,3,6,1,5,2] has 9 inversions. The total number of inversions is thus 15.

11) Show that there are n(n−1)/2 inversions in a permutation of n distinct ordered elements with respect to its transpose.

Solution: A permutation (k1,k2, … , kn) has an inversion (i, j) with respect to its transpose if the order between ki and kj in the permutation is the opposite of the order of the same elements in the transpose.

It is easy to see that all elements change the order in the transpose, so the number of inversions is the number of distinct pairs (i, j). As shown in Section A.7, the number of n! combinations of n objects taken k at a time is . Making k = 2 and simplifying, we k!(n  k )! obtain n(n-1)/2.

12) Show that the total number of inversions in a permutation and its transpose is n(n−1)/2. Use this to find the total number of inversions in the permutation in Exercise 10 and its transpose.

Solution: The proof of Theorem 7.1 states: --Let r and s be integers between 1 and n such that s >r. Showing that there are n(n−1)/2 such pairs of integers between 1 and n is left as an exercise.-As shown in Section A.7, the number of combinations of n objects taken k at a time is n! . Making k = 2 and simplifying, we obtain n(n-1)/2. k!(n  k )!

6

Ch7: Introduction to Computational Complexity: Sorting

Solutions

In exercise 10, n is 6, so our formula yields: n(n  1) 6(6  1)  2 2 65   15 inversions.

2

Section 7.4

13) Implement the different Mergesort algorithms discussed in Section 2.2 and Section7.4, run them on your system, and study their best-case, average-case, and worst-case performances using several problem instances.

Solution: The implementations and performances will vary.

14) Show that the time complexity for the number of assignments of records for the Mergesort algorithm (Algorithms 2.2 and 2.4) is approximated by T (n) =2n∙lg n.

Solution: In Section 2.2, it was shown that the worst-case comparison complexity of Mergesort is n∙lg n, and from Algorithm 2.3 we see that every comparison in Merge is followed by a record assignment, so we have  n∙lg n assignments in the Merge part of the algorithm. To this, we add the copy operations in Algorithm 2.2, which are also  n∙lg n, because the division tree is lg n levels deep.

Adding the two previous numbers, we have  2n∙lg n.

Example: Consider the following input for the section 2.2 Algorithm: 2, 5, 4, 6, where n = 4.

To get to the singleton list in the middle, we did 8 assignments.

Then we performed this Merge: U 2, 5 2, 5 2, 5 2, 5

V 4, 6 4, 6 4, 6 4, 6

S 2 2, 4 2, 4, 5 2, 4, 5, 6

The Merge required another 8 assignments, for a total of 16.

The

formula

2n∙lgn

yields

2x4xlog24

=

2x4x2

=

16,

the

same

number!

7

Ch7: Introduction to Computational Complexity: Sorting

Solutions

15) Write an in-place, linear-time algorithm that takes as input the linked list constructed by the Mergesort 4 algorithm (Algorithm 7.4) and stores the records in the contiguous array slots in nondecreasing order according to the values of their keys.

Solution: With the notation in Algorithm 7.4, the merged list is identified by the index of its head, called mergedlist. The whole list is stored in the array S, whose elements have type struct node, i.e. they are pairs (key, link). We need the local variable next_index. The pseudocode is: void contiguous(S, n, mergedlist) { next_index = mergedlist; for (int i =1; i<=n; i++) { swap S[i] with S[next_index]; next_index = S[i].link; } } Notes:   

The link members are not updated, since they are redundant once the array is physically sorted.

Swapping requires a temporary variable of type struct node.

The every-time complexity is (n), due to the existence of only one for loop.

16) Use the divide-and-conquer approach to write a nonrecursive Mergesort algorithm.

Analyze your algorithm, and show the results using order notation. Note that it will be necessary to explicitly maintain a stack in your algorithm.

Solution: The non-Recursive Mergesort Algorithm is modeled after Algorithm 2.1, but instead of using recursive calls, the stack details are manipulated explicitely. The array S is global. struct Stack { int lo, hi; bool new; }; void non_rec_Mergesort (int n) { struct Stack S[100]; int top, low, hig, mid; top = 0; S[top].lo = 1; S[top].hi = n; S[top].new = true;

//n array elements

//first array index and //last array index are being

8

Ch7: Introduction to Computational Complexity: Sorting top++;

Solutions

//push on the stack

while (top > 0) { low = s[top].lo; hig = s[top].hig; mid = floor((lo + hi)/2); if (S[top].new) { if (low >= hig) { top--; continue; } else { S[top].new = false; top++; S[top].lo = low; S[top].hi = mid; S[top].new = true; top++; S[top].lo = mid+1; S[top].hi = hig; S[top].new = true; } } else { merge2(low, mid, hig); top--; } }// end of while

//stack is not empty

//first time processing subarray //boundary case: 0 or 1 elements //pull off stack //go to next top of stack //non-trivial subarray //push left subarray

//push right subarray

//not new subarray, must merge //See Algorithm 2.3 //pull off stack once merged

}

The operation is identical to that of the recursive Mergesort from Algorithm 2.2, so it has the same complexity: (n∙lg n) in the worst and average cases.

17) Implement the nonrecursive Mergesort algorithm of Exercise 16, run it on your system using the problem instances of Exercise 13, and compare the results to the results of the recursive versions of Mergesort in Exercise 13.

Solution: Implementations and comparisons will vary.

9

Ch7: Introduction to Computational Complexity: Sorting

Solutions

18) Write a version of mergesort3 (Algorithm 7.3), and a corresponding version of merge3, that reverses the roles of two arrays S and U in each pass through the repeat loop.

Solution: void mergesort3_reverse_SU (int n, keytype S[]) { int m; index low , mid, high , size ; keytype U[n]; //U is now local to mergesort3 bool S_to_U = true; //keeps track of roles m = pow(2, ceil(log(n)); size = 1; //start with singletons repeat (lg m times) { for (low = 1; low <= m− 2∗size + 1; low = low + 2∗size) { mid = low + size − 1; high = minimum(low + 2∗size − 1, n); merge3(low , mid, high , S_to_U); S_to_U = !S_to_U; //reverse roles of S and U } size = 2∗ size ; } The function merge3 is similar to merge2 from Algorithm 2.5, but it makes a decision based on S_to_U: if true, it works exactly as merge2, if not, the roles of S and U in the code (comparisons, assignments) are reversed.

Section 7.5 19) Implement the Quicksort algorithm (Algorithm 2.6) discussed in Section 2.4, run it on your system, and study its best-case, average-case, and worst-case performances using several problem instances.

Solution: #include<stdio.h> void quicksort(int [10], int, int); int main(){ int x[20], size, i; printf (“Enter size of the an(array: ”); scanf(“%d”,&size); printf(“Enter %d elements: “,size); for(i=0;i<size; i++) scanf(“%d”, &x[i]); 10

Ch7: Introduction to Computational Complexity: Sorting

Solutions

quicksort(x,0,size-1); printf(“Sorted elements: ”); for(i=0;i<size; i++) printf(“%d”, x[i]); return 0; } void quicksort(int x[10], int first, int last) { int pivot, j, temp, i; if(first<last){ pivot=first; i=first; j=last; while(i<j){ while(x[i]<=x[pivot]&&i<last) i++; while(x[j]>x[pivot]) j--; if(i<j){ temp=x[i]; x[i]=x[j]; x[j]=temp; } } temp=x [pivot]; x[pivot]=x[j]; x[j]=temp; quicksort(x, first, j -1); quicksort(x, j+l, last); } } Worst Case Analysis: T(n) = n+ T(n-l) T(n) = Θ(n2) Best Case and Average Case: T(n) = n+2T(n/2) T(n) = Θ (n log n) Analysis n 10 50 100 200

n2 100 2500 10000 22500

n log n 34 288 670 1095 11

Ch7: Introduction to Computational Complexity: Sorting

Solutions

20) Show that the time complexity for the average number of exchanges performed by the Quicksort algorithm is approximated by 0.69(n+1)lg n.

Solution: At the end of Section 2.4, the average number of comparisons in Quicksort is proven to be A(n)  1.38(n+1)lg n. Since the exchanges are performed only in the partition function (Algorithm 2.7), we reproduce it here: void partition (index low, index high, index& pivotpoint) { index i , j ; keytype pivotitem ; pivotitem = S[low ]; j = low; for (i = low + 1; i<= high; i++) if (S[ i ] < pivotitem) { j++; exchange S[ i ] and S[ j ]; } pivotpoint = j ; exchange S[low] and S[pivotpoint]; } At each comparison (if), in the average case there are equal probabilities for the result to be true or false, so on average there are half as many exchanges: 1.386(n+1)lg n / 2 = 0.693(n+1)lg n.

Note that the bulk of the exchanges are performed inside the for loop. The exchange on the last line is only performed once per partition call, and on average there are (n-1) such calls, so for asymptotic analysis we can neglect the linear term in comparison to n lg n.

21) Write a non-recursive Quicksort algorithm. Analyze your algorithm, and show the results using order notation. Note that it will be necessary to explicitly maintain a stack in your algorithm.

Solution: void quicksort (item * a, int start, int stop) { int i, s = 0, stack [64]; stack [s++] = start; stack [s++] = stop; while (s > 0) { stop = stack [--s]; start = stack [--s]; if (start >= stop) continue; i = partition (a, start, stop); if (i-start > stop-i) { stack [s++] = start; stack [s++] = i - 1; stack [s++] = i + 1; stack [s++] = stop; 12

Ch7: Introduction to Computational Complexity: Sorting

Solutions

} else { stack [s++] = i + 1; stack [s++] = stop; stack [s++] = start; stack [s++] = i -1; } }

}

Complexity analysis: All the comparisons and record assignments are done in partition, which has not changed, so these complexities are the same as in the recursive algorithm: worst-case: Θ(n2), best-case: Θ (n log n)

22) Implement the non-recursive Quicksort algorithm of Exercise21, run it on your system using the same problem instances you used in Exercise19, and compare the results to the results of the recursive version of Quicksort in Exercise19.

Solution: Implementation and performance will vary, but, all other things being equal, the non-recursive algorithm performs a little faster than the recursive one because it lacks the overhead incurred by the recursive function calls.

23) Show that with the new partition procedure, the time complexity for the number of assignments of records performed by Quicksort is given by A(n) ≈ 0.69(n+1)lg n.

Show further that the average-case time complexity for the number of comparisons of keys is about the same as before.

Solution: For the average–case analysis, we assume that all possible orderings are equal likely. For simplicity, we take the keys to be just the integers from 1 to n.

Average nr. of assignments of records When we select the pivot in the function partition, it is equally likely to be any one of the n keys. Denote by p whatever key is selected as pivot. After the partition, key p is guaranteed to be in position p, since the keys 1, …p-1 are all to its left (and p+1 , … , n are all to its right). The number of exchanges inside the loop is therefore equal to the number N(p) of keys larger than p that were initially positioned to the left of index p. (Compare this with the initial partition in Algorithm 2.7, where all p-2 numbers between low and p need to be exchanged in!) What is the average E[N(p)] of N(p)? Since the permutation is random, we can assume that the (n-p) numbers larger than p are uniformly distributed, so the number of them in the interval [2 … p] is proportional to the length of this interval: E[N(p)] =(p1)(n-p)/(n-1). The “minus ones” are there because we already know that p is in position 1, so we don’t count this position.

13

Ch7: Introduction to Computational Complexity: Sorting

Solutions

Using the 3-assignment exchange technique, this means 3E[N(p)] assignments.

Outside the loop, we have one assignment for pivotitem = S[low], and three for the exchange on the last line, for a total of 4.

Summing up, there are 3E[N(p)] + 4 assignments in the first call to partition if the pivot (first element) is p.

The solution continues similarly to the average-case analysis at the end of Section 2.4: Denote by A(n) the average number of assignments done by quicksort on a list of length n.

Since each p is equally likely, we have n

1 A( p  1)  E[ N ( p)]  4  A(n  p) = p 1 n

A(n) = 

1 n 3 E[ N ( p)]  4n  n  p1 4+

n



p 1



[ A( p  1)  A(n  p)] =

3 n ( p  1)( n  p) 2 n A( p  1)  n 1  n  n p1 p 1

With the tools in Section A.3, the first sum can be shown to be

n ( n  2) , so we have: 6

n2 2 n A(n) = 4 +   A( p  1) 2 n p1 A(n) =

n6 2 n   A( p  1) 2 n p1

(*)

Multiply (*) by n, then write a second relation where n is substituted by (n-1), and then subtract the latter from the former to obtain

A(n) A(n  1) n  2.5   n 1 n n(n  1) Denote A(n)/(n+1) by xn and solve the recursion for xn by direct substitution, as in Example B.22, to obtain xn  ln n, and, accordingly, A(n)  (n+1)ln n.

Changing the base of the logarithm to binary, we use ln n = ln 2 ∙ lg n  0.69lg n, so A(n)  0.69(n+1)lg n Average nr. of comparison of keys i and j start, respectively, from 1 and n, and take steps moving towards each other until they meet (or overlap by 1 position). Each move, either i++ or j--, is always done as a result of a key comparison, so the number of key comparisons is every time about n. Therefore the recursion at the end of Section 2.4 is essentially the same, and has the same approximate complexity: 1.38(n+1)lg n.

14

Ch7: Introduction to Computational Complexity: Sorting

Solutions

24) Give two instances for which Quicksort algorithm is the most appropriate choice.

Solution: The quadratic worst-case complexity of Quicksort (obtained in Section 2.4) is not often a practical issue, because, even in applications where the array is already sorted (or almost sorted) in non-decreasing order, randomized Quicksort can be applied.

The only competition from (n2) algorithms could come in applications where the extra memory space is an issue, since those algorithms (e.g. insertion sort) are able to sort in place, whereas the best Quicksort still needs lg n extra locations on the stack. However, in order for lg n to be significant, n itself must be large, so (n2) will make the running time prohibitive. Shellsort with a gap sequence that yields (n1.5) is the only practical competition, for n up to  10,000 records.

When comparing Quicksort with Mergesort, one significant difference in performance comes from the extra memory space:  The best Quicksort needs only lg n extra records, whereas the best Mergesort needs n.  True, if the records are large, Mergesort can be implemented with a linked list version, which is in-place, but it is more complex to code and still has overhead for the links.

However, if the data to be sorted is already stored as a linked-list, Mergesort becomes more promising.  Mergesort is better when access to the individual records is slow, which happens when the array is too large for RAM, and therefore stored in external memory (HDD, tape); for this case, so-called “external sorting” algorithms exist base on Mergesort. Quicksort is not as easily applicable to external sorting, because the final position of the pivot is not easy to determine before partitioning. There seems to be a loose consensus that Quicksort is better for RAM-based sorting, whereas Mergesort is better for external sorting.

Real-life examples of Quicksort being the “best” implementation:  The qsort library function for the C/C++ language was initially based on Quicksort (the Berkeley 1983 implementation), although the standards do not mandate it. Most of today’s implementations still use it, with various improvements. One notable exception is the GNU library qsort, which is based on Mergesort instead.  Parallel version of Quicksort on a cache-coherent shared address space multiprocessor outperforms best previous algorithm: P. Tsigas, Y. Zhang, “A Simple, Fast Parallel Implementation of Quicksort and its Performance Evaluation on SUN Enterprise 10000”, Parallel, Distributed and Network-Based Processing, 2003. Proceedings of the Eleventh Euromicro Conference on, 2003, pp. 372 –381.  Parallel multi-core graphics processors (GPU): D. Cederman, P. Tsigasy, “A Practical Quicksort Algorithm”, Technical Report no. 2008-01, Goteborg University.

15

Ch7: Introduction to Computational Complexity: Sorting

Solutions

25) Another way to sort a list by exchanging out-of-order keys is called Bubble Sort. Bubble Sort scans adjacent pairs of records and exchanges those found to have out-of-order keys.

After the first time through the list, the record with the largest key (or the smallest key) is moved to its proper position. This process is done repeatedly on the remaining, unsorted part of the list until the list is completely sorted. Write the Bubble Sort algorithm. Analyze your algorithm, and show the results using order notation. Compare the performance of the Bubble Sort algorithm to those of Insertion Sort, Exchange Sort, and Selection Sort.

Solution: void bubbleSort(int n, keytype& S[]) { index i,j; for(i=n; i>0; i--) { for(j=1; j<=n; j++) { if(S[j] > S[j+1]) { keytype temp = S[j]; S[j] = S[j+1]; S[j+1] = temp; } } } } The first iteration performs n-1 comparisons, and each subsequent iteration performs one less comparison than the previous. The total number of comparisons is thus given by the sum of the integers from 1 to n-1, which is equal to n(n-1)/2. The complexity is thus (n2). The number of comparisons is always the same regardless of the input.

Section 7.6

26) Write an algorithm that checks if an essentially complete binary tree is a heap. Analyze your algorithm and show the results using order notation.

Solution: boolean isHeap(node n) { 16

Ch7: Introduction to Computational Complexity: Sorting

Solutions

if(n.left==null && n.right==null) return true; if(n.right==null) { if(n.left > n) return false; return true; } if(n.left > n || n.right < n) return false; return isHeap(n.left) && isHeap(n.right); } Initial call uses root of tree as input parameter. Assumes nodes have member variables left, right, and key, representing the left and right children of the node and its associated key, respectively. The left and right variables of a node may be null, indicating that the node has no such child. Note that a node in an essentially complete binary tree cannot have a null left child unless the right child is also null.

This algorithm performs a pre-order traversal of the tree, in which each node is visited exactly once. Thus, the time complexity is O(n), where n is the number of nodes in the tree.

27) Show that there are 2j nodes with depth j for j < d in a heap having n (a power of 2) nodes. Here d is the depth of the heap.

Solution: Induction on the depth d: Base case: When d = 1, we only have j=0: 20 = 1, and, indeed, there is only one node – the root – on level 0.

Induction step: Assuming the property holds for d, we prove it for d+1. For all j up to d-1, we have 2j because of the assumption. For j = d, we use the fact that the heap is an essentially complete binary tree, meaning that the tree down to level d is complete; this means that each node on level d-1 has 2 children, so the number of those children is 2∙2 d-1 = 2d.

Q.e.d.

Example: In this heap we have: 20 = 1 node at depth 0 → 55 21 = 2 nodes at depth 1 → 54, 53 22 = 4 nodes at depth 2 → 52, 51, 50, 49

17

Ch7: Introduction to Computational Complexity: Sorting

Solutions

28) Show that a heap with n nodes has ⎾n/2⏋ leaves.

Solution: As in Exercise 27, let d be the depth of the heap. The number of nodes can be written as n = 20 + 21 + 2d-1 + N(d), where N(d) is the number of nodes on the last level, d.

There are two cases:  

N(d) is odd N(d) is even

→ →

Base case: When n = 1, the only leaf is the root, and ⎾n/2⏋= ⎾1/2⏋= 1.

Induction step: We assume the property to hold for n, and prove it for n+1. We distinguish between two cases: 

If n = 2k – 1, we use the fact that a complete tree always has a number of nodes of the form 2k – 1. Because the heap is an essentially complete tree, then ⎾n/2⏋= 2k-1.

Because a complete tree always has a number of nodes of the form 2 k – 1 , we conclude that

Example: In this heap we have: n = 8 nodes, and 4 leaves.

The number of leaves is equal to⎾n/2⏋= ⎾8/2⏋= 4

18

Ch7: Introduction to Computational Complexity: Sorting

Solutions

29) Implement the Heapsort algorithm (Algorithm 7.5), run it on your system, and study its best-case, average-case, and worst-case performances using several problem instances.

Solution: Implementation and performance measures will vary.

30) Show that there is a case for Heapsort in which we get the worst-case time complexity of W (n)  2nlg n ∊ (n lg n).

Solution: Consider this heap with n = 8: 2n lg n = 2 x 8 x 3 = 48 comparisons.

The first call to function root causes 30 to be extracted, and performs 2 swaps = 4 comparisons,

producing this heap: Extract 25 and perform 4 comparisons,

producing this heap: Extract 20, 19 and 18, with 4 comparisons for each,

producing this heap: Extracting 17 costs 2 comparisons, and then 16 and 12 need no comparisons.

Total: 22, which is quite different from 48!

However, asymptotic analysis only deals with the limit when n →∞. The point of the example is to show that, for general n = 2k, we have the following tallies of comparisons:   

For about half the nodes 2k-1, k swaps or 2k comparisons are performed → k2k.

For half of the remaining half 2k-2, k-1 swaps or 2(k-1) comparisons → (k-1)2k-1. ……

19

Ch7: Introduction to Computational Complexity: Sorting 

Solutions

Finally, when only 2 nodes are left, zero comparisons are needed.

Total: According to Example A.5, the sum is (k-1)2k+1 2nlg n.

31) Show that the worst-case time complexity of the number of exchanges of records for Heapsort is approximated by W (n)  nlg n.

Solution: From the while loop in the function siftdown (p.309), we see that one iteration of the loop contains two comparisons and one exchange. Since the worst case for comparisons is the same as the worst case for exchanges (i.e. after each extraction the root has to sift all the way down), the ratio comparisons/exchanges is 2/1. Using the result from the previous exercise, we have 2nlg n/2 = nlg n.

32) Modify Heapsort so that it stops after it finds the k largest keys in non-increasing order.

Analyze your algorithm, and show the results using order notation.

Solution: void heapsort (int n, int k, heap & H) { remove keys (n, k, H, S); //removes the largest k keys } void removekeys (int n, int k, heap & H, keytype s []) { index i; if (k > n) { cout << “Not enough elements!” <<endl; return; } for (i = n; i >= n-k+1; i--) s[i] = root (H); } The other functions are identical.

Complexity analysis: The for loop in removekeys executes k times, and each time it calls root, which, as before, performs  2lg n comparisons. The entire loop therefore has  2klg n comparisons, so the algorithm is in (k∙lg n).

20

Ch7: Introduction to Computational Complexity: Sorting

Solutions

Section 7.7

33) List all the advantages and disadvantages of all the sorting algorithms discussed in this chapter based on the comparisons of keys and the assignments of records.

Solution: Among the quadratic algorithms (Table 7.1), Exchange Sort loses out (and is almost never used in practice), since all its performance measures are worse than those of the other two. Between Insertion and Selection Sort, Insertion is faster in terms of comparisons, but Selection is much faster (linear vs. quadratic) in terms of assignments.

Insertion sort can be made a lot more efficient (down to nlg2n) in both comparisons and assignments by using diminishing gap sequences (Shellsort), as explained in Exercise 7.8, thereby becoming much faster than Selection in comparisons, and only slightly slower in assignments.

Among the (nlg n) algorithms (Table 7.2), the order for both assignments and comparisons is the same: Mergesort is best, followed by Quicksort (with Hoare’s partition Exercise 23), followed by Heapsort. However, the order in terms of extra space is exactly the opposite, which is why all three algorithms are used in practice.

34) Run the implementations of all the sorting algorithms discussed in this chapter on your system using several problem instances. Use the results, and the information provided in Exercise33, to give a detailed comparison of these sorting algorithms.

Solution: Implementations and results will vary.

35) Among Selection Sort, Insertion Sort, Mergesort, Quicksort, and Heapsort, which algorithm would you choose in each list-sorting situation below? Justify your answers. (a) The list has several hundred records. The records are quite long, but the keys are very short.

Solution: For long records, the complexity of assignments is of primary concern, with Selection and Merge as prime candidates. Since n is small, the quadratic algorithm will win out because of its simplicity of implementation, since speed is not an issue. (b) The list has about 45,000 records. It is necessary that the sort be completed reasonably quickly in all cases. There is barely enough memory to hold the 45,000 records.

21

Ch7: Introduction to Computational Complexity: Sorting

Solutions

Solution: For this n, quadratic algorithms are most likely too slow. The memory restriction recommends Heapsort, because it’s in-place. (c) The list has about 45,000 records, but it starts off only slightly out of order.

Solution: Again, we consider only (nlg n) algorithms for speed. With the list only slightly out of order, Quicksort will “go quadratic”, so it’s not recommended (unless we choose a randomized version). Since extra space is not an issue, Mergesort wins over Heapsort b/c it’s twice as fast in comparisons and can be also made better in assignments by choosing the linked list variation. (d) The list has about 25,000 records. It is desirable to complete the sort as quickly as possible on the average, but it is not critical that the sort be completed quickly in every single case.

Solution: With 25,000 records, Insert Sort (fastest quadratic) requires 156,250,000 comparisons, whereas Mergesort only requires 365,241, so Mergesort wins. The statement of this case seems to suggest that Quicksort is also acceptable, although its nr. of comparisons is slightly (1.38 times) larger.

36) Give at least two instances for each of the sorting algorithms (based on the comparisons of keys) discussed in this chapter for which the algorithm is the most appropriate choice.

Solution: 







Insertion Sort: Relatively small number (n  1,000) of keys, when only average performance is important e.g. repeatedly sorting 10-minute interval weather data collected in one day from a number of weather stations, or sorting someone’s tweets by length in real-time (it is also an “online” algorithm, i.e. it can sort partial data as it receives it).

Selection sort is preferred to Insertion Sort in real-time applications where it is important for the run-time to be identical irrespective of the order of the elements, e.g. in an embedded system that needs to perform some critical task periodically, or when there is little extra memory space available.

Mergesort is used when there are many (n>10,000) keys with long records, when the records are stored externally, e.g. for sorting databases located on tape or network storage (NFS, SAN), or when the records are stored in caches, b/c it accesses blocks of contiguous data (external sorting). Mergesort is now used by several scripting languages as internal sorting algorithm instead of Quicksort, e.g. Python uses Timsort, which is a combination of Mergesort and Insertion Sort.

Quicksort is used for in-RAM sorting, when we need a good compromise between the complexity of comparisons and that of assignments; it is the basis of most qsort library implementations for C/C++, and of efficient GPU sorting algorithms.

22

Ch7: Introduction to Computational Complexity: Sorting 

Solutions

Heapsort is the slowest of the (nlg n) algorithms, but it has the great merit of sorting in-place, so it’s used in embedded systems when the number of keys is large (n>10,000), e.g. in image-processing. One great application is the priority queue (introduced for Huffman’s algorithm in Ch.4), because it supports insertion and extraction efficiently ((lg n)). Priority queues are used in their own right in many other algorithms, but also in operating systems (e.g. queue of ready processes), eventdriven simulators (queue of events sorted by their time of occurrence), and Internet routers (queues of packets sorted by class of service).

Section 7.8

37) Write a linear-time sorting algorithm that sorts a permutation of integers 1 through n, inclusive. (Hint: Use an n-element array.)

Solution: Inputs: positive integer n, array of integers S containing a permutation of the integers from 1 to n, inclusive.

Outputs: The array S sorted in nondecreasing order. void nSort(int n, int& S[]) { index i; int A[1…n]; for(i=1; i<n; i++) A[S[i]] = S[i]; S = A; }

38) Does the linear-time performance of your algorithm in Exercise 37 violate the lower bound for sorting only by comparisons of keys? Justify your answer.

Solution: The lower bound for sorting based on comparison of keys is not violated by the algorithm of Exercise 37 because there are no comparisons performed by the algorithm.

Strong assumptions are made about the input that allow for linear time complexity.

23

Ch7: Introduction to Computational Complexity: Sorting

Solutions

39) Prove the general case of Lemma 7.9 when the number of leaves m is not a power of 2.

Solution: Lemma: 7.9: For any 2 tree that has m leaves and whose EPL equals min EPL(m), the depth d is given by d= lg m . The proof is modeled after the one for Lemma 7.9: If m is not a power of 2, then, for some positive integers k and z, we have m = 2k + z,

with z < 2k

Let d be the depth of a minimizing tree. By Lemma 7.8, there are r= 2d − m = 2d – 2k – z leaves on the level d – 1, and, since r ≥ 0, we must have d ≥ k + 1.

We show that assuming d > k+1 leads to a contradiction. If d > k+1, then d ≥ k+2, and, by Lemma 7.8, the number of leaves on level d is 2m – 2d = 2(2k + z) – 2d = 2k+1 + 2z – 2d < 2k+1 + 2∙2k – 2d = 2k+2 – 2d ≤ 2k+2 – 2k+2 = 0.

This means that the number of leaves on level d is negative – contradiction.

Therefore, d = k + 1.

On the other hand = lg m = lg(2k + z ) = k + 1, since z < 2k.

We established that d = k + 1 = lg m. Q.e.d.

Section 7.9 40) Implement the Radix Sort algorithm (Algorithm 7.6), run it on your system, and study its best-case, average-case, and worst-case performances using several problem instances.

Solution: Implementations and performances will vary.

Best Case performance occurs when we have the smallest number of digits, i.e. the numbers do not have (big) gaps. Ideally, we could have no gaps, meaning consecutive numbers. Here is an example with the keys being one-digit numbers: 1

5

4

2

3

First pass 1 5 4 2 3 24

Ch7: Introduction to Computational Complexity: Sorting

0 1



1

2



2

3



3

4



4

5



5

Solutions

6 7 8 9 items are moved back to the master list: 1 2 3 4 5 Worst Case performance occurs when we have a mix of numbers, of which some are very small, e.g. one-digit, and a few (just one?) very large, e.g. 100 digits. The length of the for loop in the function radixsort, numdigits will have to be 100 to accommodate the few large numbers, but most of the time only the zeroes in the small numbers will be distributed.

For the average case, the concrete class of applications should be considered. If we have a given finite precision (e.g. integers on 32 bits), and the numbers are uniformly distributed, simply use the library random number generator to generate lists for sorting. If, on the other hand, the number of digits is variable within large bounds (e.g. strings, or “infinite-precision” integers), then the meaning of “average” has to be clarified first for the given application.

41) Show that when all the keys are distinct the best-case time complexity of Radix Sort (Algorithm 7.6) is in Θ(nlgn).

Solution: When all keys are distinct, the best case occurs when the keys consist of the first n integers (i.e., 1 through n), since in this case n is the smallest possible maximum value of the set of keys. Since the complexity of Radix Sort is  (nnumdigits), and the number of digits, it takes to represent the number n is logB n, the best-case complexity in this case is thus (nlogB n). The base does not matter in order notation, since logB n = (log2n)/ (log2B), and log2B is just a constant factor, therefore the best-case complexity is also in (nlg n).

25

Ch7: Introduction to Computational Complexity: Sorting

Solutions

42) In the process of rebuilding the master list, the Radix Sort algorithm (Algorithm 7.6) wastes a lot of time examining empty sublists when the number of piles (radix) is large. Is it possible to check only the sublists that are not empty?

Solution: We see in the function coalesce that the sublists are implemented as an array of lists named list: for (j = 0; j <= 9; j++) link the nodes in list[ j ] to the end of masterlist ; With this data structure, it’s impossible to keep track of the empty sublists.

The problem can be fixed at the price of using a more complicated data structure, e.g. adding to the array element list a boolean field that keep track of whether the list stored at that position is empty or not. The for loop in coalesce will now read: for (j = 0; j <= 9; j++) if (list[j].notEmpty) link the nodes in list[ j ].actualList to the end of masterlist ; The notEmpty field has to be always initialized with false, and it is turned to true in the function distribute: while (p != NULL) { j = value of ith digit (from the right) in p−> key; link p to the end of list[ j ].actualList; list[j].notEmpty = true; p = p−> link ; }

Additional Exercises

43) Write an algorithm that sorts a list of n elements in nonincreasing order by finding the largest and smallest elements and exchanges those elements with the elements in the first and last positions. Then the size of the list is reduced by 2, excluding the two elements that are already in the proper positions, and the process is repeated on the remaining part of the list until the entire list is sorted. Analyze your algorithm and show the results using order notation.

26

Ch7: Introduction to Computational Complexity: Sorting

Solutions

Solution: Finding the minimum and maximum elements of an unsorted array of n elements can be done in the worst case in 2(n-1) comparisons, by initializing a pair of variables max and min with a[1] and then comparing them in a loop with all the remaining n-1 elements.

The recursion is T(n) = 2(n-1) + T(n-2), easily solve by repeated substitution.

For even n, we have T(n) = 2(n-1) + 2(n-3) + … + 2(3) + T(2), and T(2) =1, because sorting two elements only requires one comparison. Using the summation techniques from Appendix A.3, we obtain T(n) (3/2)n2.

For odd n, we obtain similarly T(n) (3/2)n2, so for all n we have T(n) (3/2)n2), which is in (n2).

The algorithm sorts in place, needing only 2 extra variables = (1) extra space.

44) Implement the Quicksort algorithm using different strategies for choosing a pivot item, run it on your system, and study its best-case, average-case, and worst-case performances for different strategies using several problem instances.

Solution: Different ways to select the pivot:  First element (Program was written in Exercise 19): We have the worst case when the array is already sorted (either ascending or descending) → (n2). We say that the algorithm “goes quadratic”.  Last element: We also have the worst case above.  Median-of-three elements: Calculate the “middle value”, a.k.a. the median of the first, middle and last element of the partition. Use median as the pivot. This counters both the sorted-ascending and sorted-descending quadratic cases, but there are still rare cases of “natural” data on which Quicksort goes quadratic.  Random element: Randomly pick an element as a pivot. Although the worst-case is still (n2) as above, it is very unlikely to obtain it!  If we always picked a pivot value that exactly cuts the array in half, a.k.a. the array median, then, half of the values in the array are larger and half are smaller, making Quicksort optimal. Unfortunately, the only sure way to find the true median is to examine the entire array. There are tradeoff methods that estimate the median that are less expensive.

Ultimately, there is no foolproof way to avoid Quicksort going quadratic. In practice, we can simply detect the quadratic condition and switch to a different algorithm if this happens, as done in Introsort. According to Wikipedia, “[Introsort] begins with quicksort and switches to heapsort when the recursion depth exceeds a level based on (the logarithm of) the number of elements being sorted. This combines the good parts of both algorithms, with practical performance comparable to quicksort on typical data sets and worst-case O(nlog n) runtime due to the heapsort. “

27

Ch7: Introduction to Computational Complexity: Sorting

Solutions

45) Study the idea of designing a sorting algorithm based on a ternary heap. A ternary heap is like an ordinary heap except that each internal node has three children.

Solution: The number of levels d in a ternary heap is (log3n) = (lg n), although the multiplicative constant is smaller: d  (lg n)/(lg 3)  0.63lg n, so, for the same number of elements, a ternary heap is only about 63% as high as its binary counterpart.

Insertions in a ternary heap are done exactly as in the binary heap, they still require only one comparison per exchange (between child and parent), and so they are O(log3n). This still leaves the number of comparisons in makeheap linear, O(n), although the multiplicative constant is this time larger: 3 instead of 2.

Removing the root key is almost the same, except that determining the largest child for siftdown requires two comparisons instead of one, so for every exchange we need 3 comparisons instead of 2. We can obtain an upper bound similar to the one on p.314: 3nlog3n, which is also an upper bound for the entire sorting algorithm.

Based on these upper bounds, ternary heaps should be asymptotically (2nlog2n – 3nlog3n)/ 3nlog3n 0.0566, or 5.66% faster than binary heaps. Practical experiments with long arrays (n = 100,000 – 1,000,000) indicate an even larger speedup, in the range of 8-10%.

46) Suppose we are to find the k smallest elements in a list of n elements, and we are not interested in their relative order. Can a linear-time algorithm be found when k is a constant? Justify your answer.

Solution: This problem can be solved by running Exchange Sort or Selection Sort for k iterations. After k iterations, the list is guaranteed to contain the k smallest elements in the first k indices of the list. Each iteration takes O(n) time, and so the whole process takes O(kn) time, which is linear in n when k is a constant.

47) Suppose we have a very large list stored in external memory that needs to be sorted.

Assuming that this list is too large for internal memory, what major factor(s) should be considered in designing an external sorting algorithm?

Solution: The CPU can access in-RAM data very fast (e.g. tens of nanoseconds), but transferring data between RAM and the external memory (disk, tape) is about a million 28

Ch7: Introduction to Computational Complexity: Sorting

Solutions

times slower (e.g. tens of milliseconds or worse). For this reason, we should minimize the number of accesses (reads and writes) to the external memory. The general plan is to:  



Load from the external memory chunks of data that fit in the RAM. Sort each chunk in the RAM with an “internal” sorting algorithm that has “locality of reference”, herein abbreviated LoL. An algorithm having LoL uses only data items in the local chunk. o Mergesort has very good LoL, since it always uses only a well-defined half of the array. It is the main algorithm used in external sorting. o Heapsort, in contrast, has poor LoL, because the sifting requires exchanges between parent-child pairs that can be stored in vastly different positions in the physical array. o Quicksort also has good LoL: the function partition (algorithm 2.7) compares elements in linear, orderly fashion, by moving through the array, so, as long as the indices i and j are staying reasonably close to each other, they will remain in the same chunk.

Merge smaller chunks together in RAM (2, 3, or k at a time) and move the merged parts back to external memory. Here Quicksort has the upper hand: unlike Mergesort, merging the chunks is trivial!

48) Classify the sorting algorithms discussed in this chapter based on the ideas behind the algorithms. For example, Heapsort and Selection Sort find the largest (or smallest) key and exchange it with the last (or first) element according to the desired order.

Solution:   

Insertion Sort finds for each new element the final place among the already sorted ones, and inserts it there, after shifting other elements to make room for it.

Quicksort is more subtle, in that it does not require the other elements to be sorted, but still manages to find the final position of each new element (the pivot). The other important idea is divide-and-conquer.

Mergesort also uses divide-and-conquer, but without actually finding the final place of any element until the final merge. Unlike Quicksort, this relaxation “buys” the ability to always split the array into deterministic halves, which is important in parallel environments and for external sorting.

49) A stable sorting algorithm is one that preserves the original order of equal keys. Which of the sorting algorithms discussed in this chapter are stable? Which are unstable? Justify your answer.

29

Ch7: Introduction to Computational Complexity: Sorting

Solutions

Solution: Stable: 

Insertion Sort: In algorithm 7.1, if 42A comes before 42B in the original array, then 42A will be inserted first, and, when the turn comes for 42B, only elements strictly larger than 42 are shifted down, so 42B will end up under (after) 42A.

Unstable: 

  

Heapsort: Example: Consider the initial array [42A, 42B, 42C], in which the keys used for sorting are all 42, and the labels A, B, C will help us to distinguish them. The resulting heap has 42A in the root, with 42B as left child and 42C as right child, so they will be extracted in the reverse order [42A] -> [42C, 42A] -> [42B, 42C, 42A], so the initial order was not preserved.

Selection Sort: Example: Consider the initial array [42A, 42B, 1]; after the first iteration of the outer loop, the order is [1, 42B, 42A], therefore the relative order of 42A and 42B has changed.

Mergesort: The pseudocode from Algorithms 2.3 and 2.5 actually inverts the positions of identical elements.

Quicksort: Consider the initial array [10, 42A, 42B, 1]; during the first partition (we use Hoare’s partition from Exercise 23), 42A swaps with 1, so, after swapping the pivot 10 with 1, the order is [1, 10, 42B, 42A], therefore the relative order of 42A and 42B has changed.

50) Which of the sorting algorithms identified as unstable in Exercise 49 can easily be changed to stable sorting algorithms?

Solution: 

Mergesort: In the pseudocode from Algorithms 2.3 and 2.5, we replace one < with <= thus:

while (i<= h && j<= m) { if (U[ i ] <= V[ j ]) { S[k] = U[ i ]; i++; } else { S[k] = V[ j ]; j++; } k++; }

//this is the only change!

30

Ch7: Introduction to Computational Complexity: Sorting 

Solutions

Quicksort can be made stable if we insert the elements during the partitioning at the right end of the sublists, rather than swap them (the swap can change the relative order, as shown in the solution of Exercise 49). Insertions in an array are, however, too expensive, and they will ruin the (nlg n) complexity; the only way to preserve the low complexity is to have the data in a linked list. It can also be made “almost stable” if we are willing to spend more time choosing a better pivot.

31

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

Section 8.1 1) Let us assume that a search does not start at the beginning of a list when the Sequential Search algorithm (Algorithm 1.1) is used, but rather starts wherever the list index was left at the termination of the preceding search. Let us further assume that the item for which we are searching is selected randomly and independently of the destinations of previous searches. Under these assumptions, what would be the average number of comparisons?

Since the item is selected randomly, it has equal probabilities to be in any position. If it’s not found after the current element, the search will continue at the beginning of the array, but on the average n/2 elements will have to be searched, as in Algorithm 1.1.

2) Problem: Find all of the comment elements of two arrays S and T of sizes m and n respectively.

Inputs: positive integers m and n, arrays S[] and T[] indexed from 1 to m and 1 to n, respectively.

Outputs: Array U[] containing all of the common elements of S and T. keytype[] common(int m, int n, keytype S[], keytype T[]) { index i, j=1; keytype H[1…2*m]; keytype U[1…min(n,m)]; for(i=1; i<m; i++) { int hash = hashCode(S[i]) % (2*m); while(H[hash] != null) hash++; H[hash] = S[i]; } for(i=1; i<n; i++) { int hash = hashCode(T[i]) % (2*m); while(H[hash] != null) { if(H[hash]==T[i]) { U[j] = T[i]; j++; break;

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

} hash++; } //end while } //end for return U; } The algorithm places the elements of the first array into a hash table using linear probing. This takes (m) time in the average case for a suitable load factor (<0.5 is used in the algorithm). Then each element in the second array is searched for in the hash table.

This takes (n) time again in the average case for a reasonable size hash table. The total runtime is thus (m+n). It is assumed that a suitable, constant-time hash function hashCode() is available to generate a hash code for an element appropriate to the element’s keytype.

3) Improve the Binary Search algorithm (Algorithm 1.5) assuming a successful search.

Analyze your algorithm and show the results using order notation.

If we know in advance that the searched key will be found in the array, the test low <= high does not need to be performed, because by the time low becomes greater than mid, the key has been found. void binsearch (int n, const keytype S[] , keytype x, index& location) { index low , high , mid; low = 1; high = n; location = 0; while (low <= high && location = = 0) { mid = floor((low+high)/2); if (x = = S[mid]) location = mid; else if (x < S[mid]) high = mid − 1; else low = mid + 1; } }

//test skipped

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

Since the complexity is derived in the text by counting only comparisons that involve array elements, this change does not impact it at all.

4) Show that, if x is in the array and is equally probable to be in each of the array slots, the average-case time complexity for Binary Search (Algorithm 1.5) is bounded approximately by

Let k-1 be the deepest complete level in the binary tree. From Lemma 3, we know that the tree is nearly complete, so there can be only one more level after that: k. The number of nodes down to level k-1 is 2k -1, and, following the hint, the number of nodes at the bottom level, k, is n – (2k – 1).

The contributions to the TND are as follows:  

As in the proof on p.344, for the complete tree (down to level k-1) we have (k-1)2k + 1.

The bottom level is reached through k + 1 nodes starting from the root, so we have (k+1)( n – (2k – 1)).

Adding up the two terms above, we obtain the TND for the whole tree, and dividing by n we obtain the average: A(n) = TND/n = k + 1 –

2k 1 k  2  n n

Since the last term tends to 0 when n tends to infinity, we can neglect it: A(n)  k + 1 –

2 k 1 n

(*)

Since the bottom level can have from 0 to 2k – 1 nodes, the total number of nodes in the tree n is between 2 k – 1 and 2(2k – 1), so we have the double inequality 2k – 1 ≤ n ≤ 2k+1 – 1, which can be processed in two ways to apply to (*):  

Divide by n to obtain approximately Take log2 to obtain approximately

2k/n ≤ 1 ≤ 2k+1/n (**) k ≤ lg n < k + 1, which shows that

⎿lg n⏌= k By using (***) and right-hand-side of (**) in (*), we obtain A(n) ≤ ⎿lg n⏌

()

Similarly, using (***) and left-hand-side of (**) in (*), we obtain ⎿lg n⏌ – 1 ≤ A(n)

()

() and () are the two sides of the result we needed to prove.

(***)

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

5) Suppose that all of the following 2n+1 possibilities are equally probable:

Show that the average-case time complexity of the Binary Search algorithm (Algorithm 1.5) is bounded approximately by

Let k-1 be the deepest complete level in the binary tree. From Lemma 3, we know that the tree is nearly complete, so there can be only one more level after that: k. The number of nodes down to level k-1 is 2k -1, and, following the hint, the number of nodes at the bottom level, k, is n – (2k – 1). It is easy to see that the position of the nodes on the bottom level does not change the number of successful or unsuccessful searches on either level k-1 or k, so we can assume, for instance, an essentially complete binary tree (Fig. 7.4).

The contributions to the TND are as follows:  

For the complete tree (down to level k-1) we have (k-1)2k + 1 + k(n + 1 – [n – 2k + 1]).

For clarity, the square brackets above include the subtracted term that is not in the proof for the complete case on p.345.

The bottom level is reached through k + 1 nodes starting from the root, so we have (k+1)[n – 2k + 1]).

Adding up the two terms above, we obtain the TND for the whole tree, and dividing by 2n+1 we obtain the average: A(n) = TND/(2n+1) =

(k  1)n  k 2k  2k 1 k  2  2n  1 2n  1

Since the last term tends to 0 when n tends to infinity, we can neglect it. Then divide both numerator and denominator by n and neglect the vanishing terms. We have: A(n) 

k  1 k 2k 2k   2 2n n

(*)

Since the bottom level can have from 0 to 2k – 1 nodes, the total number of nodes in the tree n is between 2 k – 1 and 2k+1 – 1, so we have the double inequality 2k – 1 ≤ n ≤ 2k+1 – 1, which can be processed in two ways to apply to (*):  

Divide by n to obtain approximately Take log2 to obtain approximately ⎿lg n⏌= k

By using (***) in (*), we obtain

2k/n ≤ 1 ≤ 2k+1/n (**) k ≤ lg n < k + 1, which shows that (***)

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

lg n  1 lg n 2k 2k A(n)       2 2 n n and now application of (***) yields the result we needed to prove.

6) Complete the proof of Lemma 8.6. n

To prove the Lemma, it is left to show that

 c  minTND(n), where c is the number of i 1

i

i

nodes in the shortest path to a node that compares the search key x to the array item si. The reason we say “shortest path” is that there may be other comparisons of x to si in the tree, located at a larger depth, therefore when we add the length of the paths to those nodes, we n

find that

 c  TND(n), which, in turn, must be by definition ≥ minTND(n), and the i 1

i

inequality follows by transitivity.

Section 8.2 7) Implement the Binary Search, Interpolation Search, and Robust Interpolation Search algorithms on your system and study their best-case, average-case, and worst-case performances using several problem instances.

The following is an implementation of Interpolation Search: #include <iostream> #include <random> #include <ctime> using namespace std; #define N 30 int S[N] = { -420, -40, -38, -37, -36, -35, -34, -30, -29, -28, -10, -5, 1, 2, 3, 4, 5, 6, 100, 102, 110, 120, 130, 135, 136, 138, 140, 143, 150, 151}; int counter = 0; int interpsrch(int n, int S[], int x) { int low, mid, high; low = 0; high = n-1; while (low <= high) { counter++; mid = low + floor( (high-low) * (x-S[low]) / (float)(S[high]-S[low])

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

); if (x == S[mid]) return mid; if (x < S[mid]) high = mid - 1; else low = mid + 1; } return -1; //if x not found in array } int main() { srand(time(0)); int i = rand() % N;

//seed the random number generator //generate random index to search

cout <<"Searching for element at index: " <<i <<endl; cout <<"Nr. " <<S[i] <<" found at index " <<interpsrch(N, S, S[i]) <<endl; cout <<"nr. operations = " <<counter <<endl; return 0; }

8) Show that the average-case time complexity of Interpolation Search is in Θ(lg(lgn)), assuming the keys are uniformly distributed and that search key x is equally probable to be in each of the array slots.

Solutions will vary.

9) Show that the worst-case time complexity of Interpolation Search is in Θ((lgn)2), assuming the keys are uniformly distributed and that search key x is equally probable to be in each of the array slots.

Solutions will vary.

Section 8.3 10) Problem: Find the largest key in a binary search tree.

Inputs: Reference to a node n, representing the root of the tree.

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

Outputs: an integer m representing the maximum key of the tree. int maxKey(const node& n) { if(n.right == null) return n.key; return maxKey(n.right); } The algorithm traverses the rightmost branch of the tree. The every-case complexity is O(h), where h is the height of the tree. If there are no restrictions on the balance of the tree, this could have a worst-case complexity of O(n), where n is the number of nodes in the tree.

If the tree is restricted to be balanced, then the every-case complexity is (log n).

11) Theorem 8.3 states that, for a successful search, the average search time over all inputs containing n keys, using binary search trees, is in Θ(lg n). Show that this result still holds if we consider an unsuccessful search as well.

If key k is in the root, there are still k-1 keys in the left subtree and n-k in the right subtree, but now we also have k of the intervals for unsuccessful search on the left and n-k+1 on the right; the total cases on the left is 2k-1, and on the right 2n-2k+1.

The derivations now closely parallel those in Theorem 8.3, but with a slightly different recursion. The conditional average is A(n|k) = A(k-1)

2n  2k  1 2k  1 + A(n-k) +1 2n  1 2n  1

And therefore the entire average is

1 n 2n  2k  1 2k  1 A(n) =  ( A(k-1) + A(n-k) +1) n k 1 2n  1 2n  1 We denote A(m)∙(2m+1) by C(m), and, after manipulations, obtain

C (n) C (n  1) 4n  1   n 1 n n(n  1) We denote C(n)/(n+1) by an, and write

5 1 4n  1  to obtain the recursion = n(n  1) n  1 n

an = an-1 +

5 1  n 1 n

We “unroll” the recurrence by successive substitutions and use the logarithm formula from Example A.9 to get the closed-form expression an  4ln n – 5, which means C(n)  2.76(n+1)lg n, and A(n)  1.38lg n.

Chapter 8: More Computational Complexity: The Searching Problem

12) void delete(node root&, keytype k) { node current = root; node parent = root; boolean leftChild = true; //search for node to delete while(current.key != k) { parent = current; if(k < current.key) { leftChild = true; current = current.left; } else { leftChild = false; current = current.right; } if(current == null) return; //return if not found } //node to delete has no children if(current.left == null && current.right == null) { if(current == root) root = null; else if(leftChild) parent.left = null; else parent.right = null; } //node to delete has only a left child else if(current.right == null) { if(current == root) root = current.left; else if(leftChild) parent.left = current.left; else parent.right = current.left; } //node to delete has only a right child else if(current.left == null) { if(current == root) root = current.right;

Solutions

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

else if(leftChild) parent.left = current.right; else parent.right = current.right; } //node to delete has two children else { node successor = getSuccessor(current); if(current == root) root = successor; else if(leftChild) parent.left = successor; else parent.right = successor; successor.left = current.left; } return true; } node getSuccessor(node n&) { node sParent = n; node s = n; node current = n.right; while(current != null) { sParent = s; s = current; current = current.left; } if(s != n.right) { sParent.left = s.right; s.right = n.right; } return s; } The algorithm considers the four possible cases when deleting a node: the node has no children, the node has only a left child, the node has only a right child, and the node has two children. The algorithm traverses at most one branch of the tree for any given key. If the

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

tree is balanced, the complexity is thus O(log n). If there are no balance restrictions the complexity can degenerate to O(n) in the worst case.

13) Write an algorithm that creates a 3–2 tree from a list of keys. Analyze your algorithm and show the results using order notation.

The keys from the list L(1..n) are inserted successively in the tree T, starting with an empty tree: int main() { T = root containing L(1);

//insert first key

Insert L(2) in root, either left or right of L(1)

//insert second key

Insert L(3) in root, in one of 3 positions relative to L(1) and L(2);//start inserting third key //This is not a valid 2-3 tree!

Leave only middle key in root; Create 2 children with left and right keys;

//Finished inserting third key

for (i = 4; i <= n; i++) { w = find(T, L(i)); recursive_insert(T, w, L(i)); return 0; } void general_insert(T, w, x) { if (w has 2 children) {

//w contains only one key

Insert x as either w.left or w.right; Insert k as the appropriate child of w; } else if (w has 3 children) { Make x the appropriate new child of w;

//w is full: it contains two keys //w has 4 children!

Create a new internal node v.

Give w's two rightmost children to v; Place either child’s key in v. Let this key be z; recursive_insert(T, w’s parent, z); }

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

The function find(T, x) navigates the tree T from the root to the internal node under which the key x should be inserted.

In the worst case, the insertion of a key requires moving down lg n levels, then, recursively, up lg n levels, so the complexity is O(nlg n).

14) void inorder(node n) { if(n.left != null) inorder(n.left); print(n.leftKey); if(n.mid != null) inorder(node.mid); if(n.rightKey != null) print(n.rightKey); if(n.right != null) inorder(n.right); } } Assumes each node contains two key variables, leftKey and rightKey, and three child variables, left, mid, and right. If a node has only 1 key, then rightKey and mid are null.

15) If the three elements with the smallest hash values are inserted in the order 567-217100, 267-89-5401, 234-76-8100, then: Element 567-21-7100 is placed in bucket 0.

Element 267-89-5401 is placed in bucket 1.

Element 234-76-8100 is placed in bucket 2.

Element 330-38-0172 is placed in bucket 72.

Element 435-87-9872 is placed in bucket 73.

Element 897-38-6872 is placed in bucket 74.

Element 432-87-0199 is placed in bucket 99.

16) Open hashing incurs the overhead of maintaining a linked list structure at each nonempty spot in the hash array and traversing it each time a search is performed. However, its performance is not as seriously dependent on load factor as linear probing, and thus a smaller hash array size can generally be used than with linear probing, saving space. Linear probing is easier to implement since there is no overhead of maintaining lists, but a high load factor can seriously degrade performance, and so care must be taken to have a large enough hash array size, which takes up more memory.

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

17) void delete(keytype H[], keytype key) { int hash = hashCode(key); while(H[hash] != null) { if(H[hash]== key) { H[hash] = *special value*; return; } hash++; } } The algorithm takes the hash array in which the element resides and searches for the element in the array using linear probing. The hashCode() function is the same hash function that was used to insert the element into the array. If the element it found, the position in the array in which it is found is set to a special value that is known not to be a possible key. For example, if the array contains only non-negative integers, this value could be -1. In this way, the search function would have to be modified to skip over (but not terminate on) this value. Additionally, the insert function would insert a new element into a slot that was either null or contained this special value. Doing this, insertion, search, and deletion all take O(1) time assuming suitable load factors and hash functions as usual.

18) Let the second hash function be s(key) = (key/10000)%100. Assuming that the three elements with the smallest hash values are inserted in the order 567-21-7100, 267-895401, 234-76-8100, the elements would be mapped as follows: Element 567-21-7100 is placed in bucket 0.

Element 267-89-5401 is placed in bucket 1.

Element 234-76-8100 is placed in bucket 76.

Element 330-38-0172 is placed in bucket 72.

Element 435-87-9872 is placed in bucket 59.

Element 897-38-6872 is placed in bucket 10.

Element 432-87-0199 is placed in bucket 99.

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

Section 8.5 19) Modify Algorithm 8.4 (Find Smallest and Largest Keys by Pairing Keys) so that it works when n (the number of keys in the given array) is odd and show that its time complexity is given by 3n/2 −2 if n is even and 3n/2 – 3/2 if n is odd.

The only change for n odd is in the if statement before the for loop. Instead of if (S[1] < S[2]) { small = S[1]; large = S[2]; } else { small = S[2]; large = S[1]; } we “consume” only one array element, so the remainder is left even for the for loop: small = S[1]; large = S[1]; For even n we have 1 + 3(n/2 – 1) = 3n/2 – 2 comparisons.

For odd n, we have 0 + 3(n-1)/2 = 3n/2 – 3/2 comparisons.

20) Complete the proof of Theorem 8.8. That is, show that a deterministic algorithm that finds the smallest and largest of n keys only by comparisons of keys must in the worst case do at least (3n−3)/2 comparisons if n is odd.

If n is odd, let n = 2k + 1. The algorithm can make at most k = (n – 1)/2 “two unit” comparisons, which gain 2k = n – 1 units of information. This leaves (2n – 2) – (n – 1) = n – 1 units to be gained from “one unit” comparisons, which requires n – 1 comparisons. The total nr. of comparisons is (n – 1)/2 + n – 1 = (3n – 3)/2.

21) Write an algorithm for the method discussed in Section 8.5.3 for finding the secondlargest key in a given array.

We need two specialized structures for this algorithm:

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

struct list_element { struct array_element * array_ptr; struct list_element * list_ptr; }; struct array_element { int data; bool loser;

//true if node lost

struct list_element * list_ptr; }; The array is stored in struct array_element a[1..n]; whose loser fields are initially all set to false, and pointers are initially all set to NULL. The tournament rounds are implemented with a for loop: for (int i = 0; i < lg n; i++){ int j = 0; while (j < n){ choose next non-loser pair in a; compare pair; update loser field of the loser; append element in winner’s list with array+ptr = &loser; } } After the for loop, we find the sole winner (non-loser) left, and perform linear search in its list.

22) Show that for n in general, the total number of comparisons needed by the method discussed in Section 8.5.3 for finding the second-largest key in a given array is T (n) = n + lg n −2.

As indicated in the text, when n = 2k + m is not a power of 2, we pad the array with 2 k – m “minus infinity” elements in order for the tournament scheme to work, however we don’t need to physically compare any of the pad elements – the algorithm knows by their index if they’re pads or actual elements. When two pads need to be compared, any of them (say, the

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

first) can be declared the winner, and when a pad is compared with a real element, the real element is the winner. This way, the complexity of the tournament part is still n – 1 comparisons.

The nr. of levels in the tournament tree is lg n, so their sequential search requires lg n 1 comparisons.

The total is T (n) = n + lg n −2.

23) int medianOf5(int a[]) { if(a[1] > a[2]) swap(a, 1, 2); if(a[4] > a[5]) swap(a, 4, 5); if(a[1] > a[4]) swap(a, 1, 4); if(a[3] > a[2]) { if(a[2] < a[4]) return min(a[3], a[4]); else return min(a[2], a[5]); } else { if(a[3] > a[4]) return min(a[3], a[5]); else return min(a[2], a[4]); } } void swap(int a[], int x, int y) { int temp = a[x]; a[x] = a[y]; a[y] = temp; } Note that each call to min() counts as one comparison. Thus, each possible control execution path has exactly 6 comparisons.

24) Use induction to show that the worst-case time complexity of Algorithm 8.6 (Selection Using the Median) is bounded approximately as W (n) ≤ 22n.

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

We use the (approximate) recursion formula on p.384: W(n)  W(7n/10) + W(n/5) + 11n/5, valid for all positive integers n.

The base case is already provided in the text: W(n) ≤ 22n is true for n up to 5. (T(5) = 6 comparisons).

Induction hypothesis: W(k) ≤ 22k is true for all k up to n.

Induction step: We prove that W(k) ≤ 22k is true for n + 1. Indeed, W(n+1)  W(7(n+1)/10) + W((n+1)/5) + 11(n+1)/5, and, since 7(n+1)/10 < n, and (n+1)/5 < n, we can apply the hypothesis: W(n+1) ≤ 22∙7(n+1)/10 + 22∙(n+1)/5 + 11(n+1)/5 = 22n +22 ≤ 22n. Q.e.d.

25) Show that for an arbitrary m (group size), the recurrence for the worst-case time complexity of Algorithm 8.6 (Selection Using the Median) is given by . where a is a constant. This is Recurrence 8.2 in Section 8.5.4.

If we use groups of m instead of 5, the nr. of groups is n/m, and all but one of them have elements that could lie on either side of the median: n/m – 1 groups. In each of them, there are ⎿m/2⏌such elements, so instead of 2(n/5 – 1) we have ⎿m/2⏌∙(n/m – 1) elements that could lie on either side of the median (see p.381 of text).

The maximum nr. of elements on one side of the median of medians is

1 1 m n m n  m  n  1 m n ( n  1    (  1) ) +    1 = ( n  1    (  1) ) ≤ ( n  1   (  1) ) = 2 2  2  m  2 2 m 2 m 2 m 3n  m 1  .

4 2 As done in the text analysis, we drop the constant term, and keep only

3n  m .

4

The number of comparisons required when calling selection2 from partition2 is n/m.

The number of comparisons needed to find the medians in the groups of m and to partition the array is linear in n.

Conclusion: W(n)  W(

n 3n  m ) + W( ) + an.

4 m

Note: The text formula for Recurrence 8.2 is wrong; it should be changed to the one above.

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

26) Use induction to show that W (n) ∈ Ω(nlg n) for the following recurrence.

This is Recurrence 8.2 in Section 8.5.4 where m (group size) is 3.

For simplicity, we use log3 here, knowing that changing the base to 2 means only a constant multiplicative factor.

Base case: For n = 2 we have W(2) = W(1) + W(1) + 10/3 = 0 + 0 + 10/3 = 10/3, and nlg n= 2, so the maximum C that would satisfy 10/3 = W(2) ≥ C2log32 is Cmax = 5/3  2.64. (*) Induction hypothesis: W(k) ≥ Cklg k, for k from 2 to n.

Induction step: We prove that W(n+1) ≥ C(n+1)lg(n+1):  2(n  1)   n  1  5(n  1) W(n+1) = W  , and, because each of the arguments of W on the  W  3  3   3  right-hand-side are ≤ n, we can apply the hypothesis:

n 1 2n  2 n  1 5n  5 2n  2 log 3 log 3  +C , which after simple manipulations 3 3 3 3 3 2 log 3 2 5 1 becomes C(n+1)log3(n+1) + C(n+1)[ ], and the sum in the square brackets is 3 3C 2n  2 2n  2 log 3 negative for C ≥ 0.77 (**), making W(n+1) ≥ C .

3 3 W(n+1) ≥ C

From the inequalities (*) and (**) above, we conclude that the induction works with any C in the interval [0.77 … 2.64].

27) Show that the constant c in the inequality E(n,k) ≤ cn in the expected-value time complexity analysis of Algorithm 8.7 (Probabilistic Selection) cannot be less than 4.

We redo all the derivations following formula (8.3) on p.389 with an unknown constant C instead of 4. We obtain E(n, k) < C∙3n/4 + n – 1. In order for the induction step to work, we need this to be less than C∙n, which resolves to C > 4 – 1/n, and since 1/n tends to zero, this means C ≥

28) Implement Algorithms 8.5, 8.6, and 8.7 (Selection algorithms for finding the kthsmallest key in an array) on your system and study their best-case, average-case, and worst-case performances using several problem instances.

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

Implementations and performances will vary.

29) Write a probabilistic algorithm that determines whether an array of n elements has a majority element (the element that appears the most). Analyze your algorithm and show the results using order notation.

One solution is based on the observation that, if the majority element exists, it is also the median (if n = 2k+1 is odd), or immediately to the right of the median (k+1, if n = 2k is even). For simplicity, we assume n odd here: n = 2k+1, and we need the k+1-smallest key.

We then apply the probabilistic algorithm for the median (Algorithm 8.7) to find the median M, and, once we’ve found it, we make one more pass through the array to count the number of occurrences of M. If that number is ≥ k+1, M is the majority element; if not, there is no majority element.

The complexity is  4n + n = 5n, therefore linear, (n).

The simplest solution, however, is a “flow counting” algorithm invented in 1980 by Boyer and Moore. (It can be found in R. S. Boyer and J. S. Moore, MJRTY: A fast majority vote algorithm, in Automated Reasoning: Essays in Honor of Woody Bledsoe, 1991. ) //Boyer-Moore majority algorithm int MJRTY(int* S, int n) { int count_major = 0; int major; for (int i = 0; i < size; i++) { if (count_major == 0) major = arr[i]; if (arr[i] == major) count_major ++; else count_major --; } //end for (*) count_major = 0; for (i = 0; i < n; i++) if (arr[i] == major)

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

count_major ++; if (count_major > n/2) return major; return -1; //there’s no majority element } Note: If we know in advance that there is a majority element, the algorithm can stop at point (*), because major contains that majority element. If, however, there is no majority element, at point (*) major contains a spurious, “late majority” element of S, as can be seen in this example: S = {1, 1, 1, 2, 2, 2, 3} – here the three 1s and three 2s cancel each other and 3 gets stored in major.

In general, then, we need a second pass through the array to make sure that a majority element exists; this is done with the second for loop.

The every-time complexity is T(n) = 2n, so we don’t need a probabilistic version of it.

Additional Exercises 30) Suppose a very large sorted list is stored in external storage. Assuming that this list cannot be brought into internal memory, develop a searching algorithm that looks for a key in this list. What major factor(s) should be considered when an external search algorithm is developed? Define the major factor(s), analyze your algorithm, and show the results using order notation.

Since accessing the external storage is about a million times slower than accessing the RAM, we want to minimize the number of external accesses (only reads if our problem is just searching). The complexity in this case is determined by counting only the external read operations.  

If all we know about the data is that it is sorted, the best algorithm is binary search (BS), with deterministic complexity lg n.

If the data has even a modest degree of uniformity in distribution, interpolation search (IS) should be tried; although the index calculations are more complex than for BS, those calculations are very fast, since they are performed entirely in the RAM. True, the worst-case for IS is linear, but the average case is (lg lg n), which is a great improvement.

Other factors are the particular mechanism used to transfer data from external storage to RAM, and the size of the keys: All external storage devices (e.g. hard-disk drives – HDD) have a minimum block of information (e.g. 4KB or 16 KB) that is transferred in one access. If the keys are relatively

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

small compared to the block size, then many of them can fit in one block, and the access speed can be improved by using a sparse index. 



The first key in each block is stored in RAM, with a pointer to the physical external location; when a key k is searched, its block is found through a fast RAM search, and then the block is read in 1 operation, (1). Let m be the number of keys stored in one block. If n/m records can fit in the RAM, this method works.

If n/m keys are still too many for the RAM, a hierarchical index can be used: We build a B-tree of the keys (or just the first key in each block, as described above), but only the first few levels of the tree are stored in the RAM – the subtrees under the deepest RAM level are stored in external storage. When a key k is searched, its subtree is first determined through a fast RAM search, and then we perform a read to bring that subtree in the RAM. Now we search the subtree, and another read will bring the key in the RAM. The total of read operations is 2, still (1).

In real-life applications, the keys are only one part of a larger record, and the entire record must be retrieved, not just the key. If the rest of the record is large compared with the key, we can again use an index that stores only the keys, with pointers to the physical location of the record in external storage. Again, the index can be sparse, it can use B-trees, or hashing.

31) Discuss the advantages of using each of the following instead of the other: (a) A binary search tree with a balancing mechanism (b) A 3–2 tree For equal number of keys n, the height of a balanced BST is  log2n, whereas the height of a 3-2 tree is  log3n. Since log2n / log3n  1.44, the BST is about 44% deeper, resulting in 44% longer search time on average if the time to access a node dominates (e.g. nodes are in external storage). In practice, the branching factor of B-trees is a lot higher (100-99 trees are not uncommon), leading to much smaller depth than BSTs: log 2n / log100n  6.64.

Because 3 keys can fit inside a node instead of 1, 3-2 trees need to be rebalanced less often than BSTs. Consider, however, this example where both trees are entirely in the RAM: We have n = 220  1 million keys, and each key is stored in one word of memory. Pointers are also one word long.  

From the analysis of Algorithm 8.1, the average BST search examines 20 nodes, with 3 words per node (one key and 2 pointers), for a total of 60 words.

The algorithm and analysis of B-trees are not presented in our text, but it is known that a 3-2 tree is on the average 75% full, so if it stores n actual keys, there is room in its nodes for N = 222/3 keys. The average search examines log3N = log3(222/3) nodes, with 5 words per node (2 keys and 3 pointers) for a total of 5∙22∙log 3(2) – 5 = 64.4 words.

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

If the operations counted for complexity are word accesses in the RAM, the 3-2 tree has more complexity.

32) 1. Any situation in which the records need to be accessed in sorted order according to key: an example would be a student roster in which records are hashed by student last name that must be printed out in alphabetical order.

2. Any situation in which the data are likely to be or known to be non-uniformly distributed or in which there are many duplicate keys such that many keys will hash to the same value.

33) Let S and T be two arrays of n numbers that are already in nondecreasing order. Write an algorithm that finds the median of all 2n numbers whose time complexity is in Θ(lg n).

1. Find the medians of Sm of S and Tm of T. 2. If Sm == Tm, we found the overall median.

3. If Sm > Tm, the overall median is the median of the array composed of the left half of S and the right half of T. (recursive call) 4. If Sm < Tm, the overall median is the median of the array composed of the right half of S and the left half of T. (recursive call) Analysis: The algorithm is similar to the binary search, in that it approximately halves the number of elements to search at each step (divide-and-conquer), and therefore its complexity is lg(2n) = lg 2 + lg n (lg n).

34) Write a probabilistic algorithm that factors any integer using the functions prime and factor. Function prime is a boolean function that returns “true” if a given integer is a prime number and returns “false” if it is not. Function factor is a function that returns a nontrivial factor of a given composite integer. Analyze your algorithm, and show the results using order notation.

Finding a non-trivial factor of N can be done with a Monte Carlo algorithm called “Pollard’s Rho”: int pollard_rho_factor(long N) { int a, b, s; a = random(); b = a;

Chapter 8: More Computational Complexity: The Searching Problem a = f(a); b = f(f(b)); while (a != b) { a = a % N; b = b % N; s = gcd(a – b, N); if (s != 1 && s != N) return s; a = f(a); b = f(f(b)); }

Solutions

//greatest common divisor //this is a non-trivial factor! //update a //update b

} int f(x) { return x*x+1; }

//other functions can be used

Pollard, J. M. (1975), “A Monte Carlo method for factorization”, BIT Numerical Mathematics 15 (3): 331–334

Primality testing can be performed with the Miller-Rabin algorithm: bool MR_prime(long N, int nr_iter) { if ( N == 1 || (N!=2 && N%2==0) ) return false; if ( N == 2 || N == 3) return true; long d = N - 1; while(d%2 == 0) d/=2; for(int i=0; i<nr_iter; i++){ long a = rand() % (N-3) + 2; //pick a random number between 2 and N-2 long x = pow(a, s) % N; if (x == 1) continue; for (int i = 1; i <= s-1; i++){ x = x*x % N; if (x == 1) return false; if (n == N-1) continue; } return false; } return true; } Analysis:

Chapter 8: More Computational Complexity: The Searching Problem  



Solutions

Pollard’s algorithm has an average complexity A(n) O(n1/4), and it has to be called at most lg n times.

The Miller-Rabin algorithm decreases the probability that its argument be not prime by a factor of ¼ every time it iterates the for loop, so nr_iter is chosen such that (¼)nr_iter < p-dependent-threshold; in most applications, nr_iter = 20 is sufficient. Using efficient implementations for the power and modulus operations, the M-R algorithm has complexity O(nr_iter ∙ lg3N). It has to be called before each call to Pollard’s algorithm, i.e. also < lg n times.

The complexity of the entire algorithm is then O(n1/4lg n + nr_iter ∙ lg4n) = O(n1/4lg n).

35) List the advantages and disadvantages of all the searching algorithms discussed in this chapter. 







Binary Search (BS): o Advantage: Very fast, and worst-case practically the same as average case (lg n). o Disadvantages: Data must be stored in array (which is not conducive to fast insertions and deletions), and array must be sorted.

Interpolation Search (IS): o Advantage: Under the right conditions, it is even faster than BS! o Disadvantages:  As BS, data must be stored in array, and array must be sorted.  It has larger overhead than BS, due to more complex index calculations. It “goes logarithmic” or worse (worst-case is Q(n)) if distribution of keys is non-uniform. In experiments, it beats BS when the number of keys is very large (> 106) or the time to access the keys is large (external memory), and the distribution of the keys is close to uniform, but we are not aware of any useful real-life application that uses IS. B-Tree: o Advantages:  Appropriate for dynamic data, stored in a tree structure that permits fast insertions and deletions.  (n) on average, and in worst-case.  Compared with a binary tree, the larger index of branching allows for “flatter” trees, which is good for external searches. o Disadvantages:  If data is not stored in a tree structure already, time must be spent building one.  Compared with a binary tree, the larger index of branching makes nodes bulkier, so it’s slower if the tree is entirely stored in RAM. Hashing: o Advantages: Data need not be sorted! With a good hash function and few duplicates, it is the fastest search: Q(1)!.

Chapter 8: More Computational Complexity: The Searching Problem

Solutions

o Disadvantages: Deals poorly with many duplicate keys. Does not allow searches on partial keys. No easy way to traverse all keys in sorted order. Needs to allocate memory for non-existing keys (although this can be alleviated with more advanced algorithms).

36) For each of the searching algorithms discussed in this chapter, give at least two examples of situations in which the algorithm is the most appropriate. 







Binary Search: o Small databases that store data mostly in RAM use a version of balanced BST called T-Tree (e.g. MySQL, Oracle Times-Ten). o Network routers use binary search to locate entries in their routing tables.

Interpolation Search: o It’s a direct competitor to binary search. It beats binary search only in applications where the number of keys is very large (> 10 6) or the time to access the keys is large (external memory), and the distribution of the keys is close to uniform. Dictionary lookup by a human is an example (accessing a key takes  1 second!). o Versions of interpolation search exist that can work efficiently (i.e. (log log n) on average) with non-uniform distributions, if those distributions are known in advance.

B-Tree: o Most modern relational database systems use a version of B-Trees (B+ Trees) for table indices (e.g. IBM DB2, Informix, PostgreSQL, MySQL, Microsoft SQL Server, Oracle, Sybase ASE, SQLite), along with other indexing methods. A B+ Tree index is more efficient than a hash index when ranges of records are to be retrieved, or when searches are performed on leftmost key characters rather than entire keys. o Many file systems (e.g. NTFS, ReiserFS, NSS, XFS, JFS, ReFS) use B-trees for metadata indexing.

Hashing: o Most modern relational database systems use hashing for table indices, along with other indexing methods. A hash index is more efficient than a B-tree index when individual records are to be retrieved, and when the searches are performed on entire keys rather than partial keys. o The Linux Virtual File System (VFS) uses a hash table with overflow lists for its Information Nodes (inodes).

Chapter 9: Computational Complexity and Intractability

Solutions

Sections 9.1, 9.2, 9.3 1) List three problems that have polynomial-time algorithms. Justify your answer.

1. The Single Source Shortest Path problem; i.e., find the shortest path between a given vertex and each other vertex in a weighted digraph, which may contain negative weights.

There exists an O(V + E) algorithm that solves this problem (Bellman-Ford), where V is the number of vertices in the graph and E is the number of edges.

2. Standard linear programming problems: there exist generalized algorithms for solving them that have polynomial complexity (O(n4)).

3. Finding the greatest common divisor of two integers: the Euclidian algorithm for solving this problem is polynomial in the size of the smaller input number.

2) Give a problem and two encoding schemes for its input. Express its performance using your encoding schemes.

Consider the sequence alignment algorithm (Algorithm 3.12) from Section 3.7:

The integers i, j, m, n are always represented on 32 bits, but we have two encoding schemes for the base characters (A, C, G, T):

Chapter 9: Computational Complexity and Intractability

Solutions

1. 8-bit ASCII characters 2. 2-bit custom representation, with 4 bases stored in one Byte for storage efficiency Because of the dynamic programming algorithm, the number of values of opt to be computed is bounded by m∙n, and the most expensive integer manipulation performed for each of them is the assignment on the last line: opt = min(opt(i + 1, j + 1) + penalty , opt(i + 1, j) + 2, opt(i , j + 1) + 2); Denote by c the number of assembly instructions required to perform the operation above and all the integer operations (e.g. the test i == m), plus the stack operations required by the three recursive calls. The total number of steps is bounded by the expression N2(c + b), Where N = max(n,m), and b is the nr. of steps required for the test x[i] == y[j] in the third if statement.  For the encoding scheme #1, a commercial CPU will probably use two LOAD instructions to bring the array elements from RAM into CPU registers, followed by a BRE (branch if equal) or BNE (branch if not equal), for a total of 3 assembly instructions.  For the encoding scheme #2, the CPU will additionally have to apply bit-level masks to the Bytes loaded, in order to extract the appropriate 2-bit representations of the bases; assuming for simplicity that the 4 possible masks are precomputed and stored in registers, this means two extra AND instructions, for a total of 5 assembly instructions.

In conclusion, encoding scheme #1 has a maximum nr. of N2(c + 3) steps, and encoding scheme #2 has a maximum nr. of N2(c + 5) steps; they are both O(N2).

3) 3. Show that a graph problem using the number of vertices as the measure of the size of an instance is polynomially equivalent to one using the number of edges as the measure of the size of an instance.

For a connected graph, the lower bound on the number of edges E in terms of the number of vertices V is V-1. The upper bound is V2, a fully-connected graph. Thus, the number of edges is a polynomial function of the number of vertices, and the two input representations are polynomially equivalent.

Chapter 9: Computational Complexity and Intractability

Solutions

4) In which of the three general categories discussed in Section 9.3 does the problem of computing the nth Fibonacci term belong? Justify your answer.

Computing the nth Fibonacci number is in P, since for any n, one only needs the values of the previous two numbers in the sequence. Thus, an iterative algorithm that only stores two numbers and computes the next number at each step (beginning with 0 and 1) can compute the nth number in n steps. Since the complexity of the problem is based on the input size, the bound on such an algorithm is O(n log n), since it takes log n bits to store a number of size n and log n times steps to add two numbers of size n. The space complexity is thus O(log n).

5) A graph has an Euler Circuit if and only if (a) the graph is connected and (b) the degree of every vertex is even. Find a lower bound for the time complexity of all algorithms that determine if a graph has an Euler Circuit. In which of the three general categories discussed in Section 9.3 does this problem belong? Justify your answer.

Checking to see if a graph is connected can be performed in O(V+E) time using breadth-first search, where V is the number of vertices and E is the number of edges. Additionally, during this process the degree of each vertex can be recorded to verify whether it is even or not. Since each vertex must be visited and its degree determined by examining each of its outgoing edges, this constitutes a lower bound on the problem of determining whether a given graph has an Euler circuit. The problem is thus in P. 6) List at least two problems that belong in each of the three general categories discussed in Section 9.3.   

Polynomial-time algorithms have been found: (n2) algorithm for sequence alignment, (nlg n) for the fractional knapsack.

Proven intractable: generating all permutations od a string of n characters (unreasonable), Presburger Arithmetic (reasonable), Halting Problem (undecidable).

Not proven intractable, and no polynomial algorithm found: finding longest path in a graph, 0-1 Knapsack where W cannot be used as input size parameter, deciding if a number n is prime.

Chapter 9: Computational Complexity and Intractability

Solutions

Section 9.4 7) Implement the verification algorithm for the Traveling Salesperson Decision problem discussed in Section 9.4.1 on your system, and study its polynomial time performance. #include <iostream> #include <time.h> using namespace std; #define n 15 int W[n+1][n+1] = { { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}, { 1, 0, 29, 82, 46, 0, 0, 0, 0, 51, 55, 29, 74, 23, 72, 0}, { 2, 29, 0, 0, 0, 0, 0, 43, 23, 23, 31, 0, 51, 11, 52, 21}, { 3, 82, 0, 0, 0, 46, 55, 23, 43, 41, 29, 79, 21, 64, 0, 51}, { 4, 46, 46, 68, 0, 82, 15, 72, 31, 0, 0, 21, 51, 0, 0, 64}, { 5, 0, 0, 46, 82, 0, 0, 23, 52, 21, 46, 82, 58, 46, 65, 23}, { 6, 52, 43, 55, 15, 74, 0, 0, 0, 0, 0, 33, 37, 51, 29, 59}, { 7, 72, 43, 23, 72, 23, 61, 0, 42, 23, 31, 77, 37, 51, 46, 33}, { 8, 42, 23, 43, 31, 52, 23, 42, 0, 33, 0, 0, 0, 33, 0, 37}, { 9, 51, 23, 41, 62, 21, 55, 0, 0, 0, 29, 62, 46, 29, 51, 11}, {10, 55, 0, 29, 42, 46, 31, 31, 15, 29, 0, 51, 21, 41, 23, 37}, {11, 29, 41, 79, 21, 82, 33, 77, 0, 0, 51, 0, 65, 42, 0, 0}, {12, 0, 0, 0, 0, 58, 37, 0, 33, 46, 21, 65, 0, 61, 0, 0}, {13, 23, 11, 64, 0, 0, 0, 0, 0, 29, 41, 42, 61, 0, 62, 23}, {14, 2, 0, 0, 0, 65, 29, 46, 31, 51, 23, 59, 11, 62, 0, 0}, {15, 46, 0, 51, 0, 23, 59, 0, 0, 11, 0, 61, 0, 23, 59, 0}}; int S[n+2] = {0, 1, 2, 9, 4, 5, 7, 6, 3, 8, 15, 11, 12, 13, 10, 14, 1}; //claimed tour #define D 500 //weight of tour not greater than this bool visited(int key, int arr[n+1]){ bool found = false; int i = 1; while (i<=n && !found) if (key == arr[i]) found = true; else i++; return found; } bool verify(int w[][n+1], int d, int s[]){ int sum = 0; int i = 1; bool OK_tour = true; int v[n+2] = {0}; //keeps track of nodes visited while (i<=n && OK_tour){ if (w[s[i]][s[i+1]] <= 0){ OK_tour = false; cout <<"Not a tour!"; cout <<"\tnode " <<s[i] <<" does not connect to node "; cout <<s[i+1] <<endl; } else if (visited(s[i+1], v)){ OK_tour = false; cout <<"Not a tour!"; cout <<"\tnode " <<s[i+1] <<" is a cycle" <<endl; } else{

Chapter 9: Computational Complexity and Intractability

Solutions

sum += w[s[i]][s[i+1]]; if (sum > d){ OK_tour = false; cout <<"Weight too great: " <<sum <<endl; } else{ i++; v[i] = s[i]; cout <<"node " <<s[i] <<" OK "; cout <<"sum = " <<sum <<endl; } } } return OK_tour; } int main(){ time_t begin, end; begin = time(NULL); if (verify(W, D, S)){ cout <<"\nThis is a tour, and its weight is "; cout <<"no greater than " <<D <<endl; } end = time(NULL); cout << "\ttime = " << difftime(end, begin) << " s" << endl; return 0; }

Below are outputs obtained for d = 500, 600, and 700, respectively:

Chapter 9: Computational Complexity and Intractability

Solutions

8) Establish that the problems in Examples 9.2 to 9.5 are in NP. 9.2 Traveling Salesperson: In the program from Exercise 7 above, function verify has a loop that executes n times in the worst case, calling each time the function visited, which also has a loop that executes n times in the worst case, therefore the verification algorithm is in O(n2).

9.3 0-1 Knapsack: Similar to the TP algorithm, this verification algorithm adds at most n weights, checking each time if the current item has not been already used, and checking at the end if the sum is ≥ P, therefore O(n2).

9.4 Graph-Coloring: In the worst case, the color of every vertex has to be compared with that of all the n-1 remaining vertices, therefore the verification algorithm is in O(n2).

9.5 Clique: To verify a clique of size k, the algorithm has to check if each of the k vertices has edges to each of the remaining k-1, therefore O(k2), and, since k ≤ n, also in O(n2).

9) A verification algorithm for the Clique Decision problem takes as input a graph G = (V,E), a subset of vertices U, and an integer k. The algorithm must return true if W is of size k and constitutes a clique, and return false or not halt otherwise. boolean verifyClique(int W[][], int U[], int k) { if(U.size != k) return false; index i, j; for(i = 1; i < k; i++) for(j = 1; j < k; j++) if(W[U[i]][U[j]] == infinity) return false; return true; } The algorithm checks to see that the size of the input clique U is equal to k, and then checks each element of U to see that it is adjacent to each other element in U. If this is the case, the algorithm returns true; otherwise, it returns false. The algorithm has quadratic time complexity in the size of the clique U (O(U2)), which is a subset of the input graph, and is thus polynomial in the size of the input.

10) Show that the reduction of the CNF-Satisfiability problem to the Clique Decision problem can be done in polynomial time.

Here is the relevant fragment from Example 9.9:

Chapter 9: Computational Complexity and Intractability

Solutions

“Let B = C1 ∧ C2 ∧ ··· ∧ Ck be a logical expression in CNF, where each Ci is a clause of B, and let x1,x2,...,xn be the variables in B. Transform B to a graph G = (V,E), as follows: V = {(y,i) such that y is a literal in clause Ci} E = {((y,i),(z,j)) such that i = j and z = y}.”

Since there are n literals, setting up the vertices in V takes c 1n steps, where c1 is a constant.

There are at most n(n-1)/2 edges, and each takes a constant nr. of steps c2 to set up.

The reduction algorithm therefore takes c 1n + c2n(n-1)/2 ≤ 2cn2 steps, where c = max(c1, c2/2), so it is in O(n2).

11) Write a polynomial-time verification algorithm for the Hamiltonian Circuits Decision problem.

A verification algorithm for the Hamiltonian Circuits Decision problem takes as input a digraph G = (V,E), and a path P. The algorithm must return true if P begins and ends with the same vertex and visits each vertex exactly once, and return false or not halt otherwise. boolean verifyHam(int n, int W[][], int P[]) { if(P.size != n+1) return false; if(P[1] != P[n+1]) return false; index i; for(i = 1; i <= n; i++) if(W[i][i+1] == infinity) return false; return true; } The algorithm checks to see that the path is of appropriate length and that it begins and ends with the same vertex. It then goes through the vertices of the path and checks to see that the sequence of vertices is a valid path. The first two checks are constant time operations, and the third is linear in the length of the input path P, which has an upper bound equal to the number of vertices in the graph (O(V)). The algorithm is thus polynomial in the size of the input.

Chapter 9: Computational Complexity and Intractability

Solutions

12) Show that the reduction of the Hamiltonian Circuits Decision problem to the Traveling Salesperson (Undirected) Decision problem can be done in polynomial time.

Reducing the Hamiltonian Circuits Decision problem to the Traveling Salesman (Undirected) Decision problem involves creating a graph with the same number of vertices, V, as the input to the Hamiltonian Circuits problem, and adding an edge of appropriate weight (1 if the edge is in the original graph, and 2 otherwise) between every pair of vertices . The number of edges in a complete graph with V vertices is given by V 2, which is greater than or equal to the number of edges, E, in the original graph. Thus the complexity of the transformation is O(V2), which is polynomial in the size of the input.

13) Show that the reduction of the Traveling Salesperson (Undirected) Decision problem to the Traveling Salesperson Decision problem can be done in polynomial time.

Let V be the number of vertices in the graph. The number of edges in the undirected graph is ≤ V∙(V-1)/2 < V2/2, so the number of edges to be created in the directed graph is < V2, thus the complexity of the transformation is O(V2), which is polynomial in the size of the input.

14) Show that a problem is NP-easy if and only if it [Turing-] reduces to an NP-complete problem.

Remark: From the remark right after the definition of NP-easy (p.425), it’s clear that Turing-reduction is meant.

We prove the double implication: i] Problem A is NP-easy ⇨ A Turing-reduces to B (NP-complete) Proof: A  NP-easy ⇨ A T B NP, but, according to the definition of NP-complete, any NP problem, including B, reduces to any NP-complete problem, for example CNF: B T CNF. Since Turing-reduction implied many-to-one reduction, we have that B T CNF, so by transitivity A T CNF NP-complete. ii] Problem A Turing-reduces to B (NP-complete) ⇨ A is NP-easy Proof: By definition, any NP-complete problem is also NP, so B is NP, so A Turing-reduces to B  NP.

Chapter 9: Computational Complexity and Intractability

Solutions

15) Suppose that problem A and problem B are two different decision problems.

Furthermore, assume that problem A is polynomial-time many-one reducible to problem B. If problem A is NP-complete, is problem B NP-complete? Justify your answer.

The definition of NP-completeness (p.415) also requires for B to be in NP, which does not follow from our hypothesis. In fact, any decision problem A (in P or NP) can be trivially reduced to the halting problem by appending a while (true) loop right after the instruction that made the result true (or false), so B can be the halting problem, which is not in NP.

16) When all instances of the CNF-Satisfiability problem have exactly three literals per clause, it is called the 3-Satisfiability problem. Knowing that the 3-Satisfiability problem is NP-complete, show that the Graph 3-Coloring problem is also NP-complete.

Let’s call the three colors T (True), F (False) and B (base). We describe how to construct a graph G(B) that is 3-Col if and only if the problem B = C1∧C2∧···∧Ck is 3-Sat:   



Create a “triangle” graph with 3 vertices T, F, and B, fully connected.

Create pairs of vertices for each literal and its inverted version (e.g. x 1 and x1’), and connect all of them to the B vertex.

Connect together the vertices in each pair; this ensures that, in each pair, one vertex is T and the other F. The graph so far is shown below:

For each clause Ci in B, we add to the graph above a group of 6 nodes, known in the literature as a “gadget”. As an example, we shall assume that we have C 1 = x1’  x2’  x3.

The corresponding gadget is shown below in the dashed box:

Chapter 9: Computational Complexity and Intractability

 

Solutions

It is left to the reader to prove that the graph is 3-Colorable if and only if at least one literal in the clause is true.

The gadget construction is repeated for every other clause in B.

17) Show that if a problem is not in NP, it is not NP-easy. Therefore, Presburger Arithmetic and the Halting problem are not NP-easy.

Reasoning by contradiction, let’s assume that the problem A is not in NP, but it is NP-easy.

This means that A has no polynomial-time verification algorithm, but it can be Turingreduced to a decision problem B that does have a polynomial-time verification algorithm, i.e. to an easier problem – contradiction.

Section 9.5 18) Implement the approximation algorithms for the Traveling Salesperson problem, run them on your system, and study their performances using several problem instances.

Implementations and performances will vary.

Chapter 9: Computational Complexity and Intractability

Solutions

19) Write a detailed algorithm of the approximation algorithm for the Bin-Packing problem given in Section 9.5.2, and show that its time complexity is in Θ(n2). int binPack(int n, number S[]) { //assumes S is sorted in nonincreasing order number B[1…n]; //initialized to zeroes index i, j, k = 1; for(i=1; i <= n; i++) { boolean packed = false; for(j=1; j <= k; j++) if(1-B[j] >= S[i]) { B[j] = S[i]; packed = true; } if(!packed) { k++; B[k] = S[i]; } } return k; } For each item, the algorithm searches through the bins that have already been opened to find a bin into which to place the item. In the worst case, each item must be placed in a separate bin, and thus at each iteration all of the open bins must be checked before opening a new one. The time complexity of this worst-case scenario is thus equal to the sum of the integers from 1 to n, which is n(n+1)/2; therefore, the algorithm has (n2) complexity.

20) For the Sum-of-Subsets problem discussed in Chapter 5, can you develop an approximation algorithm that runs in polynomial time?

As explained in Section 5.4, the Sum-of-Subsets is a particular case of the Knapsack problem, when we just want to find a subset of objects whose total weight equals the capacity W (irrespective of their value, or assuming they all have the same value per unit of weight). There, the problem was solved with backtracking, but here we use a different idea:

Chapter 9: Computational Complexity and Intractability  

  

Solutions

Let’s start with a list containing just the number zero: L0 = [0].

We define the following operation to be applied to a list L: f(L) is another list, obtained by: o Taking all the distinct sums between an element of L and an element of the array of weights w: L[i] + w[j]  Note that there can be length(L)∙length(w) = m∙n such sums in the worst case. o Taking the union between L and the sums above  The union can contain (m+1)n numbers in the worst case o Sorting the new list o Dropping the sums that are > W.  There can still be (m+1)n numbers in the worst case If we apply the operation f to L0 n times, we obtain a sequence of lists L1, …, Ln, The last of them, Ln, contains all the candidates o There can be (nn) numbers in Ln in the worst case (we neglected the “+1”) The max(Ln) is the solution.

The problem with the algorithm above is that it is worse than exponential in n, however, we can do better if all we need is an approximation: Instead of keeping all the sums < W at each step, we only keep those that are “about” a small constant  from each other. One way to accomplish this is to split the interval [0..W] into bins of size  and only retain one number in each bin. This guarantees that the least at each step has no more than W/ elements, avoiding the combinatorial explosion.

It can be easily proved that, by doing this, the error we make only increases by at most  at each step, so it is bounded by n for the entire algorithm. If we have a target error E for the entire algorithm, we choose  = E/n, so the length of the list is at most Wn/E at each step, so the additions and sorting take polynomial time at each step, therefore polynomial time for the n iterations.

21) Can you develop an approximation algorithm for the CNF-Satisfiability problem by stating it as an optimization problem—that is, by finding a truth assignment of the literals in the expression that makes the maximum possible number of clauses true?

Let’s call S1 the assignment of True values to all the literals, and S2 the assignment of False values to all literals. We then choose between S1 and S2 the one that makes the most clauses True. In the worst case, no clauses have mixed inverted and non-inverted literals (i.e. in a clause the literals are either all inverted or all non-inverted), and therefore the numbers of True clauses in S1and S2 add up to n. This proves that the algorithm makes at least 50% of the clauses true.

Remark: A better polynomial algorithm exists (Karloff-Zwick), that satisfies at least 7/8 of the clauses.

Chapter 9: Computational Complexity and Intractability

Solutions

Additional Exercises 22) Can an algorithm be a polynomial-time algorithm for a problem using one encoding scheme and be an exponential-time algorithm for the same problem using another encoding scheme? Justify your answer.

Yes. Consider the case of an O(n) algorithm in which the input is represented in binary. The same algorithm given the same input in unary would be O(2n), since a number n in binary takes 2n symbols to write in unary.

23) Write a verification algorithm for the composite number problem (Example 9.14) and analyze it to show that it is a polynomial-time algorithm.

The verification algorithm is simply the long division algorithm from grade school. If we divide a/b, its complexity is bounded by n (nr. of digits of a) times m (nr. of digits of b), O(nm): there are at most n-m steps (round it up to n, assuming n >>m), and at each step we have to multiply the m-digit number b by the current digit qi found in the quotient, then subtract bqi, which is O(m).

24) boolean composite(int n) { int i = 2; while(i <= floor(n0.5)) { if(n % i == 0) return true; i++; } return false; } The algorithm goes through at most floor(n0.5) iterations of the while loop, and the modulus operation takes O(log2n) time (division of two numbers each with log n bits).

The algorithm is thus O(n0.5 log2n), a polynomial algorithm.

Chapter 9: Computational Complexity and Intractability

Solutions

25) Is the Towers of Hanoi problem an NP-complete problem? Is it an NP-easy problem? Is it an NP-hard problem? Is it an NP-equivalent problem? Justify your answers. This problem is presented in Exercise 17 in Chapter 2.

It is not NP, because the solution has 2n – 1 moves, and in the worst case the verification algorithm has to examine all of them.

It is not NP-easy (because NP-easy requires NP), and therefore not NP-equivalent.

It is (trivially) NP-hard, because the definition of NP-hard is “if a polynomial algorithm did exist …”. Since we know that a polynomial algorithm does not exist for ToH, the hypothesis is false, and therefore it can imply anything.

26) Given a list of n positive integers, divide the list into two sublists such that the absolute value of difference between the sums of the integers in the two sublists is minimized. Is this problem an NP-complete problem? Is this problem an NP-hard problem?

This is known as the Partition Problem. As a decision problem (call it A), it requires to find out if the list can be partitioned such that the difference is 0, and it has been proved to be NP-complete, however, the problem as stated in the Exercise (call it B) is not a decision problem, and therefore cannot be NP. If we had a polynomial-time solution to B, A could be solved easily (in (1) steps) by simply examining the solution and returning True if the difference is zero and False if it is not. Therefore the NP-complete problem A is Turing-reducible to B, so B is NP-hard.

A polynomial-time approximation algorithm exists, similar to the nonincreasing first fit presented in the text for the bin packing problem: Iterate through the numbers in nonincreasing order, assigning then to the set currently having the smaller sum. The complexity is (nlg n) due to sorting, and it can be proved that the result is at the most 4/3 of the optimal solution.

Chapter 10: Genetic Algorithms

Solutions

Section 10.1 1) Describe the difference between sexual reproduction in diploid organisms, binary fission in haploid organisms, and fusion in haploid organisms.

In diploid organisms, each parent produces a haploid gamete, and the two gametes unite to produce a diploid zygote, which then grows into an adult. The zygote has half of the genome from each parent.

Some haploid organisms (unicellular) reproduce through fusion: two parent cells combine to produce a transient meiocyte (diploid), which then undergoes meiosis, producing four haploid descendant cells. Each descendant has a mixture of genes from each parent, which makes this type of reproduction sexual.

Most haploid organisms reproduce asexually, through fusion: one parent cell splits into two copies with identical genomes.

2) Suppose a diploid organism has 10 chromosomes in one of its genomes. (a) How many chromosomes are in each of its somatic cells?

Somatic cells are diploid, therefore they have two genomes, or 20 chromosomes. (b) How many chromosomes are in each of its gametes?

Gametes are haploid, therefore they have only one genome, or 10 chromosomes.

3. Suppose two adult haploid organisms reproduce by fusion. (a) How many children will be produced?

Four. (b) Will the genetic content of the children all be the same?

No, in general each of the four children is genetically distinct (each inherits differnet combinations of genes from each parent).

Chapter 10: Genetic Algorithms

Solutions

4. Consider the eye color of a human being as determined by the bey2 gene. Recall that the allele for brown eyes is dominant. For each of the following parent allele combinations, determine the eye color of the individual.

Father

Mother

Child

BLUE

BLUE

BLUE

BLUE

BROWN

BROWN

BROWN

BLUE

BROWN

BROWN

BROWN

BROWN

Section 10.2 5) Consider Table 10.1. Suppose the fitnesses of the 8 individuals are .61, .23, .85, .11, .27, .36, .55, and .44. Compute the normed fitnesses and the cumulative normed fitnesses.

Fitness

Normed fitness

Cumulative normed fitness

.61

.178

.178

.23

.067

.245

.85

.249

.494

.11

.032

.526

.27

.079

.605

.36

.105

.710

.55

.161

.871

.44

.129

1.000

Sum 3.42

Chapter 10: Genetic Algorithms

Solutions

6) Suppose we perform basic crossover as illustrated in Table 10.3, the parents are 01101110 and 11010101, and the starting and ending points for crossover are 3 and 7.

Show the two children produced.

We assume that the points for crossover are numbered from left to right, from 1 to 8.

The children are:

0101 0100 1110 1111

7) Implement the genetic algorithm for finding the value of x that maximizes f(x) = sin(xπ/256), which is discussed in Section 10.2.2.

Implementations will vary, however, in C/C++ efficient mutation and crossover operations should take advantage of the bitwise operators. Below is a short example showing how the bit in position x can be swapped (crossover) between two 8-bit numbers a and b. The bit positions are 0 on the right and 7 on the left.

To swap an entire crossover sequence, a mask with consecutive ones should be constructed first, and then the process is similar.

Chapter 10: Genetic Algorithms

Solutions

8) Consider the instance of the TSP in Figure 10.12. Assume the weights in both directions on an edge are the same. Find the shortest tour.

We use the implementation from Exercise 6-10, with a 5-by-5 matrix: #include <iostream> using namespace std; #define n 5 int W[n+1][n+1] = { {0, 0, 0, 0, 0, 0}, {0, 0, 4, 5, 2, 0}, {0, 4, 0, 1, 3, 7}, {0, 5, 1, 0, 6, 2}, {0, 2, 3, 6, 0, 3}, {0, 0, 7, 2, 3, 0} }; int best_vindex[n] = {-1}; int vindex[n] = {-1}; int minimum = 1000; bool promising(int i){ int j; bool switchie; if (i==n-1 && !W[vindex[n-1]][vindex[0]]) //No edge to close circuit switchie = false; else if (i>0 && !W[vindex[i-1]][vindex[i]]) //No edge to current node switchie = false; else{ //Check if vertex has already been selected switchie = true; j = 1; while (j<i && switchie){ if (vindex[i] == vindex[j]) switchie = false; j++; } } return switchie; } int evaluate(int vindex[]){ int sum = 0; for (int i=0; i<n-1 ;i++) sum += W[vindex[i]][vindex[i+1]]; sum += W[vindex[n-1]][vindex[0]]; return sum; } void hamiltonian(int i) { int j; if(promising(i)) { if (i == n-1){ int eva = evaluate(vindex); if (eva <= minimum) {

//closing the circuit

Chapter 10: Genetic Algorithms

Solutions minimum = eva; cout <<"Found new minimum tour = " <<minimum <<" : "; for (int k=0; k<n; k++){ cout << vindex[k] <<" "; best_vindex[k] = vindex[k]; //update min. circuit } cout <<endl;

} } else for(j=2; j<=n; j++) { vindex[i+1] = j; hamiltonian(i+1); } } } int main(){ vindex[0] = 1; hamiltonian(0); return 0; }

As expected, there are two minimal tours, one being the reverse of the other, since all edges are bi-directional (undirected graph).

9) Suppose we perform order crossover, the parents are 3 5 2 1 4 6 8 7 9 and 5 3 2 6 9 1 8 7 4, and the starting and ending points for the pick are 4 and 7. Show the two children produced.

We assume that the points for crossover are numbered from left to right, from 1 to 8.

The children are:

975146832 473691852

The picks are shaded.

10) Consider the instance of the TSP in Figure 10.12. Apply the Nearest Neighbor Algorithm starting with each of the vertices. Do any of them yield the shortest tour?

The only two starting vertices that yield tours are v2 and v4: v2 → v3 → v5 → v4 → v1 → v2

Length is 12

v4 → v1 → v2 → v3 → v5 → v4

Length is 12

They are both the same tour (after a circular permutation), namely the first shortest tour found in Exercise 8.

Chapter 10: Genetic Algorithms

Solutions

11) Form the union graph of the two tours shown in Figure 10.13, and apply the Nearest Neighbor Algorithm to the resultant graph starting at vertex v5.

The union graph is:

1

1

2 7

6

4 5

3

2

4

2

5

8

3

5

6

3

4

The sequence of nodes starting at v5 is: v5 → v6 → v2 → v4 → v3 → STOP, since there is no edge to the remaining node v1.

If, however, we start at v1, we obtain the tour v1 → v2 → v4 → v3 → v5 → v6 → v1, of cost 22.

12) Apply the Greedy Edge Algorithm to the instance of the TSP in Figure 10.12. Does it yield a shortest tour?

The first three shortest edges are 2-3 (cost 1), 1-4 (cost 2), and 3-5 (cost 2). They can all be added without violating the conditions for a tour. The next two edges are 2-4 and 4-5, both of cost 3, but they cannot be both added, because this would create a cycle composed of only the nodes 2 → 3→ 5 → 4. If we try to add 2-4, no tour can be found, but if we add 4-5, we obtain a tour after the addition of the next edge, 1-2 (cost 4).

The final tour is 1 → 2 → 3 → 5 → 4 → 1, of cost 12.

Section 10.3 13) Consider the two trees in Figure 10.8. Show the new trees that would be obtained if we exchanged the subtree starting with the “4” in the left tree with the subtree starting with the “+” in the right tree. /

+ ‒

+

x

/

+

2

5

3

*

x

3

x

*

4

x

2

Chapter 10: Genetic Algorithms

Solutions

14) Consider the individual (program) in Figure 10.10. Show the moves produced by that program when negotiating the Santa Fe trail for the first 10 time steps.

Step nr.

Actions

End position and orientation

1

Move (and eat)

(1,2)→

2

Move (and eat)

(1,3)→

3

Move (and eat)

(1,4)→

4

L, R, R, L, R, move (and eat), move (and eat)

(3,4)↓

5

Move (and eat)

(4,4)↓

6

Move (and eat)

(5,4)↓

7

Move (and eat)

(6,4)↓

8

L, move (and eat), R, L, R, L, move (and eat)

(6,6)→

9

Move (and eat)

(6,7)→

10

L, R, R, L, R, L, move (not eat)

(6,8)→

15) Implement the genetic programming algorithm for the Santa Fe trail discussed in Section 10.3.2.

Implementations will vary.

Chapter 11: Number-Theoretic Algorithms

Solutions

Section 11.1 1) Find the positive divisors of the following integers. a) 72 : 1, 2, 3, 4, 6, 9, 8, 24, 36, and 72. b) 31 : 1 and 31 c) 123 : 1, 3, 41, and 123

2) Prove that if h|m and m|n, then h|n. h|m means that ah = m, and m|n means bm = n, therefore (ab)h = b(ah) = bm = n, so h|n.

3) Show that two positive integers divide each other if and only if they are equal.

If m and n are positive integers, m|n means am = n, and n|m means bn = m, with and b positive integers. am = n implies that m ≤ n, and bn = m implies that n ≤ m, therefore n = m, q.e.d.

4) Let p and q be two prime numbers. If p = q + 2, then p and q are called “twin prime numbers.” Find two pairs of twin prime numbers.

The first pairs of twin primes are (3, 5), (5, 7), (11, 13), (17, 19).

5) Prove that gcd(n,m) = gcd(m,n).

The definition of gcd does not rely in any way on the order of the numbers: gcd(n,m) is the largest integer that divides both of them.

Chapter 11: Number-Theoretic Algorithms

Solutions

6) Prove that if m and n are both even, then gcd (m,n) = 2 gcd (m/2,n/2).

Denote d = gcd (m,n) and d’ = gcd (m/2,n/2).

We have that d'|m/2 and d’|n/2, so 2d’|m and 2d’|n, so 2d’|d.

(*)

On the other hand, 2|m and 2|n, so 2/d, so d = 2a. Since d = 2a divides 2*m/2, a divides m/2, and similarly a divides n/2, so a divides d’, so d = 2a divides 2d’ (**) From (*) and (**) we have that d and d’ are equal in absolute value. Considering in turn all possibilities for the signs of m and n, we easily prove that d and d’ are actually equal.

7) Prove that if n ≥ m > 0, then gcd (m,n) = gcd (m,n−m).

Denote d = gcd (m,n), which means d|m and d|n, which means ad = m and bd = n, therefore (b-a)d = n-m, so d|(n-m). From d|m and d|(n-m), we have that d|gcd (m,n-m). Denoting d’ = gcd (m,n-m), this is rewritten d|d’. (*) On the other hand, a’d’ = m, and b’d’ = n-m, so (a’+b’)d’ = m+n-m = n, so d’|n. Therefore d’|gcd (m,n) = d. (**) From (*) and (**), we have that |d| = |d’|, and from the fact that n ≥ m > 0, we have d = d’.

8) Prove that if p is a prime number and 0<h<p, then gcd(p,h) = 1.

Proof by contradiction: assume that gcd(p,h) = d > 1. Since d|h, d≤h<p, so d<p. But also d|p, which contradicts the primality of p.

9) Use Corollary 11.2 to show that the prime factorization of an integer, as discussed in Theorem 11.5, is unique.

Let n = p1k1…piki = q1h1…qjkj be two factorizations of n. For them to be different means that: A. a factor p is not on the right-hand side OR B. a factor q is not on the left-hand side OR C. the factors themselves are identical, but at least one of the powers is different.

Chapter 11: Number-Theoretic Algorithms

Solutions

Cases A and B are easily dismissed using the Corollary: If, for instance, p 1 is not found among the q, the Corollary implies that, nevertheless, p 1 divides q1h1 or … or qjkj. Let’s choose for example that p1|q1h1; the same corollary now implies that p1|q1, which is a contradiction.

We are left then with case C: n = p1k1…piki = p1h1…pihi.

Let’s say that k 1 < h1; we simplify by dividing left and right by p1k1, and we are left with p1h1k1 on the right-hand side, whereas the left-has side has no factors of p1. From the Collorary, this is absurd.

10) Write each of the following integers as a product of prime numbers. a) 123 = 31411 b) 375 = 3153 c) 927 = 32103

11) Prove Theorem 11.6.

Let d = gcd(n,m) and let d’ be the product of primes in the statement of Th.11.6. Since the power of each prime in d’ is the smaller of the two, that power divides both n and m, so d’|n and d’|m, therefore d’|gcd(n,m) = d. (*) Conversely, d cannot have any extra factors beyond those in d’; assuming by contradiction that it did have an extra factor q, it wouldn’t divide either n or m (or both of them), therefore d|d’. (**) From (*) and (**) it follows that |d| = |d’|, and an examination of all the possible sign combinations of n and m shows that actually d = d’.

Chapter 11: Number-Theoretic Algorithms

Solutions

12) Prove Theorem 11.7.

The correct statement of Th. 11.7 is: The lcm(n,m) is a product of all the primes that are in either n or m, where the power of each prime in the product is the larger of its powers in n and m.

Proof: Let M = lcm(n,m) and let M’ be the product of primes in the statement of Th.11.7.

Since the power of each prime in M’ is the larger of the two, M’ has all the factors of n, and all the factors of m, hence it is a multiple of n and a multiple of m, therefore it is a common multiple, so M|M’.

On the other hand, we cannot take away any of the factors p in M’ and still have it be a common multiple, since then M’ would not be a multiple of the number n or m in which the largest power of p occurs. Therefore, M’ is the least common multiple.

13) Prove that for positive integers m and n, gcd(m,n) = lcm(m,n) iff m =n.

We use the characterizations of gcd and lcm from Theorems 11.6 and 11.7. Gcd and lcm are equal if and only if they have the same factors, raised to the same powers. From Th.11.7, this can happen iff that there are no uncommon prime factors for n and m. From both Th. 11.6 and 11.7, the smallest and largest power for each prime factor are equal, so the powers are identical.

Section 11.2 14) Illustrate the flow of Algorithm 11.1 when the top-level call is gcd (68, 40).

68 % 40 = 28 40 % 28 = 12 28 % 12 = 4 12 % 4 = 0

⇨

gcd = 4

Chapter 11: Number-Theoretic Algorithms

Solutions

15) Write an iterative version of Algorithm 11.1. Your algorithm should only use a constant amount of memory [i.e., the space complexity function is inθ(1)]. int gcd(int n, int m) { while (m>0) { int q = n mod m; n = m; m = q; } return n; }

16) Write an algorithm that uses Algorithm 11.1 to express a rational number in its lowest terms. You may assume that this rational number is given in the form of a fraction m/n where m and n are integers.

We apply Algorithm 11.1 to find the gcd, and then divide both m and n by the gcd.

17) Illustrate the flow of Algorithm 11.2 when the top-level call is Euclid(64,40,gcd,i,j).

We show the results in a table similar to Table 11.1: Call

n

m

gcd

i

j

0

64

40

8

2

-3

1

40

24

8

-1

2

2

24

16

8

1

-1

3

16

8

8

0

1

4

8

0

8

1

0

Indeed, 8 = 2∙64 – 3∙40.

Chapter 11: Number-Theoretic Algorithms

Solutions

18) Write an algorithm that uses subtraction to compute the greatest common divisor. (See Exercise 7.) Analyze your algorithm.

In Exercise 7, it was proved that, if n ≥ m > 0, then gcd (m,n) = gcd (m,n−m). We use this property in the following algorithm: int gcd_by_sub(int n, int m) { if (m > n) swap(m, n); if (m == 0) return n; return gcd_by_sub(n-m, m); } Analysis: The worst case is when m = 1 and n is a large number, when the number of recursive calls is n. In each call, the algorithm performs one subtraction, which needs lg n bit manipulations, so our algorithm is O(n lg n).

Section 11.3 19) Show that (S,∗) of Example 11.21 is a group.

Proving associativity: x*(y*z) = (x*y)*z. Since this group has only 3 elements, we consider all 3*3*3 = 27 possible cases.

For example, take x = a, y = a, z = b: We must prove that a*(a*b) = (a*a)*b ⇔ a*e = b*b ⇔ a = a, which is true. All remaining 26 cases are solved similarly, by applying the rules given.

Proving the existence of identity element: From the fourth and fifth properties, e is the identity.

Proving the existence of an inverse for each element: From the first property, a and b are each other’s inverse. From the sixth property, e is its own inverse.

20) Prove Theorem 11.12.

Chapter 11: Number-Theoretic Algorithms

Solutions

Proving implication “⇨” By definition, m ≡ k mod n iff n|(m-k) iff an integer a exists such that an = m – k , or m = an + k. (*) (m mod n) is an integer between 0 and n-1 such that m = bn + (m mod n). We plug in m from (*) and we get an + k = bn + (m mod n) ⇔ k = (b-a)n + (m mod n) (**) (k mod n) is an integer between 0 and n-1 such that k = cn + (k mod n). Comparing with (**) and using the uniqueness of the integer division, we have that b – a = c, and (m mod n) = (k mod n).

Proving implication “⇦”.

Denote (m mod n) = (k mod n) = R, with R between 0 and n-1.

We have m = bn + R and k = cn + R, so m – k = (b-c)n, therefore n | (m-k).

21) The following was left as an exercise in the proof of Theorem 11.13. Show that there exists an integer c such that h1 = c∙n2n3 ···nj.

In the proof of Th. 11.13, we have h 1n1 =h2n2 = ··· = hjnj. Let’s focus on h 1n1 =h2n2 , which implies that n2|h1n1. Since n2 and n1 are relatively prime, Th. 11.4 implies that n2 | h1. In the same manner, we can show that any n i | h1. Since they’re all pairwise relatively prime, each ni is a separate factor in h1, so h1 = c∙n2n3 ···nj.

22) Prove Theorem 11.14.

Reflexivity:

m ≡ m mod n is true, because m – m = 0, so n|0 (any integer divides 0).

Symmetry:

Also true, because is n|(n-k), then n also divides the negative (k-n).

Transitivity: If n|(m-k), an = m-k. If n|(k-j), bn = k-j. Adding the identities, we have (a+b)n = m-k+k-j = m – j, so n|(m-j).

23) Show that if s ∈ [m]n and t ∈ [k]n, then s×t ∈ [m×k]n. s – m = an, and t – k = bn ⇨ s∙t = (m + an)(k + bn) = m∙k + n(ak + bm + abn), which shown that s∙t ≡ m∙k mod n.

Chapter 11: Number-Theoretic Algorithms

Solutions

24) Show that if G=(S,∗) is a finite group and a ∈S, then there exists integers k, m ≥ 1 such that ak =akam.

Consider the sequence of powers of a: {a, a2, a3, …, an, …} Since S is finite, let k and k+m the first pair of duplicates in the sequence.

25) Show that if S={[0]12, [3]12, [6]12, [9]12}, then (S,+) is a subgroup of (Z 12,+).

Associativity follows from the associativity of integer addition.

The null element is [0]12.

The inverse of [3] is [9], and the inverse of [6] is itself.

The fact that S is closed under addition is easily verified, e.g. [9] + [6] = [9+6] = [15] = [12+3] = [3].

26) Use Theorem 11.19 to prove Corollary 11.3.

From the theorem, |S’| divides |S|, so |S| / |S’| must equal a positive integer a. a cannot be 1, because S’ is a proper subgroup, so a ≥ 2, so |S| / |S’| ≥ 2, or |S| / 2 ≥ |S’|.

27) Consider the group (Z9*,×). Show that <[2]9> = Z 9*.

Z9* is made up of the classes of integers that are relatively prime with 9. As shown in Example 11.31, these are [1], [2], [4], [5], [7], [8].

The powers of [2] are: [2], [2 2] = [4], [23] = [8], [24] = [16] = [9+7] = [7], [25] = [32] = [27+5] = [5], and [26] = [64] = [63+1] = [1]. From [27], the elements repeat themselves. Therefore we have obtained exactly the set of classes in Z 9*.

Chapter 11: Number-Theoretic Algorithms

Solutions

Section 11.4 28) Solve the following modular equations. (a) [8]10 x = [4]10 Gcd(8,10) is 2, and 2|4, so, according to Corollary 11.6, the equation has 2 solutions. We can find them by simply multiplying the class [8]: 2[8] = [16] = [6] 3[8] = [24] = [20+4] = [4], so 3 is solution 4[8] = [32] = [30+2] = 2,

5[8] = [40] = [0],

6[8] = [48] = [8],

7[8] = [56] = [6]

8[8] = [64] = [60+4] = [4], so 8 is solution The reader is encouraged to also find the solutions using Algorithm 11.3. (b) [4]17 x = [5]17 Gcd(4,17) = 1, so the equation has a unique solution. We find it by applying Algorithm 11.3: We apply the extended Euclid’s algorithm Euclid(n,m,d,i,j) to the integers 4 and 17: Call

n

m

gcd

i

j

0

17

4

1

1

-4

1

4

1

1

0

1

2

1

0

1

1

0

Check: 1 = 1∙17 – 4∙4.

According to Th. 11.24 / Alg. 11.3, the solution is [jk/d] n = [-4∙5/1]17 = [-20]17 = [-3]17 = [14]17. Check: [14∙4]17 = [56]17 = [3∙17+5]17 = [5]17.

29) Implement Algorithm 11.3 and run it on various problem instances.

Implementations will vary.

Chapter 11: Number-Theoretic Algorithms

Solutions

30) Find all solutions to the equations: [1]7 x = [3]7 Gcd(1,7) = 1, so there is a unique solution. It is obvious that the solution is x = [3] 7.

[12]9 x = [6]9 Gcd(12,9) = 3, so there are 3 solutions. First we find the integers i and j using Euclid’s algorithm: Call

n

m

gcd

i

j

0

12

9

3

1

-1

1

9

3

3

0

1

2

3

0

3

1

0

Check: 3 = 1∙12 - 1∙9.

According to Th. 11.24 / Alg. 11.3, the solutions are:   

[-1∙12/3 + 0∙9/3]9 = [-4 + 0]9 = [-4]9 = [5]9 . Check: [12∙5]9 = [60]9 = [9∙ 6+6]9 = [6]9.

[-1∙12/3 + 1∙9/3]9 = [-4 + 3]9 = [-1]9 = [8]9 . Check: [12∙8]9 = [96]9 = [9∙10+6]9 = [6]9.

[-1∙12/3 + 2∙9/3]9 = [-4 + 6]9 = [ 2]9 .

Check: [12∙2]9 = [24]9 = [9∙ 2+6]9 = [6]9.

Section 11.5 31) Compute ([3]73)12 by raising 3 to the 12th power. ([3]73)12 = [1]73.

32) Compute ([7]73)15 by raising 7 to the 15th power. ([7]73)15 = [22]73.

Chapter 11: Number-Theoretic Algorithms

Solutions

33) Use Algorithm 11.4 to compute ([3] 73)12.

12 is 1100 in binary. We use a table similar to Table 11.2: i

3

2

1

0

bi

1

1

0

0

ki

1

3

6

12

a

[3]73

[9∙3]73 = [27]73

[72]73

[1]73

34) Use Algorithm 11.4 to compute ([7] 73)15.

15 is 1111 in binary. We use a table similar to Table 11.2: i

3

2

1

0

bi

1

1

1

1

ki

1

3

6

12

a

[7]73

[343]73 = [51]73

[30]73

[22]73

35) Implement Algorithm 11.4, and run it on various problem instances.

Implementations will vary.

Section 11.6 36) Find the number of prime numbers that are less than or equal to 100.

There are 25 primes less than or equal to 100: 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, and 97.

Chapter 11: Number-Theoretic Algorithms

Solutions

37) If an integer between 1 and 10,000 is randomly chosen according to the uniform distribution, approximately what is the probability of it being prime?

From Theorem 11.27, there are approximately 10000/ln 10000  1085 primes less than or equal to 10000. Randomly choosing an integer according to the uniform distribution gives P(prime) = 1/ln 10 000  0.1085.

38) Suppose we randomly choose 100 numbers between 1 and 10,000 according to the uniform distribution. Approximately, what is the probability of all of them not being prime?

We shall also assume that the numbers are chosen independently of each other, so each has an independent probability of being prime or not prime. p = P(prime) was calculated in the exercise above to be  0.1085, so P(not prime) is the complementary probability q = 1 – p  0.8914. The probability of all of them not being prime is q 100  10-5.

39) Show that if n is prime and 1<k ≤n−1, then B(n,k) ≡ 0 mod n, where B(n,k) denotes the binomial coefficient.

B(n,k) ≡ 0 mod n means that B(n,k) is divisible by n. By definition, B(n,k) = n! / (k!(n-k)!) = n∙[(n-1)! / (k!(n-k)!)]. Since n is prime, the number in brackets is an integer, so B(n,k) = n∙a, therefore divisible by n.

40) Show that if q is a factor of n and k is the order of q in n, then qk | B(n,q), where B(n,q) denotes the binomial coefficient.

B(n,q) = n! / (q!(n-q)!), and, after simplifying (n-q)!, we obtain

(𝑛−𝑞)(𝑛−𝑞+1)…(𝑛−1)𝑛 1∙2∙3∙…∙(𝑞−1)∙𝑞

.

Writing n = a∙qk, with k > 0, we can factor out q in the first and last factor of the numerator: 𝑞(𝑎𝑞 𝑘−1 −1)(𝑛−𝑞+1)…(𝑛−1)𝑎𝑞 𝑘 1∙2∙3∙…∙(𝑞−1)∙𝑞

=

(𝑎𝑞 𝑘−1 −1)(𝑛−𝑞+1)…(𝑛−1)𝑎𝑞 𝑘

. Since q is prime, none of the

1∙2∙3∙…∙(𝑞−1) (𝑎𝑞 𝑘−1 −1)(𝑛−𝑞+1)…(𝑛−1)𝑎

factors in the denominator can divide qk, so we write integer, and B(n,q) = bqk, which proves the result.

1∙2∙3∙…∙(𝑞−1)

= b is an

Chapter 11: Number-Theoretic Algorithms

Solutions

41) Are 9x3+2x and x2−4 congruent modulo 2?

The coefficients of x3 in the two polynomials are 9 and 0, which are not congruent modulo 2 (9-0 is not divisible by 2), therefore the polynomials aren’t either.

42) Show that (x−9) 4 is not congruent to (x4−9) modulo 4. (x-9)4 = ((x-9)2)2 = (x2 – 18x +81)2 = x4 – 36x 3 + 486x2 – 2916x + 812 486 is not congruent with 0 modulo 4, because 486 is not divisible by 4.

43) Show that (x−5) 3 is congruent to (x3 −5) modulo 3.

This must be so by Theorem 11.28, since 3 is prime.

We can also prove it directly, by expanding (x−5) 3 = x 3 – 15x2 + 75x – 125, whose coefficients are all congruent with the respective coefficients in (x 3 −5) modulo 3. For example, - 15 = (-5)3 ≡ 0 mod 3.

44) Prove Theorem 11.29.

A useful re-statement of this congruence modulo (x r – 1, n) is to say that (x – m)n – (x n – m) = nP(x) + (xr – 1)Q(x),

(*)

where P and Q are polynomials in x. From Lemma 11.2, we have that (x – m)n – (xn – m) = nP(x), so in (*) we simply take Q(x) = 0.

45) Implement Algorithm 11.5 and run it on different problem instances.

Implementations will vary.

Chapter 11: Number-Theoretic Algorithms

Solutions

46) Use Lemma 11.3 to prove Lemma 11.4.

A useful re-statement of this congruence modulo (x r – 1, n) is to say that [g(x)]n – g(xn) = nP(x) + (xr – 1)Q(x),

(*)

where P and Q are polynomials in x. From Lemma 11.3, we have that [g(x)] n – g(xn) = nP(x), so in (*) we simply take Q(x) = 0.

47) The following was left as an exercise in the proof of Lemma 11.6. Show ordr(n) | lcm[ordr(p1), ordr(p2), ..., ordr(pk)].

For simplicity, we first give the proof for only two prime factors p 1 and p2, both having powers equal to 1, i.e. n = p1p2 : To make the notation simpler, denote a = ordr(p1), and b = ordr(p2). We want to prove ordr(p1p2) | lcm(a,b).

Lcm has the property of being divisible by either argument: a | lcm(a,b), and b | lcm(a,b), therefore p1lcm(a,b) ≡ 1 mod r, and p2lcm(a,b) ≡ 1 mod r. When we multiply these congruences, we obtain (p1p2)lcm(a,b) ≡ 1 mod r. (*) At this stage, we need a Lemma: If nx ≡ 1 mod r, then x is a multiple of ordr(n).

Proof of lemma: We apply the division algorithm to x: x = y∙ord r(n) + z, with 0≤z<ordr(n).

Therefore nx = n y∙ordr(n) + z = [nordr(n)]y ∙nz = [1]y∙nz = nz ≡ 1 mod r. But since 0≤z<ordr(n), the congruence is possible iff z = 0, iff x = y∙ord r(n), which proves the lemma ∎ Apply the lemma to (*) to obtain that ordr(p1p2) | lcm(a,b).

The next step is to deal with arbitrary powers of the factors p 1 and p2. This is achieved by means of another Lemma: ordr(p1)k1 =

𝑜𝑟𝑑𝑟 𝑝1 gcd⁡(𝑘1 ,⁡𝑜𝑟𝑑𝑟𝑝1 )

Proof ??

The next step is to apply induction on the number of factors k.

Proof ??

Chapter 11: Number-Theoretic Algorithms

Solutions

48) Use Inequality 11.22 to obtain the inequality that follows it.

Since the numerators of the right-hand sides are the same in Inequality 11.22 and the next, we only have to prove that the second denominator is larger than the first: 7(lg(lg n)) ≥ lg(c2(lg n)6) ⇔

⇔

lg(lg n)7 ≥ lg(c2(lg n)6)

⇔

(lg n)7 ≥ c2(lg n)6

lg n ≥ c2, which is true if we choose lg N ≥ c 2

Section 11.7 49) What is the difference between a public key and a secret key?

The public key of a recipient is known to everyone, but the secret key of the recipient is only known to her/himself. Nevertheless, the recipient can use the secret key to decipher any message that was encrypted with their public key.

50) For an RSA cryptosystem using p = 7, q = 11, and g = 13: a) n = 7*11 = 77. b) (n) = 6*10 = 60. c) h = 37.

51) Given that p = 23, q = 41, and g = 3, then n = 943, (n) = 880, and h = 587.

The message [847]943 is encrypted as ([847]943)3 = [741]943.

52) Using the cryptosystem in Exercise 51, the cipher [741]943 is decrypted as ([741] 943)587 = [847]943.

Chapter 11: Number-Theoretic Algorithms

Solutions

53) In an RSA cryptosystem, show that if ϕ(n) can be discovered, then the cryptosystem may be compromised.

The number n is already public, as part of the public key, so, if ϕ(n) is also known, we have a system of 2 equations with the two unknowns p and q: pq = n (p-1)(q-1) = ϕ(n) Simple algebraic manipulations lead to the solutions p and q being the two roots of the quadratic equation x 2 – (n+1- ϕ(n))x + n = 0, which is easily solved.

Additional Exercises 54) Prove that there are infinitely many prime numbers.

Suppose that there are a finite number, n, of primes denoted p 1, p2, …, pn.

Let q = p1p2p3…pn + 1. Then q, which is not prime by our initial assumption, must contain a prime factor in p1 through pn. However, if we divide q by any of p1 through pn, we obtain a remainder of 1, which is a contradiction. Therefore, there must be infinitely many primes.

55) Show that the gcd operator is associative. That is, for all integers m, n, and h, we have gcd (m, gcd(n,h)) = gcd (gcd(m,n),h).

The simplest proof uses the characterization of gcd in terms of the prime factors (Th. 11.2 in the text): gcd (m, gcd(n,h)) has all the prime factors common to m and gcd(n,h), with their lowest powers, and gcd(n,h) has all the prime factors common to n and h, with their lowest powers. Since “common to” is an associative relation, it follows that gcd (m, gcd(n,h)) has all the prime factors common to m, n, and h, with their lowest powers.

The second part of the last statement does not depend on the order of operation, so it’s clear that gcd (gcd(m,n),h) can be brought to the same form.

Chapter 11: Number-Theoretic Algorithms

Solutions

56) Prove that if m is odd and n is even, then gcd (m, n) = gcd (m,n/2).

We use the characterization of gcd in terms of the prime factors (Th. 11.2 in the text): Since m is odd, m does not have 2 as a prime factor, therefore gcd(m,n) does not have it either, therefore it’s OK to remove not just one, but all factors of 2 from n: gcd (m, n) = gcd (m,n/2k), where k is the highest power of 2 that divides n.

57) Prove that if m and n are both odd, then gcd (m, n) = gcd ((m−n)/2,n).

We already know from Exercise 7 that gcd (m, n) = gcd (m−n,n). Since m and n are both odd, m – n is even, so we can apply Exercise 56.

58) Find the necessary condition to have equation mx ≡ my mod n imply x≡y mod n. mx ≡ my mod n ⇔ (mx – my) = an ⇔ m(x – y) = an . When does the last equality imply that x≡y mod n ⇔ (x – y) = bn?

We substitute (x – y) into the first one: mbn = an, which, for non-zero n, implies mb = a, i.e. m is a factor of a.

59) Assuming that p is a prime number, find the solutions of the equation x 2 =[1]p. x2 =[1]p means x 2 – 1 = kp, or (x-1)(x+1) = kp, therefore p divides x – 1 or p divides x + 1. In the first case, x = ap + 1, and in the second, x = ap – 1. Using classes modulo p, the first solution is x = [1]p, and the second is x = [-1]p = [p-1]p.

60) In an RSA cryptosystem, let p and q be the large primes, let n = pq, and let pub be the public key. Show that pub(a)pub(b) is congruent to pub(ab) modulo n. pub(a) = [ag]n, and pub(b) = [bg]n, so pub(a)pub(b) = (ag + in)(bg + jn) = (ab)g + kn ≡ (ab)g mod n.

Chapter 12: Introduction to Parallel Algorithms

Solutions

Section 12.1 1) If we assume that one person can add two numbers in ta time, how long will it take that person to add two n×n matrices, if we consider the operation of addition as the basic operation? Justify your answer.

It would take one person ta*n2 time to add two n  n matrices, since the addition of a pair of elements would have to be done once for each position in the resulting matrix sequentially.

2) If we have two people add numbers and it takes ta time for one person to add two numbers, how long will the two people take to add two n×n matrices, if we consider the operation of addition as the basic operation? Justify your answer.

It would take two people (ta*n 2)/2 time to add two n  n matrices. It would take half the amount of time as it would for one person, since none of the values either person computes depends on the values the other person computes.

3) Consider the problem of adding two n×n matrices. If it takes ta time for one person to add two numbers, how many people do we need to minimize the total time spent to get the final answer? What will be the minimum amount of time needed to find the answer, if we assume that we have enough people? Justify your answers.

Having n2 people adding two n  n matrices would minimize the time for performing the addition. With n2 people, it would take just one time step (ta time), since none of the values are dependent on the results of others. More people would not be needed since they could not reduce the time any further.

4) Assuming that one person can add two numbers in ta time, how long will it take that person to add all n numbers of a list, if we consider the operation of addition as the basic operation? Justify your answer.

It would take one person ta*(n-1) time to add all the numbers in a list of size n, since after the addition of the first two elements they would have to add the current value of the running sum to the next element in the list sequentially.

Chapter 12: Introduction to Parallel Algorithms

Solutions

5) If we have two people add n numbers in a list and it takes ta time for one person to add two numbers, how long will it take the two people to add all n numbers in the list, if we consider the operation of addition as the basic operation and include tp time for passing the result of an addition from one person to the other? Justify your answer.

It would take two people ta*(n/2) time to add all the numbers in a list of size n, since each person must add up half the list in the same manner as Exercise 4 (n/2 – 1 additions), and then one of them must make the final addition of both of their results at the end.

6) Consider the problem of adding n numbers in a list. If it takes ta time for one person to add two numbers and it takes no time to pass the result of an addition from one person to another, how many people do we need to minimize the total time spent to get the final answer? What will be the minimum amount of time needed to find the answer, if we assume we have enough people? Justify your answer.

Having n/2 people adding a list of n numbers would minimize the time for performing the addition. With n/2 people, it would take ta*log n time since the sum of each pair of numbers added by one person in the first iteration could be added to the result of another person’s calculation by each of half as many people in the second iteration, and so on, reducing the number of additions by half at each iteration. More people would not reduce the time any further since the result of each addition operation is necessary to compute the result of the next addition.

Section 12.2 7) Write a CREW PRAM algorithm for adding all n numbers in a list in Θ(lg n) time.

Problem: Find the sum of all n numbers in a list.

Inputs: positive integer n, array of numbers S[] indexed from 1 to n.

Outputs: the sum of all the numbers in S. Comment: It is assumed that n is a power of 2 and that we have n/2 processors indexed from 1 to n/2. number sum(int n, number S[]) { index step, size = 1; local index p = index of this processor;

Chapter 12: Introduction to Parallel Algorithms

Solutions

local number num1, num2; for(step = 1; step <= log n; step++) { if((p-1) % size == 0) { read S[2*p – 1] into num1; read S[2*p – 1 + size] into num2; write num1 + num2 into S[2*p – 1]; size = size * 2; } } return S[1]; } The loop runs for log n steps for the first processor and so the complexity is (log n).

8) Write a CREW PRAM algorithm that uses n2 processors to multiply two n×n matrices.

Your algorithm should perform better than the standard Θ(n3)-time serial algorithm.

Problem: Multiply two n  n matrices.

Inputs: positive integer n, two n  n arrays of numbers, S[][] and T[][].

Outputs: An n  n array of numbers U that is product of S and T. Comments: We assume there are n2 processors available, each with a unique pair of indices in {1, 2, …, n }2. void matMult(int n, number S[][], number T[][], number& U[][]) { local index i, j, k; local int a, b; i = the first index of this processor; j = the second index of this processor; local int sum = 0; for(k=1; k<=n; k++) { read S[i][k] into a; read T[k][j] into b; sum = sum + a*b; } write sum into U[i][j];

Chapter 12: Introduction to Parallel Algorithms

Solutions

} Each processor computes a sum of n numbers and writes the result to the position in the U array to which is corresponds. Computing this sum takes O(n) time since it must loop through n numbers to do so. Thus, the whole operation takes O(n) time with n2 processors.

9) Write a PRAM algorithm for Quicksort using n processors to sort a list of n elements. void parQuicksort(int N, index left, index right, keytype S[], keytype T[]) { keytype pivot = S[left]; bool small; local index p; local element; index crt_left = left; index crt_right = right; p = index of this processor; read S[p] into element; if (element < pivot) small = True; else small = False; for (i = 1; i <= n; i++) if (small) { write element into T[crt_left]; crt_left++; } else { write element into T[crt_right]; crt_right++; } //at this point, crt_left + 1 equals crt_right – 1 Write pivot into T[crt_right+1]; parQuicksort(crt_left – left + 1,

left,

crt_left, S[], T[]);

parQuicksort(right – crt_right + 1, crt_right, right,

S[], T[])

Chapter 12: Introduction to Parallel Algorithms

Solutions

} 10) Write a sequential algorithm that implements the Tournament method to find the largest key in an array of n keys. Show that this algorithm is no more efficient than the standard sequential algorithm. for (step = 1; step <= lg n; step = 2*step) for (i = 1; i <= n; i = i + step*2) { a1 = S[i]; a2 = S[i+step]; if (a1 < a2) S[i] = a2; //largest goes in left member of the pair } //At the end, S[1] will store the largest element.

Analysis: In the first iteration of the outer for loop, there are n/2 comparisons, in the second iteration n/4, then n/8, … , until we reach 1 comparison in the last round. The total is n/2 + n/4 + n/8 + … + 1 = n-1, exactly as in the standard sequential algorithm developed in Ch.1.

11) Write a PRAM algorithm using n3 processors to multiply two n×n matrices. Your algorithm should run in Θ(lg n) time.

Problem: Multiply two n  n matrices.

Inputs: positive integer n, two n  n arrays of numbers, S[][] and T[][].

Outputs: An n  n array of numbers U that is product of S and T. Comments: We assume there are n3 processors available, each with a unique triple of indices in {1, 2, …, n }3. void matMult(int n, number S[][], number T[][], number& U[][]) { local index i, j, k; local int a, b; i = the first index of this processor; j = the second index of this processor; k = the third index of this processor;

Chapter 12: Introduction to Parallel Algorithms

Solutions

int step, size = 2; int V[1…n][1…n][1…n]; read S[i][k] into a; read T[k][j] into b; write a*b into V[i][j][k]; for(step=1; step<=log n; step++) { if((k-1)%size==0) { read V[i][j][2*k – 1] into a; read V[i][j][2*k – 1 +size] into b; write a + b into S[2*k – 1]; size = size *2; } } write V[i][j][1] into U[i][j]; } Each processor first writes the product of two numbers associated with its indices to its corresponding spot in array V. This takes constant time with n 3 processors. The loop is then executed to compute the sum of the list of elements in V[i][j] for each pair of indices i and j. This sum can be computed in log n time using n/2 processors for each sum. Thus, only half of the processors are needed for this step. The method is the same as that used in Exercise 7, except that size starts out with value 2, meaning that half of the processors never do anything during the loop. The loop thus takes log n time and the whole operation takes O(log n) time with n3 processors.

12) Write a PRAM algorithm for the Shortest Paths problem of Section3.2. Compare the performance of your algorithm against the performance of Floyd’s algorithm (Algorithm 3.3).

We use n2 processors to implement the inner two for loops in parallel: void parFloyd(int n, number D[][]) { local index i, j, k; local int a, b, c; i = the first index of this processor; j = the second index of this processor; for (k = 1; k <= n; k++) {

Chapter 12: Introduction to Parallel Algorithms

Solutions

read D[i][j] into a; read D[i][k] into b; read D[k][j] into c; a = minimum(a, b + c); write a into D[i][j]; } Analysis: Since there is only one for loop, this algorithm is (n).

13) Write a PRAM algorithm for the Chained Matrix Multiplication problem of Section 3.4.

Compare the performance of your algorithm against the performance of the Minimum Multiplications algorithm (Algorithm 3.6).

Refer to Figure 3.8 for explanations:

The parallel algorithm will proceed in a loop with n iterations, implementing first the calculations in the main diagonal of the matrix, then the secondary diagonal above it, and so on until the upper-right corner. Note that initially n processors are needed, but their number decreases by one in each iteration, until the upper-left corner only requires one processor.

Chapter 12: Introduction to Parallel Algorithms

Solutions

int parCMM(int n, const int d[], int M[][]) { index diagonal; local index p = index of this processor; local min, i, j, k, a, b, c, d, e; write 0 to M[p][p];

//zeroes in parallel on main diagonal

for (diagonal=1; diagonal<=n-1; diagonal++) if (p <= n–diagonal+1) {

//if processor p is being used in this iteration

i = p; j = p + diagonal -1;

//coordinates of the position that processor //p is updating in this iteration

min = large number; //calculate the minimum for (k=1; k<=j-1; k++) { read M[i][k] into a;

read M[k+1][j] into b;

read d[i-1] into c;

read d[k] into d;

read d[j] into e;

if (a + b + c*d*e < min) min = a + b + c*d*e; } Write min into M[i][j]; } //end if return M[1][n]; } For simplicity, the algorithm above does not calculate P[i][j], the value that gives the minimum. The reader is encouraged to complete the code.

Analysis: Iteration diagonal of the outer for loop requires calculating the minimum of k elements, which is done with k – 1 comparisons, so the every-case time complexity is (0+1+2+…+n-1) = (n2), compared to (n3) for the serial algorithm.

Chapter 12: Introduction to Parallel Algorithms

Solutions

14) Write a PRAM algorithm for the Optimal Binary Search Tree problem of Section 3.5.

Compare the performance of your algorithm against the performance of the Optimal Binary Search Tree algorithm (Algorithm 3.9). float parOST() {int n, const float p[]) { index diagonal; float A[1..n+1][0..n]; local min, i, j, k, a, b, c; local index p = index of this processor; for (i=1; i<=n; i++) {

//initialize the boundary elements

A[i][i-1] = 0; A[i][i] = p[i]; } A[n+1][n] = 0; for (diagonal=1; diagonal<=n-1; diagonal++) if (p <= n–diagonal+1) {

//if processor p is being used in this iteration

i = p; j = p + diagonal;

//coordinates of the position that processor //p is updating in this iteration

min = large number; //calculate the minimum for (k=i; k<=j; k++) { read A[i][k-1] into a; read A[k+1][j] into b; read [sum of p(m) for i<=m<=j] into c; if (a + b + c < min) min = a + b + c; } Write min into A[i][j]; } //end if return A[1][n]; } For simplicity, the algorithm above does not:  

Show how to efficiently calculate [sum of p(m) for i<=m<=j]. Take a look at Exercise 23 in Ch.3 to remember how it’s done (dynamic programming!)

Calculate R[i][j], the index that gives the minimum. The reader is encouraged to complete the code.

Chapter 12: Introduction to Parallel Algorithms

Solutions

Analysis: Iteration diagonal of the outer for loop requires calculating the minimum of k elements, which is done with k – 1 comparisons, so the every-case time complexity is (0+1+2+…+n-1) = (n2), compared to (n3) for the serial algorithm.

Additional Exercises 15) Consider the problem of adding the numbers in a list of n numbers. If it takes ta(n−1) time for one person to add all n numbers, is it possible for m people to compute the sum in less than [ta(n−1)]/m time? Justify your answer.

No. This is not possible because the result of additions performed by one person must be combined at the next time step with the result of additions performed by another person.

Thus, even if there were enough people (n/2) to add each pair of numbers in one step, the sums of each of those pairs would then have to be added, and so on. The lower bound on the time to add n numbers is O(log n), which can be achieved when m = n/2. This would not produce a time cost better than ta(n-1)/m since ta*(n/2)/(n/2) = ta < ta*log n. Adding more people cannot make the operation any faster.

16. Write a PRAM algorithm that runs in Θ((lg n)2) time for the problem of mergesorting.

(Hint: Use n processors, and assign each processor to a key to determine the position of the key in the final list by binary searching.)

The algorithm is called odd-even sorting or bitonic mergesort, and it was published by Batcher in 1968 (K. E. Batcher, "Sorting Networks and their applications", Proc. AFIPS Spring Joint Summer Computer Conference, 32:307-314, 1968.) void parMergesort(int n, intA[]) { local index p = index of this processor;

//between 0 and n-1

local index counterpart; for (k=2; k<=N; k=2*k) for (j=k/2; j>0; j=j/2) { counterpart = p^j; if (counterpart > p) { if ((p&k)==0 && A[i] > A[counterpart]) exchange(p, counterpart, A);

Chapter 12: Introduction to Parallel Algorithms

Solutions

if ((p&k)!=0 && A[p] < A[counterpart]) exchange(p, counterpart, A); } //end if } //end inner for } void exchange(index i, index j, int A[]) { int temp; temp = A[i]; A[i] = A[j]; A[j] = temp; } Analysis: The outer for loop (on k) iterates lg n times, and, in each iteration, the inner for loop iterates lg k times, so the total complexity is 1 + 2 + 3 + … + lg n = (lg n)∙(lg n + 1)/2  Q((lg n)2).

17. Write a PRAM algorithm for the Traveling Salesperson problem of Section 3.6. Compare the performance of your algorithm against the performance of the Traveling Salesperson algorithm (Algorithm 3.11).

For reference, we reproduce below the pseudocode for Algorithm 3.11:

Chapter 12: Introduction to Parallel Algorithms

Solutions

We implement the outer for loop (on k) in parallel, on n – 2 processors, by distributing each value of k to a processor p; each processor executes in parallel the block made up of the inner two loops.

Since we have split the task into (n) parallel tasks, our algorithm is (n2n), compared to (n22n) for the serial Algorithm 3.11.