Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Multicore & GPU Programming: An Integrated Approach, 2e Instructor’s Manual Gerassimos Barlas

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Contents Contents

2

1 Introduction

5

2 Multicore and Parallel Program Design

9

3 Threads and Concurrency in standard C++

13

4 Parallel data structures

57

5 Distributed memory programming

61

6 GPU Programming

117

7 GPU and Accelerator Programming : OpenCL

143

8 Shared-memory programming : OpenMP

169

9 The Thrust Template Library

183

10 High-level multi-threaded programming with the Qt library

199

11 Load Balancing

205

3

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

for more solution manuals,visit Library Genesis: libgen.is, libgen.st, libgen.rs, and forum.mhut.org

Chapter 1

Introduction Exercises 1. Study one of the top 10 most powerful supercomputers in the world. Discover: What kind of operating system does it run?

How many CPUs/GPUs is it made of?

What is its total memory capacity?

What kind of software tools can be used to program it?

Answer Students should research the answer by visiting the Top 500 site and -if available- the site of one of the reported systems.

2. How many cores are inside the top GPU offerings from NVidia and AMD? What is the GFlop rating of these chips?

Answer N/A. 3. The performance of the most powerful supercomputers in the world is usually reported as two numbers Rpeak and Rmax, both in TFlops (tera floating point operations per second) units. Why is this done? What are the factors reducing performance from Rpeak to Rmax? Would it be possible to ever achieve Rpeak?

Answer This is done because the peak performance is unattainable. Sustained, measured performance on specific benchmarks, is a better indicator of the true machine potential.

The reason these are different is communication overhead.

Rpeak and Rmax could never be equal. Extremely compute-heavy applications, that have no inter-node communications, could asymptotically approach Rpeak if they were to run for a very long time. A very long execution time is required to diminish the influence of the start-up costs.

5

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

6

CHAPTER 1.

INTRODUCTION

4. A sequential application with a 20% part that must be executed sequentially, is required to be accelerated five-fold. How many CPUs are required for this task?

Answer This requires the application of Amdahl’s law. The part that can be parallelized is α = 1 − 20% = 80%. The speedup predicted by Amdahl’s 1 law is speedup = α .

1−α+ N

Achieving a three-fold speedup requires that: 1 1 − α +N

α

=3⇒

1 0.8

0.2 + N

=3⇒

0.8 N

1 =

3

− 0.2 ⇒ N = 1

0.8

3 − 0.2

Achieving a 5-fold speedup requires that: 0.8 0.8 1 = =5⇒ N = 1 =∞ 0.8 0 0.2 + N 5 − 0.2

=6 (1.1)

(1.2)

So, it is impossible to achieve a 5-fold speedup, according to Amdahl’s law.

5. A parallel application running on 5 identical machines, has a 10% sequential part. What is the speedup relative to a sequential execution on one of the machines? If we would like to double that speedup, how many CPU would be required?

Answer This requires the application of Gustafson-Barsis’ law as the information relates to a parallel application. The parallel part is α = 1— 10% = 90%.

The speedup over a single machine is speedup — = 1 α+N · α = .1+5· 0.9 = 4.6.

9.1 Doubling the speedup would require .1 + ·N 0.9 = 9.2 ⇒ N = 0.9 = 10.1 machines. As N has to be an integer, we have to round-up to the closest integer, i.e. N = 11.

6. An application with a 5% non-parallelizable part, is to be modified for parallel execution. Currently on the market there are two parallel machines available: machine X with 4 CPUs, each CPU capable of executing the application in 1hr on its own, and, machine Y with 16 CPUs, with each CPU capable of executing the application in 2hr on its own. Which is the machine you should buy, if the minimum execution time is required?

Answer As the information provided relates to a sequential application, we have to apply Amdahl’s law. The execution time for machine X is: αT = 0.05 1hr + 0.95 · 1hr = 0.2875hr (1.3) ∗ tX = (1 − α)T + 4 N The execution time for machine Y is:

αT = 0.05 2hr + 0.95 · 2hr = 0.21875hr ∗ tY = (1 − α)T + 16 N

(1.4)

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

7 So we should buy machine Y. 7. Create a simple sorting application that uses the mergesort algorithm to sort a large collection (e.g. 107) of 32-bit integers. The input data and output results should be stored in files, and the I/O operations should be considered a sequential part of the application. Mergesort is an algorithm that is considered appropriate for parallel execution, although it cannot be equally divided between an arbitrary number of processors, as Amdahl’s and Gustafson-Barsis’ laws require.

Assuming that this equal division is possible, estimate α, i.e. the part of the program that can be parallelized, by using a profiler like gprof or valgrind to measure the duration of mergesort’s execution relative to the overall execution time. Use this number to estimate the predicted speedup for your program.

Does α depend on the size of the input? If it does, how should you modify your predictions and their graphical illustration?

Answer N/A 8. A parallel application running on 10 CPUs, spends 15% of its total time, in sequential execution. What kind of CPU (how much faster) would we need to run this application completely sequentially, while keeping the same total time?

Answer This is an application of Gustafson-Barsis’ law. If T is the parallel execution time on the 10 CPUs, the sequential execution time on a single one would be Ts = (1 − α)T + N · α · T . The fast CPU should match the parallel time, i.e. Tf = T which means if should be TTfs = (1 − α) + N · α = 0.15 + 10 · 0.85 = 8.65 times faster.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Chapter 2

Multicore and Parallel Program Design 1. Perform a 2D agglomeration step for the image convolution problem of Section 2.2. What is the resulting number of communication operations?

Answer Assuming that we target a grid of task groups forming K rows x M columns and that K and M divide the corresponding dimensions evenly, each group will hold IMGX IMGY tasks.

M

K

The number of communication operations are: Four for “internal” groups. There are (K− 1)(M − 1) internal groups.

Each group sends IMGX M pixel values to its top and bottom neighbors, and IMGY pixels to its left and right neighbors.

K Two for corner groups. There are four corner groups.

Three for “boundary” groups. There are 2(K − 2)+2(M −2) boundary groups.

The total data volume communicated is : IMGY IMGX +2 totalComm = (2 )(K − 1)(M − 1) + M K IMGY IMGX (2 + )2(M − 2) + M K IMGY IMGX ( +2 )2(K − 2) + M K IMGX IMGY ( + )4 M K

(2.1) (2.2) (2.3) (2.4)

2. Perform the comparison between the 1D and 2D decompositions of the heat diffusion example in Section 2.3.3, by assuming that (a) half-duplex communication links are available and (b) n-port communications are possible, i.e. all communications can take place at the same time over all the links.

Answer 9

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

10

CHAPTER 2. MULTICORE AND PARALLEL PROGRAM DESIGN (a) If half-duplex communication links where used, then the time spend on communication would be doubled: comp1D + comm1D =

comp2D + comm2D =

N2 · tcomp + 4 · (tstart + tcommN ) P

N2

N tcomp + 8 tstart + tcomm · √ P P

(2.5)

which means that comm1D + comp1D < comm2D + comp2D ⇒ 2 1− √ P So there is no change in the condition that favors 1D over 2D. tstart > tcommN

(2.6)

(b) In the n-port case, the communication time per time step will be: comm1D = tstart + tcommN N comm2D = tstart + tcomm · √ P so the comparison between the two decompositions would be based on comm1D + comp1D < comm2D + comp2D ⇒

√

P <1

which is obviously false. So, 2D is always better than 1D. 3. How would communication costs affect the pipeline performance? Derive variations of Equations 2.16 to 2.18 that take into account a constant communication overhead between the pipeline stages.

Answer Let’s assume that communication between the stages costs a fixed amount of time tc. Then we would have an additional overall time of tc(N + M ), assuming that the last stage also sends data back to whoever is controlling the execution. Thus: Σ

M −1

l−1

ttotal =

Σ

tj + N · tl +

j=0

tj + tc(N + M )

j=l+1

The processing rate of the pipeline is: rate =

Σl−1

j=0 tj + N · tl +

N ΣM −1

j=l+1 tj + tc(N + M )

The latency of the pipeline is: M −2

latency =

Σ

tj + tcM

j=0

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

11 4. The total number of tasks calculated in Section 2.4.5 for the parallel quicksort of Listing 2.8, is based on the best-case assumption that the input is split in equal halves by every call to the PartitionData function. What would be the result if the worst-case (i.e. one part gets N — 1 elements and the other part 0) were considered?

Answer If we assume that the PartitionData function produces a zero sized part and a part with N − 1 elements, then: T (N ) =

0 if N ≤ THRES 1 + T (N − 1) if N > THRES

as one of the calls to QuickSort would always have no input.

Backward substitution can provide the answer: T (N ) = 1 + T (N − 1) = 1 + 1 + T (N − 2) = k + T (N − k) T (N−k) can be eliminated when N THRES. − N k = THRES ⇒ k = − Substituting this value of k in the previous equation yields: T (N ) = N − THRES + T (THRES) = N − THRES as T (THRES) = 0. This is obviously a poor result. What this formula does not convey is that parallelism is also eliminated as a result: there is effectively no overlap between the generated tasks.

5. Use a simple problem instance (e.g. a small array of integers) to trace the execution of the parallel quicksort of Listing 2.8. Create a Gantt graph for the tasks generated, assuming an infinite number of compute nodes is available for executing them. Can you calculate an upper bound for the speedup that can be achieved?

Answer Given N input elements, PartitionData executes between N— 1 and N +1 key comparisons. For simplicity we will assume that N comparisons are done and that the array is evenly split into parts equal in size to N2−1 (taking out the pivot element).

The first initial task that starts the sorting operation, will perform N comparisons, before spawning a task for N−21 elements and continuing with the remaining N−21 . The comparisons are done exclusively inside the PartitionData function, which means we have the following sequence of task executions and corresponding comparison steps: 1 task, N steps 2 tasks,

N−1

2

steps each 2 N −22+1 22

steps each

3

N−2 +1

steps each

2 tasks, 2 tasks,

3

23

And so on...

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

12

CHAPTER 2. MULTICORE AND PARALLEL PROGRAM DESIGN Size of data block to be sorted N −1 N −3 N

Threads

0 2 1 3 Time

Figure 2.1: The first three steps (assuming an ideal scenario of a perfect partition each time) in the parallel quicksort algorithm of Listing 2.8.

Figure 2.1 illustrates this process.

So, the overall duration in number of comparisons is : L−1

totalComp = Σ i=0

N − 2i + 1 2i

+ THRES

where L is associated with when the size become smaller or equal to THRES and no more tasks are spawned.

The value of L is: N − 2L + 1 L L = THRES ⇒ N − 2 + 1 = 2 THRES ⇒ L 2 N +1 N +1 2L = ⇒ L = lg( ) THRES + 1 THRES + 1 Thus: L−1

totalComp = Σ i=0

N −2 +1 i

2i

Σ 1i Σ ( ) — 1 + THRES = 2 i=0

L−1

(N + 1)

+ THRES = L−1

i=0

( ) −1 — L + THRES = (N + 1) 21 2 −1 THRES + 1 2(N + 1)(1 − ) − L + THRES = N +1 2(N + 1) − 2(THRES + 1) − L + THRES = 1 L

2N − THRES − L ≈ 2N So the maximum speedup that could be ever achieved, given that the sequential quicksort performs NlgN comparisons, is: NlgN lgN = speedupmax = 2N 2

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Chapter 3

Threads and Concurrency in standard C++ Exercises 1. Enumerate and create the other timing diagrams that show the alternatives of Figure 3.4, when it comes to the final balance of the bank account.

Answer All the possible permutations of the four events are allowed, as long as 1 proceeds 3 and 2 proceeds 4. So we have: 1, 2, 3, 4 : produces wrong result 1, 3, 2, 4 : produces correct result 1, 2, 4, 3 : produces wrong result 2, 1, 3, 4 : produces wrong result 2, 1, 4, 3 : produces wrong result 2, 4, 1, 3 : produces correct result 2. Research the term “fork bomb” and write a program that performs as such.

Answer #i n c l u d e < s t d l i b . h> #i n c l u d e < u n i s t d . h> #i n c l u d e < l i m i t s . h> i n t main ( i n t argc , c ha r ** a rgv ) { i n t N = a to i ( ar gv [ 1 ] ) ; f o r ( i n t i=0; i<N ; i++) p id _t c ID = for k ( ) ;

}

sl eep ( IN T_ M A X ) ; re t ur n 0 ;

13

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

14 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ 3. Modify the producer-consumer example shown in Listing 3.11, so that the threads terminate after the number 100 is generated.

Answer The only modification required affects the consume method, where the detection of the exit condition is done. It should become: bool co n su m e ( i n t i ) { // to be implemented cout << ”@” ; // j u s t to show something i s ha ppening i f ( i == 100 ) re t ur n t ru e ; e l s e re tu rn f a l s e ; }

4. Suggest a modification to the program of Listing 3.12 so that the IntegrCalc threads can use any function that return a double and takes a double as a parameter.

Answer A pointer to such a function needs to be passed to the initClass method, so that it can be called by the threads. The modifications are highlighted in the following code: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

. . . //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // a c t s as a consumer c l a s s In te gr C a lc : p u b l i c Q Th re ad { private : i n t ID ; s t a t i c Q Se m a p h o re * slo ts A va il ; s t a t i c Q Se m a p h o re * re s A v ail ; s t a t i c Q M u tex l2 ; s t a t i c Q M u tex re s Lo ck ; s t a t i c S li c e * b u ffer ; s t a t i c i n t out ; s t a t i c double * resu lt ; s t a t i c do u bl e ( * f ) ( d ou bl e ) ; // <======= s t a t i c Q Se m a p h o re n u m P ro d u cts ; public : s t a t i c voi d in it C l a s s ( Q Se m a p h o re * s , Q Se m a p h o re doubl e * r , doubl e ( * f ) ( doubl e ) ) ;

Slice

* b , →֓

void In te gr C a lc : : in it C l as s ( Q Se m a p h o re * s , Q Se m a p h o re * a , S lice , d ou bl e * res , d o ubl e ( * g ) ( do u bl e ) ) { // <======= slo ts A va il = s ; re s A v ail = a ; b u ffer = b ; resu lt = res ; * r esu lt = 0 ; f = g ; // <======= } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

* b →֓

*a ,

18 19 20 21 22 23 24 25

In te gr C a lc ( i n t i ) : ID ( i ) { }; void run ( ) ;

}; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− . . . double ( * In te gr C a lc : : f ) ( double ) ; // <=======

26 27

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

void In te gr C a lc : : run ( ) { while ( 1 ) { res A vai l −>a cq u ire ( ) ; // wait f o r an a v a i l a b l e item l2 . lock ( ) ; i n t tm p O u t = out ; out = ( out + 1 ) % B U F F S I Z E ; // update the out index

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

15 45 46

l2 . u n lo ck ( ) ;

47

// take the item out double st = b u ffer [ tm p O u t ] . st a r t ; double en = b u ffer [ tm p O u t ] . end ; double div = b u ffer [ tm p O u t ] . d i v i s i o n s ;

48 49 50 51

slo ts A vail −>r e le as e ( ) ; // s i g n a l f o r a new empty s l o t

52 53

i f ( div == 0 ) br eak ; // e x i t

54 55

// c a l c u l a t e are a doubl e lo c al R e s = 0 ; double step = ( en − st ) / div ; double x ; x = st ; lo cal R e s = f ( st ) + f ( en ) ; // <======= lo ca l R e s /= 2 ; f o r ( i n t i=1; i< div ; i++) { x += st ep ; lo cal R e s += f ( x ) ; // <======= } lo cal R e s *= step ;

56 57 58 59 60 61 62 63 64 65 66 67 68

// add i t to r e s u l t re s Lo ck . loc k ( ) ; * r esu lt += lo cal R e s ; re s Lo ck . u n lo ck ( ) ;

69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88

} } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t ma in ( i n t argc , c har * argv [ ] ) { i f ( a r gc == 1 ) { cer r << ” Usage ” << ar gv [ 0 ] << ” #t hr e ads #jo bs \n” ; exit ( 1 ) ; } i n t N = at oi ( ar gv [ 1 ] ) ; i n t J = at oi ( ar gv [ 2 ] ) ; S lice * b u ff er = new S lic e [ B U F F SI ZE ] ; Q Se m a p h o re a va il , b u ff Slo ts ( B U F F S I Z E ) ; i n t in = 0 ; double resu lt ; In te gr C a lc : : in it C l ass (& buff Slo ts , &avail , buffer , &result , f unc ) ; // <=======

89

→֓

90 91

.

.

.

5. In a remote region of Siberia there are single tracks joining railroad stations. Obviously only one train can use a piece of track between two stations. The other trains can wait at the stations before they do their crossings. The following graph indicates the track and station layout: A

B

C

D

E

F

G

Write a Qt program that simulates the journey of 3 trains with the following schedules: A→B→E→C D→B→E→G C→E→B→D→F

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

16 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ As each trains arrives at a station display a relative message. You can assume that a station can hold any number of trains waiting.

Answer The Train class constructor expects two vectors : one containing the names of the stations that will be traversed, and one containing references to the binary semaphores controlling access to the lines connecting the stations.

1 2 3 4 5

#i n c l u d e <QThread> #i n c l u d e <QMutex> #i n c l u d e <i os t r e am > #i n c l u d e <s t r i n g > #i n c l u d e <ve ct or >

6 7

u si ng namespace std ;

8 9 10 11 12 13 14 15 16 17 18

c l a s s Tra in : p u b l i c Q Th re ad { private : i n t ID ; v ect or <string > * s t a t i o n s ; vector <Q M u tex *> * li n e Lo cks ; public : Tr a i n ( i n t i , v ect or <string > * s , v ect or <Q M u t ex s ta ti o n s ( s ) , li n e Lo cks ( l ) {} void run ( ) ; };

*> * l )

: ID ( i ) , →֓

19 20 21 22 23 24 25 26 27 28 29 30 31 32

void Train : : run ( ) { int i ; f o r ( i=0; i<line Lo cks −>s ize ( ) ; i++) { cout << ” Train ” << ID << ” i s in s t a t i o n ” << st a ti o ns −>→֓ at ( i ) << endl ; line Locks −>a t ( i )−>l ock ( ) ; cout << ” Train ” << ID << ” i s t r a v e l i n g to s t a t i o n ” << →֓ st a t i o n s −>a t ( i+1) << endl ; line Locks −>a t ( i )−>u n lo ck ( ) ; } cout << ” Train ” << ID << ” a r r i v e d in s t a t i o n ” << st a ti o ns →֓ −>a t ( i ) << endl ; }

33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59

int {

main ( i n t argc , char

* ar gv [ ] )

Q M u t ex AB , BD , DF , BE , CE , EG ; // m u t i c es // one f o r each t r a i n vec t or <Q M u t ex *> ro u t es [ 3 ] ; vect or <string > st [ 3 ] ;

f o r the

lines

ro u t es [ 0 ] . p u sh _ b ack (& AB ) ; ro u tes [ 0 ] . p u sh _ b ack (& BE ) ; ro u tes [ 0 ] . p u sh _ b ack (& CE ) ; st [ 0 ] . p u sh _ b ac k ( ”A” ) ; st [ 0 ] . p u sh _ b ac k ( ”B” ) ; st [ 0 ] . p u sh _ b ack ( ”E” ) ; st [ 0 ] . p u sh _ b ac k ( ”C” ) ; ro u tes [ 1 ] . p u sh _ b ack (& BD ) ; ro u tes [ 1 ] . p u sh _ b ack (& BE ) ; ro u t es [ 1 ] . p u sh _ b ack (& EG ) ; st [ 1 ] . p u sh _ b ack ( ”D” ) ; st [ 1 ] . p u sh _ b ac k ( ”B” ) ; st [ 1 ] . p u sh _ b ack ( ”E” ) ; st [ 1 ] . p u sh _ b ac k ( ”G” ) ; ro u tes [ 2 ] . p u sh _ b ack (& CE ) ; ro u tes [ 2 ] . p u sh _ b ack (& BE ) ; ro u tes [ 2 ] . p u sh _ b ack (& BD ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

17 ro u tes [ 2 ] . p u sh _ b ack (& DF ) ; st [ 2 ] . p u sh _ b ac k ( ”C” ) ; st [ 2 ] . p u sh _ b ack ( ”E” ) ; st [ 2 ] . p u sh _ b ac k ( ”B” ) ; st [ 2 ] . p u sh _ b ack ( ”D” ) ; st [ 2 ] . p u sh _ b ac k ( ”F” ) ;

60 61 62 63 64 65 66

// t hr e a d spawning Tr a i n * t [ 3 ] ; f o r ( i n t i=0; i <3; i++) { t [ i ] = new Trai n ( i , &st [ i ] , &ro u tes [ i ] ) ; t [ i]−> s ta rt ( ) ; }

67 68 69 70 71 72 73 74 75 76 77 78

}

f o r ( i n t i=0; i <3; i++) t [ i]−> wait ( ) ; re tu rn 0 ;

6. Modify the program of the previous exercise so that each station can hold only 2 trains. Can this lead to deadlocks?

If you have not done so already, make sure that your program uses only one thread class.

Answer This cannot lead to deadlocks as long as we have three or less trains.

For four or more we could have pairs of trains trying to cross opposite directions of the same line, leading to deadlock.

The only required changes involves controlling access to the stations via a set of general semaphores.

1 2 3 4 5 6

#i n c l u d e <QThread> #i n c l u d e <QMutex> #i n c l u d e <QSemaphore> #i n c l u d e <i os t r ea m > #i n c l u d e <s t r i n g > #i n c l u d e <ve ct or >

7 8

u si ng namespace std ;

9 10 11 12 13 14 15 16 17 18

19 20 21 22 23 24 25 26 27 28 29 30 31

c l a s s Tra in : p u b l i c Q Th re ad { private : i n t ID ; v ect or <string > * s t a t i o n s ; vector <Q Se m a p h o re *> * sta tio n s C tr l ; vector <Q M u tex *> * li n e Lo cks ; public : Tr a i n ( i n t i , v ect or <string > * s , v ect or <Q M u t ex *> * l , v ect or <→֓ Q Se m a p h o re *> * c ) : ID ( i ) , s t a ti o n s ( s ) , li n e Lo cks ( l ) , →֓ st a ti o n s C trl ( c ) {} void run ( ) ; }; void Train : : run ( ) { int i ; s t a t i o n s C tr l −>a t ( 0 )−>a cq u i r e ( ) ; f o r ( i=0; i<line Lo cks −>s ize ( ) ; i++) { cout << ” Tr ain ” << ID << ” i s in s t a t i o n ” << st a ti o ns −>→֓ at ( i ) << endl ; s ta ti o n s C t r l −>a t ( i+1)−>a cq u ire ( ) ; // r e s e r v e s t a t i o n →֓ b e f o r e l o c k i n g the l i n e line Locks −>a t ( i )−>l ock ( ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

18 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ cout << ” Train ” << ID << ” i s t r a v e l i n g to s t a t i o n ” << →֓ st a t i o n s −>a t ( i+1) << endl ; line Locks −>a t ( i )−>u n lo ck ( ) ;

32 33 34

s ta ti o n s C trl −>a t ( i )−>r e le ase ( ) ; // r e l e a s e pre vi o us →֓ station reservation cout<<ID <<” ”<< s t a t i o n s C trl −>a t ( i )−>a v a i l a b le ( ) << endl →֓ ;

35 36 37 38 39 40

}

} cout << ” Train ” << ID << ” a r r i v e d in s t a t i o n ” << st a ti o ns →֓ −>a t ( i ) << endl ;

41 42 43

int {

* ar gv [ ] )

Q M u t ex AB , BD , DF , BE , CE , EG ; Q Se m a p h o re A ( 2 ) , B ( 2 ) , C ( 2 ) , D ( 2 ) , E ( 2 ) , F ( 2 ) , G ( 2 ) ; // one f o r each t r a i n vec t or <Q M u t ex *> ro u t es [ 3 ] ; vec t or <Q Se m a p h o re *> st C ap ac [ 3 ] ; vect or <string > st [ 3 ] ;

44 45 46 47 48 49 50

ro u t es [ 0 ] . p u sh _ b ack (& AB ) ; ro u tes [ 0 ] . p u sh _ b ack (& BE ) ; ro u tes [ 0 ] . p u sh _ b ack (& CE ) ; st [ 0 ] . p u sh _ b ac k ( ”A” ) ; st [ 0 ] . p u sh _ b ac k ( ”B” ) ; st [ 0 ] . p u sh _ b ack ( ”E” ) ; st [ 0 ] . p u sh _ b ac k ( ”C” ) ; st C ap ac [ 0 ] . p u sh _ b ac k (&A ) ; st C ap ac [ 0 ] . p u sh _ b ac k (&B ) ; st C ap ac [ 0 ] . p u sh _ b ac k (&E ) ; st C ap ac [ 0 ] . p u sh _ b ack (&C ) ;

51 52 53 54 55 56 57 58 59 60 61 62

ro u tes [ 1 ] . p u sh _ b ack (& BD ) ; ro u tes [ 1 ] . p u sh _ b ack (& BE ) ; ro u t es [ 1 ] . p u sh _ b ack (& EG ) ; st [ 1 ] . p u sh _ b ack ( ”D” ) ; st [ 1 ] . p u sh _ b ac k ( ”B” ) ; st [ 1 ] . p u sh _ b ack ( ”E” ) ; st [ 1 ] . p u sh _ b ac k ( ”G” ) ; st C ap ac [ 1 ] . p u sh _ b ac k (&D ) ; st C ap ac [ 1 ] . p u sh _ b ac k (&B ) ; st C ap ac [ 1 ] . p u sh _ b ac k (&E ) ; st C ap ac [ 1 ] . p u sh _ b ac k (&G ) ;

63 64 65 66 67 68 69 70 71 72 73 74

ro u tes [ 2 ] . p u sh _ b ack (& CE ) ; ro u tes [ 2 ] . p u sh _ b ack (& BE ) ; ro u tes [ 2 ] . p u sh _ b ack (& BD ) ; ro u tes [ 2 ] . p u sh _ b ack (& DF ) ; st [ 2 ] . p u sh _ b ac k ( ”C” ) ; st [ 2 ] . p u sh _ b ack ( ”E” ) ; st [ 2 ] . p u sh _ b ac k ( ”B” ) ; st [ 2 ] . p u sh _ b ack ( ”D” ) ; st [ 2 ] . p u sh _ b ac k ( ”F” ) ; st C ap ac [ 2 ] . p u sh _ b ack (&C ) ; st C ap ac [ 2 ] . p u sh _ b ac k (&E ) ; st C ap ac [ 2 ] . p u sh _ b ac k (&B ) ; st C ap ac [ 2 ] . p u sh _ b ac k (&D ) ; st C ap ac [ 2 ] . p u sh _ b ac k (&F ) ;

75 76 77 78 79 80 81 82 83 84 85 86 87 88 89

Tr a i n * t [ 3 ] ; f o r ( i n t i=0; i <3; i++) { t [ i ] = new Trai n ( i , &st [ i ] , &ro u tes [ i ] , &st C ap ac [ i ] ) ; t [ i]−> s ta rt ( ) ; }

90 91 92 93 94 95 96 97 98 99 100

main ( i n t argc , char

}

f o r ( i n t i=0; i <3; i++) t [ i]−> wait ( ) ; re tu rn 0 ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

19 7. A desktop publishing (like PageMaker) application has two threads running: one for running the GUI and one for doing background work. Simulate this application in Qt. Your implementation should have the thread corresponding to the GUI, send requests to the other thread to run tasks on its behalf. The tasks should be (obviously just printing a message is enough for the simulation): Printing Mail merging PDF generation After performing each requested task, the second thread should wait for a new request to be send to it. Make sure that the first thread does not have to wait for the second thread to finish before making new requests.

Answer The following solution is a producer-consumer derivative, where just two counting semaphores are needed for each pair of interacting threads. There are two buffers, one for handling the interaction between GUI and mailmerge and one for the interaction between GUI and PDF generation.

1 2 3 4 5

#i n c l u d e <QSemaphore> #i n c l u d e <QThread> #i n c l u d e < s t d l i b . h> #i n c l u d e < u n i s t d . h> #i n c l u d e < ios t r ea m >

6 7

u si ng namespace std ;

8 9 10 11

co ns t i n t T E R M IN FLA G = −1; co nst i n t RUN S = 2 0 ; co nst i n t B U F F S I ZE = 5 ;

12 13 14 15 16 17

// the f o l l o w i n g shoul d not be g l o b a l v a r i a b l e s // but they are so to re d u ce the le ng t h o f the code Q Se m a p h o re m ail R e q ( 0 ) , m ail Sp ace ( B U F F S I ZE ) ; i n t m ail Jo b [ B U F F S I ZE ] ; // b u f f e r i n t m_i n = 0 , mout = 0 ;

18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

Q Se m a p h o re p d f R eq ( 0 ) , p d f Sp ace ( B U F F S I ZE ) ; i n t p d f Jo b [ B U F F S I Z E ] ; // b u f f e r i n t pin = 0 , pout = 0 ; //============================================= c l a s s GUI : p u b l i c Q Th re ad { public : void run ( ) ; }; //−−−−−−−−−−−−−− void GUI : : run ( ) { f o r ( i n t i = 0 ; i < RUNS ; i++) { i n t ch o i c e = rand ( ) % 2 ; i f ( ch o i ce ) // mail merge { m ail Sp ace . acq u ire ( ) ; m ail Jo b [ m_i n ] = i ; m_in = ( m_in + 1 ) % B U F F S I ZE ; m ail R e q . re le ase ( ) ; } else // PDF ge n e ra t io n { p d f Sp ace . acq u ir e ( ) ; p d f Jo b [ pin ] = i ; pin = ( pin + 1 ) % B U F F S I Z E ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

20 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ 46 47 48 49 50

m

51 52

m

53 54 55

M

G

RM

BU G

56 57 58 59 60 61 62 63 64 65

M

66

M

67 68 69

m

70

m

71 72 73 74 75

m

76 77 78 79 80

D FG

81

Q

82 83 84 85 86 87 88 89 90 91 92 93

RM

94

BU G

95 96 97 98 99 100 101 102 103 104 105 106 107 108 109

m

110 111 112 113 114 115

8. A popular bakery has a baker that cooks a loaf of bread at a time and

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

21 deposits it on a counter. Incoming customers pick up a loaf from the counter and exit the bakery. The counter can hold 20 loafs. If it is full the baker stops baking bread. If it is empty, a customer waits. Use semaphores to solve the coordination problem between the baker and the customers.

Answer This is an instance of the producer-consumers problem. Because the baker thread is producing just simple integers in sequence, we can use a QAtomicInt variable for retrieving loaf IDs on the customers’ side.

1 2 3 4 5

#i n c l u d e <QThread> #i n c l u d e <QSemaphore> #i n c l u d e <QAtomicInt> #i n c l u d e <i os t r ea m > #i n c l u d e < s t d l i b . h>

6 7

u si ng namespace std ;

8 9

co nst i n t SPA C E =20;

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Q Se m a p h o re lo afs ( 0 ) , co u n te r Sp a ce ( SP A C E ) ; Q A to m ic In t lo af ID ; //============================ c l a s s B aker : p u b l i c Q Th re ad { private : i n t to ta l B re a d ; public: B aker ( i n t t ) : to ta l B re a d ( t ) {} void run ( ) ; }; //−−−−−−−−−−−−−−−−−− void B aker : : run ( ) { f o r ( i n t i=0; i<to ta l B re a d ; i++) { co u n te r Sp a ce . acq u ire ( ) ; cout << ” Baker baked # ” << i << endl ; lo afs . re le ase ( ) ; } } //============================ c l a s s C u sto m e r : p u b l i c Q Th re ad { private : i n t ID ; public: C u sto m e r ( i n t i ) : ID ( i ) {} void run ( ) ; }; //−−−−−−−−−−−−−−−−−− void C u sto m e r : : run ( ) { lo afs . acq u i r e ( ) ; i n t m y B re ad = lo af ID . fe tch A n d A d d O rd e re d ( 1 ) ; co u n te r Sp a ce . re le ase ( ) ; cout << ” Customer ” << ID << ” got bread #” << m y B re ad << endl →֓ ; } //============================ i n t main ( i n t argc , char * ar gv [ ] ) { i n t to ta l C u st o m e rs = at oi ( a r gv [ 1 ] ) ; C u st o m e r * c [ to ta l C u s t o m e rs ] ; f o r ( i n t i=0; i<to ta l C u sto m e rs ; i++) { c [ i]=new C u sto m e r ( i ) ; c [ i]−> sta rt ( ) ; } B aker b ( to ta l C u s t o m e rs ) ; b . run ( ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

22 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ 61 62

m

63 64 65 66 67

9. Because of customer demand, the bakery owner is considering the following enhancements to his shop: (a) Increase the capacity of the counter to 1000 (b) Hire 3 more bakers Modify the solution of the previous exercise to accommodate these changes.

Which is the easiest to implement?

Answer (a) In this case, the only change required is to modify line 9 in the previous listing to: co ns t

i n t SP A C E =1000;

(b) The complication in this case is the termination of the baker threads.

For this purpose, the totalCustomers variable is used to initialize a general semaphore that is tested and decremented prior to the execution of each iteration of a baker thread (line 27): 1 2 3 4 5

#i n c l u d e <QThread> #i n c l u d e <QSemaphore> #i n c l u d e <QAtomicInt> #i n c l u d e <i os t r ea m > #i n c l u d e < s t d l i b . h>

6 7

u si ng namespace std ;

8 9

co nst i n t SPA C E =20;

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

Q Se m a p h o re lo afs ( 0 ) , co u n te r Sp a ce ( SP A C E ) ; Q Se m a p h o re m o re Lo afs ( 0 ) ; Q A to m ic In t lo af ID ; //============================ c l a s s B aker : p u b l i c Q Th re ad { private : i n t ID ; public: B aker ( i n t i ) : ID ( i ) {} void run ( ) ; }; //−−−−−−−−−−−−−−−−−− void B aker : : run ( ) { i n t m y P ro d =0; w h i l e ( m o re Lo af s . t r y A cq u ire ( ) ) { co u n te r Sp a ce . acq u ire ( ) ; lo afs . re le ase ( ) ; m y P ro d++; } cout << ” Baker #” << ID << ” baked a t o t a l o f ” << m y P ro d << →֓ ” br e a d s ” << endl ; } //============================ c l a s s C u sto m e r : p u b l i c Q Th re ad { private :

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

23 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62

i n t ID ; public : C u sto m e r ( i n t i ) : ID ( i ) {} void run ( ) ; }; //−−−−−−−−−−−−−−−−−− void C u sto m e r : : run ( ) { lo afs . acq u ir e ( ) ; i n t m y B re ad = lo af ID . fe tch A n d A d d O rd e re d ( 1 ) ; co u n te r Sp a ce . re le ase ( ) ; cout << ” Customer ” << ID << ” got bread #” << m y B re ad << →֓ endl ; } //============================ i n t main ( i n t argc , char * ar gv [ ] ) { i n t to ta l C u st o m e rs = at oi ( a r gv [ 1 ] ) ; C u st o m e r * c [ to ta l C u s t o m e rs ] ; m o re Lo a fs . re le ase ( to ta l C u sto m e rs ) ; f o r ( i n t i=0; i<to ta l C u sto m e rs ; i++) { c [ i]=new C u sto m e r ( i ) ; c [ i]−> sta rt ( ) ; }

63

B aker * b [ 4 ] ; f o r ( i n t i=0; i <4; i++) { b [ i ] = new B aker ( i ) ; b [ i ] −> st a r t ( ) ; }

64 65 66 67 68 69 70 71

// wait f o r te r m i n a t i o n f o r ( i n t i=0; i<to ta l C u sto m e rs ; i++) c [ i]−> wait ( ) ; f o r ( i n t i=0; i <4; i++) b [ i]−> wait ( ) ;

72 73 74 75 76 77 78 79

}

re tu rn

0;

10. A bank account class is defined as follows: c l a s s B a n k A cc o u n t { pr o te c t e d : doubl e b al an ce ; strin g h o ld e r N a m e ; public : doubl e ge t B a la n ce ( ) ; void d e p o sit ( doubl e ) ; void w i th d r aw ( double , i n t ) ; // the h i g h e s t the s ec o n d →֓ argument , the h i g h e r the p r i o r i t y o f the re q u e s t };

Write the implementation of the three methods given above, so that withdraw operations are prioritized: if there are not enough funds in the account for all, the withdrawals must be done in order of priority regardless if there are some that can be performed with the available funds. You can assume that the priority level in the withdraw method is by default equal to 0, and that it can is upper bounded by a fixed constant MAXPRIORITY. Answer

for more solution manuals,visit libgen.is, libgen.st, libgen.rs, and forum.mhut.org

Solution is monitor based, using a separate wait condition for each level of priority.

1 2

// Monitor− based s o l u t i o n #i n c l u d e <QThread>

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

24 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ 3 4 5 6 7 8 9

#i n c l u d e <QMutexLocker> #i n c l u d e <QWaitCondition> #i n c l u d e <c s t r i n g > #i n c l u d e < u ni st d . h> #i n c l u d e <s t r i n g > #i n c l u d e < ios t r ea m >

10 11

u si ng namespace std ;

12

co ns t i n t M A X P R IO R IT Y = 1 0 ; co nst i n t N U M A G E N T S = 2 0 ;

13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s B a n k A cc o u n t { pr o te c t e d : doubl e b al an ce ; st r i n g h o ld e r N a m e ; Q M u tex l ; Q W a it C o n d it io n p ri C o n d [ M A X P R IO R IT Y ] ; i n t w a iti n g C o u n t [ M A X P R IO R IT Y ] ; public : B a n k A c c o u n t ( st r i n g name , do ubl e init ) ; doubl e ge t B a la n ce ( ) ; void d e p o sit ( doubl e ) ; void w ith d r aw ( double , i n t ) ; // the h i g h e s t the s econd →֓ argument , the h i g h e r the p r i o r i t y o f the re q u e s t }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− B a n k A c c o u n t : : B a n k A cc o u n t ( s t r i n g name , d ou bl e init ) { h o ld e r N a m e = na me ; b a l a n ce = init ; m em set ( w ai ti n g C o u n t , 0 , M A X P R IO R IT Y * s i z e o f ( i n t ) ) ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− do u ble B a n k A cc o u n t : : ge t B a la n ce ( ) { re t ur n b alan ce ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void B a n k A cc o u n t : : d e p o si t ( dou bl e x ) { Q M u te x Lo cke r ml (&l ) ; b a la n ce += x ; cerr << ” D e p o s it e d ” << x << endl ;

52 53 54 55 56 57 58 59 60 61 62 63

// now d e t e r m i n e i f any t h r e a d i s w a i t i n g to withdraw i n t p ri L vl = M A X P R IO R IT Y − 1 ; w h i l e ( p ri Lvl >= 0 ) { i f ( w a i t i n g C o u n t [ p ri L vl ] > 0 ) { cerr << ”Waking l e v e l ” << p ri Lvl << endl ; p ri C o n d [ p ri L vl ] . w ake A ll ( ) ; // wake up a l l the at t hat p r i o r i t y l e v e l re tu rn ; } pri Lvl −−; }

64 65

}

66

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // the h i g h e s t the s ec o nd argument , the h i g h e r the re q u e s t void B a n k A c c o u n t : : w ith d r aw ( do ubl e x , i n t lvl ) { Q M u te x Lo cke r ml (&l ) ; lvl = ( lvl >= M A X P R IO R IT Y ) ? M A X P R IO R IT Y − 1 s u r e p r i o r i t y i s not too high

67 68 69 70 71

t h r e ads →֓

the p r i o r i t y o f →֓

: lvl ;

//make →֓

72

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

25 // d e t e r m i n e i f o t h e r s at the same or h i g h e r p r i o r i t y are w ait i n g i n t o th e rs W a it i n g = 0 ; i n t p ri L vl = M A X P R IO R IT Y − 1 ; w h i l e ( p ri Lvl >= lvl ) { o th e rs W a i t i n g += w a i t i n g C o u n t [ p ri L vl ] ; pri Lvl −−; }

73 74 75 76 77 78 79 80 81

// i f they are or the funds are not enough wai t i f ( o th e rs W a i t i n g > 0 | | b alan ce < x ) { w a it i n g C o u n t [ lvl ]++; w hi l e ( x > b ala n ce ) p ri C o n d [ lvl ] . wait (&l ) ;

82 83 84 85 86 87 88

also

b alan ce −= x ; w a it i n g C o u n t [ lvl ]−−; i f ( w a it i n g C o u n t [ lvl ] == 0 ) { p ri Lvl = lvl ; whi l e ( p ri Lvl >= 0 && w a it in g C o u n t [ p ri Lvl ] == 0 ) pri Lvl −−; i f ( p ri L vl >= 0 ) p ri C o n d [ p ri L vl ] . w ake A ll ( ) ; }

89 90 91 92 93 94 95 96 97 98

} else b alan ce −= x ;

99 100 101 102 103

cerr << ” Withdrew ” << x << ” with p r i ” << lvl << endl ;

104 105

}

106

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // c l a s s f o r t e s t i n g pur p o se s c l a s s A gen t : p u b l i c Q Th re ad { private : B a n k A c co u n t * ba ; double am o u n t ; i n t p ri o r i t y ; public : A gen t ( B a n k A cco u n t * b , doubl e x , i n t pri ) : ba ( b ) , am o u n t p rio ri t y ( pri ) {} void run ( ) ; };

107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142

→֓

( x ) , →֓

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void A gen t : : run ( ) { i f ( am o u n t > 0 ) ba−>d e p o sit ( am o u n t ) ; else ba−>w it h d r aw (− amount , p r io r it y ) ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t main ( i n t argc , char * ar gv [ ] ) { B a n k A cco u n t b ( ” John Doe” , 0 ) ; A gen t * ag [ N U M A G E N T S ] ; f o r ( i n t i = 0 ; i < N U M A G E N T S ; i++) { ag [ i ] = new A gen t (&b , − rand ( ) % 20 , i / 2 ) ; ag [ i]−> st a rt ( ) ; } sleep ( 1 ) ; b . d e p o sit ( N U M A G E N T S * 20 ) ; // make s u r e enough fun ds available f o r ( i n t i = 0 ; i < N U M A G E N T S ; i++) ag [ i]−> wai t ( ) ;

a re →֓

143

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

26 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ 144 145

}

re tu rn

0;

11. The IT department of a big corporation is equipped with 5 high speed printers that are used by a multitude of threads. The threads are part of the same accounting process. Each of the threads is supposed to perform the following (pseudocode) sequence in order to printout any material: ... p rin te r ID = g e t_ a va i la b le _ p ri n te r ( ) ; // p r i n t to p r i n t e r ID p r i n t e r re le a se P rin te r ( p rin te r ID ) ; ...

Write an appropriate implementation for the two functions listed above using semaphores. You can assume that the available printer IDs are stored in a shared buffer.

Answer Solution is based on the producers-consumers pattern. A buffer with the available printer IDs is setup and managed. Function get available printer() operates as a consumer and function releasePrinter() operates as a producer. Only one counting semaphore is required as the buffer is as big as the overall number of printers.

1 2 3 4 5

#i n c l u d e <QThread> #i n c l u d e <QMutex> #i n c l u d e <QSemaphore> #i n c l u d e < u n i s t d . h> #i n c l u d e <i os t r ea m >

6 7

u si ng namespace std ;

8 9 10

co ns t i n t P R IN T E R S = 5 ; co nst i n t N U M A G E N T S = 2 0 ;

11 12 13 14 15

i n t p ri n t ID b u ffe r [ P R IN T E R S ] ={ 0 , 1 , 2 , 3 , 4 }; Q Se m a p h o re p rn A vail ( P R IN T E R S ) ; i n t in =0 , out =0; Q M u tex l1 , l2 ;

16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t g e t_ a va ila b le _ p ri n te r ( ) { p rn A v a i l . acq u i r e ( ) ; l1 . lock ( ) ; i n t tmp = p ri n t ID b u ffe r [ out ] ; out = ( out +1)%P R IN T E R S ; l1 . u n lo ck ( ) ; re t ur n tmp ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void re le a se P rin te r ( i n t p rin te r ID ) { l2 . lock ( ) ; p rin t ID b u ffe r [ in ]= p rin te r ID ; in = ( in +1)%P R IN T E R S ; l2 . u n lo ck ( ) ; p rn A va i l . re le a se ( ) ; }

36 37 38 39 40 41 42

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // c l a s s f o r t e s t i n g p ur po se s c l a s s A gen t : p u b l i c Q Th re ad { private :

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

27 43 44 45 46 47

i n t ID ; public : A g en t ( i n t i ) : ID ( i ) {} void run ( ) ; };

48 49 50 51 52 53

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void A gen t : : run ( ) { s lee p ( rand ( ) %3) ; i n t p ri n te r ID = g e t_ a va i l a b le _ p r i n te r ( ) ;

54 55 56 57 58

}

// p r i n t to p r i n t e r ID p r i n t e r cout << ID << ” i s p r i n t i n g to ” << p rin te r ID << endl ; re le a se P rin te r ( p rin te r ID ) ;

59 60 61 62 63 64 65 66 67 68

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t main ( i n t argc , char * ar gv [ ] ) { A gen t * ag [ N U M A G E N T S ] ; f o r ( i n t i = 0 ; i < N U M A G E N T S ; i++) { ag [ i ] = new A g en t ( i ) ; ag [ i]−> st a rt ( ) ; }

69

f o r ( i n t i = 0 ; i < N U M A G E N T S ; i++) ag [ i]−> wai t ( ) ;

70 71 72 73 74

}

re tu rn

0;

12. Create 3 threads, each printing out the letters ’A’, ’B’ and ’C’. The printing must adhere to these rules: The total number of ’B’s and ’C’s that have been output at any point in the output string cannot exceed the total number of ’A’s that have been output at that point.

After a ’C’ has been output, another ’C’ cannot be output until one or more ’B’s have been output.

Use semaphores to solve the problem.

Answer 1 2 3 4 5

#i n c l u d e <QThread> #i n c l u d e <QMutex> #i n c l u d e <QSemaphore> #i n c l u d e <i os t r ea m > #i n c l u d e < u n i s t d . h>

6 7

u si ng namespace std ;

8 9 10 11

Q Se m a p h o re p er m it ( 0 ) ; Q M u te x all o w C ; v o l a t i l e bool B f la g=true ;

12 13 14 15 16 17 18 19 20 21 22 23

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr A : p u b l i c Q Th re ad { public : void run ( ) ; }; //−−−−−−−−−−−−−−− void Thr A : : run ( ) { while ( 1 ) {

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

28 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ 24 25

}

26 27

cout << ’A ’ ; p er m it . re le as e ( ) ;

}

28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr B : p u b l i c Q Th re ad { public : void run ( ) ; }; //−−−−−−−−−−−−−−− void T hr B : : run ( ) { while ( 1 ) { p erm it . acq u ir e ( ) ; cout << ’B ’ ; i f ( B flag==f a l s e ) { a ll o w C . u n lo ck ( ) ; B fla g=t r ue ; } } } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr C : p u b l i c Q Th re ad { public : void run ( ) ; }; //−−−−−−−−−−−−−−− void T hr C : : run ( ) { while ( 1 ) { all o w C . loc k ( ) ; p erm it . acq u ire ( ) ; cout << ’C ’ ; B flag=f a l s e ; } } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t main ( i n t argc , char * ar gv [ ] ) { Thr A a ; Thr B b ; Thr C c ;

72

a . sta rt ( ) ; b . sta rt ( ) ; c . run ( ) ;

73 74 75 76 77 78

}

re tu rn

0;

13. Modify the previous exercise so that the printing is governed by this set of rules: One ’C’ must be output after two ’A’s and three ’B’s are output.

While there is no restriction on the order of printing ’A’ and ’B’, the corresponding threads must wait for a ’C’ to be printed when the previous condition is met.

Use a monitor to solve the problem.

Answer The following solution uses the “critical section outside the monitor” approach.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

29 1 2 3 4 5 6 7

#i n c l u d e <QThread> #i n c l u d e <QMutex> #i n c l u d e <QMutexLocker> #i n c l u d e <QWaitCondition> #i n c l u d e < ios t r ea m > #i n c l u d e < u n i s t d . h>

8 9

u si ng namespace std ;

10

c l a s s M o n it o r { private : i n t cntA , cnt B ; Q M u tex l ; Q W a it C o n d it io n cAB , cC ; public : M o n ito r ( ) : cnt A ( 0 ) , cnt B ( 0 ) {} void all o w A ( ) ; void all o w B ( ) ; void al lo w C ( ) ; void d o n e A ( ) ; void d o n e B ( ) ; void d o n e C ( ) ; }; //−−−−−−−−−−−−−−− void M o n i to r : : a ll o w A ( ) { Q M u te x Lo cke r ml(&l ) ; w hi l e ( cnt A == 3 ) c AB . wa it( &l ) ; } //−−−−−−−−−−−−−−− void M o n ito r : : all o w B ( ) { Q M u te x Lo cke r ml(&l ) ; w hi l e ( cnt B == 2 ) c AB . wa it( &l ) ; } //−−−−−−−−−−−−−−− void M o n ito r : : all o w C ( ) { Q M u te x Lo cke r ml(&l ) ; w h i l e ( cnt A != 3 | | cnt B ! =2 ) cC . wait( &l ) ; } //−−−−−−−−−−−−−−− void M o n ito r : : d o n e A ( ) { Q M u te x Lo cke r ml(&l ) ; cnt A++; i f ( cnt A==3) cC . w ake O n e ( ) ; } //−−−−−−−−−−−−−−− void M o n it o r : : d o n e B ( ) { Q M u te x Lo cke r ml(&l ) ; cnt B++; i f ( cnt B==2) cC . w ake O n e ( ) ; } //−−−−−−−−−−−−−−− void M o n it o r : : d o n e C ( ) { Q M u te x Lo cke r ml(&l ) ; cnt A = c nt B = 0 ; c AB . w ake A ll ( ) ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr A : p u b l i c Q Th re ad { private : M o n ito r * m ; public :

11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

30 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ Thr A ( M o n it o r * x ) : m ( x ) {} void run ( ) ;

75 76 77 78 79 80 81 82 83 84 85 86 87

}; //−−−−−−−−−−−−−−− void T hr A : : run ( ) { while ( 1 ) { m−>a llo w A ( ) ; cout<< ’A ’ ; m−>d o n e A ( ) ; } }

88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr B : p u b l i c Q Th re ad { private : M o n ito r * m ; public : Thr B ( M o n ito r * x ) : m ( x ) {} void run ( ) ; }; //−−−−−−−−−−−−−−− void T hr B : : run ( ) { while ( 1 ) { m−>a llo w B ( ) ; cout<< ’B ’ ; m−>d o n e B ( ) ; } } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr C : p u b l i c Q Th re ad { private : M o n ito r * m ; public : Thr C ( M o n ito r * x ) : m ( x ) {} void run ( ) ; }; //−−−−−−−−−−−−−−− void T hr C : : run ( ) { while ( 1 ) { m−>a llo w C ( ) ; cout<< ’C ’ ; m−>d o n e C ( ) ; } } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t m ain ( i n t argc , char * ar gv [ ] ) { M o n it o r m ; Thr A a(&m ) ; Thr B b(&m ) ; Thr C c(&m ) ;

134

a . sta rt ( ) ; b . sta rt ( ) ; c . run ( ) ;

135 136 137 138 139 140

}

re tu rn

0;

14. Address the termination problem in the previous exercise. How can the three threads terminate after e.g. a fixed number of As has been output?

Or when a fixed total number of character has been output?

Answer

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

31 The following code solves the termination problem for a fixed number of total characters. A character is reserved at the output before the termination of the allow?() methods, which compared to the previous exercise, return a boolean to indicate whether to continue execution or not. If the maximum number of characters has been reached, appropriate signals wake up any waiting threads.

1 2 3 4 5 6

#i n c l u d e <QThread> #i n c l u d e <QMutex> #i n c l u d e <QMutexLocker> #i n c l u d e <QWaitCondition> #i n c l u d e < iostrea m > #i n c l u d e < u n i s t d . h>

7 8

u si ng namespace std ;

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51

c l a s s M o n ito r { private : i n t cntA , cntB , to tal ; Q M u tex l ; Q W a it C o n d it io n cAB , cC ; public : M o n ito r ( i n t t ) : cnt A ( 0 ) , cnt B ( 0 ) , to t al ( t ) {} bool al lo w A ( ) ; bool al lo w B ( ) ; bool all o w C ( ) ; void d o n e A ( ) ; void d o n e B ( ) ; void d o n e C ( ) ; }; //−−−−−−−−−−−−−−− bool M o n it o r : : a l l o w A ( ) { Q M u te x Lo cke r ml (&l ) ; w h i l e ( cnt A == 3 && to t al > 0 ) c AB . wait (&l ) ; i f ( to tal ) { t ot al −−; re tu rn tr ue ; } else { c AB . w ake O n e ( ) ; cC . w ake O n e ( ) ; re tu rn f a l s e ; } } //−−−−−−−−−−−−−−− bool M o n ito r : : all o w B ( ) { Q M u te x Lo cke r ml (&l ) ; w h i l e ( cnt B == 2 && to t al > 0 ) c AB . wait (&l ) ; i f ( to tal ) { t ot al −−; re tu rn tr ue ; } else { c AB . w ake O n e ( ) ; cC . w ake O n e ( ) ; re tu rn f a l s e ; }

52 53 54 55 56 57 58 59 60 61 62 63 64

}

65

//−−−−−−−−−−−−−−−

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

32 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ 66 67 68 69 70

bool M o n i to r : : a ll o w C ( ) { Q M u te x Lo cke r ml (&l ) ; w h i l e ( ( cnt A != 3 | | cnt B != 2 ) && ( to tal > 0 ) ) cC . wait (&l ) ;

71

i f ( to tal ) { t ot al −−; re tu rn tr ue ; } else { c AB . w ake O n e ( ) ; cC . w ak e O n e ( ) ; re t ur n f a l s e ; }

72 73 74 75 76 77 78 79 80 81 82 83

}

84 85 86 87 88 89 90 91 92

//−−−−−−−−−−−−−−− void M o n i to r : : d o n e A ( ) { Q M u te x Lo cke r ml (&l ) ; cnt A++; i f ( cnt A == 3 ) cC . w ake O n e ( ) ; }

93 94 95 96 97 98 99 100 101

//−−−−−−−−−−−−−−− void M o n ito r : : d o n e B ( ) { Q M u te x Lo cke r ml (&l ) ; cnt B++; i f ( cnt B == 2 ) cC . w ake O n e ( ) ; }

102 103 104 105 106 107 108 109

//−−−−−−−−−−−−−−− void M o n ito r : : d o n e C ( ) { Q M u te x Lo cke r ml (&l ) ; cnt A = c nt B = 0 ; c AB . w ake A ll ( ) ; }

110 111 112 113 114 115 116 117 118 119 120 121

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr A : p u b l i c Q Th re ad { private : M o n ito r * m ; public : Thr A ( M o n ito r * x ) : m ( x ) { } void run ( ) ; };

122 123 124 125 126 127 128 129 130 131

//−−−−−−−−−−−−−−− void T hr A : : run ( ) { while ( m−>a llo w A ( ) ) { cout << ’A ’ ; m−>d o n e A ( ) ; } }

132 133 134 135 136 137 138 139

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr B : p u b l i c Q Th re ad { private : M o n ito r * m ; public : Thr B ( M o n it o r * x ) : m ( x )

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

33 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159

M

m

160

M

161

m

162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180

M

m

181 182 183 184 185 186 187 188 189 190 191 192

15. Create 4 threads, each printing out the letters ’A’, ’B’, ’C’, and ’D’. The printing must adhere to these rules: The total number of ’A’s and ’B’s that have been output at any point in the output string cannot exceed the total number of ’C’s and ’D’s that have been output at that point.

The total number of ’A’s that have been output at any point in the output string cannot exceed twice the number of ’B’s that have been output at that point.

After a ’C’ has been output, another ’C’ cannot be output until one or more ’D’s have been output.

Solve the problem using (a) semaphores and (b) a monitor.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

34 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ Answer (a) 1 2 3 4 5 6

#in clu d e <QT hread > #i n c l u d e <QMutex> #i n c l u d e <QSemaphore> #i n c l u d e < ios t r ea m > #i n c l u d e < u ni st d . h>

7 8

u si ng namespace std ;

9

Q Se m a p h o re allo w A B ( 0 ) , p e rm it A ( 0 ) , to tal Q M u t ex p e rm it C ; bool D is O u t = tr u e ;

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70

( 100 ) ;

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr A : p u b l i c Q Th re ad { public : void run ( ) ; }; //−−−−−−−−−−−−−−− void T hr A : : run ( ) { w hi l e ( 1 ) { p e rm it A . acq u i re ( ) ; // f o r r u l e #2 a l l o w A B . ac q u ir e ( ) ; // f o r r u l e #1 i f ( ! to tal . try A cq u i re ( ) ) break ; cout << ’A ’ ; u s le e p ( 1 ) ; } } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr B : p u b l i c Q Th re ad { public : void run ( ) ; }; //−−−−−−−−−−−−−−− void T hr B : : run ( ) { w hi l e ( 1 ) { all o w A B . ac q u ire ( ) ; // f o r r u l e #1 i f ( ! to tal . try A cq u i re ( ) ) break ; cout << ’B ’ ; p e rm it A . re le ase ( 2 ) ; // f o r r u l e #2 u s le e p ( 1 ) ; } } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr C : p u b l i c Q Th re ad { public : void run ( ) ; }; //−−−−−−−−−−−−−−− void T hr C : : run ( ) { w hi l e ( 1 ) { p e rm it C . lock ( ) ; // f o r r u l e #3 i f ( ! to tal . try A cq u i re ( ) ) break ; cout << ’C ’ ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

35 71 72 73

}

74 75

D is O u t = f a l s e ; all o w A B . re le a se u s le e p ( 1 ) ;

( ) ; // f o r

r u l e #1

}

76 77 78 79 80 81 82

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr D : p u b l i c Q Th re ad { public : void run ( ) ; };

83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99

//−−−−−−−−−−−−−−− void T hr D : : run ( ) { w hi l e ( 1 ) { cout << ’D ’ ; i f ( ! to tal . try A cq u ir e ( ) ) brea k ; all o w A B . re le a se ( ) ; // f o r r u l e #1 i f ( D is O u t == f a l s e ) // f o r r u l e #3 { D is O u t = tru e ; p e rm it C . u n lo ck ( ) ; } u s le e p ( 1 ) ; }

100

// Make sur e a l l te rmina te // This i s the most a p p r o p r i a t e t hr e a d to do t h i s , as ThrD does →֓ not wait f o r any o t he r all o w A B . re le as e ( 2 ) ; p e rm it A . re le ase ( ) ; i f ( D is O u t == f a l s e ) { p e rm it C . u n lo ck ( ) ; }

101 102 103 104 105 106 107 108 109 110

}

111 112 113 114 115 116 117 118

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t main ( i n t argc , char * ar gv [ ] ) { Thr A a ; Thr B b ; Thr C c ; Thr D d ;

119

a . start ( ) ; b . sta rt ( ) ; c . st a r t ( ) ; d . run ( ) ;

120 121 122 123 124 125 126 127 128 129

}

a . wa it ( ) ; b . w ait ( ) ; c . wa it ( ) ; re t ur n 0 ;

(b) In this case, the Monitor class could be equipped with generic allow() and done() methods, that would accept as a parameter the character that is supposed to be printed, i.e. the type of thread calling. In the following code we stick with separate pairs of Monitor methods for each type of thread, to make the program easier to read.

1 2 3 4 5

#i n c l u d e <QThread> #i n c l u d e <QMutex> #i n c l u d e <QMutexLocker> #i n c l u d e <QWaitCondition> #i n c l u d e < iostrea m >

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

36 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++

M

1 1 1 1 1 1 1 1 1

M

w

1 2

wA w w wD A

2 2 2 2 2 2 2

D

2 2 3 3 3

M

w

3 3

QM

3 3 3 3 3 4 4 4 4 4 4 4 4 4 4

w

5

w

5

O O

5 5 5 5 5

M

5

w

5 5

QM

6 6 6 6 6 6 6 6 6 6 7

w

7 7

w

O O

7 7 7 7 7

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

37 78 79 80 81 82 83

bool M o n i to r : : a ll o w C ( ) { Q M u te x Lo cke r ml (&l ) ; w hi l e ( p e rm it C == f a l s e && to tal > 0 ) cC . wait (&l ) ; i f ( to tal ) { t ot al −−; re tu rn tr ue ; } else { c AB . w ake A ll ( ) ; cA . w ake O n e ( ) ; re tu rn f a l s e ; }

84 85 86 87 88 89 90 91 92 93 94 95 96

}

97

//−−−−−−−−−−−−−−− bool M o n i to r : : a ll o w D ( ) { Q M u te x Lo cke r ml (&l ) ; i f ( to tal ) { t ot al −−; re tu rn tr ue ; } else { c AB . w ake A ll ( ) ; cA . w ake O n e ( ) ; cC . w ake O n e ( ) ; re tu rn f a l s e ; } }

98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151

//−−−−−−−−−−−−−−− void M o n i to r : : d o n e A ( ) { // nothi ng to be done he re } //−−−−−−−−−−−−−−− void M o n ito r : : d o n e B ( ) { Q M u te x Lo cke r ml (&l ) ; p e rm it A += 2 ; cA . w ake O n e ( ) ; } //−−−−−−−−−−−−−−− void M o n ito r : : d o n e C ( ) { Q M u te x Lo cke r ml (&l ) ; allo w A B++; p e rm it C = f a l s e ; c AB . w ake A ll ( ) ; } //−−−−−−−−−−−−−−− void M o n ito r : : d o n e D ( ) { Q M u te x Lo cke r ml (&l ) ; i f ( p e rm it C == f a l s e ) { p e rm it C = tr ue ; cC . w ake O n e ( ) ; } allo w A B++; c AB . w ake A ll ( ) ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

38 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ 152 153 154 155 156 157 158 159 160 161

c l a s s Thr A : p u b l i c Q Th re ad { private : M o n i to r * m ; public : Thr A ( M o n ito r * x ) : m ( x ) { } void run ( ) ; };

162 163 164 165 166 167 168 169 170 171 172

//−−−−−−−−−−−−−−− void T hr A : : run ( ) { while ( m−>a llo w A ( ) ) { cout << ’A ’ ; m−>d o n e A ( ) ; u sleep ( 1 ) ; } }

173 174 175 176 177 178 179 180 181 182 183 184

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr B : p u b l i c Q Th re ad { private : M o n ito r * m ; public : Thr B ( M o n it o r * x ) : m ( x ) { } void run ( ) ; };

185 186 187 188 189 190 191 192 193 194 195

//−−−−−−−−−−−−−−− void T hr B : : run ( ) { while ( m−>a llo w B ( ) ) { cout << ’B ’ ; m−>d o n e B ( ) ; u sleep ( 1 ) ; } }

196 197 198 199 200 201 202 203 204 205 206 207

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr C : p u b l i c Q Th re ad { private : M o n ito r * m ; public : Thr C ( M o n it o r * x ) : m ( x ) { } void run ( ) ; };

208 209 210 211 212 213 214 215 216 217 218

//−−−−−−−−−−−−−−− void T hr C : : run ( ) { while ( m−>a llo w C ( ) ) { cout << ’C ’ ; m−>d o n e C ( ) ; u sleep ( 1 ) ; } }

219 220 221 222 223 224 225

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− c l a s s Thr D : p u b l i c Q Th re ad { private : M o n ito r * m ; public :

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

39 226 227 228 229 230

};

Thr D ( M o n it o r * x ) : m ( x ) { } void run ( ) ;

231 232 233 234 235 236 237 238 239 240 241

//−−−−−−−−−−−−−−− void T hr D : : run ( ) { while ( m−>a llo w D ( ) ) { cout << ’D ’ ; m−>d o n e D ( ) ; u sleep ( 1 ) ; } }

242 243 244 245 246 247 248 249 250

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t main ( i n t argc , char * ar gv [ ] ) { M o n it o r m ( 100 ) ; Thr A a (&m ) ; Thr B b (&m ) ; Thr C c (&m ) ; Thr D d (&m ) ;

251

a . start ( ) ; b . sta rt ( ) ; c . st a r t ( ) ; d . run ( ) ;

252 253 254 255 256 257 258 259 260 261

}

a . wa it ( ) ; b . w ait ( ) ; c . wa it ( ) ; re t ur n 0 ;

16. Use semaphores to solve the typical cigarette smokers problem, where the agent signals directly the smoker missing the two ingredients placed on the table.

Answer 1 2 3 4 5 6 7 8 9 10 11 12 13 14

Q

m

m w

15

A

n

16 17 18 19 20 21 22

m

23 24 25 26 27 28

m

m

m

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

40 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void Sm o ker : : run ( ) { while ( 1 ) { m i s s i n g In gr [ m i s s i n g ] . acq u ire ( ) ; // wait f o r ag ent to send →֓ signal i f ( te rm F l a g ) br eak ; // te r m i n a t i o n check cout << ” Smoker ” << msg [ m i ssi n g ] << ” i s smoking\n” ; m s l e e p ( r and ( ) % M A X S LE E P ) ; w ak e A ge n t . re le as e ( ) ; // wake up agent } } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * c l a s s A g en t : p u b l i c Q Th re ad { private : i n t r uns ; public : A gen t ( i n t ) ; void run ( ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− A gen t : : A gen t ( i n t r ) : runs ( r ) {} //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void A gen t : : run ( ) { f o r ( i n t i=0; i<runs ; i++) { i n t in gr e d s = r and ( ) % 3 ; m i s s i n g In gr [ in g re d s ] . re le ase ( ) ; w ak e A ge n t . ac q u ire ( ) ; } // s e t te r m i n a t i o n f l a g and wake up a l l t h r e ads te rm F l ag =true ; m i s s i n g In gr [ 0 ] . re le as e ( ) ; m i s s i n g In gr [ 1 ] . re le as e ( ) ; m i s s i n g In gr [ 2 ] . re le as e ( ) ; } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t main ( i n t argc , c ha r ** argv ) { Sm o ker * s [ 3 ] ; f o r ( i n t i=0; i <3; i++) { s [ i ] = new Sm o ker ( i ) ; s [ i]−> s ta rt ( ) ; } A gen t a ( atoi ( argv [ 1 ] ) ) ; a . run ( ) ;

78

f o r ( i n t i=0; i <3; i++) s [ i]−> wait ( ) ;

79 80 81 82 83

}

re t ur n E X IT _ SU C C E SS ;

17. Solve the cigarette smokers problem as described in Section 3.6.2, using semaphores.

Answer In the following solution all smoker threads wake up and check the table contents.

1 2 3 4 5

#i n c l u d e <QThread> #i n c l u d e <QMutex> #i n c l u d e <QSemaphore> #i n c l u d e <i os t r ea m > #i n c l u d e < s t d l i b . h>

6 7

u si ng namespace std ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

41 8 9 10 11 12

#d e f i n e MAXSLEEP 10 #d e f i n e TOBACCO PAPER 0 #d e f i n e TOBACCO MATHCES 1 #d e f i n e PAPER MATHCES 2

13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80

Q Se m a p h o re w a ke U p Sm o ke r [ 3 ] ; Q Se m a p h o re w ake A ge n t ( 0 ) ; v o l a t i l e bool te rm F lag=f a l s e ; i n t t ab le ; co ns t char * msg [ ] ={ ” having matches ” , ” having paper ” , ” having to bac c o ” } ; // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * c l a s s Sm o ker : p u b l i c Q Th re ad { private : i n t missin g ; public : Sm o ker ( i n t ) ; void run ( ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− Sm o ker : : Sm o ker ( i n t m ) : m i s s i n g ( m ) {} //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void Sm o ker : : run ( ) { while ( 1 ) { do { w a ke U p Sm o ke r [ m i s si n g ] . acq u ir e ( ) ; } w hi l e ( ta b le != m is s i n g && ! te rm F la g ) ;

→֓

i f ( te rm F la g ) br eak ; // te r m i n a t i o n check cout << ” Smoker ” << ms g [ m i ss i n g ] << ” i s s moking\n” ; m sl ee p ( rand ( ) % M A X SLE E P ) ; w ak e A ge n t . re le as e ( ) ; // wake up agent

} } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * c l a s s A g en t : p u b l i c Q Th re ad { private : i n t r uns ; public : A gen t ( i n t ) ; void run ( ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− A gen t : : A gen t ( i n t r ) : runs ( r ) {} //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void A gen t : : run ( ) { f o r ( i n t i=0; i<runs ; i++) { tab le = r and ( ) % 3 ; w a ke U p Sm o ke r [ 0 ] . re le as e ( ) ; w a ke U p Sm o ke r [ 1 ] . re le as e ( ) ; w a ke U p Sm o ke r [ 2 ] . re le as e ( ) ; w ak e A ge n t . ac q u ire ( ) ; } // s e t te r m i n a t i o n f l a g and wake up a l l t h r e ads te rm F l ag =true ; w a ke U p Sm o ke r [ 0 ] . re le as e ( ) ; w a ke U p Sm o ke r [ 1 ] . re le as e ( ) ; w a ke U p Sm o ke r [ 2 ] . re le as e ( ) ; } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t main ( i n t argc , c har ** argv ) { Sm o ker * s [ 3 ] ; f o r ( i n t i=0; i <3; i++) {

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

42 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ s [ i ] = new Sm o ker ( i ) ; s [ i]−> s ta rt ( ) ;

81 82

} A gen t a ( atoi ( argv [ 1 ] ) ) ; a . run ( ) ;

83 84 85 86

f o r ( i n t i=0; i <3; i++) s [ i]−> wai t ( ) ;

87 88 89 90 91

}

re t ur n E X IT _ SU C C E SS ;

18. Model the movie-going process at a multiplex cinema using a monitor.

Assume the following conditions: There are 3 different movies played at the same time in 3 halls. The capacity of each hall is 4, 5 and 7 respectively.

One hundred customers are waiting to see a randomly chosen movie.

A cashier issues the tickets.

If a hall is full a movie begins to play.

A customer cannot enter a hall while a movie is playing or while the previous viewers are exiting the hall.

A movie will play for the last customers even if the corresponding hall is not full.

Answer 1 2 3 4 5 6

#i n c l u d e <QThread> #i n c l u d e <QMutex> #i n c l u d e <QWaitCondition> #i n c l u d e <i os t r e am > #i n c l u d e <ve ct or > #i n c l u d e < s t d l i b . h>

7 8

u si ng namespace std ;

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37

co ns t i n t N U M C U ST =100; // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * c l a s s M o n it o r { private : i n t n u m H al l s ; vector <int > &cap a c i ty ; i n t * in sid e ; i n t * left ; bool * fi n i sh e d ; i n t n u m C u s to m e rs R e m a in ; Q M u tex l ; Q W a it C o n d it io n * co n d E ; // ’ e nt e r ’ ar r a y o f c o n d i t i o n s . One →֓ f o r each h a l l Q W a it C o n d it io n * co n d L ; // ’ l e a v e ’ ar r a y o f c o n d i t i o n s public : M o n ito r ( vect or <int > &cap , i n t n u m C u st ) ; ˜M o n it o r ( ) ; void e n te r T h e a te r ( i n t m o vi e ) ; void le a ve T h e a te r ( i n t m o vi e ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− M o n ito r : : M o n ito r ( vector <int > &cap , i n t n u m C u st ) : cap a ci t y ( cap ) { n u m H alls=ca p . si ze ( ) ; n u m C u st o m e rs R e m a in = n u m C u st ; in sid e = new i n t [ n u m H al ls ] ; f o r ( i n t i=0; i<n u m H a l ls ; i++) in si d e [ i ] = 0 ; left = new i n t [ n u m H al ls ] ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

43 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110

f o r ( i n t i=0; i<n u m H a l ls ; i++) le ft [ i ] = 0 ; fin is h e d = new bool [ n u m H a lls ] ; f o r ( i n t i=0; i<n u m H al l s ; i++) fi n is h e d [ i]= f a l s e ; co n d E = new Q W a it C o n d it io n [ n u m H a ll s ] ; co n d L = new Q W a it C o n d it io n [ n u m H a lls ] ;

} //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− M o n ito r : : ˜ M o n ito r ( ) { d e l e t e [ ] in si d e ; d e l e t e [ ] left ; d e l e t e [ ] fi n is h e d ; d e l e t e [ ] co n d E ; d e l e t e [ ] co n d L ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void M o n ito r : : e n te r T h e a te r ( i n t m ) { Q M u te x Lo cke r ml(&l ) ; w h i l e ( in si d e [ m ] == cap a c i t y [ m ] ) co n d E [ m ] . w ait( &l ) ; in si d e [ m ]++; i f ( in sid e [ m ] == cap a ci t y [ m ] ) // i s h a l l f u l l ? { cout << ” Movie ” << m << ” s t a r t e d \n” ; fin is h e d [ m ]= t ru e ; // s e t end−of−movie f l a g left [ m ] = 0 ; co n d L [ m ] . w ak e A ll ( ) ; // l e t a l l c u s t o m e r s l e a v e } n u m C u sto m e rs R e m ain −−; i f ( n u m C u sto m e rs R e m a in==0) // s t a r t the r e mai ni ng shows { f o r ( i n t i=0; i<n u m H al l s ; i++) i f ( in si d e [ i ] >0) { cout << ” Movie ” << i << ” s t a r t e d \n” ; fi n i sh e d [ i]= t r u e ; left [ i ] = 0 ; co n d L [ i ] . w ake A ll ( ) ; } }

} //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void M o n ito r : : le a ve T h e a te r ( i n t m ) { Q M u te x Lo cke r ml(&l ) ; w h i l e ( f in is h e d [ m]== f a l s e ) co n d L [ m ] . wai t (&l ) ; left [ m ]++; i f ( l eft [ m ] == cap a c i t y [ m ] ) // i s h a l l empty { fi n is h e d [ m ]= f a l s e ; in s id e [ m ] = 0 ; left [ m ] = 0 ; co n d E [ m ] . w ake A ll ( ) ; } } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * c l a s s C u sto m e r : p u b l i c Q Th re ad { private : i n t ID ; i n t m o v ie ID ; M o n ito r * m ; public : C u sto m e r ( i n t i , M o n i to r * c , i n t m ID ) : ID ( i ) , m o vie ID ( m ID ) , m→֓ ( c ) {}; void run ( ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void C u sto m e r : : run ( ) { cout << ” Customer ” << ID << ” wants to s e e ” << m o vie ID << ”→֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

44 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++

m

111 112

m

113 114 115 116 117 118 119 120 121 122 123 124 125

UM U

126 127

Cu

128

m

&m

129 130 131

M

132 133 134 135 136

19. Write a multi-threaded password cracker based on the Producer-Consumer paradigm. The producer should generate plain-text passwords according to a set of rules and the consumers should be hashing each password and checking whether it matches a target signature. All the threads should terminate upon the discovery of a matching password. You can use the MD5 cryptographic hash function for this exercise.

Answer The following listing shows a monitor based solution to the problem. There are two classes whose instances act as a producer (class PassGenerator) and consumers (class PassChecker).

The main program is expecting two command-line parameters : the number of password checkers and the maximum length of the brute-force generated passwords to be examined. Once a password with the same MD5 hash value is found, the finishUp method is called, which in turn terminates all remaining threads upon their next interaction with the Monitor.

1 2 3 4 5 6 7 8

#i n c l u d e < ios t r ea m > #i n c l u d e <s t r i n g > #i n c l u d e < s t r i n g . h> #i n c l u d e ”md5 . h” #i n c l u d e <QThread> #i n c l u d e <QMutex> #i n c l u d e <QMutexLocker> #i n c l u d e <QWaitCondition>

9 10

u si ng namespace std ;

11 12 13 14 15 16 17 18 19 20 21

char * al p h ab e t=” 1234567890 a b c d e f g h i j k l m n o p q r s t u v w x y z ” ; st r i n g ta rg e t ( ” 5912 d 7 bfd 10 f 631 f 1715 bf 85 bbb 72 d 97 ” ) ; // c i p h e r t e x t co nst i n t P A S S S T O R E SIZ E =100; // s i z e o f password b u f f e r // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * c l a s s M o n it o r { private : Q M u tex l ; Q W a it C o n d it io n full ; Q W a it C o n d i t i o n em p ty ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

45 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94

char * buff [ P A S S S T O R E S I Z E ] ; i n t in , out , N , N co n s ; i n t to tal ; bool fo u n d ; public : M o n it o r ( i n t c o n s u m e r s , i n t m ax L en ) ; ˜M o n it o r ( ) ; bool p u t C a n d id a te ( char * ) ; // both re tu rn f a l s e when i t time to stop bool g e t C a n d id a te ( char * ) ; v oi d fi n i sh U p ( ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− M o n ito r : : M o n ito r ( i n t c o ns u m e r s , i n t m ax Len ) { N co n s = co n su m e rs ; buff [ 0 ] = new char [ ( m ax Le n +1)* P A S S S T O R E SIZ E ] ; f o r ( i n t i=1; i<P A S S S T O R E SIZ E ; i++) buff [ i]= buff [ 0 ] +( m ax L en +1)* i ; in =0; out =0; N =0; to t al =0; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− M o n ito r : : ˜ M o n ito r ( ) { d e l e t e [ ] buff [ 0 ] ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void M o n ito r : : fi n is h U p ( ) { Q M u te x Lo cke r ml(&l ) ; fo u n d=true ; em p ty . w ake A ll ( ) ; full . w ake O n e ( ) ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− bool M o n ito r : : p u t C a n d id a te ( char * s ) { Q M u te x Lo cke r ml(&l ) ; w h i l e ( N==P A S S S T O R E SIZ E && ! fo u n d ) full . wait(&l ) ; strc p y ( buff [ in ] , s ) ; in=( in +1)%P A S S S T O R E SI Z E ; N++; to tal++; i f ( to tal %1000000==0) cout << ” Checki ng #” << to tal << ” : ” << s << endl ; em p ty . w ake O n e ( ) ; re t ur n ! fo u n d ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− bool M o n i to r : : g e t C a n d id a te ( char * s ) { Q M u te x Lo cke r ml(&l ) ; w h i l e ( N==0 && ! fo u n d ) em p ty . wait ( &l ) ; i f ( fo u n d ) re t ur n f a l s e ;

i s →֓

strc p y ( s , buf f [ out ] ) ; out=( out +1)%P A S S S T O R E SIZ E ; N−−; full . w ake O n e ( ) ; re t ur n ! fo u n d ;

} // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * c l a s s P a ss G e n e ra to r : p u b l i c Q Th re ad { private : char * can d id ate ; void r ec G en ( i n t pos , i n t r em a in ) ; M o n ito r * m ; i n t m ax Len ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

46 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ 95

bool s t o p F l a g ;

96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167

public : P a ss G e n e ra to r ( M o n i to r * x , i n t l ) ; void run ( ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− P a ss G e n e ra to r : : P a ss G e n e ra to r ( M o n ito r * x , i n t l ) : m ( x ) , m ax Len ( l ) →֓ , sto p Fla g ( f a l s e ) { can d id ate = new char [ m ax Len + 1 ] ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void P a ss G e n e ra to r : : rec G en ( i n t pos , i n t rem ain ) { i f ( r em ain==0) { can d id ate [ pos ] = 0 ; st o p Fla g = ! m−>p u t C a n d id a te ( can d id at e ) ; } else { f o r ( u ns ig n ed i n t i=0; i <strlen ( alp h ab e t ) && ! sto p Fla g ; i++) { can d id ate [ pos ] = al p h ab e t [ i ] ; rec G en ( pos +1 , remain −1) ; } } } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void P a ss G e n e ra to r : : run ( ) { i n t i=6; w h i l e ( st o p Flag==f a l s e && i<=m ax L en ) r ec G en ( 0 , i++); } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * c l a s s P a ss C h e ck e r : p u b l i c Q Th re ad { private : M o n ito r * m ; i n t m ax Len ; public : P a ss C h e cke r ( M o n ito r * x , i n t L ) : m ( x ) , m ax Len ( L ) {}; void run ( ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void P a ss C h e cke r : : run ( ) { c har tmp [ m ax Len + 1 ] ; w h i l e ( m−>g e t C a n d id a te ( tmp ) ) { i f ( t a r g e t == md 5 ( tmp ) ) { m−>f in ish U p ( ) ; cout << ” Password : ” << tmp << endl ; break ; } } } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t main ( i n t argc , char * ar gv [ ] ) { i n t Nc hk = at oi ( ar g v [ 1 ] ) ; int L = atoi ( argv [ 2 ] ) ; M o n ito r m ( Nchk , L ) ; P a ss C h e ck e r * chk [ Nc hk ] ; P a ss G e n e ra to r gen (&m , L ) ; gen . start ( ) ; f o r ( i n t i=0; i<Nchk ; i++) { chk [ i ] = new P a ss C h e cke r (&m , L ) ; chk [ i]−> st a r t ( ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

47 168 169

}

170

gen . wait ( ) ; f o r ( i n t i=0; i<Nchk ; i++) chk [ i]−> w ait ( ) ;

171 172 173

re tu rn

174 175

}

0;

20. Write a multi-threaded program for finding the prime numbers in a usersupplied range of numbers. Compare the following design approaches: (a) Split the range in equal pieces and assign each one to a thread. (b) Have a shared QAtomicInt variable that holds the next number to be checked. Threads should read and increment this number before testing it. (c) Have a shared “monitor” object that returns upon request, a range of numbers to be tested. This can be considered a generalization of the previous design.

Which of the designs is more efficient? Explain your findings.

Answer (a) In the following listing, the Repository class is used to provide thread-safe storage for the discovered prime numbers. The PrimeChecker class is used to create threads that will check all the odd numbers in the range that is specified in their constructor. The main thread actually runs one of the PrimeChecker objects to avoid having to idle while they are doing useful work.

1 2 3 4 5 6 7 8

#i n c l u d e <QThread> #i n c l u d e <QMutex> #i n c l u d e <QMutexLocker> #i n c l u d e <math . h> #i n c l u d e < s t d l i b . h> #i n c l u d e <i os t r e am > #i n c l u d e <ve ct or >

9 10

u si ng namespace std ;

11

//=========================================== // used f o r s t o r i n g the r e s u l t s in a t hr ead− s a f e c l a s s R e p o si t o ry { private : vector <int > resu lt ; Q M u tex l ; public : v oi d st o re ( i n t i ) ; vect or <int > * ge t R e su lt ( ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void R e p o s i t o ry : : sto re ( i n t i ) { Q M u te x Lo cke r ml(&l ) ; resu lt . p u sh _ b ack ( i ) ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− vector <int > * R e p o si t o ry : : ge t R e su lt ( ) { re t ur n &resu lt ; } //=========================================== c l a s s P rim e C h e cke r : p u b l i c Q Th re ad

12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

manner

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

48 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ 35 36 37 38 39 40

{ private : i n t firs t , la st ; R e p o si to ry * r epo ; public : P ri m e C h e cke r ( i n t f , i n t l , R e p o sit o ry las t ( l ) , r epo ( r ) {} void run ( ) ;

41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71

*r )

: first ( f ) , →֓

}; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void P ri m e C h e cke r : : run ( ) { f o r ( i n t i=first ; i<=last ; i+=2) { bool is P r im e=tr ue ; i n t j=3; i n t li m it = s qrt ( i ) ; whi l e ( j<= li m it && is P ri m e ) { i f ( i % j == 0 ) is P rim e=f a l s e ; j+=2; } i f ( is P rim e ) repo −>s to re ( i ) ; } } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t main ( i n t argc , char * ar gv [ ] ) { i n t n u m T h re a d s ; int a , b ; a=atoi ( a rgv [ 1 ] ) ; b=atoi ( ar gv [ 2 ] ) ; n u m T h re a d s=atoi ( a r gv [ 3 ] ) ; // make s ur e a and b ar e odd a = ( a % 2 == 0 ) ? a+1 : a ; b = ( b % 2 == 0 ) ? b−1 : b ;

72 73 74 75

i n t ra n g e P e r T h re a d = ( b − a ) * 1 . 0 / n u m T h re a d s ; // make s ur e r ange P er T hr ead i s even ra n g e P e r T h re a d = ( ra n g e P e r T h re a d % 2 == 0 ) ? →֓ ra n g e P e r T h re a d : ra n g e P e r T h re a d + 1 ;

76 77 78 79

R e p o si t o ry rep ; P r im e C h e c k e r * pc [ n u m T h re a d s ] ; i n t firs t , last=a −2; f o r ( i n t i=0; i<n u m Th read s −1; i++) { first =last +2; las t = firs t + ra n g e P e r T h re a d ; pc [ i]= new P ri m e C h e cke r ( first , last , & rep ) ; pc [ i]−> st a r t ( ) ; } pc [ n u m Th read s −1] = new P ri m e C h e cke r ( l a st +2 , b , & rep ) ; pc [ n u m Th r ead s −1]−>run ( ) ;

80 81 82 83 84 85 86 87 88 89 90 91 92

f o r ( i n t i=0; i<n u m Th read s −1; i++) pc [ i]−> w ait ( ) ;

93 94 95 96 97 98

}

cout << ” T ot al pr im es f ound ” << rep . ge t R e su lt ( )−>s ize ( ) →֓ << endl ; re tu rn 0 ;

To run the above program in order to find all the prime numbers in the range [103, 108] using eight threads, one has to use the following command-line (the same conventions are used by the next versions as well):

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

49

$

p ri m e C h e cke r_ V 3 1000 100000000 8

(b) In this version, a shared QAtomicInt object is used to fetch odd number to check from primality. The Repository class serves the same purpose as above.

1 2 3 4 5 6 7 8 9

#i n c l u d e <QThread> #i n c l u d e <QMutex> #i n c l u d e <QMutexLocker> #i n c l u d e <QAtomicInt> #i n c l u d e <math . h> #i n c l u d e < s t d l i b . h> #i n c l u d e <i os t r e am > #i n c l u d e <ve ct or >

10 11

u si ng namespace std ;

12

//=========================================== // used f o r s t o r i n g the r e s u l t s in a t hr ead− s a f e c l a s s R e p o si t o ry { private : vector <int > resu lt ; Q M u tex l ; public : v oi d st o re ( i n t i ) ; vect or <int > * ge t R e su lt ( ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void R e p o s i t o ry : : sto re ( i n t i ) { Q M u te x Lo cke r ml(&l ) ; resu lt . p u sh _ b ack ( i ) ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− vector <int > * R e p o si t o ry : : ge t R e su lt ( ) { re t ur n &resu lt ; } //=========================================== c l a s s P rim e C h e cke r : p u b l i c Q Th re ad { private : i n t last ; R e p o s i to ry * repo ; Q A to m ic In t * next ;

13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66

manner

public : P ri m e C h e cke r ( Q A to m ic In t * n , i n t l , R e p o s i t o ry ( l ) , repo ( r ) , next ( n ) {} void run ( ) ;

*r)

: la st →֓

}; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void P ri m e C h e cke r : : run ( ) { i n t can d id ate ; w h i l e ( ( c an d id at e = next−>f e t ch A n d A d d O rd e re d ( 2 ) ) <= l a st ) { bool is P r im e=tr ue ; i n t j=3; i n t li m it = sqrt ( ca n d id ate ) ; whi l e ( j<= li m it && is P ri m e ) { i f ( can d id at e % j == 0 ) is P rim e=f a l s e ; j+=2; } i f ( is P rim e ) repo −>s to re ( can d id at e ) ; } } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

50 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ 67 68

int {

main ( i n t argc , char

* ar gv [ ] )

i n t n u m T h re a d s ; int a , b ; a=atoi ( a rgv [ 1 ] ) ; b=atoi ( ar gv [ 2 ] ) ; n u m T h re a d s=atoi ( a r gv [ 3 ] ) ;

69 70 71 72 73 74

// make s ur e a and b ar e odd a = ( a % 2 == 0 ) ? a+1 : a ; b = ( b % 2 == 0 ) ? b−1 : b ;

75 76 77 78 79 80

Q A to m ic In t fi r st ( a ) ;

81

R e p o si t o ry rep ; P r im e C h e c k e r * pc [ n u m T h re a d s ] ; f o r ( i n t i=0; i<n u m Th read s −1; i++) { pc [ i]= new P rim e C h e cke r(& firs t , b , & rep ) ; pc [ i]−> st a r t ( ) ; } pc [ n u m Th read s −1] = new P ri m e C h e cke r(& fi rst , b , & rep ) ; pc [ n u m Th r ead s −1]−>run ( ) ;

82 83 84 85 86 87 88 89 90

f o r ( i n t i=0; i<n u m Th read s −1; i++) pc [ i]−> w ait ( ) ;

91 92 93 94 95 96

}

cout << ” T ot al pr im es f ound ” << rep . ge t R e su lt ( )−>s ize ( ) →֓ << endl ; re tu rn 0 ;

The overall load balancing achieved by this version is substantially better: As the numbers get larger, the checking procedure becomes more time consuming. This makes the thread assigned the last portion of the range in the previous version, dominate the execution time. (c) This version shares the load balancing efficiency of the second version, while reducing the cost of performing atomic operations. The Monitor effectively returns sets of 50 numbers to check, instead of the single one of the second version.

1 2 3 4 5 6 7 8

#i n c l u d e <QThread> #i n c l u d e <QMutex> #i n c l u d e <QMutexLocker> #i n c l u d e <QAtomicInt> #i n c l u d e <math . h> #i n c l u d e < s t d l i b . h> #i n c l u d e < ios t r ea m > #i n c l u d e <ve ct or >

9 10

u si ng namespace std ;

11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

co ns t i n t P A R T LE N =100; //=========================================== // used f o r s t o r i n g the r e s u l t s in a t hr ead− s a f e manner c l a s s R e p o si t o ry { private : vector <int > resu lt ; Q M u tex l ; public : v oi d st o re ( i n t i ) ; vect or <int > * ge t R e su lt ( ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void R e p o s i t o ry : : sto re ( i n t i ) { Q M u te x Lo cke r ml(&l ) ; resu lt . p u sh _ b ack ( i ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

51 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

} //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− vector <int > * R e p o si t o ry : : ge t R e su lt ( ) { re t ur n &resu lt ; } //=========================================== c l a s s M o n it o r { private : i n t first , last ; i n t cu r r e n t A s si g n ; i n t a s s i g n R a n ge ; Q M u tex l ; public : M o n ito r ( i n t a , i n t b , i n t r ) : fir s t ( a ) , las t ( b ) , a s s i g n R a n ge ( r ) { cu rre n t A s s i g n = a ; } void g e t N e xt R a n g e ( i n t * a , i n t * b ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void M o n it o r : : g e t N e xt R a n g e ( i n t * a , i n t * b ) { Q M u te x Lo cke r ml(&l ) ; i f ( cu rr e n t A s si g n >= l a st ) // te r m i n a t i n g check { * a = −1; re tu rn ; } * a = cu rre n t A s s i g n ; * b = cu rr e n t A s si g n + a s si g n R a n ge ; i f ( * b > last ) * b = last ; cu rre n t A s s i g n = * b + 2 ; }

→֓

//=========================================== c l a s s P rim e C h e cke r : p u b l i c Q Th re ad { private : R e p o s i to ry * repo ; M o n ito r * mon ; public : P ri m e C h e cke r ( M o n it o r * m , R e p o si t o ry * r ) : repo ( r ) , mon ( m→֓ ) {} void run ( ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void P ri m e C h e cke r : : run ( ) { i n t first , last ; while ( 1 ) { mon−>g e t N e xt R a n g e (& first , &last ) ; i f ( fi rst == −1) brea k ; // i f the Monitor r e t u r n s −1→֓ for f irst , exit f o r ( i n t i=first ; i<=last ; i+=2) { bool is P r im e=tr ue ; i n t j=3; i n t lim it = sqrt ( i ) ; whi l e ( j<= li m it && is P ri m e ) { i f ( i % j == 0 ) is P rim e=f a l s e ; j+=2; } i f ( is P rim e ) repo −>s to re ( i ) ; } }

96 97 98 99

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

52 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++

Table 3.1: Average execution times (in sec) over 10 runs, of the three prime checker variations. The best times in each case, are highlighted.

Version Even assignment to threads QAtomicInt based Monitor based

100

8 0.495 0.426 0.418

{ i n t n u m T h re a d s ; int a , b ; a=atoi ( a rgv [ 1 ] ) ; b=atoi ( ar gv [ 2 ] ) ; n u m T h re a d s=atoi ( a r gv [ 3 ] ) ;

101 102 103 104 105 106

// make s ur e a and b are a = ( a % 2 == 0 ) ? a+1 : b = ( b % 2 == 0 ) ? b−1 :

107 108 109 110

odd a; b;

M o n ito r mon ( a , b , P A R TLEN ) ; // r ange o f PARTLEN numbers →֓ , or PARTLEN/2 odd ones R e p o si t o ry rep ; P r im e C h e c k e r * pc [ n u m T h re a d s ] ; f o r ( i n t i=0; i<n u m Th read s −1; i++) { pc [ i]= new P ri m e C h e cke r(& mon , & rep ) ; pc [ i]−> st a r t ( ) ; } pc [ n u m Th r ead s −1] = new P ri m e C h e ck e r(& mon , & rep ) ; pc [ n u m Th r ead s −1]−>run ( ) ;

111 112 113 114 115 116 117 118 119 120 121

f o r ( i n t i=0; i<n u m Th read s −1; i++) pc [ i]−> w ait ( ) ;

122 123 124 125 126 127

Threads 2 4 1.14 0.61 1.04 0.54 0.92 0.46

1 1.826 2.048 1.837

}

cout << ” T ot al pr im es f ound ” << rep . ge t R e su lt ( )−>s ize ( ) →֓ << endl ; re tu rn 0 ;

The exact performance figures depend on the execution platform. On a Intel i7 3770K CPU, running under Linux and compiled with the GCC 4.8.2 compiler and -O2 optimization, we were able to measure the execution times shown in Table 3.1, for checking the range [103, 107]. With the exception of the single thread case, where the no-coordination-overheads version #1 wins, in all other cases the last, monitor-based version is superior.

21. Modify Listing ?? so that termination of the producer and consumer threads is accomplished via atomic integers. Answer This can be done by definining one atomic integer for each class: 1 2 3 4 5

numP

6

du

7 8 9 10 11

C

num

m

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

53 12

P ro duc er <T >( i n t i ) : ID ( i ) {} void o pe ra t o r ( ) ( ) ;

13 14 15 16 17 18 19 20 21 22 23 24 25 26

}; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− . . . templ ate <typename T> void P r o d u c e r <T > : : o pe r at o r ( ) ( ) { w h i l e ( tr ue ) { i n t temp = n u m P ro d u ct s . loa d ( ) ; w hi l e ( ! n u m P ro d u c ts . co m p a re _ e xc h a n g e _ w e a k ( temp , temp −1) ) temp = n u m P ro d u cts . load ( ) ; i f ( t em p <=0) re tu rn ;

27

T item = ( * p ro d u ce ) ( ) ; mon−>p ut ( it em ) ;

28 29 30 31 32

} } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

33 34 35 36 37 38 39 40 41 42

templ ate <typename T> c l a s s C o n su m e r { private : i n t ID ; s t a t i c Moni tor <T> * mon ; s t a t i c atomic <int > n u m P ro d u cts ; public : s t a t i c v oid ( * co n su m e ) ( T i ) ; s t a t i c v oid in it C l a s s ( i n t num P , Monitor <T> * m , v oid ( * c ons ) ( T→֓ ));

43

C o nsu m er <T >( i n t i ) : ID ( i ) {} void o pe ra t o r ( ) ( ) ;

44 45 46 47 48 49 50 51 52 53 54 55 56

}; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− . . . templ ate <typename T> voi d C o nsu m er <T > : : o pe rat o r ( ) ( ) { w h i l e ( tr ue ) { i n t temp = n u m P ro d u ct s . loa d ( ) ; w hi l e ( ! n u m P ro d u c ts . co m p a re _ e xc h a n g e _ w e a k ( temp , temp −1) ) temp = n u m P ro d u cts . load ( ) ; i f ( t em p <=0) re tu rn ;

57 58 59 60 61

}

T item = mon−>g et ( ) ; // take the item out ( * co n su m e ) ( item ) ;

}

22. Which type of thread is favored by the solution to the readers-writers problem in Listing ??? If either of the readers or writers are favored, modify the code to produce a fair solution.

Answer The readers are favored as once the shared mutex is locked in shared mode, there is no restriction to how many readers can enter their critical section. A fair solution is shown below: 1 2 3

m

4 5

m

6 7 8 9 10 11

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

54 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++ 12 13 14

}

// re ade r

critical

section

15 16 17 18 19 20

void W r i t e r : : o pe r at o r ( ) ( ) { fa ir . lo ck ( ) ; u n iq u e _ lo ck w r i t e G u a rd ( rw lo ck ) ; // e x c l u s i v e lo ck f o r w r i t e r s fair . u n lo ck ( ) ;

21 22 23

}

// w r i t e r c r i t i c a l

s ec tion

23. Justify the statement at the end of Section ?? about the use of the yield and sleep for methods in Listing ??. Answer The use of this thread::yield and this thread::sleep for methods results in calling the kernel and switching to another thread, after changing the state of the executing thread to ready and blocked respectively.

Switching to another fiber requires that the fiber library is called. And this can only happen if this fiber::yield and this fiber::sleep for are called in our case.

24. Devise a benchmark for testing how much faster spawing a fiber is compared to spawning a thread. Answer The folowing code: 1 2 3 4 5 6 7 8 9 10 11 12 13

numTe

14

[1]

15 16 17 18

o

19

numTe

20

th read

21

t(

[]()

22 23

{ }

24

re tu rn

0;

);

25 26

t . join ( ) ;

27 28

}

29

auto t2 = h ig h _ re so lu ti o n _ cl o ck : : now ( ) ; auto dur = t2 − t1 ; cout << ”Time per thr ea d : ” << d u ra ti o n _c a s t <m i l l is e c o n d s >( dur →֓ ) . co u n t ( ) * 1 . 0 L/ n u m T e s ts << endl ;

30 31 32 33 34 35 36 37 38 39 40 41 42

t1 = h ig h _ re so lu t io n _ clo ck : : now ( ) ; // c r e a t e the f i b e r s f o r ( i n t i = 0 ; i < n u m T e sts ; i++) { b o o st : : fib ers : : fib er f ( [ ] ( ) { re tu rn 0 ; } );

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

55 43 44 45 46 47 48 49

}

f . jo i n ( ) ; } t2 = h ig h _ re so lu t io n _ clo ck : : now ( ) ; dur = t2 − t1 ; cout << ”Time per f i b e r : ” << d u ra ti o n _c a st <m i l l i s e c o n d s >( dur ) →֓ . co u n t ( ) * 1 . 0 L/ n u m T e sts << endl ; re t ur n 0 ;

Produces this output on an AMD Ryzen 3900x (times in ms): . / fi b e r_ b e n ch 1000000 Time per th read : 0 . 015655 Time per fi b er : 0 . 001067

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

56 CHAPTER 3. THREADS AND CONCURRENCY IN STANDARD C++

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Chapter 4

Parallel data structures Exercises 1. Modify the queue structure of Listing 4.3, so that a thread trying to deque from an empty container will wait for a signal, instead of throwing an exception. Answer 2. In Figure 4.5 a possible scenario of list corruption is described if validation is not performed. Suggest other possible ways the list could be corrupted.

Answer For two threads any of the insert/erase combos that affect different neighboring nodes could be tested. For example, if Thread2 was erasing 7 instead of 11, then 9 would not be inserted in the list.

If Thread1 was inserting 8, then whem Thread1 completed its insertion, 9 would be removed from the list.

3. Someone suggests that in the lazy synchronization list, we could replace the marked flag by setting the next field to mullptr. Could this modification be a viable alternative? Answer This is not a viable alternative because another thread may reach the deleted node during the list traversal, and try to dereference the next pointer causing a protection fault.

4. Modify the stack implementation of Listing 4.16 to support the empty and size methods. Answer We need to add an atomic integer to count the number of items in the stack: 1 2 3 4 5 6

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− t e m p l at e < typename T > c l a s s st a ck_ lo ck_ fr e e { private : . . . std : : ato m ic < i n t > n u m It e m s { 0 };

7 8 9 10 11

public : bool em p ty ( ) ; . . . };

12 13 14 15 16 17 18

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− t e m p l at e < typename T > b ool st a ck_ lo ck_ fr e e < T > :: em p ty ( ) { re t ur n n u m It e m s . load ( ) ==0; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

57

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

58

CHAPTER 4. PARALLEL DATA STRUCTURES

19

tem pla te < typename T > i n t sta c k_ lo ck_ fre e < T > :: siz e ( ) { re t ur n n u m Ite m s . load ( ) ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− t e m p l at e < typename T > vo id st a ck_ lo ck_ fr e e < T > :: pus h ( co ns t T→֓ & i) { Node < T > * n e w N o d e = ge t F r e e ( ) ; new Node −>d ata = i ; new Node −>n ext = he a d . load ( ) ; w h i l e ( ! head . co m p a re _ e xc h a n g e _ w e a k ( new Node −>n ext , n e w N o d e ) ) ; n u m Ite m s++; }

20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− t e m p l at e < typename T > bo ol s ta ck_ lo ck_ f r e e < T > :: pop ( T &i ) { Node < T > * ca n d id ate = head . loa d ( ) ; w h i l e ( c an d id at e != n u ll p tr && ! head . co m p a re _ e xc h a n g e _ w e a k ( →֓ c a n d i d a t e , c a n d i d a t e −>n ext ) ) ; i f ( can d id ate == n u llp tr ) re tu rn f a l s e ; i = ca ndi dat e −>d at a ; re le a se N o d e ( can d id ate ) ; num I te ms −−; re tu rn tr ue ; }

5. In Listing 4.16 we have a stack that manages node memory allocation, one node at a time. Modify the code so that a bunch of Node instances are created initially as an array, and kept in the freeNodes list. What are the advantages and disadvantages of this solution? Answer We just need to modify the constructor and destructor of the class. The rest of the code can remain identical. The destructor needs to be modified because the initial nodes are allocated as an array and they cannot be deallocated as individual nodes.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

59 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49

}

tmp = fr e e No d e s ; w hi l e ( tmp != n u l l p t r ) { aux = tmp−>n ext ; i f ( tmp< i n i t S t o r e | | tmp>i n i t S t o r e+init No de s ) not part o f i n i t S t o r e c a l l d e l e t e d e l e t e tmp ; tmp = aux ; } tmp = head ; w hi l e ( tmp != n u l l p t r ) { aux = tmp−>n ext ; i f ( tmp< i n i t S t o r e | | tmp>i n i t S t o r e+init No de s ) not part o f i n i t S t o r e c a l l d e l e t e d e l e t e tmp ; tmp = aux ; } del ete [ ] i ni tS t or e ;

// only

i f →֓

// only

i f →֓

The advantage is that we can allocate a bunch of Node instances in one go, making the initial operation of the stack faster.

6. What would happen in the lazy synchronization list of Section ??, if an item was inserted to the list after it was erased by another thread? Answer When the thread calls the findLoc method as part of the insert, any nodes marked as deleted will be removed. As such, a new Node will be created to hold the item.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

60

CHAPTER 4. PARALLEL DATA STRUCTURES

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Chapter 5

Distributed memory programming Exercises 1. Write a MPMD version of the “Hello World” program of Figure 5.2, in effect eliminating the if/else structure around which the program is built. You may use the C or C++ bindings.

Answer Two source code files are needed:

M M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ; mm O O &num M M

M M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ; M

61

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

62

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

M P I_ St a tu s s t at u s ; M P I_ R e cv ( mess , MAX LEN , MP I _ C H A R , 0 , MES ST AG , M P I_ C O M M _ W O R LD , &→֓ sta t u s ) ; p r i n tf ( ”%i r e c e i v e d %s \n” , rank , mess ) ;

}

M P I_ Fi n a l iz e ( ) ; re t ur n 0 ;

The program can run with the following appfile:

and via the command sequence: $

$

$

m p icc h e llo _ n o d e 0 . c −o h e llo _ n o d e 0 m p icc h e llo _ n o d e _ i . c −o h e llo _ n o d e _ i m p ir u n −app h e ll o _ ap p

2. Write a SPMD version of the two programs shown in Listing 5.5.

Answer The only requirement is the inclusion of an if-else control logic.

1 2 3 4 5 6 7 8 9

MA

10 11 12

M

13

m

14 15 16

M

17 18

M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ;

19 20 21

Num

22 23

M

24 25 26 27 28 29

} else {

30 31 32 33 34 35 36

}

37 38 39 40

}

}

M P N Y_ TA G P CO _ O R LD i n t aux ; M P I_ G e t_ co u n t (&st , MP I _ C HA R , &aux ) ; buff [ aux ] = 0 ; p r i n tf ( ”%s \n” , buff ) ;

sra n d ( time ( 0 ) ) ; i n t gr ID = r and ( ) % 3 ; sp ri n tf ( buff , ”Node %i sa y s %s ” , rank , gre e t i n gs [ gr ID ] ) ; M P I_ Se n d ( buff , s t r l e n ( bu ff ) , M P I _ C H A R , 0 , 0 , →֓ M P I_ C O M M _ W O R LD ) ;

M P I_ Fin a l i z e ( ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

63 3. Modify the program shown in Listing 5.5 so that the master node prints out a list of the processes IDs for which the message has not been read yet. Your output should be similar to: 1 2 3 4

m p iru n −np 1 m as ter : −np 3 w o rker Node 2 s ays Hi . A w a i t i n g n o d es : 1 3 Node 3 s ays Hi . A w a i t i n g n o d es : 1 Node 1 s ays Hi . A w a i t i n g n o d es :

$

Answer visit Library Genesis for more solution manuals: libgen.is, libgen.st, libgen.rs, forum.mhut.org An STL set<int> container can be used to hold the ID’s of the processes still pending: 1 2 3 4 5

#i n c l u d e <mpi . h> #i n c l u d e <s t d i o . h> #i n c l u d e < s t r i n g . h> #i n c l u d e <se t > #i n c l u d e < ios t r ea m >

6 7

u si ng namespace std ;

8 9 10

#d e f i n e MAXLEN 100 c har buf f [ M A X LE N ] ;

11 12 13 14 15

i n t main ( i n t argc , c ha r { M P I_ St a tu s st ; i n t p ro c N u m ;

** a rgv )

16

M P I_ In it (& argc , &argv ) ; M P I_ C o m m _ s i z e ( M P I_ C O M M _ W O R LD , &p ro c N u m ) ; set <int > le ft O ve r ; f o r ( i n t i=1; i<p ro c N u m ; i++) le ft O ve r . in s ert ( i ) ;

17 18 19 20 21 22

w h i l e (−−p ro c N u m ) { M P I_ R e cv ( buff , MAXLE N , MP I _ C HA R , M P I_ A N Y_ SO U R C E , →֓ M P I_A N Y_TA G , M P I_ C O M M _ W O R LD , &st ) ; i n t aux ; M P I_ G e t_ co u n t (&st , MP I _ C HA R , &aux ) ; le ft O ve r . er as e ( st . M P I_ SO U R C E ) ;

23 24 25 26 27 28 29 30

buff [ aux ] = 0 ; p ri n tf ( ”%s . Awa iting nodes ” , buff ) ; f o r ( set<int > : : it e r a t o r it=le ft O ve r . b egin ( ) ; it != le ft O ve r . →֓ end ( ) ; ++it ) cout << ’ ’ << * it ; cout << ’ \n ’ ;

31 32 33 34 35 36 37 38 39

}

} M P I_ Fin a l i z e ( ) ;

4. Create a model of the characteristics of the communication link joining two processes running on two different machines, i.e. calculate the start-up latency and communication rate, by implementing and testing a ping-pong benchmark program. A ping-pong program measures the time elapsed between sending a message, having it bounce at its destination, and receiving it back at its origin. By varying the message size, you can use statistical methods (least-squares) to estimate the start-up latency and rate as the intercept and slope respectively, of the line fitted to the experimental data.

Answer

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

64

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

The following code will calculate the average communication time over REP number of tests between two processes, and for message sizes ranging from 0 to MAX MESG in increments of 1000.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

char

18

p ro ce sso r_ n a m e [ M P I_ M A X _ P R O C E S S O R _ N A M E ] ; m

19 20 21 22

M

23 24 25 26

M

27

P Comm_ P CO _ O R LD M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ;

28 29 30

m

31 32 33 34 35 36 37 38 39

m

40

m

M

M

m

MPI_CHAR O R LD

1

41 42 43 44

M

45

MP

46

R e cv P N Y_ TA G

M A X _ M ES G P CO _

→֓

47 48 49 50 51 52 53 54

M

M

OMM WO

55 56 57 58 59 60

MP

R e cv P CO

_ MP

61

M A X _ M ES G MPI_CHAR O R LD TA G EN D TA G

0

MP

N Y_ TA G

→

62 63 64

M M

G

&m

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

65

}

65

}

66 67 68 69 70 71

M P I_ C O M M _ W O R LD ) ;

}

d e l e t e [ ] b u f f er ; M P I_ Fin a l i z e ( ) ; re tu rn 0 ;

5. How would we need to modify the broadcasting program of Listing 5.9, if the source of the message was an arbitrary process, and not the one with rank 0?

Answer The solution follows the guidelines of Listing 5.9, i.e. treating the processes as nodes in a hypercube, and proceeding to propagate the message along the hypercube’s dimension one-by-one. However, the source-destination process pairings has to be done using an alternative approach. The key to the pairing calculation is the communication phase during which a node receives data. This is equal to the distance of a process/node from the root on the virtual hypercube, which can be calculated by counting the number of bits in the rank XOR root expression (variable recvPhase, line 35).

The sender node can be found by reversing the bit of a node’s rank in the recvPhase - 1 position (line 51). Once a node receives the message, it can start to send it to all the nodes which are adjacent to it (ranks different by a single bit). The ranks of the receivers can be determined by reversing the bits in a node’s rank from position recvPhase and onwards (lines 58-64).

1 2 3 4

# i nc l u d e <mpi . h> #inc lude <s t r i n g . h> #i nc lu d e <s t d i o . h> #d e f i n e MESSTAG 0

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // Returns the number o f s e t b i t s in i t s argument i n t b it C o u n t ( i n t i ) { i n t co u n t = 0 ; w h i l e ( i != 0 ) { int j = i ; i >>= 1 ; i f ( i != j ) co u n t++; } re tu rn co u n t ; }

20 21

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

22 23 24 25

i n t main ( i n t argc , c ha r ** a rgv ) { M P I_ In it (& argc , &argv ) ;

26 27 28 29 30 31 32

i n t rank , num , i ; M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ; M P I_ C o m m _ s i ze ( M P I_ C O M M _ W O R LD , &num ) ; M P I_ St a tu s st a t u s ; i n t r oot = at oi ( a r gv [ 1 ] ) ; d ou bl e dat a = r ank * 1 . 0 ; // s om et hi ng s i m p l e to e n a bl e →֓ v e r i f i c a t i o n of correct receipt

33

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

66

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

34 35 36 37 38 39 40 41 42 43 44 45 46

}

} else {

47 48 49

message p ri n tf ( ”#%i se n di ng to %i \n” , rank , d est ID ) ; M P I_ Se n d (& data , 1 , M PI_ D O U B LE , dest ID , ME SSTA G , →֓ M P I_ C O M M _ W O R LD ) ; p h ase++; d est ID = r ank ˆ ( 1 << p h as e ) ;

i n t sr c ID ; src ID = rank ˆ ( 1 << ( re cv P h as e − 1 ) ) ; p r i n tf ( ”SRC o f #%i : %i \n” , rank , sr c ID ) ;

50 51 52 53

M P I_ R e cv (& data , 1 , M PI_ D O U B LE , src ID , ME SST A G , →֓ M P I_ C O M M _ W O R LD , &s tatu s ) ;

54 55

// c a l c u l a t e the ID o f the node t hat w i l l r e c e i v e a copy the message i n t d es t ID = r ank ˆ ( 1 << re cv P h as e ) ; w h i l e ( d est ID < num ) { p ri n tf ( ”#%i se ndin to %i \n” , rank , d est ID ) ; M P I_ Se n d (& data , 1 , M PI_ D O U B LE , dest ID , ME SSTA G , →֓ M P I_ C O M M _ W O R LD ) ; re cv P h ase++; d est ID = r ank ˆ ( 1 << re cv P h as e ) ; }

56 57 58 59 60 61 62 63 64 65 66 67 68 69

}

o f →֓

} p r i n tf ( ”Node #%i has % l f \n” , rank , data ) ; M P I_ Fin a l i z e ( ) ; re tu rn 0 ;

6. Assuming that the execution platform of your program, consists of 4 machines with identical architecture but different CPU clocks: one with 4GHz, one with 3 GHz and 2 with 2 GHz. How should you split the matrix A used in the example of Section 5.11.1, in order to solve the matrix-vector product problem in the smallest possible time?

Answer The heterogeneity of the execution platform calls for an uneven partitioning of the problem’s data, i.e. we should assign a bigger portion of the problem to the faster nodes. If we assume that the computing speed of the machines is linearly dependent on their clock, and we ignore communication overheads, then if T is the time to complete the multiplication on a 1GHz machine, then on a C times faster-clock machine we would need time TC. If each node i were assigned parti percent of the M rows of the input matrix, then it would need time Ti = parti TCi to complete its part of the computation.

It can be easily proven by contradiction, that the overall execution time is minimized if Ti = Tj for 6i /= j, which translates to: Ti = Tj ⇒ parti

T Ci

T = partj

Cj

⇒ parti = part j

Ci Cj

(5.1)

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

67 As the sum of all parts should be equal to 1, we can derive the value of part0 and subsequently from Eq. 5.1 all the other parts: Σ

parti = 1 ⇒ part0

∀i

Ci − 1 Σ Ci = (Σ ) C = 1 ⇒ part0 C 0

∀i

(5.2)

0

∀i

From the problem description we have C0 = 4GHz, C1 = 3GHz, C2 = 2GHz and C3 = 2GHz, which when substituted in Eq. 5.1 and 5.2 return: 4 part0 = (

4

3 +

4

2 +

+

4

2 −1 4 ) = 4 11

4 3 part1 =

11 4

3 =

11

4 2 part2 = part3 =

11 4

2 =

11

The modified code for setting up the scattering of the matrix to the four nodes, is shown below: 1 2

d ou bl e part [ ] = { 4 . 0 / 11 , 3 . 0 / 11 , 2 . 0 / 11 , 2 . 0 / 11 } ;

3

i f ( ra nk == 0 ) { i n t d i sp ls [ N ] ;

4 5 6 7 8 9 10 11 12 13 14 15 16

i n t se n d cn ts [ N ] ; f o r ( i n t i = 0 ; i < N ; i++) { i n t ro w s Fo r P ro ce ss = M * part [ i ] ; se n d cn ts [ i ] = M * ro w s Fo r P ro ce ss ; d is p ls [ i ] = ( i == 0 ) ? 0 : d is p ls [ i − 1 ] + se n d cn ts [ i −→֓ 1]; i f ( i == N − 1 ) se n d cn ts [ i ] = M * M − d i sp ls [ i ] ; } M

A

M

OU

M

7. Write a program that performs gathering as efficiently as possible, using point-to-point communications, i.e. the equivalent of MPI Gather. What is the time complexity of your algorithm?

Answer The answer is very closely related to the solution of Exercise 5.5. The major thing that needs to be altered, is the reversal of data flow, i.e. replacing MPI Send with MPI Recv and vice-versa. The XOR operator can be still used to calculate the destination and source nodes in the communications.

The other change stems from the need to accumulate the data that are collected. Because the order of collection does not necessarily (and typically does not) match the node ranking, all the parts that are accumulated are prefixed in the same buffer with the rank of the node that contributed them. In this fashion, the designated root of the gathering operation, can reshuffle the parts and produce an outcome compatible with what MPI Gather would produce.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

68

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

The following program requires as a command-line parameter, the rank of the process/node that will gather the data.

1 2 3 4 5 6 7

# i nc l u d e <mpi . h> #inc lude <s t r i n g . h> #i nc lu d e <s t d i o . h> #inc lude <s t d l i b . h> #i nc lu d e <math . h> #d e f i n e MESSTAG 0 #d e f i n e MESSSIZE 10

8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // Returns the number o f s e t b i t s in i t s argument i n t b it C o u n t ( i n t i ) { i n t co u n t = 0 ; w h i l e ( i != 0 ) { int j = i ; i >>= 1 ; i f ( i != j ) co u n t++; } re tu rn co u n t ; }

23 24

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

25 26 27 28

i n t main ( i n t argc , char ** argv ) { M P I_ In it (& argc , &argv ) ;

29 30 31 32 33 34 35 36 37

i n t rank , num , i ; M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ; M P I_ C o m m _ s i ze ( M P I_ C O M M _ W O R LD , &num ) ; M P I_ St a tu s st a t u s ; i n t root = atoi ( argv [ 1 ] ) ; i n t da ta [ M E S S SI Z E ] ; f o r ( i n t i = 0 ; i < M E S S SI ZE ; i++) dat a [ i ] = r ank * 1 . 0 ; // s om et hin g s i m p l e to e n a bl e →֓ v e r i f i c a t i o n of correct receipt

38 39

i n t to ta l P h a se s = ceil ( log 2 ( num ) ) ; // t o t a l c o l l e c t i o n ph ase s

40 41 42 43 44 45 46 47 48 49 50 51 52 53

i n t se n d P h ase = b it C o u n t ( rank ˆ root ) ; // phase when a node →֓ f or w a r d s i t s part o f the data p ri n tf ( ”%i w i l l se nt to ro o t d ur i n g phas e %i \n” , rank , →֓ se n d P h ase ) ; // a l l o c a t e the temp . b u f f e r f o r ho l d i n g the data to be →֓ communicated towards the ” ro o t ” i n t b u ff Sp ace = ( M E S S S I Z E + 1 ) << ( to ta l P h a se s − se n d P h ase ) ; i f ( rank == root ) b u ff Sp ace = num * ( M E S S SI ZE + 1 ) ; i n t * co m m B u ff = new i n t [ b u ff Sp ace ] ; i n t in C o m m B u ff = M E S S S I Z E + 1 ; // s et u p l o c a l par t o f the comm data and the r e l e v a n t ” hea d er ” co m m B u ff [ 0 ] = r ank ; m em cp y ( co m m B u ff + 1 , data , s i z e o f ( i n t ) * M E S S S I ZE ) ; p ri n tf ( ”#%i has BUFF %i \n” , rank , b u ff Sp ac e ) ;

54 55 56 57 58 59 60 61 62 63 64

i f ( ra nk == root ) { i n t p h as e = to ta l P h a se s − 1 ; i n t src ID = rank ˆ ( 1 << p h ase ) ; w h i l e ( p h ase >= 0 ) { i f ( src ID < num ) { p r i n tf ( ”#%i re cv from %i SPACE %i \n” , rank , src ID →֓ , b u ff Sp ace − in C o m m B u ff ) ; M P I_ R e cv ( co m m B u ff + in C o m m B u ff , b u ff Sp ac e − →֓ in C o m m B u ff , MP I _I NT , src ID , M ESS TAG , →֓ M P I_ C O M M _ W O R LD , &s tatu s ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

69 i n t r ecv d ; M P I_ G e t_ co u n t (& status , MP I_INT , &recvd ) ; in C o m m B u ff += recvd ;

65 66 67 68 69 70

}

71 72

// now re− arrang e the c o l l e c t e d data i n t g a th e re d D a ta [ M E S S S I ZE * num ] ; i n t pos = 0 ; f o r ( i n t i = 0 ; i < num ; i++) { i n t d e st P a rt Id x = co m m B u ff [ pos ++]; f o r ( i n t j = 0 ; j < M E S S S I ZE ; j++) g a th e re d D a ta [ j + d e st P a rt Id x * M E S S S I Z E ] = co m m B u ff [ →֓ pos ++]; }

73 74 75 76 77 78 79 80 81 82 83 84 85 86

} else {

87 88 89

// pr i nt −out f o r v e r i f i c a t i o n p u r p os e s f o r ( i n t i = 0 ; i < M E S S S I ZE * num ; i++) p ri n tf ( ”%i ” , g a th e re d D a ta [ i ] ) ; p ri n tf ( ”\n” ) ;

i n t p h as e = to ta l P h a se s ; // c a l c u l a t e the ID o f the node t hat w i l l send i t s part the data to t h i s node i n t src ID = r ank ˆ ( 1 << ( p h as e − 1 ) ) ; // c o l l e c t from o t he r nodes f i r s t w h i l e ( p h ase > se n d P h as e ) { i f ( src ID < num ) { p r i n tf ( ”#%i re cv from %i \n” , rank , src ID ) ; M P I_ R e cv ( co m m B u ff + in C o m m B u ff , b u ff Sp ac e − →֓ in C o m m B u ff , MP I _I NT , src ID , MESS TAG , →֓ M P I_ C O M M _ W O R LD , &statu s ) ; i n t r ecv d ; M P I_ G e t_ co u n t (& status , MP I_INT , &recvd ) ; in C o m m B u ff += recvd ; } pha se −−; src ID = rank ˆ ( 1 << ( p h as e − 1 ) ) ; }

90 91 92 93 94 95 96 97 98 99

100 101 102 103 104 105 106 107

o f →֓

// then send towards the g a t h e r i ng ro o t i n t d est ID = r ank ˆ ( 1 << ( se n d P h as e − 1 ) ) ; p ri n tf ( ”DEST o f #%i : %i \n” , rank , d est ID ) ;

108 109 110 111 112

}

113 114 115 116 117

} pha se −−; src ID = ra nk ˆ ( 1 << p h ase ) ;

}

M P I_ Se n d ( comm B uf f , in C o m m B u ff , MP I _I NT , dest ID , M ESS TA G , →֓ M P I_ C O M M _ W O R LD ) ;

M P I_ Fin a l i z e ( ) ; re tu rn 0 ;

8. Write a program that performs scattering as efficiently as possible, using point-to-point communications, i.e. the equivalent of MPI Scatter. What is the volume of data collectively communicated, if each process is to receive K number of items? Express this number as a function of K and the number of processes N .

Answer The following program requires as a command-line parameter, the rank of the process/node that will scatter the data. The algorithm mirrors the broadcasting algorithm used in Exercise 5.5, as far as the tree of nodes

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

70

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

2 3,1,5,7

1,5

1

3

0,4

0 7

4

7

4

6

6

5

5 Figure 5.1: An example of a scattering operation initiated from node 2. The edges are labeled with the parts sent in a single operation. that is implicitly formed is concerned.

The major difference is that the data to be scattered are rearranged in the root node, as the data sent-down a branch of the scatter “tree” are not related to consecutive ranking nodes. Figure 5.1 illustrates a scattering operation initiated from node 2 in an eight-node communicator.

The recursive findOrder() function of lines 31-43, calculates the order with which the parts need to be rearranged, so that parts collectively sent down a tree branch, occupy a contiguous buffer space.

For the example shown in Figure 5.1, the parts residing in the scatteredData array are rearranged accordingly to form the sequence : 2 3 1 5 7 0 4 6.

1 2 3 4 5 6 7

# i nc l u d e <mpi . h> # i nc l u d e <s t r i n g . h> # i nc l u d e <s t d i o . h> # i nc l u d e <s t d l i b . h> #include <ios t r eam > # i nc l u d e <ve ct or > # i nc l u d e <math . h>

8 9

u s i n g namespace std ;

10 11 12

#d e f i n e MESSTAG 0 #d e f i n e MESSSIZE 10

13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // Returns the number o f s e t b i t s in i t s argument i n t b it C o u n t ( i n t i ) { i n t co u n t = 0 ; w h i l e ( i != 0 ) { int j = i ; i >>= 1 ; i f ( i != j ) co u n t++; } re tu rn co u n t ; }

28 29 30 31 32

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // used to c a l c u l a t e in what o r de r the data to be s c a t t e r e d →֓ s h o u l d be ar r ang ed void fi n d O rd e r ( i n t ID , i n t phase , i n t N , v ect o r < i n t >&v ) {

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

71 33 34 35 36 37 38 39

p h ase++;

40 41 42 43 44 45 46 47 48 49

M

50 51 52

M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ;

53 54 55 56 57 58 59 60 61 62

M

63 64 65

bu

num

M

66 67 68 69 70

M

71

num

72 73 74 75

O

76

num

O

77 78 79 80 81

de

82

Pa

O

e [

]

83

M

84 85

M 86 87 88 89

m em

D

M

90 91 92

M

93 94 95

M

96 97 98 99

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

72

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

100

M

101

D dest ID , MESS TAG , M P I_ C O M M _ W O R LD ) ;

102 103

p h ase++;

104 105 106 107 108

M

109 110 111

M

112 113 114 115

MP

116

Re P I_ C

D R LD ,

_

M ESSTA G P I_ S T A T S _ IG

RE ) ;

117

m em

118

D

M

119 120 121 122

bu

123

Po

M

M

D dest ID , MESS TAG , M P I_ C O M M _ W O R LD ) ;

124 125 126 127 128 129 130 131 132 133

p h ase++; to Sen d >>= 1 ;

134 135

} } // v e r i f y s c a t t e r i n g p r i n tf ( ”%i f i n i s h e d : ” , ra nk ) ; f o r ( i n t i = 0 ; i < M E S S SI ZE ; i++) p ri n tf ( ”%i ” , da ta [ i ] ) ; p ri n tf ( ”\n” ) ;

136 137 138 139 140 141 142 143 144 145 146 147 148

}

M P I_ Fin a l i z e ( ) ; d e l e t e [ ] dat a ; d e l e t e [ ] sca tte re d D a ta ; re tu rn 0 ;

The scattering is performed over ⌈lg(N ) ⌉ number of steps. In each step, the participating nodes grow exponentially, while the communicated parts shrink exponentially (by a factor of 2 in both cases).

If we number the communication phases i = 0, . . .⌈, lg(N⌉)− 1, then we have: Phase 0: participating nodes 20, data communicated K·N2 Phase 1: participating nodes 21, data communicated per node K·N22 Phase 2: participating nodes 22, data communicated per node K·N3 2

...

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

73 Therefore, the overall data volume is: ⌈lg(N )⌉−1

Σ i=0

2i

K·N = 2i+1

⌈lg(N )⌉−1

Σ i=0

K·N K·N = ⌈lg(N )⌉ 2 2

(5.3)

9. The amount of data exchanged during every step of the butterfly pattern in Figure 5.9, double in relation to the previous step. If initially every process had data of size K bytes to exchange, what is the total time required for the operation to complete, if we assume that each message exchange takes time ts + l· V , where ts is the link’s start-up latency, V is the volume of data to be sent and l is the inverse of the communication speed.

Answer The butterfly operation is completed in lg(N ) steps, with each step i = 0, . . . , lg(N )— 1, requiring the exchange of 2iK data moving in opposite directions. If we assume that there is no delay between the steps, and that communications are bi-directional, then the overall time would be equal to: lg(N )−1

Σ

t s + l · 2i K =

i=0 lg(N )−1

lg(N )ts + l · K

Σ

2i =

i=0

lg(N )ts + l · K(2lg(N) − 1) = lg(N )ts + l · K(N − 1) (5.4) 10. An alternative parallel bucket sort algorithm, would have the root process of a N -process run, scan the input data and split them into N buckets before scattering the buckets to the corresponding processes. Implement this alternative design and compare its performance with the version presented in Section 5.11.5.

Answer 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

D

M

18 19

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

74

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING M

20 21 22 23 24 25 26 27

M

28 29 30 31 32

M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ; P Comm_ P CO _ O R LD &N M

33 34 35 36 37 38 39 40 41 42

M m

m

M m

43

MAX

44

N

M

45 46 47 48

D

49

MAX

M m

m

m

m

N

50 51 52 53 54 55 56 57 58

O

59 60 61 62

M

63 64 65 66 67 68 69 70 71

MP

e

d

m

MPI_ I NT , 0 , M P I_ C O M M _ W O R LD ) ; 72

M m

O M

m

OMM WO

m

73 74 75 76

O

77

O

78 79

MP

G bu

e

de O

d 0

m MP

C O M M _ W O R LD

80 81 82

M

83 84 85 86 87 88

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

75 89

} else {

90 91 92 93 94 95 96 97

}

98 99 100 101 102 103

}

d e l e t e [ ] b u cke t O f f s e t ;

// ge t the s i z e o f the dat a to be s o r t e d l o c a l l y f i r s t M P I_ Sca tt e r ( NULL , 1 , MP I_INT , &d e li ve re d It e m s , 1 , MP I_INT , →֓ 0 , M P I_ C O M M _ W O R LD ) ; M P I_ S ca tt e rv ( NULL , NULL , NULL , MP I_INT , data , →֓ m ax Ite m s P e r B u cke t , MP I_ I NT , 0 , M P I_ C O M M _ W O R LD ) ; q so rt ( data , d e live re d It e m s , s i z e o f ( i n t ) , c omp ) ; M P I_ G a th e rv ( data , d e li ve re d Ite m s , MP I_IN T , NULL , NULL , →֓ NULL , M PI_I NT , 0 , M P I_ C O M M _ W O R LD ) ;

M P I_ Fin a l i z e ( ) ; d e l e t e [ ] dat a ; re tu rn 0 ;

11. Write a function that could be used for providing multicasting capabilities to a program, i.e. to be able to send a message to a subset of the processes of a communicator. Use an appropriate collective operation for the task.

Answer 1 2 3 4

mu

5

N

MP

D

O R LD ED

g

&mc ;

e

6 7 8 9 10 11

M

G

M

G

P f

Co

_ a R an k

MP

B

12 13 14 15 16

P P

CO _ UNDE

17 18

N

mc

19 20 21 22

M

23 24

12. Write the equivalent of the ping-pong program using RMA functions, and measure the communication speed achieved versus the size of the message used. Compare your results with the data rates accomplished with pointto-point communications.

Answer The following program prints out the average communication time over REP tests, for messages ranging in size from 0 to MAX MESG bytes, in increments of 1000.

1 2 3 4

#i n c l u d e <s t d i o . h> #i n c l u d e < s t d l i b . h> #i n c l u d e < s t r i n g . h> #i n c l u d e <mpi . h>

5 6

u si ng namespace std ;

7

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

76

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

1 1 1 1 1 1 1

char

1

p ro ce sso r_ n a m e [ M P I_ M A X _ P R O C E S S O R _ N A M E ] ; m

1 1 2 2

M

2 2

M

G

2 2

M

2

P Comm_ P CO _ O R LD M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ;

2 2 2

m

3 3 3 3 3 3 3 3 3

m em

MA

M M

M

3 4 4 4

M MP

G W

_

a

M A X _ M ES G

1

MP

FO _N U

43 44 45 46 47 48 49 50 51 52 53 54 55

i f ( ra nk == 0 ) { f o r ( i n t m e sg _ size = 0 ; m e sg_ si z e <= M A X _ M E SG ; m e sg_ si z e +=→֓ 1000 ) { sta rt_ tim e = M P I_ W tim e ( ) ; f o r ( i n t i = 0 ; i < REP ; i++) { // send M P I_ W in _ st a rt ( ot h er Pr oc , 0 , w ) ; M P I_ P u t ( buffer , m e sg _ si z e , M P I_ U N S IG N E D _ C H A R , 1 , →֓ 0 , m e s g _ s i z e , M P I _C H A R , w ) ; M P I_ W in _ co m p le te ( w ) ;

56 57 58

}

59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75

} else {

}

// wait f o r re s po nse M P I_ W in _ p o st ( ot h er Pr oc , 0 , w ) ; M P I_ W in _ w a it ( w ) ;

e n d _ t im e = M P I_ W tim e ( ) ; p ri n tf ( ”%i % l f \n” , m e s g _ s i z e , ( e n d _ ti m e − sta rt_ ti m e ) →֓ / 2 / REP ) ;

// o t he r p r o c e s s m i r r o rs what rank 0 i s doing f o r ( i n t m e s g_ si z e = 0 ; m e s g_ si z e <= M A X _ M E SG ; 1000 ) f o r ( i n t i = 0 ; i < REP ; i++) { M P I_ W in _ p o st ( ot h er Pr oc , 0 , w ) ; M P I_ W in _ w a it ( w ) ;

m e sg _ s iz e +=→֓

M P I_ W in _ st a rt ( ot h er Pr oc , 0 , w ) ; M P I_ P u t ( buff er , m e s g _ s i z e , M P I_ U N S I G N E D _ C H A R , 0 , 0 , →֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

77

76 77

}

78 79 80 81 82 83 84 85 86

}

}

mesg_size , MPI_CHAR , w ) ; M P I_ W in _ co m p le te ( w ) ;

d e l e t e [ ] b u f f er ; M P I_ W in _ fre e (&w ) ; M P I_ G ro u p _ f re e (& all ) ; M P I_ G ro u p _ fr e e (& o th e r P ro c ) ; M P I_ Fin a l i z e ( ) ; re tu rn 0 ;

13. Modify the program of Listing 5.7, so that the partitioning of the range depends on the relative speed of the participating nodes. One easy approach is to make the ranges proportional to the CPU operating frequency, or the calculated bogomips. Both numbers are available in the /proc/cpuinfo pseudo-file. The master can collect the numbers and reply back to the worker nodes with the calculated ranges. If we represent as mi the inode’s bogomips, the percent of the range αi that should be assigned to mi node i can be calculated as αi = Σ ∀k mk Answer The master node has to gather the bogomips of the execution platform machines, and calculate the appropriate ranges before sending them back to the nodes. The bogomips can be read from the /proc/cpuinfo pseudofile using a pipe as shown in line 23. The bogomips numbers are gathered by the master node (line 31) before calculating the parts αi (lines 35-41, identified as part[] in the code) and the ranges that should be assigned to each node (lines 43-51). The remaining code is identical to Listing 5.7.

1 2 3 4

#i n c l u d e <mpi . h> #i nc lu d e <s t d i o . h> #inc lude <s t d l i b . h> # i nc l u d e <s t r i n g . h>

5 6 7 8

#d e f i n e RANGEMIN 0 #d e f i n e RANGEMAX 1000 #d e f i n e MSGTAG 0

9 10 11 12 13 14 15 16 17 18

i n t main ( i n t argc , c ha r ** a rgv ) { i n t rank , num , i ; i n t ran ge [ 2 ] ; M P I_ In it (& argc , &argv ) ; M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ; M P I_ C o m m _ s i ze ( M P I_ C O M M _ W O R LD , &num ) ; M P I_ St a tu s st a t u s ;

19 20 21

f l o a t b o go M IP S ; f l o a t * all B o go = NULL ;

22 23

24 25

FILE * f = p o p en ( ”/ bin / cat / proc / c p uinf o usr / bin / head −n 1 | / usr / bin /gawk −F r”); fsca n f ( f , ”%f ” , &b o go M IP S ) ; p clo se ( f ) ;

| / bin / grep bogo | /→֓ ’ : ’ ’{ p r i n t $2 } ’ ” , ”→֓

26 27 28 29 30 31

// a rra y f o r g a t h e r ing i s only need in rank 0 i f ( rank == 0 ) all B o go = new f l o a t [ num ] ; M P I_ G a th e r (& bog o MIPS , 1 , MP I _ F LO AT , all B ogo , 1 , M P I _ F L O AT , 0 , →֓ M P I_ C O M M _ W O R LD ) ;

32

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

78

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

33 34

m

35 36 37 38 39 40

[ ]

41

B o go [

]

m B o go

42

num

43 44

M 0

45

1

46

o

47

1

w

0

1

num

48 49

[

50

]

num

RANGE

AX

[

]

w 51 52

MP

53

Reque

[ num

1]

54

MP

55

nd CO

MSGTAG OR

56 57 58 59 60 61 62 63 64 65 66 67 68

M

69

M

MSGTAG

M

OMM WOR

W

70 71 72 73 74

M

75 76

14. The butterfly communication scheme that is outlined in Section 5.11.4, is only one of the possible strategies for an all-to-all, or all-reduce data exchange. A different approach would be mandated if the underlying communication infrastructure did not provide the required links, making the procedure inefficient. An example of such an architecture is the ring, where each node is directly connected to just two others. (a) Write an MPI program that would implement an efficient all-to-all exchange of data on a ring of machines. (b) How many steps would be required in comparison to a butterfly scheme, if the number of nodes/processes were N = 2k? (c) If we assume that time taken to send V bytes over a communication link is given by l ·V , where l is the (inverse of the) link speed in sec/byte, how does your algorithm compare against the butterfly scheme in terms of overall communication time?

Answer

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

79 (a) The following program contains an allToAll function that is equivalent to MPI Alltoall. Assuming that N processes are organized in a ring, each process performs N—1 rounds/steps of forwarding its data to the next process (identified by destID) and receiving data from the previous process (identified by srcID). The data are rearranged so that the first block of NperNode items received, should reside with the process. The remaining N - round - 1 blocks are sent to the next process and so on. The data rearrangement is performed before commencing any communications in lines 24-26. The target data layout depends on the rank of the process.

The MPI datatype of the communicated data is needed so that heterogeneous platforms can be supported, while the native type size (parameter typeSize) is needed so that a properly-sized data buffer is allocated for handling the communication traffic.

The communication pattern for the allToAllGather function is made up from the same number of steps, but with a reverse flow of communications and each communication carries NperNode elements.

1 2 3 4 5 6 7

// All−to−a l l s c a t t e r i n g on a r i n g // Number o f p r o c e s s e s can be a r b i t r a r y # i nc l u d e <mpi . h> # i nc l u d e <s t r i n g . h> # i nc l u d e < u n i s t d . h> # i nc l u d e <math . h> #include <ios t r eam >

8 9 10 11

co nst i n t K = 1 0 ; co ns t i n t A LL G A T H E R T A G = 0 ; co nst i n t A L L SC A T T E R T A G = 0 ;

12 13

u si ng namespace std ;

14 15 16 17 18 19 20 21

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // a l l −to−a l l s c a t t e r i n g . o ut B u f f e r / i n B u f f e r ar e s up p os e d to →֓ have N * NperNode e le me nts / s pac e v oi d all T o A ll ( v oi d * out B u f f er , i n t Nper Node , i n t t y pe Si ze , →֓ M P I_ D a ta typ e t , voi d * in B u ffe r ) { i n t rank , N ; M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ; M P I_ C o m m _ s i ze ( M P I_ C O M M _ W O R LD , &N ) ;

22 23 24

// l o c a l −d e s t in e d part o f data c o p i e d f i r s t m em cp y ( in B u ffe r + N p e r N o d e * typ e S i z e * rank , o u t B u ffe r +→֓ N p e r N o d e * typ e Si z e * rank , N p e r N o d e * typ e Si z e ) ;

25 26 27 28 29

// re a r r a n g e l o c a l data b e f o r e s t a r t i n g to se nt unsi g ne d char * te m p B u ff = new un si g n ed char [ ( N − 1 ) * →֓ N p e r N o d e * ty p e Si z e ] ; m em cp y ( temp B uff , o u t B u ffe r + N p e r N o d e * typ e Si z e * ( rank →֓ + 1 ) , ( N − rank − 1 ) * N p e r N o d e * typ e Si z e ) ; m em cp y ( te m p B u ff + ( N − rank − 1 ) * N p e r N o d e * type Size , →֓ out B uf f er , ra nk * N p e r N o d e * ty p e Siz e ) ;

30 31 32 33 34 35 36 37 38 39

i n t ro u n d = 1 ; i n t to Sen d = ( N − 1 ) * N p e r N o d e ; i n t src ID = ( ra nk == 0 ) ? N − 1 : rank − 1 ; i n t d est ID = ( ra nk + 1 ) % N ; void * ptr 1 = t emp B uf f , * ptr 2 = out B uf f er , * aux ; whil e ( ro u n d < N ) { M P I_ Se n d ( ptr1 , to Send , t , dest ID , A LLS C A T T E R TA G , →֓ M P I_ C O M M _ W O R LD ) ; M P I_ R e cv ( ptr2 , to Send , t , src ID , A LLS C A T T E R TA G , →֓ M P I_ C O M M _ W O R LD , M P I_ S T A T U S _ IG N O R E ) ;

40

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

80

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

41 42

m

43

N

N

44 45 46 47 48 49 50

N

51

N

ro u n d++;

52 53 54 55 56 57 58 59 60 61

M

D

B

62 63 64 65

M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ; P Comm_ P CO _ O R LD &N

66 67 68

m

N

N

69 70 71 72 73 74 75 76 77 78 79 80

M

81

M P I_ C

_

R LD ,

P I_ S T A T S _ IG

RE ) ;

82 83 84 85 86

ro u n d++;

87 88 89 90 91 92 93 94

M

95 96 97 98 99

M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ; P Comm_ P CO _ O R LD &N M

100 101 102 103 104 105 106

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

81 f o r ( i n t i = 0 ; i < K * N ; i++) lo cal P art [ i ] = pow ( 10 , ra nk ) + i ;

107 108 109

all T o A ll ( l o ca l P art , K , s i z e o f ( doubl e ) , M P I_D O U B LE , →֓ all P a rts ) ;

110 111

sleep ( ra nk ) ; cout << r ank << ” : ” ; f o r ( i n t i = 0 ; i < K * N ; i++) cout << all P a rt s [ i ] << ” ” ; cout << endl ;

112 113 114 115 116 117 118 119

M P I_ B a r r i e r ( M P I_ C O M M _ W O R LD ) ;

120

i f ( ra nk == 0 ) cout << ” Test f o r a l l t o A l l G a t h e r \n” ; f o r ( i n t i = 0 ; i < K ; i++) lo cal P a rt [ i ] = ra nk ;

121 122 123 124

a ll T o A ll G a th e r ( l o ca l P ar t , K , s i z e o f ( do u ble ) , M P I_ D O U B LE , →֓ all P a rts ) ;

125 126

sleep ( ra nk ) ; cout << r ank << ” : ” ; f o r ( i n t i = 0 ; i < K * N ; i++) cout << all P a rt s [ i ] << ” ” ; cout << endl ;

127 128 129 130 131 132 133 134 135 136 137 138

}

M P I_ Fin a l i z e ( ) ; d e l e t e [ ] lo cal P ar t ; d e l e t e [ ] all P ar t s ; re tu rn 0 ;

(b) The number of steps/rounds required is N — 1 which is obviously much higher than the lgN of the butterfly scheme. However, the data transmitted in every step by the allToAllGather function is constant, while in the butterfly algorithm they grow exponentially in size by a factor of 2. (c) Assuming that the blocks destined for each process have a size of K bytes, and that the communications during a round take place simultaneously, then the total communication costs are: N −2 Σ (ring) tcomm = l · K = (N − 1) · l · K i=0

(5.5)

lgN −1

t(butterfly) = comm

Σ

l · K · 2i = l · K(2lgN − 1) = (N − 1) · l · K (5.6)

i=0

The t(butterfly) = t(ring) result may seem surprising, but there is an comm

comm

important factor that has been intentionally ignored in this setting : communication latency. The start-up overhead can be significant.

The butterfly pattern can offer substantially lower costs if this is factored-in as well.

15. The case study on Diffusion Limited Aggregation in Section 5.20, decomposes the problem’s data on a particle-wise basis. Explore the alternative of partitioning the 2D grid and assigning it to the processes involved.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

82

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

What are the benefits and drawbacks of this approach? Is the communication pattern involved different in this case? Does this approach produce similar results to the sequential program and alternative parallel programs?

Answer The benefit of a row-wise geometric decomposition is a more regular communication pattern that is also localized, instead of global. Changes to the crystal formation do not have to be broadcasted to all the nodes. The drawbacks are (a) a skewed load distribution, as the nodes handling the crystal region have in general fewer particles assigned to them, and (b) a potentially much higher communication cost that grows linearly with the number of columns used. Results should be identical to the sequential program, as long as the sequential program also grows the crystal at the end of each time step.

In the following listing (which is part of a project including the dla core.cpp file shown later), the grid is initialized in node 0 (line 74) and scattered in a row-wise fashion to the remaining nodes (line 79).

The grid local to each node is made up of two additional columns (to simplify boundary checking) and two additional rows that contain data handled by the neighboring processes. The two top and bottom rows of each local grid are exchanged with the previous and next process respectively (the first and last processes are obviously an exception). The particles that end-up in the top/bottom rows “migrate” to the previous/next process after the message exchanges taking place in line 92-93 and 99-100.

The loops of lines 113-115 and 120-122 collect those migrating particles.

The bounceUp and bounceDown variables control whether this migration is possible : It is obviously impossible for the very first and very last rows in the overall grid, because that would mean that the particles are leaving completely.

The final state of the grid is gathered by process 0 (line 134) and output to the console in the form of a PBM image (lines 138-149). A schematic of the problem decomposition is shown in Figure 5.2.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

#d e f i n e PIC SIZE 100 #d e f i n e PARTICLES 500 #d e f i n e MAX ITER 10000

16 17 18 19 20 21 22 23 24

N

R w

N

R w

num M M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

83

rowsPerProcess

Node i

cols

: local grid cells : boundary cells

} Overlapping rows, exchanged at the end of each iteration

Node i+1

} :managed by Node i :managed by Node i+1

Figure 5.2: Geometric problem decomposition illustration for the solution of Exercise 15.

25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

M P I_ C o m m _ s i ze ( M P I_ C O M M _ W O R LD , &num ) ; M P I_ R e q u e st req ; i f ( ar gc < 2 ) // use d e f a u l t v a l u e s does not s p e c i f y anythi ng { col s = P IC _ S I Z E ; r ows = P IC _ S I Z E ; it er = M A X _ IT E R ; p a r t i c l e s = P A R T IC LE S ; } else { co l s = at oi ( a r gv [ 1 ] ) ; r ows = a to i ( ar gv [ 2 ] ) ; p a r t i c l e s = at oi ( ar gv [ 3 ] ) ; iter = atoi ( a r gv [ 4 ] ) ; }

if

us e r →֓

// to make code s i m p l e r and avoid s p e c i a l cas e s , rows i s f o r c e d →֓ to be a m u l t i p l e o f // the number o f p r o c e s s e s r ows = ( i n t ) ceil ( r ows * 1 . 0 / num ) * num ; i n t ro w s P e r N o d e = r ows / num ;

47 48 49

// i n i t i a l i z e the random number ge ne r a t o r sran d ( ti me ( 0 ) ) ;

50 51 52 53 54 55 56 57 58 59 60

// g r i d has two e x t r a rows , one above and one below the a c t u a l →֓ rows // The e xtr a rows ” bel o ng ” to o the r nodes , so at the end o f →֓ each time st e p // t h e re s h o u l d be a pair − wis e re d u c t i o n o f t he s e rows i f ( r ank == 0 ) { pic = new i n t [ ( cols + 2 ) * ( r ows + 2 ) ] ; pic 2 = new i n t [ ( col s + 2 ) * ( r ows + 2 ) ] ; } else {

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

84 61 62 63 64 65 66 67 68

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

pic = new i n t [ ( cols + 2 ) * ( ro w s P e r N o d e + 2 ) ] ; pic 2 = new i n t [ ( cols + 2 ) * ( ro w s P e r N o d e + 2 ) ] ;

} bool b o u n ce U p = f a l s e , b o u n ce D o w n = f a l s e ; i f ( ra nk == 0 ) b o u n ce U p = t r u e ; e l s e i f ( r ank == num − 1 ) b o u n ce D o w n = t rue ;

69 70 71

n e xt N o d e R o w s = new i n t [ 2 p re v N o d e R o w s = new i n t [ 2

* *

( col s + 2 ) ] ; ( col s + 2 ) ] ;

72 73 74 75 76

i f ( ra nk == 0 ) d l a_ in it ( pic2 , rows + 2 , cols + 2 , p a r ti c l es , 1 ) ; // cle an pic b u f f e r m em set ( pic , 0 , ( cols + 2 ) * ( ro w s P e r N o d e + 2 ) * s i z e o f ( i n t ) ) ;

77 78 79

// boundary rows do not need to be communicated now .

M P I_ S ca t t e r ( pi c 2 + col s + 2 , ( col s + 2 ) * ro w s P er N o d e , MP I _IN T →֓ , pic + c ols + 2 , ( c ols + 2 ) * ro w s P er N o d e , MP I _I NT , 0 , →֓ M P I_ C O M M _ W O R LD ) ;

80 81 82 83 84 85 86 87 88 89 90 91 92 93 94

i n t n ext ID = r ank + 1 , p r ev ID = r ank − 1 ; w h i l e (−−iter >= 0 ) { d la _ e vo l ve ( pic , pic2 , ro w s P e r N o d e + 2 , col s + 2 , b o u n ce Up , →֓ b o u n ce D o w n ) ; tmp = pic 2 ; pic 2 = pic ; pic = tmp ; // exchang e i nfo r m a t i o n with ne i g h b o ri ng nodes // ex change with ” pr e vi o us ” i f ( ra nk != 0 ) { M P I_ I se n d ( pic , 2 * ( c ols + 2 ) , MPI _I NT , prev ID , iter , →֓ M P I_ C O M M _ W O R LD , &req ) ; M P I_ R e cv ( p rev N o d e R o w s , 2 * ( cols + 2 ) , MP I_INT , prev ID →֓ , iter , M P I_ C O M M _ W O R LD , M P I_ S T A T U S _ IG N O R E ) ; }

95 96 97 98 99 100 101

// ex change with ” next ” i f ( ra nk != num − 1 ) { M P I_ Ise n d ( pic + ro w s P e r N o d e * ( cols + 2 ) , 2 * ( col s + →֓ 2 ) , MP I _I NT , next ID , iter , M P I_ C O M M _ W O R LD , &req ) ; M P I_ R e cv ( n ext N o d e R o w s , 2 * ( cols + 2 ) , MP I_INT , next ID →֓ , iter , M P I_ C O M M _ W O R LD , M P I_ S T A T U S _ IG N O R E ) ; }

102 103 104 105 106 107 108 109 110 111

// re d u c t i o n i s d u p l ic a te d in the nodes so t hat t hey dont →֓ have to wait // what i s i m p o r t a nt in the ne i g h b o rs ’ rows , i s the ne g a t i v e →֓ cells i f ( ra nk != 0 ) { f o r ( i n t i = 0 ; i < col s + 2 ; i++) i f ( p re v N o d e R o w s [ i ] < 0 ) pic [ i ] = p re v N o d e R o w s [ i ] ; else pic [ i ] = 0 ; // r e s e t any p a r t i c l e s now on the →֓ o the r node

112 113 114 115 116 117 118 119 120 121 122

}

f o r ( i n t i = 0 ; i < col s + 2 ; i++) i f ( p re v N o d e R o w s [ col s + 2 + i ] > 0 ) pic [ cols + 2 + i ] += p re v N o d e R o w s [ cols + 2 + i ] ; // add p a r t i c l e s coming from the o t he r node

→֓

i f ( ra nk != num − 1 ) { f o r ( i n t i = 0 ; i < col s + 2 ; i++) i f ( n e xt N o d e R o w s [ i ] > 0 ) pic [ ro w s P e r N o d e * ( col s + 2 ) + i ] += n e xt N o d e R o w s [ →֓ i ]; // add p a r t i c l e s coming from the o t he r →֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

85 node 123 124 125 126 127 128

}

129 130

f o r ( i n t i = 0 ; i < col s + 2 ; i++) i f ( n e xt N o d e R o w s [ cols + 2 + i ] < 0 ) pic [ ( ro w s P e r N o d e + 1 ) * ( cols + 2 ) + i ] = →֓ n e xt N o d e R o w s [ col s + 2 + i ] ; else pic [ ( ro w s P e r N o d e + 1 ) * ( cols + 2 ) + i ] = 0 ; // r e s e t any p a r t i c l e s now on the o t he r node

→֓

}

131 132 133

M P I_ G a th e r ( pic + co ls + 2 , ro w s P e r N o d e * ( co ls + 2 ) , MP I_ IN T , pic + c ols + 2 , ro w s P e r N o d e * ( c ols + 2 ) , MPI _I NT , 0 , →֓ M P I_ C O M M _ W O R LD ) ; /* P r i nt to s t d o u t a PBM p i c t u r e o f the s im u l a t i o n space */ i f ( ra nk == 0 ) { p ri n tf ( ”P1\n%i %i \n” , cols , r ows ) ;

134

135 136 137 138 139

f o r ( y = 1 ; y <= r ows ; y++) { f o r ( x = 1 ; x <= col s ; x++) { i f ( pic [ y * ( cols + 2 ) + x ] < 0 ) p ri n tf ( ” 1 ” ) ; else p ri n tf ( ” 0 ” ) ; } p ri n tf ( ”\n” ) ; }

140 141 142 143 144 145 146 147 148 149 150

}

151 152 153 154 155 156 157 158 159

→֓

}

M P I_ Fin a l i z e ( ) ; d e l e t e [ ] pic ; d e l e t e [ ] pic 2 ; d e l e t e [ ] n e xt N o d e R o w s ; d e l e t e [ ] p re v N o d e R o w s ; re tu rn 0 ;

The dla core.cpp file that supplements the code above is: 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176

177 178 179 180 181 182 183 184 185 186

#i n c l u d e < s t d l i b . h> #i n c l u d e ” d l a c o r e . h” /*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− * Checks the pr e se nce o f a s t r u c t u r e in ne i g h b o r i n g c e l l s * pic p o i n t s to 2D ar r a y ho l d i n g data , ar r a n g e d in c o l s * columns . B ecaus e the a r r a y i s two columns and two rows * wi der than ne ce ss ary , the re i s no need to check f o r * boundary v a l u e s o f x and y . */ i n t ch e ck_ p ro xi m ( i n t * pic , i n t cols , i n t x , i n t y ) { i n t * row 0 , * row 1 , * row 2 ; row 0 = pic + ( y − 1 ) * co ls + x − 1 ; row 1 = r ow 0 + col s ; row 2 = r ow 1 + col s ; i f ( * row 0 < 0 | | * ( row 0 + 1 ) < 0 | | * ( row 0 + 2 ) < 0 | | * row 1 < →֓ 0 | | * ( row 1 + 1 ) < 0 | | * ( row 1 + 2 ) < 0 | | * row 2 < 0 | | * ( →֓ row 2 + 1 ) < 0 | | * ( row 2 + 2 ) < 0 ) re t ur n ( −1) ; else re t ur n ( 1 ) ; } /*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−*/ /* R et ur ns −1 ,0 and 1 with e qu al p r o b a b i l i t y */ i n l i n e i n t th re e _ w ay ( ) { re tu rn ( ran d o m ( ) % 3 ) − 1 ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

86

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

187 188

}

189

/*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−*/ /* I n i t i a l i z e s the 2D a rra y f o r the s im u l a t i o n */ voi d d la _ in it ( i n t * pic , i n t rows , i n t cols , i n t p a r ti cl es , i n t →֓ in it_ se e d ) { int i , j , x , y ; f o r ( i = 0 ; i < r ows ; i++) f o r ( j = 0 ; j < cols ; j++) pic [ i * cols + j ] = 0 ;

190 191 192 193 194 195 196 197 198

f o r ( i = 0 ; i < p a r t i c l e s ; i++) /* ge ne ra te i n i t i a l →֓ p a r t i c l e plac em e nt */ { x = ra n d o m ( ) % ( cols − 2 ) + 1 ; // c o u n t i n g s t a r t s from 1 y = r an d o m ( ) % ( r ows − 2 ) + 1 ; i f ( ( y == r ows / 2 + 1 ) && ( x == col s / 2 + 1 ) ) // re pe at →֓ i f true i−−; else pic [ y * co l s + x]++; }

199 200 201 202 203 204 205 206 207 208

i f ( in i t _ se e d ) pic [ ( r ows / 2 ) se e d */

209 210

*

col s + ( cols / 2 ) ] = −1;

/* p l a c e

i n i t i a l →֓

211 212

}

213 214 215 216

/*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− * S i n g l e s t e p e v o l u t i o n o f the s im u l a t i o n .

221

* * The c e l l v a l u e s r e p r e s e n t : * 0 : empty s pace * >0 : m u l t i p l e p a r t i c l e s * <0 : c r y s t a l *

222

* R et ur ns

217 218 219 220

the a d d r e ss o f the s t r u c t u r e ho l d i n g the l a s t update →֓

*/

223 224 225 226 227 228 229 230

i n t * d la _ e vo lve ( i n t * pic , i n t * pic2 , i n t rows , i n t cols , bool →֓ b o u n ce Up , bool b o u n ce D o w n ) { int x , y , k ; // pr e par e ar r a y to hold new s t a t e f o r ( y = 1 ; y < r ows − 1 ; y++) f o r ( x = 1 ; x < col s − 1 ; x++) pic 2 [ y * col s + x ] = pic [ y * col s + x ] > 0 ? 0 : pic [ y col s + x ] ;

*

→֓

231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251

f o r ( y = 1 ; y < r ows − 1 ; y++) f o r ( x = 1 ; x < col s − 1 ; x++) f o r ( k = 0 ; k < pic [ y * cols + x ] ; k++) { i n t n ew _x = x + th re e _ w ay ( ) ; i f ( n ew _x == 0 && b o u n ce U p ) n ew _x = 1 ; e l s e i f ( n ew _x == cols − 1 && b o u n ce D o w n ) n ew _x = col s − 2 ; i n t n ew _y = y + th re e _ w ay ( ) ; i f ( n ew _y == 0 ) n ew _y = 1 ; e l s e i f ( n ew _y == rows − 1 ) n ew _y = rows − 2 ; i f ( pic 2 [ n ew _y * cols + n ew _x ] > 0 ) // s t e p s i n t o →֓ empty spac e pic 2 [ n ew _y * cols + n ew _x ]++; e l s e i f ( pic 2 [ n ew _y * co ls + n ew _x ] == 0 ) // s t e p s →֓ i n t o unchecked space {

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

87 252 253

}

254 255 256

}

}

pic 2 [ n ew _y * cols + n ew _x ] = ch e ck_ p ro xi m ( pic2 , →֓ cols , ne w_ x , n ew _y ) ;

re t ur n pic 2 ;

16. Implement a 3D-space Diffusion Limited Aggregation simulation, by extending the solutions provided in Section 5.20.

Answer The project reported in Section 5.20 consists of two source code files: dla core.c and dla mpi.c. The transition to 3D requires significant changes only to the functions in dla core.c, which handle the particles as represented by the PartStr structure (which in turn gains a z coordinate).

The complete code (with a minor conversion to C++) for dla core.cpp is shown below, the major differences being limited to: (a) Lines 24-33 of the check proxim function, that now has to check 27 possible neighboring positions, scattered over 3 planes along the z-axis. (b) The calculation of the indices into the grid array (as seen in lines 69, 109 and 142), that need to take into consideration the third dimension.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

if

( p la n e 0 [ row 0 ] < 0 2] < 0 | | p la n e 0 [ r ow 1 ] < 0 2] < 0 | | p la n e 0 [ r ow 2 ] < 0 2] < 0 | | p la n e 1 [ r ow 0 ] < 0 2] < 0 | | p la n e 1 [ r ow 1 ] < 0 2] < 0 | | p la n e 1 [ r ow 2 ] < 0 2] < 0 | | p la n e 2 [ r ow 0 ] < 0 2] < 0 | | p la n e 2 [ r ow 1 ] < 0 2] < 0 | |

||

p lan e 0 [ r ow 0 + 1 ] < 0 | |

p l a n e 0 [ row 0 + →֓

||

p lan e 0 [ r ow 1 + 1 ] < 0 | |

p l a n e 0 [ row 1 + →֓

||

p lan e 0 [ r ow 2 + 1 ] < 0 | |

p l a n e 0 [ row 2 + →֓

||

p lan e 1 [ r ow 0 + 1 ] < 0 | |

p l a n e 1 [ row 0 + →֓

||

p lan e 1 [ r ow 1 + 1 ] < 0 | |

p l a n e 1 [ row 1 + →֓

||

p lan e 1 [ r ow 2 + 1 ] < 0 | |

p l a n e 1 [ row 2 + →֓

||

p lan e 2 [ r ow 0 + 1 ] < 0 | |

p l a n e 2 [ row 0 + →֓

||

p lan e 2 [ r ow 1 + 1 ] < 0 | |

p l a n e 2 [ row 1 + →֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

88

CHAPTER 5.

32 33 34 35 36

}

DISTRIBUTED MEMORY PROGRAMMING

p lan e 2 [ row 2 ] < 0 | | p lan e 2 [ r ow 2 + 1 ] < 0 | | p lan e 2 [ row 2 + →֓ 2] < 0) re t ur n ( −1) ; else re t ur n ( 1 ) ;

37 38 39 40 41 42 43 44 45 46 47 48 49 50

/*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−*/ /* R et ur ns −1 ,0 and 1 with e qu al p r o b a b i l i t y */ i n l i n e i n t th re e _ w ay ( ) { re tu rn ( ran d o m ( ) % 3 ) − 1 ; } /*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−*/ /* I n i t i a l i z e s the 3D ar r a y and ar r a y o f p a r t i c l e s f o r the →֓ s im u l a t i o n */ void d la _ in i t_ p l ist ( i n t * pic , i n t rows , i n t cols , i n t dept h , →֓ P art Str * p , i n t pa rti cl es , i n t in it _ se e d ) { i n t i , x , y , z , idx ; m em set ( pic , 0 , rows * cols * d ep th * s i z e o f ( i n t ) ) ;

51 52

f o r ( i = 0 ; i < p a r t i c l e s ; i++) /* ge ne ra te i n i t i a l →֓ p a r t i c l e plac em e nt */ { x = r an d o m ( ) % ( col s − 2 ) + 1 ; // c o u n t i n g s t a r t s from 1 y = r an d o m ( ) % ( r ows − 2 ) + 1 ; z = r an d o m ( ) % ( r ows − 2 ) + 1 ; i f ( ( y == r ows / 2 + 1 ) && ( x == cols / 2 + 1 ) && ( z == →֓ d ep th / 2 + 1 ) ) // re pe at i f true i−−; else { p[i ]. x = x; p[i ]. y = y; p[i ]. z = z; } } i f ( in i t _ se e d ) { idx = ( d ep th / 2 ) * r ows * co l s + ( r ows / 2 ) * col s + ( co l s →֓ / 2) ; pic [ idx ] = −1; /* p l a c e i n i t i a l se e d */ }

53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74

}

75

/*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− * S i n g l e s t e p e v o l u t i o n o f the s im u l a t i o n .

76 77 78 79 80 81 82 83 84

85 86 87 88 89 90 91 92 93 94 95 96 97

* * The c e l l v a l u e s r e p r e s e n t : * 0 : empty s pace * <0 : c r y s t a l −− f o r c o n s i s t e n c y with a l t e r n a t i v e * * P a r t i c l e s ar e he ld in a s e p a ra te a r r a y . */

fo r m u l a t i o n

void d la _ e vo l ve _ p l i s t ( i n t * pic , i n t rows , i n t cols , i n t depth , →֓ P art Str * p , i n t * p a r ti cl e s , P art Str * c ha nges , i n t *→֓ n u m C h a n ge s ) { int i ; * n u m C h a n ge s = 0 ; f o r ( i = 0 ; i < * p a r t i c l e s ; i++) { i n t n ew _x = p [ i ] . x + th re e _ w ay ( ) ; i f ( n ew _x == 0 ) // bounce o f f b o u n d a r i e s n ew _x = 1 ; e l s e i f ( n ew _x == co ls − 1 ) n ew _x = col s − 2 ; int

n ew _y = p [ i ] . y + th re e _ w ay

() ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

89 98

n ew _y

99

1

100 101 102

n ew

103

p[

]

e_w

104

ew

105 106

ew

107

ep

108 109 110 111 112

i f ( tu rn C r y s t a l < 0 ) { // re co rd c r y s t a l change ch an ge s [ * n u m C h a n ge s ] . x = n ew _x ; ch an ge s [ * n u m C h a n ge s ] . y = n ew _y ; ch an ge s [ * n u m C h a n ge s ] . z = n ew _z ; ( * n u m C h a n ge s ) ++;

113 114 115 116 117 118 119 120 121 122 123 124

} else {

125 126 127 128 129 130 131

}

132

}

// e r a s e p a r t i c l e from l i s t p [ i ] = p [ ( * p article s ) − 1 ] ; i−−; ( * p a r t i c l e s )−−; // change

position

to

particle

p [ i ] . x = n ew _x ; p [ i ] . y = n ew _y ; p [ i ] . z = n ew _z ;

133 134 135

}

136

/*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−*/ void a p p ly_ ch a n g e s ( i n t * pic , i n t rows , i n t cols , P art S tr chang es , i n t n u m C h a n ge s ) { int i ; f o r ( i = 0 ; i < n u m C h a n ge s ; i++) { i n t idx = ch an ge s [ i ] . z * r ows * cols + ch an ge s [ i ] . y + ch an ge s [ i ] . x ; pic [ idx ] = −1; } }

137 138 139 140 141 142 143 144 145

*

→֓

*

cols →֓

On the other hand, the contents of dla mpi.cpp file shown below, require only minor changes. These are limited to: (a) Lines 206-220, where the derived MPI datatype for PartStr is defined. (b) Lines 247-258 that output the resulting structure as a CSV file. This CSV file can be imported in ParaView (http://www.paraview.org/ Wiki/ParaView/Data_formats) for viewing purposes.

146 147 148 149 150 151 152 153 154

/* DLA 3D MPI program * P a r t i c l e s ar e e ve nly d i s t r i b u t e d among nodes / p r o c e s s e s * G. B a rla s , 2014 */ #i n c l u d e <s t d i o . h> #i n c l u d e < s t d l i b . h> #i n c l u d e < a s s e r t . h> #i n c l u d e ” d l a c o r e . h”

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

90

CHAPTER 5.

155 156

#i n c l u d e <mpi . h>

157

#d e f i n e PIC SIZE 100 #d e f i n e PARTICLES 1000 #d e f i n e MAX ITER 10000

158 159

DISTRIBUTED MEMORY PROGRAMMING

160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189

/*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−*/ i n t main ( i n t argc , char ** argv ) { i n t cols , rows , depth , iter , p a r ti cl es , x , y ; i n t * pic ; P art S tr * p , * chang es , * to ta l C h a n g e s ; i n t rank , num , i , n u m C h an ges , n u m T o ta l C h a n g e s ; i n t * ch an ge s P e r N o d e , * b u ff D isp l ; M P I_ In it (& argc , &argv ) ; M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ; M P I_ C o m m _ s i ze ( M P I_ C O M M _ W O R LD , &num ) ; i f ( ar gc < 2 ) // use d e f a u l t v a l u e s does not s p e c i f y a n y t hi n g { col s = P IC _ S I Z E + 2 ; r ows = P IC _ S I Z E + 2 ; d ep th = P IC _ S IZ E + 2 ; it er = M A X _ IT E R ; p a r t i c l e s = P A R T IC LE S ; } else { co l s = a to i ( a r gv [ 1 ] ) + 2 ; r ows = at oi ( ar gv [ 2 ] ) + 2 ; d ep th = at oi ( arg v [ 3 ] ) + 2 ; p a r t i c l e s = at oi ( ar gv [ 4 ] ) ; iter = atoi ( a r gv [ 5 ] ) ; }

use r →֓

if

190 191 192 193 194 195 196

m

197

m

198

N m

199

Ch n e

200

P

m

P

m

201 202

e

bu

D

M M

D

M

D

m

eo

num

203 204 205 206 207 208

M

209 210 211 212 213 214

M M M

215 216 217 218 219

M

220 221 222

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

91 w hi l e (−−iter ) { d la _ e vo l ve _ p l i st ( pic , rows , cols , depth , p , &→֓ p a r t i c l e s P e r N o d e , c ha ng es , &n u m C h a n ge s ) ;

223 224 225 226

// exchang e i nf o r m a t i o n with o the r nodes M P I_ A llg a th e r (& n u m C h an g es , 1 , MP I _I NT , ch an ge s P e r N o d e , MPI_INT , M P I_ C O M M _ W O R LD ) ; // c a l c u l a t e o f f s e t s n u m T o ta l C h a n g e s = 0 ; f o r ( i = 0 ; i < num ; i++) { b u ff D isp l [ i ] = n u m T o ta l C h a n g e s ; n u m T o ta l C h a n g e s += ch a n g e s P e r N o d e [ i ] ; }

227 228 229 230 231 232 233 234 235 236

i f ( n u m T o ta l C h a n g e s > 0 ) { M P I_ A llg a th e rv ( c hang es , n u m C h an g es , P oi nt , →֓ to t al C h an ges , ch an ge s P e r N o d e , buff Di spl , P oi nt , M P I_ C O M M _ W O R LD ) ; a p p l y_ ch a n g e s ( pic , rows , cols , to t al C h an g es , →֓ n u m T o ta l C h a n g e s ) ; }

237 238 239

240 241

1 , →֓

→֓

}

242 243

i f ( ra nk == 0 ) { // s ave data p o i n t s as a CSV f i l e t hat can be imp or t ed Paraview FILE * f = fo p en ( ” dl a . csv ” , ”w+t ” ) ; fp ri n tf ( f , ”x coord , y coord , z coord , s c a l a r \n” ) ; i n t idx = 0 ; f o r ( i n t z = 0 ; z < d ep th ; z++) f o r ( i n t y = 0 ; y < r ows ; y++) f o r ( i n t x = 0 ; x < co l s ; x++) { i f ( pic [ idx ] == −1) fp r i n tf ( f , ”%i , %i , %i , 1\n” , x , y , z ) ; idx++; } fcl o se ( f ) ; }

244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260

in →֓

M P I_ R e d u ce (& p a r t i c l e s P e r N o d e , &pa rti cl es , 1 , MP I _I N T , MP I _S U M , →֓ 0 , M P I_ C O M M _ W O R LD ) ; i f ( ra nk == 0 ) fp ri n tf ( stderr , ” Remaining p a r t i c l e s %i \n” , p a r t i c l e s ) ;

261 262 263 264 265 266 267 268 269 270 271 272

}

free ( pic ) ; free ( p ) ; fr ee ( ch an ge s ) ; fr ee ( ch a n g e s P e r N o d e ) ; fr ee ( b u ff D i sp l ) ; M P I_ Fin a l i z e ( ) ; re tu rn 0 ;

17. It is not unusual in NoWs setups, to have machines with different capabilities. Create an MPI program that would have each process read from a file a number of integers that is proportional to its CPU speed as indicated by the operating frequency (its “clock”). Use appropriate filetypes and views to perform the data distribution in a cyclic-block manner.

Answer Assuming that the input file is broken into BLOCKSIZE pieces 1, the solution 1Actually, it is an implicit requirement that the number of etype elements in the file is a multiple of the filetype which is in our case equal to BLOCKSIZE in length. Otherwise, the “excess” part of the file will not be read by any process.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

92

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

is assign to each node, a part of a block proportional to its clock speed.

The mathematical formulation is identical to the one used in Exercise 6 and Equations 5.1 and 5.2. Each node i creates a filetype that corresponds to a part of block equal to localSizei = ⌊parti · BLOCKSIZE⌋

(5.7)

in size, that starts at offset localOffi =

i−1 Σ

⌊partj · BLOCKSIZE⌋

(5.8)

j=0

from the beginning of a block.

In order to make sure that the sum of all parts equals BLOCKSIZE, the last of N nodes is assigned a part equal to N −2

localSizeN−1 = BLOCKSIZE −

Σ

⌊partj · BLOCKSIZE⌋

(5.9)

j=0

The resulting source code is shown below. Lines 24-31 deal with the local retrieval and global all-to-all distribution of the clock frequencies. The all-to-all scattering operation is required so that the computation of the individual parts can be duplicated in all nodes, without the need to have them communicated by the master node.

Lines 33-57 implements Equations 5.1, 5.2, 5.7 and 5.8. Lines 55, 56, executed solely on the last node of the communicator, deal with the truncations issues depicted by Equation 5.9.

Lines 65 and 66 declare a different local filetype based on the characteristics of each node. This in turn is used to read the data, one filetype at a time, in lines 77-90.

Commented-out lines 67, 81 and 87 represent alternative ways for accomplishing the desired results.

1 2 3 4 5 6 7

// Works only i f the a c t u a l Blo ck #i n c l u d e <mpi . h> # i nc l u d e <s t d i o . h> # i nc l u d e <s t d l i b . h> # i nc l u d e <s t r i n g . h> # i nc l u d e < u n i s t d . h> #i nc lu d e <ve c tor >

filesize

i s a m u l t i p l e o f the

c a l c u l a t e d →֓

8 9

u s i n g namespace std ;

10 11

#d e f i n e BLOCKSIZE 1000

12 13 14 15 16 17 18 19

i n t main ( i n t argc , char ** argv ) { i n t rank , num , i ; M P I_ In it (& argc , &argv ) ; M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ; M P I_ C o m m _ s i ze ( M P I_ C O M M _ W O R LD , &num ) ; M P I_ St a tu s st a t u s ;

20

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

93 MH

21 22 23 24

MH

25 26 27

C

28

new f o

[ num ]

29 30 31

MP

A

MHz

32 33 34

o o

mC

C

[

]

num m

35 36 37 38

[ ]

39

C

[ ]

mC

[ ]

B LO C K

40 41 42 43 44 45

O

46 47 48 49 50 51 52 53 54 55 56 57 58 59

M

60

M 61 62

M

63 64 65

MP

p

b

M

B

D

66 67 68

M

w

O M

NU

69 70 71 72 73 74

M M

75 76 77 78 79

M

80 81 82 83

M

G

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

94

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

84 85 86 87

//

88 89

}

90

pos += a ctu a l B lo ck ; f o r ( i n t i = 0 ; i < l o c a l S i z e ; i++) f o r ( i n t i = 0 ; i < cnt ; i++) data . p u sh _ b ack ( temp [ i ] ) ;

91

sleep ( ra nk ) ; cout << ra nk << ” rea d ” << data . siz e ( ) << ” numbers . ” << endl →֓ ; f o r ( i n t i = 0 ; i < 3 0 ; i++) cout << dat a [ i ] << ” ” ; cout << ” . . . . L ast one i s : ” << dat a [ dat a . siz e ( ) − 1 ] ; cout << endl ;

92 93 94 95 96 97 98 99 100 101

}

M P I_ Fin a l i z e ( ) ; re tu rn 0 ;

18. The details of the trapezoidal rule for computing a function integral are discussed in Section 3.5.2. Implement an MPI-based version of the trapezoidal rule using: (a) dynamic partitioning and (b) static partitioning.

For part (a) you can base your solution on master-worker implementation of Section 5.22.1.

Answer (a) The following listing is a derivation of the master-worker implementation of Section 5.22.1. Given the simple nature of the problem and the associated I/O, the resulting code is much simpler than the original. The only essential customizations are: the adaptation of the derived datatype used to communicate the description of a “job”, and the elimination of the “job assignment tracking” feature (supported by the assignedPart array in Listing 5.38), as in this case the results are scalar, floating point numbers that are accumulated by the master node (line 121).

1 2 3 4 5 6

#i n c l u d e <s t d i o . h> #i n c l u d e < s t d l i b . h> #i n c l u d e <i os t r e am > #i n c l u d e <math . h> #i n c l u d e <mpi . h> u si ng namespace std ;

7 8 9 10 11 12 13

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // Communication ta gs #d e f i n e NULLRESULTTAG 0 #d e f i n e RESULTTAG 1 #d e f i n e WORKITEMTAG 2 #d e f i n e ENDTAG 3

14 15 16 17 18 19 20 21

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * s t r u c t W o rk It e m { doubl e start ; doubl e end ; i n t divisio ns ; };

22 23 24 25 26 27

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− d ou bl e f unc ( d ou bl e x ) { re t u r n fa bs ( sin ( x ) ) ; }

28 29 30 31

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− doubl e in te gr C a lc ( doubl e st , doubl e en , i n t div ) {

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

95 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55

M

D

M

OUB

M

OUB

M

56 57 58 59 60 61

M

62 63 64

M

w

m

65 66 67 68 69 70 71 72 73

M

74 75 76

M

77

P Comm_ P CO _ O R LD &N M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ;

78 79 80

M

D

w

m

81 82 83 84

W

85

U

86 87

N

88

B

89 90 91 92 93 94 95 96 97 98 99 100 101 102

o

w

o U W

[N

B ] W

N

103 104 105

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

96

CHAPTER 5.

106 107 108 109 110

}

111 112

DISTRIBUTED MEMORY PROGRAMMING

w [ i ] . s ta rt = jo b St ; w [ i ] . end = jo b En d ; w [ i ] . d iv i si o n s = d iv P e r Jo b ; jo b St = jo b En d ; jo b En d += jo b S te p ;

// now d i s t r i b u t e the work item to the wor ker nodes f o r ( i n t i = 0 ; i < N JO B S ; i++) { M P I_ R e cv (& p a r t i a l R es , 1 , M P I_D O U B LE , M P I_ A N Y_ SO U R C E , →֓ M P I_A N Y_ TA G , M P I_ C O M M _ W O R LD , &s tatu s ) ; i n t w o r ke r ID = st a t u s . M P I_ SO U R C E ; i n t tag = st a t u s . M P I_ TA G ; M P I_ Ise n d (&( w [ i ] ) , 1 , w o rk Item T yp e , w ork er ID , →֓ W O R K I TEM TA G , M P I_ C O M M _ W O R LD , &re q u e st ) ; i f ( tag == R E SU LT T A G ) to t al += p a r t i a l R e s ; }

113 114 115 116 117 118 119 120 121 122 123

// now send te r m i n a t i o n messages f o r ( i n t i = 1 ; i < N ; i++) { M P I_ R e cv (& p a r t i a l R es , 1 , M P I_D O U B LE , M P I_ A N Y_ SO U R C E , →֓ M P I_A N Y_ TA G , M P I_ C O M M _ W O R LD , &s tatu s ) ; i n t w o r ke r ID = st a t u s . M P I_ SO U R C E ; i n t tag = sta tu s . M P I_ TA G ;

124 125 126 127 128 129 130 131 132 133

}

134 135 136 137 138 139 140

} else {

141

142 143 144 145

i f ( tag == R E SU LT T A G ) to t al += p a r t i a l R e s ; M P I_ Ise n d ( NULL , 0 , w o rk It e m T y p e , w or ke r ID , END TAG , →֓ M P I_ C O M M _ W O R LD , &re q u e st ) ;

cout << ” Re sult i s ” << to tal << endl ; delete [ ] w ; // worker

code

M P I_ Se n d ( NULL , 0 , M P I_D O U B LE , 0 , N U LLR E S U LT T A G , →֓ M P I_ C O M M _ W O R LD ) ; // e s t a b l i s h communication with →֓ master w hi l e ( 1 ) { W o rk It e m w ; M P I_ R e cv (&w , 1 , w o rk It e m T yp e , 0 , M P I_A N Y_ T A G , →֓ P CO _ O R LD MP EN D TA G

146 147

TA G

148 149 150

M

151

M

152 153 154 155

M

156 157

(b) The static partitioning is a much simpler proposition. Each node can query the command-line parameters and determine (without receiving any information from the “master”) the part of the integration range that is dedicate to it (lines 211, 212). Following the partial integration of line 213, with a reduction (line 216) completes the calculation.

158 159 160

#i n c l u d e <s t d i o . h> #i n c l u d e < s t d l i b . h> #i n c l u d e < ios t r ea m >

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

97 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197

#i n c l u d e <math . h> #i n c l u d e <mpi . h> u si ng namespace std ; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− d ou bl e f unc ( d ou bl e x ) { re t u r n fa bs ( sin ( x ) ) ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− doubl e in te gr C a lc ( doubl e st , doubl e en , i n t div ) { // c a l c u l a t e ar e a d ou bl e lo cal R e s = 0 ; double step = ( en − st ) / div ; doubl e x ; x = st ; lo cal R e s = func ( st ) + fun c ( en ) ; lo ca l R e s /= 2 ; f o r ( i n t i = 1 ; i < div ; i++) { x += st ep ; lo cal R e s += func ( x ) ; } lo cal R e s *= step ; re t ur n lo cal R e s ; } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t main ( i n t argc , char * ar gv [ ] ) { i n t N , rank ; do u ble LO W E R LIM IT = 0 ; doubl e U P P E R LIM IT = 1 0 ; i n t ND IV = 1 00 0 0 00 ; M P I_ In it (& argc , &argv ) ; M P I_ C o m m _ s i ze ( M P I_ C O M M _ W O R LD , &N ) ; M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ;

198 199 200 201

i f ( argc > 3 ) { LO W E R L I M IT = at of ( ar gv [ 1 ] ) ; U P P E R LI M IT = at of ( ar gv [ 2 ] ) ; NDI V = at oi ( ar gv [ 3 ] ) ; }

202 203 204 205 206 207 208

i n t d iv P e r N o d e = NDIV / N ; doubl e n o d e Ste p = ( U P P E R LIM IT − LO W E R LIM IT ) / N ; doubl e lo cal St = LO W E R LIM IT + rank * n o d e Ste p ; d ou bl e lo cal E n d = lo cal St + n o d e S t e p ; doubl e lo c al R e s = in te gr C a lc ( local St , lo c al End , d iv P e r N o d e ) ; double to tal ;

209 210 211 212 213 214 215

M P I_ R e d u ce (& lo cal Res , &total , 1 , M PI_D O U B LE , MP I _S U M , 0 , →֓ M P I_ C O M M _ W O R LD ) ; i f ( ra nk == 0 ) cout << ” Re sult i s ” << to tal << endl ;

216 217 218 219 220 221 222

}

M P I_ Fin a l i z e ( ) ; re tu rn 0 ;

19. Use the master-worker code provided in Section 5.22, as a basis for the creation of a hierarchical master-worker configuration, where nodes are organized in a 3-level (or higher) tree instead of the 2-level tree of the simple setup. The secondary master nodes of the middle tree layer(s) should be responsible for managing the load distribution in their subtrees, while the primary master at the root of the tree should be responsible for

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

98

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

the overall workload distribution.

Answer The calculation of the Mandelbrot fractal is by no means a suitable target application for this kind of setup. So this solution is more of a proof-ofconcept, than a practical way to speed-up the particular calculation.

In the listing that follows, nodes form a 3-level hierarchy, with SECONDARYMASTER nodes serving in the intermediate layer. The worker nodes calculate the secondary master they are reporting to, with the simple calculation of line 347. The major points can be summarized as follows: The master node initially distributes groups of BATCHSIZE number of work items (lines 234-242). Subsequently, it sends to each secondary master a single work item for every partial result received (lines 245255). When all the work items have been distributed, the master sends termination messages for each partial result received (lines 257266).

The secondary master nodes maintain a local pool of work items, that is initialized by the bulk work items sent by the primary master (lines 287-289). The work items are sent one-by-one to the workers that communicate with the secondary masters. The image parts/results that are sent back, are forwarded to the primary master (lines 296 and 316).

The secondary master nodes -expect to- receive a new work item for each result they send back. The new work item is appended to the local pool (line 320), so it is the next one to be sent to a worker.

The worker assignment is delayed, until the work item in question is received from the master node (lines 301-307).

Obviously, a proper, circular, FIFO queue setup would be more beneficial for the handling of each local work item pool. To avoid making an already lengthy code even longer, this is not pursued.

The worker node code is identical to the one reported in Section 5.22, with the exception of line 347.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

19

m m

20

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

99 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49

#d e f i n e SECONDARYMASTER 2 /* Number o f s e c o n d a r y master →֓ nodes */ // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * t y p e de f s t r u c t W o rk Ite m { doubl e upper X , upper Y , lower X , lo w er Y ; i n t pi x el s X , pi x el s Y , imag e X , i mag e Y , im g P art ID ; } W o rk Ite m ; // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // Cl as s f o r computing a f r a c t a l s e t part c l a s s M a n d e l C o m p u te { private : doubl e upper X , upper Y , lower X , lo w er Y ; i n t pi x el s X , pi xel s Y , im g ID ; i n t * img ; s t a t i c i n t M A X IT ER ; i n t d iv e rge ( double cx , double cy ) ; public : M a n d e l C o m p u te ( ) ; void init ( W o rk It e m * wi ) ; ˜ M a n d e l C o m p u te ( ) ; i n t * co m p u te ( ) ; }; i n t M a n d e l C o m p u te : : M A X IT E R = 2 5 5 ;

50 51

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

52

i n t M a n d e l C o m p u te : : d iv e rg e ( doubl e cx , doubl e cy ) { i n t iter = 0 ; double vx = cx , vy = cy , tx , ty ; w hi l e ( it er < M A X I TER && ( vx * vx + vy * vy ) < 4 ) { tx = vx * vx − vy * vy + cx ; ty = 2 * vx * vy + cy ; vx = tx ; vy = ty ; iter++; } re t u r n i t er ; }

53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− M a n d e l C o m p u te : : ˜ M a n d e l C o m p u te ( ) { i f ( img != N ULL ) d e l e t e [ ] img ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− M a n d e l C o m p u te : : M a n d e l C o m p u te ( ) { img = NULL ; }

80 81

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

82

void M a n d e l C o m p u te : : i nit ( W o rk Ite m * wi ) { u p p er X = wi−>u p p er X ; u p p er Y = wi−>u p p er Y ; lo w er X = wi−>l o w er X ; lo w er Y = wi−>l o w er Y ;

83 84 85 86 87 88 89 90 91 92

i f ( img == NULL | | p i xe ls X != wi−>p i xe ls X | | p i xe ls Y p ixe ls Y ) { i f ( img != N ULL ) d e l e t e [ ] img ;

!= wi−>→֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

100

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

// img has an e xtra el emen t f o r s t o r i n g the ID o f the →֓ p i c t u r e part computed img = new i n t [ ( wi−>p ixe ls X ) * ( wi−>p ixe ls Y ) + 1 ] ;

93 94 95 96 97 98

} p i xe ls X = wi−>p ixe ls X ; p i xe ls Y = wi−>p ixe ls Y ; im g ID = wi−>i m g P art ID ;

99 100

}

101 102

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

103

i n t * M a n d e l C o m p u te : : co m p u te ( ) { doubl e step x = ( lo w er X − u p p er X ) / p ixe ls X ; doubl e step y = ( u p p er Y − lo w er Y ) / p ixe ls Y ;

104 105 106 107 108 109 110 111 112 113 114 115 116 117

f o r ( i n t i = 0 ; i < p ixe ls X ; i++) f o r ( i n t j = 0 ; j < p ixe ls Y ; j++) { do u ble t empx , t em p y ; tem p x = u p p er X + i * step x ; tem p y = u p p er Y − j * step y ; img [ j * p ixe ls X + i ] = d ive rge ( tempx , tem p y ) ; } img [ p ixe ls X * p ixe ls Y ] = im g ID ; re tu rn img ;

118 119

}

120

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * void re g i s t e r W o rk I te m ( M P I_ D a ta ty p e * w o rk Ite m T yp e ) { s t r u c t W o rk It e m s am p le ;

121 122 123 124

i n t b l k l en [ 2 ] ; M P I_ A in t d isp l [ 2 ] , off , base ; M P I_ D a ta ty p e ty p es [ 2 ] ;

125 126 127 128

blklen [ 0 ] = 4 ; blklen [ 1 ] = 5 ; communicated r e g a r d l e s s

129 130

// the part ’ s l o c a t i o n

i s →֓

131

typ es [ 0 ] = M P I_ D O U B LE ; typ es [ 1 ] = M P I_ IN T ;

132 133 134

d is p l [ 0 ] = 0 ; M P I_ G e t_ a d d re ss (&( sam p le . u p p er X ) , &base ) ; M P I_ G e t_ a d d re ss (&( sa m p le . p i xe ls X ) , &off ) ; d is p l [ 1 ] = off − base ;

135 136 137 138 139 140 141

M P I_ T y p e _ cr e a te _ s t r u ct ( 2 , bl kl en , di spl , ty pes , w o rk It e m T yp e ) ; M P I_ T yp e _ co m m it ( w o rk I t e m T yp e ) ;

142 143

}

144

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // Uses the d i v e rg e nce i t e r a t i o n s to pse udo co lo r the f r a c t a l s e t void sa ve P ixe ls ( Q Im age * img , i n t * img P art , i n t image X , i n t →֓ ima ge Y , i n t height , i n t w id th ) { f o r ( i n t i = 0 ; i < w id th ; i++) f o r ( i n t j = 0 ; j < h ei g h t ; j++) { i n t co lo r = im g P art [ j * w id th + i ] ; img−>s e t P ixe l ( im ag e X + i , im ag e Y + j , q Rgb ( 256 − color , →֓ 256 − c olor , 256 − co lo r ) ) ; } }

145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t main ( i n t argc , char * ar gv [ ] ) { i n t N , rank ; double s t a r t _ t i m e , e n d _ t im e ; M P I_ St a tu s s t at u s ; M P I_ R e q u e st re q u e st ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

101 163 164

st a rt _ tim e = M P I_ W tim e ( ) ;

165

// visit Library Genesis for more solution manuals: libgen.is, libgen.st, libgen.rs, forum.mhut.org

166

M P I_ In it (& argc , &argv ) ; M P I_ C o m m _ s iz e ( M P I_ C O M M _ W O R LD , &N ) ; M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ;

167 168 169 170 171 172 173 174

i f ( N < SE C O N D A R Y M A ST E R * 2 + 1 ) { cer r << arg v [ 0 ] << ” needs at l e a s t ” << SE C O N D A R Y M A ST E R 2 + 1 << ” nodes to run\n” ; M P I_ A b o rt ( M P I_ C O M M _ W O R LD , 2 ) ; }

*

→֓

175 176 177 178 179 180 181 182 183 184

185 186

M P I_ D a ta ty p e w o rk It e m T yp e ; re g i s t e r W o rk I t e m (& w o rk I t e m T yp e ) ; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i f ( rank == 0 ) // prima ry ma ster code { i f ( argc < 6 ) { cerr << argv [ 0 ] << ” upperCornerX upperCornerY →֓ lowerCor nerX lower Cor nerY wor k Ite m P ix e ls Pe r S ide \n”→֓ ; M P I_ A b o rt ( M P I_ C O M M _ W O R LD , 1 ) ; }

187 188 189 190 191 192 193

do u ble u p p er C o rn er X , u p p e r C o rn e r Y ; doubl e lo w er C o rn er X , lo w e r C o rn e r Y ; doubl e part XS pan , p art Y Sp an ; i n t w o rk Ite m P i xe ls P e r S id e ; i n t Xpar ts , Yp ar t s ; i n t i mg X = 1024 , i mg Y = 7 6 8 ;

194 195 196 197 198 199

u p p e r C o rn e r X = atof ( a rgv [ 1 ] ) ; u p p e r C o rn e r Y = a tof ( a rgv [ 2 ] ) ; lo w e r C o rn e r X = a tof ( a rgv [ 3 ] ) ; lo w e r C o rn e r Y = a tof ( argv [ 4 ] ) ; w o rk I t e m P i x e ls P e r S id e = a toi ( argv [ 5 ] ) ;

200 201 202 203 204 205

// make s ur e t hat the image s i z e i s e ve nly d i v i d e d in work →֓ items X p a rts = ( i n t ) ceil ( img X * 1 . 0 / w o rk Ite m P ixe ls P e r S id e ) ; Yp a rts = ( i n t ) ceil ( img Y * 1 . 0 / w o rk It e m P ixe ls P e r S id e ) ; img X = X p a rts * w o rk It e m P ixe ls P e r S id e ; img Y = Yp a rts * w o rk Ite m P ixe ls P e r S id e ;

206 207 208 209

p a rt X Sp an = ( lo w e r C o rn e r X − u p p e r C o rn e r X ) / X p a rts ; p art Y Sp an = ( u p p e r C o rn e r Y − lo w e r C o rn e r Y ) / Yp a rts ; Q Im ag e * img = new Q Im ag e ( img X , img Y , Q Im ag e : : Fo rm a t_ R G B 3 2 ) →֓ ;

210 211 212 213 214 215 216

// pr e par e the work it ems in i n d i v i d u a l s t r u c t u r e s W o rk It e m * w = new W o rk It e m [ X p ar t s * Yp ar t s ] ; f o r ( i n t i = 0 ; i < X p ar t s ; i++) f o r ( i n t j = 0 ; j < Yp a r t s ; j++) { i n t idx = j * X p a rts + i ;

217

w [ idx ] . u p p er X = u p p e r C o rn e r X + i * p a rt X Sp an ; w [ idx ] . u p p er Y = u p p e r C o rn e r Y − j * p art Y Sp an ; w [ idx ] . lo w er X = u p p e r C o rn e r X + ( i + 1 ) * p a rt X Sp an ; w [ idx ] . lo w er Y = u p p e r C o rn e r Y − ( j + 1 ) * p ar t Y Sp an ;

218 219 220 221 222 223 224 225 226 227 228

}

w [ idx ] . im age X = i * w o rk I t e m P i x e ls P e r S id e ; w [ idx ] . im age Y = j * w o rk I t e m P i x e ls P e r S id e ; w [ idx ] . p ixe ls X = w o rk Ite m P ixe ls P e r S id e ; w [ idx ] . p ixe ls Y = w o rk Ite m P ixe ls P e r S id e ; w [ idx ] . im g P art ID = idx ;

229 230 231

// now d i s t r i b u t e the work item to the wor ker nodes i n t * a s s i g n e d P a rt = new i n t [ SE C O N D A R Y M A ST E R ] ; // keep →֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

102

232

233 234 235 236 237 238 239 240

241 242 243 244 245 246 247

248 249

250

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

tr a c k o f how many jo bs each s e c o n d ar y master i s →֓ as s i g n e d i n t * im g P ar t = new i n t [ w o rk I t e m P i x e ls P e r S id e * →֓ w o rk Ite m P i xe ls P e r S id e + 1 ] ; // f o r c o l l e c t i n g →֓ results i n t a s si g n e d C o u n te r = 0 ; f o r ( i n t i = 1 ; i <= SE C O N D A R Y M A ST E R ; i++) { M P I_ R e cv ( NULL , 0 , MP I _I NT , M P I_ A N Y_ SO U R C E , →֓ N U LL R ESU LTTA G , M P I_ C O M M _ W O R LD , &s tatu s ) ; i n t se c M a ste r ID = st at u s . M P I_ SO U R C E ; i n t to B e A s s i g n e d = min ( B A T C H S I Z E , X p ar t s * Yp ar t s − →֓ a s s i g n e d C o u n te r ) ; a ssi g n e d P a rt [ se c M a ste r ID ] = to B e A ss ig n e d ; M P I_ Ise n d ( w + a s s i g n e d C o u n te r , to B e A s si g n ed , →֓ w o rk It e m Typ e , s ec M as t e r ID , W O R K IT E M TA G , →֓ M P I_ C O M M _ W O R LD , &re q u e st ) ; a ssi g n e d C o u n te r += to B e A ssig n e d ; } i n t re m a in P a rt T o C o lle ct = X p a r t s * Yp a r t s ; w h i l e ( a s si g n e d C o u n te r < X p a rt s * Yp ar t s ) { M P I_ R e cv ( img P ar t , w o rk Ite m P i xe ls P e r S id e * →֓ w o rk I te m P ixe ls P e r S id e + 1 , MP I _I NT , M P I_ A N Y_ SO U R C E →֓ , R E S U L T T A G , M P I_ C O M M _ W O R LD , &st at u s ) ; i n t se c M a ste r ID = st at u s . M P I_ SO U R C E ; i n t widx = im g P a rt [ w o rk It e m P ixe ls P e r S id e * →֓ w o rk I t e m P ix e ls P e r S id e ] ; // a s s i g n e d P a rt [ workerID→֓ ]; M P I_ I se n d (&( w [ a ssi g n e d C o u n te r ] ) , 1 , w o rk I tem Ty p e , →֓ sec M as te r ID , W O R K IT E M TA G , M P I_ C O M M _ W O R LD , &re q u e st →֓ );

251 252

253 254 255 256 257 258 259

260 261 262

}

sa ve P i xe ls ( img , img P ar t , w [ widx ] . image X , w [ widx ] . →֓ image Y , w o rk It e m P i xe ls P e r Sid e , →֓ w o rk I te m P ixe ls P e r S id e ) ; a s s i g n e d C o u n te r++; re m a in P a rt T o C o lle ct −−;

w h i l e ( re m a in P a rt T o C o lle ct > 0 ) { M P I_ R e cv ( img P ar t , w o rk Ite m P i xe ls P e r S id e * →֓ w o rk I te m P ixe ls P e r S id e + 1 , MP I _I NT , M P I_ A N Y_ SO U R C E →֓ , R E S U L T T A G , M P I_ C O M M _ W O R LD , &st at u s ) ; i n t se c M a ste r ID = st at u s . M P I_ SO U R C E ; i n t widx = im g P a rt [ w o rk It e m P ixe ls P e r S id e * →֓ w o rk I t e m P ix e ls P e r S id e ] ; M P I_ Ise n d ( NULL , 0 , w o rk It em T y p e , se c M a st er ID , END TAG , →֓ M P I_ C O M M _ W O R LD , &re q u e st ) ;

263 264

265

sa ve P i xe ls ( img , img P ar t , w [ widx ] . image X , w [ widx ] . →֓ w m w m ma Pa ToCo

266 267 268 269 270

w

271 272 273 274 275 276 277 278

OND

279

M

280 281

w

m

282

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

103 w

283

w

m

m

284 285

int poolCount = 0;

286

// ge t batch o f l oad from pri mary master M P I_ Se n d ( NULL , 0 , MP I _I NT , 0 , N U LLR E S U LT T A G , →֓ M P I_ C O M M _ W O R LD ) ; // e s t a b l i s h communication with pri mary master M P I_ R e cv ( l o ca l P ool , B AT C HS I Z E , w o rk Item T yp e , 0 , →֓ W O R K I TEM TA G , M P I_ C O M M _ W O R LD , &statu s ) ; M P I_ G e t_ co u n t (& status , w o rk Item T yp e , &p o o l C o u n t ) ; // ge t the s i z e o f the assignment

287

288 289

→֓

→֓

290 291 292 293 294 295 296

297 298 299

M P I_ R e q u e st jo b R eq ; bool w ait Fla g = f a l s e ; i n t p e n d in g R e su lt s = 0 ; while ( p oo lC o u nt > 0) { M P I_ R e cv ( img P ar t , w o rk Ite m P i xe ls P e r S id e * →֓ w o rk I te m P ixe ls P e r S id e + 1 , MP I _I NT , M P I_ A N Y_ SO U R C E →֓ , M P I_A N Y_ TA G , M P I_ C O M M _ W O R LD , &statu s ) ; i n t w o r ke r ID = st a t u s . M P I_ SO U R C E ; i n t tag = sta tu s . M P I_ TA G ; // wait f o r j ob ( s ) to be r e c e i v e d from the MPI Ir ecv →֓ c a l l f o l l o w i n g below i f ( w ait Fl a g ) { M P I_ St a tu s st 2 ; M P I_ W ait (& job Req , &st 2 ) ; i f ( st 2 . M P I_ TA G == W O R K IT E M T A G ) p o o l C o u n t++; } pool C ount −−; p e n d in g R e su lts++; M P I_ Ise n d ( lo cal P o o l + poo l C ount , 1 , w o rk It e m T yp e , →֓ wo rk e r ID , W O R K I T E M TA G , M P I_ C O M M _ W O R LD , &re q u e st ) ; i f ( tag == R E SU LT T A G ) { p e n d in g R e su lt s −−;

300 301 302 303 304 305 306 307 308 309 310 311 312 313 314

// f or war d r e s u l t s to pr imar y master M P I_ Se n d ( img P ar t , w o rk Ite m P ixe ls P e r S id e * →֓ w o rk Ite m P i xe ls P e r S id e + 1 , MPI_INT , 0 , →֓ R E S U L T T A G , M P I_ C O M M _ W O R LD ) ;

315 316

317 318 319 320 321

}

322 323

// ge t new work item a f t e r the co rre spo n d i ng memory→֓ i s f r e e to acce pt new data M P I_ W ait (& r eq u es t , &st at u s ) ; M P I_ Ir e cv ( lo cal P o o l + poo l C ount , 1 , w o rk It e m Typ e , →֓ 0 , M P I_A N Y_ TA G , M P I_ C O M M _ W O R LD , &jo b R eq ) ; w ait Fla g = tr ue ;

}

324 325 326 327 328

329 330 331 332 333 334 335 336 337 338

// now send te r m i n a t i o n messages w h i l e ( p e n d in g R e su lts > 0 ) { M P I_ R e cv ( img P ar t , w o rk Ite m P i xe ls P e r S id e * →֓ w o rk I te m P ixe ls P e r S id e + 1 , MP I _I NT , M P I_ A N Y_ SO U R C E →֓ , M P I_A N Y_ TA G , M P I_ C O M M _ W O R LD , &statu s ) ; i n t w o r ke r ID = st a t u s . M P I_ SO U R C E ; i n t tag = sta tu s . M P I_ TA G ; M P I_ Ise n d ( NULL , 0 , w o rk It e m T y p e , w or ke r ID , END TAG , →֓ M P I_ C O M M _ W O R LD , &re q u e st ) ; i f ( tag == R E SU LT T A G ) { p e n d in g R e su lt s −−; // f or war d r e s u l t s to pr imar y master M P I_ Se n d ( img P art , w o rk It e m P i xe ls P e r S id e * →֓ w o rk I te m P i xe ls P e r S id e + 1 , MP I_IN T , 0 , →֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

104

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

RESULTTAG ,

M P I_ C O M M _ W O R LD ) ;

339 340

d e l e t e [ ] im g P art ; d e l e t e [ ] lo c al P o o l ;

341 342 343 344 345 346 347 348 349

350 351 352 353

354 355 356

} //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− else // worker code { i n t m y M a st e r ID = ( r ank % SE C O N D A R Y M A ST E R ) + 1 ; // which →֓ master to r e p o r t to M a n d e l C o m p u te c ; M P I_ Se n d ( NULL , 0 , MP I _I NT , m y M as t er ID , N U LLR E S U LT T A G , →֓ M P I_ C O M M _ W O R LD ) ; // e s t a b l i s h communication with →֓ master w hi l e ( 1 ) { W o rk It e m w ; M P I_ R e cv (&w , 1 , w o rk It e m T y p e , m y M as t er ID , M P I_A N Y_ TA G , →֓ M P I_ C O M M _ W O R LD , & st at u s ) ; // ge t a new work →֓ item i n t tag = st a t u s . M P I_ TA G ; i f ( tag == EN D TA G ) break ;

357 358 359 360

361

}

362

}

c . init (&w ) ; i n t * res = c . co m p u te ( ) ; M P I_ Se n d ( res , w . p i xe ls X * w . p i xe ls Y + 1 , MP I _I NT , →֓ m y M a st er ID , R E S U L T T A G , M P I_ C O M M _ W O R LD ) ; // →֓ re t ur n the r e s u l t s

363 364 365 366

}

M P I_ Fi n a l iz e ( ) ; re t ur n 0 ;

20. Modify the multi-threaded master-worker code of Section 5.22.2 so that there are two separate threads in each worker process for communication : one for receiving work items and one for sending back results. What are the benefits and drawbacks of this arrangement?

Answer The benefits are two-fold: (a) The decoupled “sender” and “receiver” codes are simpler to write and maintain. The less complicated logic is evident, as a simple comparison to Listing 5.40 can attest. (b) The two communication operations can proceed at different speeds, potentially improving the utilization of the computing resources.

The modifications required are limited to the part of the main function that deal with worker communications (lines 391-436 of Listing 5.40). The additional ResultSender class shown below, is used to spawn a thread that handles communication to the master node exclusively, leaving the main thread to handle communication from the master node.

In terms of data items, the only change affects the assigned counter, which now needs to be changed from two threads. Instead of using a mutex to control access to it, its type is changed to QAtomicInt. The following listing contains only the changes/additions to Listing 5.40, in order to reduce clutter.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

105 1 2 3 4 5 6 7 8 9

10 11

... c l a s s R e su lt Se n d e r : p u b l i c Q Th re ad { private : Q u e u e M o n ito r < M a n d e l R e su lt * >* o u tq u e ; i n t n u m W o r k e r T h re a d s ; Q A to m ic In t * a ssi gn e d ; public : R e su lt Se n d e r ( Q u e u e M o n ito r < M a n d e l R e su lt * >*oq , i n t nw , →֓ Q A to m ic In t * as ) : o u tq u e ( oq ) , n u m W o rke r T h re a d s ( nw ) , →֓ a s s i g n e d ( as ) {} void run ( ) ; };

12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * void R e su lt Se n d e r : : run ( ) { w hi l e ( n u m W o rke r T h re a d s ) { M a n d e l R e su lt * res ; i f ( outque −>a va il Ite m s ( ) > 0 ) { res = outque −>d eq u e ( ) ; i f ( res == N ULL ) { n u m W o rke r T h re ad s −−; } else { M P I_ R e q u e st r ; M P I_ Sta tu s s ;

30

M P I_ Ise n d ( res−>g e t R e su lt A d d re ss ( ) , res−>→֓ g e t R e su lt Si z e ( ) , M a n d e l R e su lt : : type , 0 , →֓ R ES U L T TA G , M P I_ C O M M _ W O R LD , &r ) ; // →֓ re t ur n the r e s u l t s M P I_ W ait (&r , &s ) ;

31

32 33 34 35 36 37

}

38 39

}

}

outque −>r e le ase ( res ) ; assi g n ed −>f e tch A n d A d d O rd e re d ( −1) ;

}

40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61

62 63 64 65 66 67

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t main ( i n t argc , char * ar gv [ ] ) { ... } else // worker code { ... // one loop f o r se n di ng and re cv messages bool e n d O f W o rk = f a l s e ; i n t n u m W o r k e r T h re a d s = n u m C o re s ; Q A to m ic In t a s s i g n e d ( 0 ) ; // spawn the thre ad f o r se ndi ng r e s u l t s back R e su lt Se n d e r rs ( outque , n u m W o rke r T h re ad s , &as si g n e d ) ; rs . st a r t ( ) ; // j ob r e c e i v i n g part w h i l e ( ! e n d O f W o rk ) { M a n d e l W o rk I te m * w = inque −>r e se rv e ( ) ; M P I_ R e cv ( w , 1 , M a n d e l W o rk Ite m : : type , 0 , M P I_A N Y _T A G , →֓ M P I_ C O M M _ W O R LD , & st at u s ) ; // ge t a new work →֓ item i n t tag = sta tu s . M P I_ TA G ; i f ( tag == EN D TA G ) { f o r ( i n t i = 0 ; i < n u m C o re s ; i++) inque −>e n q u e ( NULL ) ; e n d O f W o rk = t r u e ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

106

CHAPTER 5. } else {

68 69 70 71 72 73

}

74

}

DISTRIBUTED MEMORY PROGRAMMING

inque −>e n q u e ( w ) ; a s s i g n e d . fe t ch A n d A d d O rd e re d ( 1 ) ;

75

f o r ( i n t i = 0 ; i < n u m C o re s ; i++) thr [ i]−> wait ( ) ; rs . wai t ( ) ;

76 77 78 79

...

21. Conway’s Game of Life is played on a rectangular grid of cells that may or may not contain an organism. The state of the cells is updated at each time step by applying the following set of rules: Every organism with 2-3 neighbors survives.

Every organism with 4 or more neighbors dies from overpopulation.

Every organism with 0 - 1 neighbors dies from isolation.

Every empty cell adjacent to 3 organisms gives birth to a new one.

Create an MPI program that evolves a board of arbitrary size (dimensions could be specified at the command-line) over several iterations. The board could be randomly generated or read from a file.

Try applying the geometric decomposition pattern to partition the work among your processes. One example could be to evenly split the board row-wise. It is clear that each process can update its part of the board only by knowing the state of the bottom board row resident in the previous process, and the state of the top board row resident in the next process (the boundary processes being an exception).

Answer visit Library Genesis for more solution manuals: libgen.is, libgen.st, libgen.rs, forum.mhut.org The following solution uses geometric decomposition to distribute the board evenly among the MPI processes, in a row-wise fashion. At the end of each time step, the processes communicate their top and bottom rows with their neighbors, to allow proper calculation of neighbor cell counts. The arrangement is similar to the one used in Exercise 15, for the DLA problem. The difference with the depiction shown in Figure 5.2 is that only one instead of two rows is communicated per message.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

#d e f i n e PIC SIZE 100 #d e f i n e PARTICLES 500 #d e f i n e MAX ITER 10 #d e f i n e DUMPIMAGES /*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−*/ i n t co u n t N e ig h b o rs ( u nsi g ne d c har * pic , i n t cols , i n t x , i n t y ) { i n t co u n t = − pic [ cols * y + x ] ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

107 f o r ( i n t i = −1; i < 2 ; i++) f o r ( i n t j = −1; j < 2 ; j++) co u n t += pic [ cols * ( y + j ) + ( x + i ) ] ;

20 21 22 23 24

re tu rn co u n t ;

25 26

}

27

/*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−*/ /* I n i t i a l i z e s the 2D a r ra y f o r the s im u l a t i o n */ void b o a rd _ in it ( u ns ig ne d char * pic , i n t rows , i n t cols , i n t →֓ p articl e s ) { int i , j , x , y ; f o r ( i = 0 ; i < r ows ; i++) f o r ( j = 0 ; j < cols ; j++) pic [ i * cols + j ] = f a l s e ;

28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

}

f o r ( i = 0 ; i < p a r t i c l e s ; i++) p a r t i c l e plac em e nt */ { x = ra n d o m ( ) % ( col s − 2 ) + 1 ; y = ran d o m ( ) % ( r ows − 2 ) + 1 ; pic [ y * cols + x ] = tr u e ; }

/* ge ne ra te

i n i t i a l →֓

// c o u n t i n g

s t a r t s from

1

44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89

/*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− * S i n g l e s t e p e v o l u t i o n o f the s im u l a t i o n . */ void b o a rd _ e vo lv e ( un si gn ed c ha r * pic , u nsi g ne d c har * pic2 , i n t rows , i n t col s ) { // pr e par e ar r a y to hold new s t a t e m em set ( pic2 , 0 , s i z e o f ( u nsi g ne d c ha r ) * rows * co ls ) ;

→֓

f o r ( i n t y = 1 ; y < r ows − 1 ; y++) f o r ( i n t x = 1 ; x < co l s − 1 ; x++) { i n t idx = y * cols + x ; i n t c = co u n t N e ig h b o rs ( pic , cols , x , y ) ; i f ( pic [ idx ] ) // c e l l in− plac e { sw i tc h ( c ) { // Cases are commented out be ca us e pi c 2 i s r e s e t alre ady // ca s e 0 : // ca s e 1 : // deat h − l o n i l e s s // pic 2 [ idx ]= f a l s e ; // break ; c as e 2 : ca s e 3 : // s u r v i v a l pic 2 [ idx ] = t r u e ; break ; default : // deat h − overcrowded // pic 2 [ idx ]= f a l s e ; break ; } } e l s e i f ( c == 3 ) // b i r t h in empty c e l l pic 2 [ idx ] = t r u e ; } } /*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−*/ void sa v e Im a ge ( u nsi g ne d cha r * pic , i n t rows , i n t cols , c har fn am e ) { FIL E * f = fo p en ( fname , ”w+t ” ) ; fp ri n tf ( f , ”P1\n%i %i \n” , cols , r ows ) ;

*→֓

f o r ( i n t y = 1 ; y <= r ows ; y++) { f o r ( i n t x = 1 ; x <= co l s ; x++)

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

108

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

{

90

i f ( pic [ y * ( cols + 2 ) + x ] ) fp ri n tf ( f , ” 1 ” ) ; else fp ri n tf ( f , ” 0 ” ) ;

91 92 93 94

} fp ri n tf

95 96 97 98

} fcl o se ( f ) ;

( f , ”\n” ) ;

99 100

}

101

/*−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−*/ i n t main ( i n t argc , char ** argv ) { i n t cols , rows , iter , p a r t i c l e s , x , y ; un si gn e d char * pic , * pic2 , * tmp ; i n t rank , num , i ; M P I_ In it (& argc , &argv ) ; M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ; M P I_ C o m m _ s i ze ( M P I_ C O M M _ W O R LD , &num ) ; M P I_ R e q u e st req , req 2 , r eq 3 ;

102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131

i f ( ar gc < 2 ) // use d e f a u l t v a l u e s i f us e r does s p e c i f y a n yt h i n g { col s = P IC _ S I Z E ; r ows = P IC _ S I Z E ; it er = M A X _ IT E R ; p a r t i c l e s = P A R T IC LE S ; } else { co l s = at oi ( a r gv [ 1 ] ) ; r ows = a to i ( ar gv [ 2 ] ) ; p a r t i c l e s = at oi ( ar gv [ 3 ] ) ; iter = atoi ( a r gv [ 4 ] ) ; }

not →֓

// to make code s i m p l e r and avoid s p e c i a l cas e s , rows to be a m u l t i p l e o f // the number o f p r o c e s s e s r ows = ( i n t ) ceil ( r ows * 1 . 0 / num ) * num ; i n t ro w s P e r N o d e = rows / num ;

i s f o r c e d →֓

132 133

sra n d ( time ( 0 ) ) ; // i n i t i a l i z e

134

// g r i d has two e x t r a rows , one above and one below the a c t u a l →֓ rows // The e xtr a rows ” b el ong ” to o the r nodes , so at the end o f →֓ each time st e p // t hey s h o u l d be exchange d i f ( r ank == 0 ) { pic = new u n sig n ed c har [ ( cols + 2 ) * ( row s + 2 ) ] ; pic 2 = new un si gn e d c ha r [ ( cols + 2 ) * ( rows + 2 ) ] ; } else { pic = new u ns ig ne d char [ ( cols + 2 ) * ( ro w s P e r N o d e + 2 ) ] ; pic 2 = new un si gn e d char [ ( cols + 2 ) * ( ro w s P e r N o d e + 2 ) ] ; }

135 136 137 138 139 140 141 142 143 144 145 146

the

random number

ge ne r a t o r

147 148 149 150 151

i f ( ra nk == 0 ) b o a rd _ in it ( pic2 , r ows + 2 , co ls + 2 , p a r t i c l e s ) ; // cle an pic b u f f e r m em set ( pic , 0 , ( co ls + 2 ) * ( ro w s P e r N o d e + 2 ) * s i z e o f ( →֓ un si gn e d c ha r ) ) ;

152 153 154

155 156

// boundary rows do not need to be communicated now .

M P I_ Sc a tte r ( pic 2 + cols + 2 , ( cols + 2 ) * ro w s P er N o d e , →֓ M P I_ U N SIG N E D _ C H A R , pic + cols + 2 , ( cols + 2 ) * →֓ ro w s P er N o d e , M P I_ U N S IG N E D _ C H A R , 0 , M P I_ C O M M _ W O R LD ) ; i n t n ext ID = r ank + 1 , p r ev ID = r ank − 1 ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

109 157 158 159 160 161

162 163 164 165 166 167 168 169

i n t st ep = 0 ; w hi l e (++step < iter ) { #i f d e f DUMPIMAGES M P I_ G a th e r ( pic + cols + 2 , ro w s P e r N o d e * ( cols + 2 ) , →֓ M P I_ U N SIG N E D _ C H A R , pic + cols + 2 , ro w s P e r N o d e * ( cols →֓ + 2 ) , M P I_ U N SIG N E D _ C H A R , 0 , M P I_ C O M M _ W O R LD ) ; i f ( ra nk == 0 ) { char fn am e [ 2 0 ] ; sp ri n tf ( f name , ”%05 i . pbm” , st ep ) ; s a v e Im age ( pic , rows , cols , fn am e ) ; } #e n d i f b o a rd _ e vo l ve ( pic , pic2 , ro w s P e r N o d e + 2 , co l s + 2 ) ; tmp = pic 2 ; pic 2 = pic ; pic = tmp ; // ex change i nf o r m a t i o n with ” pr e vio us ” node i f ( ra nk != 0 ) { M P I_ I se n d ( pic + co l s + 2 , co l s + 2 , M P I_ U N SI G N E D _ C H A R , →֓ pr ev ID , step , M P I_ C O M M _ W O R LD , &req ) ; M P I_ Ir e cv ( pic , col s + 2 , M P I_ U N S IG N E D _ C H A R , prev ID , →֓ step , M P I_ C O M M _ W O R LD , &req 2 ) ; }

170 171 172 173 174 175 176 177 178 179 180

// exc ha nge with ” next ” i f ( ra nk != num − 1 ) { M P I_ Ise n d ( pic + ro w s P e r N o d e * ( cols + 2 ) , cols + 2 , →֓ M P I_ U N SIG N E D _ C H A R , next ID , step , M P I_ C O M M _ W O R LD , &→֓ req ) ; M P I_ Ire cv ( pic + ( ro w s P e r N o d e + 1 ) * ( cols + 2 ) , cols +→֓ 2 , M P I_ U N SIG N E D _ C H A R , next ID , step , →֓ M P I_ C O M M _ W O R LD , &req 3 ) ; }

181 182 183 184

185

186 187 188 189 190 191 192

}

193 194

M P I_ G a th e r ( pic + cols + 2 , ro w s P e r N o d e * ( cols + 2 ) , →֓ M P I_ U N SIG N E D _ C H A R , pic + cols + 2 , ro w s P e r N o d e * ( cols + →֓ 2 ) , M P I_ U N SIG N E D _ C H A R , 0 , M P I_ C O M M _ W O R LD ) ; i f ( ra nk == 0 ) sa v e Im a ge ( pic , rows , cols , ” out . pbm” ) ;

195

196 197 198 199 200 201 202 203

// bl oc k u n t i l messag es are r e c e i v e d i f ( ra nk != 0 ) M P I_ W ait( & req 2 , M P I_ S T A T U S _ IG N O R E ) ; i f ( ra nk != num − 1 ) M P I_ W ait( & req 3 , M P I_ S T A T U S _ IG N O R E ) ;

}

M P I_ Fin a l i z e ( ) ; d e l e t e [ ] pic ; d e l e t e [ ] pic 2 ; re tu rn 0 ;

22. Radix sort is a linear complexity non-comparison-based sorting algorithm that is suitable for concurrent execution. Radix sort sorts data by separating them into groups based on their digits (for integers) or characters (for strings). The data must be of fixed range, i.e. the number of bits or characters used must be known a priori.

Radix sort comes in two forms: Least-Significant Digit radix sort (LSD) or Most-Significant Digit radix sort (MSD). The latter is suitable for parallel execution, as data which are partitioned in groups can be operated independently in subsequent phases of the algorithm. The MSD algorithm,

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

110

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

which is very close to bucket-sort 2, can be implemented recursively as shown in the following pseudocode, for binary data. An extension for data with non-binary digits (strings) is straightforward. The use of the auxiliary array B allows the sorting to be stable.

1 2 3 4 5 6 7

// I nput : a r r ay A, with N e l e m e nt s , each D b i t s lon g . // Output : A ho lds s o r t e d data ra d ix_ so rt ( A , N ) al l o cat e m em o ry B eq u al in size to A tmp <− rad ix_ au x ( A , N , B , D −1) i f tmp <> A // i f s o r t e d data end up in B array copy B to A

8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37

// Au x i l i a ry r e c u r s i v e f u n c t i o n // Use o f temporary arr ay B f o r a s t a b l e s o r t // R et ur ns l o c a t i o n o f s o r t e d data rad ix_ au x ( A , N , B , k ) i f k = −1 Or N<2 // bas e c as e f o r te r m i n a t i o n re t u r n A else let r be the n u m b er of item s in A , with k−th bit set to 0 re se t Id x <− 0 set Id x <− r f o r i <− 0 to N−1 // s e p a ra te data in two b i ns i f k−th bit of A [ i ] is set sto re A [ i ] in B [ set Id x ] set Id x <− set Id x + 1 else sto re A [ i ] in B [ re se t Id x ] re se t Id x <− re se t Id x + 1 tmp 1 <− rad ix_ au x ( B , r , A , k−1) // s o r t bin with a 0 →֓ k−th bit , u s i n g the ( k−1)−th b i t tmp 2 <− r ad i x_ au x ( B+r , N−r , A+r , k−1) // s o r t bin with a 1 →֓ k−th bit , u s i n g the ( k−1)−th b i t i f tmp 1 + r <> t mp 2 // p o i n t e r compar is on if r > N − r // make the s m a l l e s t copy copy N−r e le m e n ts from tmp 2 to tmp 1+r re t ur n t mp 1 else copy r e le m e n ts from tmp 1 to tmp 2 −r re t ur n tmp 2 −r else re t ur n t mp 1

Use divide & conquer decomposition as your decomposition pattern (see Section 2.3.2) to design and implement an MPI radix sort.

Answer The algorithm design is influenced by the initial distribution of data, in the same way that bucketsort does. For example, it could be possible that all the nodes get a portion of the initial data, sort it according to one of the bits and exchange the resulting two parts. In this solution we consider the alternative where the data are originally resident at the 0ranked node, which also performs the first sorting phase according to the most significant bit of the data values.

The following listing implements the divide & conquer decomposition pattern, whereas the data after each pass/phase of the algorithm that splits them into two groups, are split between two nodes. The overall communication pattern resembles a scattering operation. Node 0 first splits the 2The major difference between radix sort and bucket sort is in how the keys are examined: piece-wise in radix sort and as a whole in bucket sort. In the latter, an arbitrary number of buckets can be prescribed.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

111 input into two pieces according to the value of the most significant bit, keeps one part for itself and sends the other part to node 1. During the second pass/phase, node 0 splits the remaining data and sends one of the parts to node 2, while node 1 performs the same operation for its part of the data and sends one of the parts to node 3. The process is repeated as many times as the number of bits used by the data values, but the migration of data to other nodes is limited by the number of MPI processes (check of line 120).

The key points of the following listing are: The data are initialized in node 0, and are sorted by having all the nodes call the radixsort function in line 195. The radixsort function sets-up the auxiliary memory needed for a stable sort (line 171) and calls the recursive function radixSort aux which performs the actual sorting.

In radixSort aux each node determines the phase at which it will join the sorting operation (line 84), as well as the node from which it will receive its assignment (line 108), and the nodes to which it will subsequently assign parts of its workload (line 118).

As with the quicksort algorithm, the split of parts is not even. In the event that a node does not have data to operate on (base case M<=0 of line 88), it terminates, but not before informing all its sub-ordinate nodes that they should terminate as well (loop of lines 94 to 99).

The radixSort aux behaves differently during its first invocation, which is flagged by setting the phase parameter to -1. Namely, data are received from another node (line 109), and sent back to it following the completion of the “local” sort operation (line 153).

The auxiliary array pointed to by aux is used to hold the data during the split operation carried out by function radixPhaseK. Data are shifted between the two memory locations (data and aux) during the different phases, with radixSort aux maintaining a pointer (tmp1) to the actual location. The data should eventually return to their original location. If for any reason (e.g. an odd number of bits is used in the sorting) this is not done, the check in line 174 restores them.

The radixSort aux function also makes sure that the two data portions produced by the application of radixPhaseK in line 117, end up in the same location after they are independently sorted (lines 120-128 for the “left” part and line 131 for the “right” part). The necessary move is minimized by the logic of lines 134-148.

1 2 3 4 5 6 7

# i nc l u d e <mpi . h> # i nc l u d e <s t r i n g . h> # i nc l u d e <s t d i o . h> # i nc l u d e <s t d l i b . h> #include <ios t r eam > # i nc l u d e <math . h> # i nc l u d e < u n i s t d . h>

8 9

u s i n g namespace std ;

10 11

#d e f i n e ASSIGNTAG 0

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

112

CHAPTER 5.

12 13

#d e f i n e SORTEDRESTAG 1

14

co nst i n t MINV = 0 ; co nst i n t MAX V = 10 000 ;

15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52

DISTRIBUTED MEMORY PROGRAMMING

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // f i r s t b i t i s at p o s i t i o n 1 i n t msb ( i n t i ) { i n t res = 0 ; while ( i ) { i >>= 1 ; res++; } re t ur n res ; } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // Au x i l i a ry f u n c t i o n t hat p er f o r ms a phase // o f a r a d i x s o r t al g o r it h m , bas ed on the // k−th b i t . Temporary a r r a y B i s used f o r // a s t a b l e s o r t . // R et ur ns l o c a t i o n the l o c a t i o n where the // s o r t e d data ” s p l i t apa r t ” . So B[ r ] i s the // f i r s t el em e nt with a 1−b i t at p o s i t i o n k i n t ra d ix P h a se K ( i n t * A , i n t N , i n t * B , i n t k ) { i f ( ( k == −1) | | ( N < 2 ) ) // bas e c a s e f o r te r m i n a t i o n { m em cp y ( B , A , N * s i z e o f ( i n t ) ) ; // data ar e e x p e ct e d to move to B re tu rn 0 ; } else { int r = 0; i n t b it M ask = ( 1 << k ) ; f o r ( i n t i = 0 ; i < N ; i++) i f ( ( A [ i ] & b it M a sk ) == 0 ) r++;

53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68

}

→֓

i n t re se t Id x = 0 , set Id x = r ; f o r ( i n t i = 0 ; i < N ; i++) { i f ( ( A [ i ] & b it M as k ) > 0 ) { B [ set Id x ] = A [ i ] ; set Id x++; } else { B [ re se t Id x ] = A [ i ] ; re se t Id x++; } } re t ur n r ;

69 70

}

71

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // Re c u r s i v e r a d i s o r t f u n c t i o n . Based on the // phase , data maybe se nt to o t he r nodes f o r // the s u b s e q u e n t ph ase s . // t o t a l B i t s c o u nt s the b i t s l e f t f o r compar is on // phase==−1 f l a g s the f i r s t c a l l to t h i s f u n c t i o n i n t * ra d ix So rt_ a u x ( i n t * data , i n t M , i n t * aux , i n t to ta l Bits , →֓ i n t rank , i n t num , i n t p h ase = −1) { i n t n e w P h as e ; i n t so u r ce ID ; i n t * tmp 1 , * tmp 2 ; i f ( p h ase == −1) // i f f i r s t c a l l to r e c u r s i v e →֓ function

72 73 74 75 76 77 78 79 80 81 82

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

113 83

{

84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111

}

n e w P h as e = msb ( r ank ) ; to tal B its −= n e w P h ase ; // co rre spo nds w i l l j o i n the s o r t i n g

to when the node →֓

i f ( to t al B its <= 0 | | M <= 0 ) // base case s , no more s o r t i n g to →֓ be done { // have to te r m i n at e any nodes w a i t i n g f o r data i f ( p h ase != −1) { i n t p a rt n e r ID = rank ˆ ( 1 << p h ase ) ; w h i l e ( p ar t n e r ID < num ) { M P I_ Se n d ( NULL , 0 , MP I_IN T , pa rt ner ID , A S S IG N T AG , →֓ M P I_ C O M M _ W O R LD ) ; p h ase++; p a rt n e r ID = ra nk ˆ ( 1 << p h ase ) ; } } re tu rn data ; } // f i r s t ge t data as signm ent M P I_ St a tu s st ; i f ( p h ase == −1 && ra nk != 0 ) { so u r ce ID = r ank ˆ ( 1 << ( n e w P h as e − 1 ) ) ; M P I_ R e cv ( data , M , MP I_IN T , so ur ce ID , AS S I G N T A G , →֓ M P I_ C O M M _ W O R LD , &st ) ; M P I_ G e t_ co u n t (&st , MP I _I N T , &M ) ; // ge t how many where →֓ actually received }

112 113 114

i f ( p h ase != −1) n e w P h ase = p h ase ;

115 116 117 118 119 120 121 122 123 124 125 126 127 128 129

// now s p l i t the data in two gr ou ps bas ed on a b i t v al u e i n t sp lit = ra d ix P h a se K ( data , M , aux , to t al B its − 1 ) ; i n t p a rt n e r ID = rank ˆ ( 1 << n e w P h ase ) ; // e i t h e r send one part to an ot he r node i f ( p ar tn e r ID < num ) { M P I_ Se n d ( aux , s plit , MP I_INT , pa rt ner ID , AS S IG N T AG , →֓ M P I_ C O M M _ W O R LD ) ; } // or s o r t i t l o c a l l y else { tmp 1 = ra d ix So rt _ a u x ( aux , s plit , data , to t al B its − 1 , rank →֓ , num , n e w P h as e + 1 ) ; }

130 131

tmp 2 = ra d ix So rt_ a u x ( aux + split , M − split , data + split , →֓ to t al B its − 1 , rank , num , n e w P h ase + 1 ) ;

132 133 134 135 136 137 138 139 140 141 142 143 144 145 146

// make s ur e a l l data end−up at the same plac e , p o i n t e d by tmp1 i f ( p a rtn e r ID < num ) // no need to move data i f the o the r →֓ part has been se nt o f f tmp 1 = tmp 2 − sp lit ; e l s e i f ( t mp 2 != t mp 1 + sp lit ) { // mi ni miz e movement by moving the s m a l l e r part i f ( sp lit > M / 2 ) { m em cp y ( tmp 1 + split , tmp 2 , ( M − sp lit ) * s i z e o f ( i n t ) ) →֓ ; } else { m em cp y ( tmp 2 − split , tmp 1 , sp lit * s i z e o f ( i n t ) ) ; tmp 1 = t mp 2 ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

114

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

147 148 149 150 151 152

M

153

G P

CO

_

O R LD

154 155 156 157

M

158

M

G

159 160 161 162 163

M

164 165 166 167 168

M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ; P Comm_ P CO _ O R LD &num

169

m

170

M

171 172

M

173 174

m em

175

M

176 177 178 179 180 181 182 183 184

M

185 186 187

M P I_ C o m m _ ra n k ( M P I_ C O M M _ W O R LD , &rank ) ;

188 189

M

190 191 192

M

193 194 195 196 197 198 199 200

M

201 202

MAXV

203 204 205 206

M

207 208 209 210 211

M

212 213 214 215 216 217

M

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

115 218 219 220

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

116

CHAPTER 5.

DISTRIBUTED MEMORY PROGRAMMING

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Chapter 6

GPU Programming Exercises 1. An array of type float elements is to be processed in a one-elementper-thread fashion by a GPU. Suggest an execution configuration for the following scenarios: (a) The array is 1-D and of size N . The target GPU has 8 SMs, each with 16 SPs. (b) The array is 2-D and of size NxN . The target GPU has 5 SMs, each with 48 SPs. For each of the above scenario, calculate what is the minimum size that N should satisfy to make the GPU computation a desirable alternative to CPU computation.

Answer The number of data items should be high enough to provide work for all SPs. (a) A total of 8·16 = 128 SPs require at least N≥ 128. Ideally, N should be a multiple of 128. An execution configuration can be obtained with the following code: i n t b lo ck = 16 * 4 ; // 64 thr ea ds , a m u l t i p l e o f 16 i n t gr i d = ( ( N − 1 ) / b lo ck ) + 1 ; foo<<< grid , block >>>(...) ;

(b) A total of 5· 48 = 240 SPs require at least N≥2 240 ⇒ N ≥ 15.5.

Ideally, N 2 should be a multiple of 240. An execution configuration can be obtained with the following code: dim 3 b lo ck ( 16 , 12 ) ; // 192 t h r e a ds , a m u l t i p l e o f 48 dim 3 gri d ( ( N − 1 ) / 16 + 1 , ( N − 1 ) / 12 + 1 ) ; foo<<< grid , block >>>(...) ;

2. A reduction is an operation frequently encountered in a many algorithms: summing-up the elements of an array, finding the minimum, maximum, etc. One possible solution to a CUDA kernel that calculates the sum of an array would be: 117

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

118

CHAPTER 6. GPU PROGRAMMING _ _ glo b a l_ _ v oid sum ( f l o a t * in , f l o a t * out ) { _ _ sh a re d _ _ f l o a t lo ca l S t o re [ ] ; // to s peedup data a c c e s s i n t g lo b al ID = th re ad Id x . x + b lo ck Id x . x * b lo ck D im . x ; i n t lo cal ID = th re ad Id x . x ;

}

lo ca l S to re [ lo cal ID ] = in [ glo b al ID ] ; // copy to s har ed memory f o r ( i n t i=1; i< b lo ck D im . x ; i *=2) { i f ( lo cal ID % ( 2 * i ) == 0 ) lo ca l St o re [ lo cal ID ] += lo ca l St o re [ lo cal ID + i ] ; _ _ s yn ct h re a d s ( ) ; } i f ( lo cal ID == 0 ) out [ b lo ck Id x . x ] = lo ca l Sto re [ 0 ] ;

The above needs to be called multiple times, each time reducing the size of the data by the number of threads in a block. Data at each kernel launch should be multiples of the block size.

Analyze the above code in relation to the following criteria and how they reflect on the execution speed: thread divergence, memory coalescing, use of SPs within a warp. Suggest modifications that would improve the performance.

Answer The shown kernel scores low in all the mentioned areas, i.e. thread divergence, memory coalescing, and use of SPs within a warp. Analytically, the issues can be identified as follows: Thread divergence : The if condition inside the for-loop causes threads to diverge. During the first iteration only the threads with an ID which is a multiple of 2 can execute the if code body. During the second iteration only the ones with an ID which is a multiple of 4 can do this, and so on.

Memory coalescing : There is no regard to memory coalescing inside the for loop. In fact, as the loop progresses, the number of bank conflicts increases.

SP utilization : As soon as the for loop starts executing, an exponentially growing number of SPs with each iteration, become inactive, as they do not satisfy the if condition.

The following program can produce the same result with only one kernel invocation, with maximum SP utilization and minimum thread divergence: 1 2 3

#i n c l u d e <s t d i o . h> #i n c l u d e < s t d l i b . h> #i n c l u d e <cuda . h>

4 5

#d e f i n e MAXVALUE 10000

6 7 8 9 10 11 12 13

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void n u m b e r G e n ( i n t N , i n t max , i n t * sto re ) { int i ; sra n d ( time ( 0 ) ) ; f o r ( i = 0 ; i < N ; i++) ra nd ( ) % max ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

119 14 15

}

16

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− _ _ glo b a l_ _ voi d sum ( i n t * d , i n t N , i n t * odds ) { e xte rn _ _ sh a re d _ _ i n t co u n t [ ] ;

17 18 19 20

i n t my ID = b lo ck Id x . x * b lo ck D im . x + th re ad Id x . x ; i n t n u m Th r = b lo ck D im . x * gr i d D im . x ; i n t lo cal ID = th re ad Id x . x ; co u n t [ lo ca l ID ] = 0 ; i f ( my ID < N ) { co u n t [ lo ca l ID ] = d [ my ID ] ; i n t idx = my ID + n u m Th r ; // p a r t i a l sum w hi l e ( idx < N ) { co u n t [ lo ca l ID ] += d [ idx ] ; idx += n u m Th r ; }

21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

_ _ s yn ct h re a d s ( ) ; // block− wide p a r t i a l sum c a l c u l a t i o n i n t st ep = 1 ; w h i l e ( ( ( lo cal ID | st ep ) < b lo ck D im . x ) && ( ( lo ca l ID == 0 ) ) { co u n t [ lo cal ID ] += co u n t [ lo cal ID | step ] ; st ep *= 2 ; _ _ s yn ct h re a d s ( ) ; }

36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

}

// now the block− wide p a r t i a l sum i s in count [ 0 ] // add to g l o b a l c o u n t e r i f ( lo cal ID == 0 ) ato m ic A d d ( odds , co u n t [ 0 ] ) ;

51 52

}

53

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t main ( i n t argc , c ha r ** a rgv ) { i n t N = a to i ( ar gv [ 1 ] ) ;

54 55 56 57 58

int

& step ) →֓

* ha ,

* hres , pointers

* da ,

* dres ;

// host

( h * ) and de v ic e

( d * ) →֓

59 60 61

ha = new i n t [ N ] ; hres = new i n t [ 1 ] ;

62 63 64

cu d a M a ll o c ( ( void cu d a M a ll o c ( ( v oid

* * ) &da , s i z e o f ( i n t ) * * * ) &dres , s i z e o f ( i n t )

N) ; * 1) ;

65 66

n u m b e r G e n ( N , M A X V AL U E , ha ) ;

67 68 69

cu d a M e m cp y ( da , ha , s i z e o f ( i n t ) * N , cu d a M e m cp y H o st T o D e v i ce ) ; cu d a M e m se t ( dres , 0 , s i z e o f ( i n t ) ) ;

70 71 72 73 74

i n t bl o ck Si ze , gr i d Si z e ; b lo ck S i ze = 2 5 6 ; gr i d Si z e = 1 6 ; sum <<< gri d Si ze , bl o ck Si ze , b lo ck Si z e N , dres ) ;

*

s i z e o f ( i n t ) >>> ( da , →֓

75 76 77 78

cu d a M e m cp y ( hres , dres , s i z e o f ( i n t ) , cu d a M e m cp y D e v ic e T o H o st ) ; p rin tf ( ”%i \n” ,

* hres ) ;

79 80 81 82 83 84

cu d a Fr e e ( ( void * ) da ) ; cu d a Fre e ( ( v oid * ) dres ) ; d e l e t e [ ] ha ; d e l e t e [ ] hr es ; cu d a D e v ic e R e se t ( ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

120

CHAPTER 6. GPU PROGRAMMING

85 86 87

}

re tu rn

0;

The kernel of lines 17-51 allocates shared memory in the form of one integer per block thread. This memory is dynamically allocated during the kernel invocation (line 74). Each thread initially spends its time calculating a partial sum by going over the input data with a stride equal to the total number of threads (while loop of lines 30-34). Subsequently, a block-wide partial sum is calculated by reducing the shared memory counters with the loop of lines 39-44. The resulting sum in count[0] is added to the global memory counter with an atomic operation.

3. The reduction operation discussed in the previous exercise, is a special case of the “scan” or prefix-sum operation that can be applied to the elements of a vector or list. In general, the operator applied can be any of summation, subtraction, minimum, maximum, etc.. Implement a CUDA kernel capable of performing a prefix-sum operation. The “Prefix-Sums and Their Applications” paper by Guy Blelloch, available at http:// www.cs.cmu.edu/~guyb/papers/Ble93.pdf, is a wonderful resource for learning more on the topic.

Answer The solution is an extension of the one given in the previous exercise.

The only differences lie in how the initial partial result for each thread is initialized (lines 34-45), in the operator used to perform the reduction (lines 56-67 and 78-89), and in the atomic operation used to consolidate the block-wide partial results with the global memory result (lines 98-109).

This program can be easily extended to handle any prefix-sum operation by introducing another symbolic constant representing it (following the ones defined in lines 8-10), and appropriately augmenting the switch statements.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

numbe Gen

N

max

16 17 18 19 20 21 22 23 24 25 26 27 28

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

121 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69

i n t my ID = b lo ck Id x . x * b lo ck D im . x + th re ad Id x . x ; i n t n u m Th r = b lo ck D im . x * gri d D im . x ; i n t lo cal ID = th re ad Id x . x ; // i n i t i a l i z e a l l the t hr e ads ’ s t o r a g e r e g a r d l e s s swi tch ( OP ) { c ase SU M _O P : p art R e s [ lo cal ID ] = 0 ; break ; ca se M IN _O P : p ar t R e s [ lo ca l ID ] = IN T_ M A X ; break ; c a s e M A X _O P : p art R e s [ lo ca l ID ] = IN T_ M IN ; break ; }; i f ( my ID < N ) { p a rt R e s [ lo cal ID ] = d [ my ID ] ; i n t idx = my ID + n u m Th r ; // p a r t i a l r e s u l t w hi l e ( idx < N ) { i n t tmp = d [ idx ] ; s wi t c h ( OP ) { c a s e SU M _O P : p a rt R e s [ lo cal ID ] += tmp ; break ; ca se M IN _O P : p art R e s [ lo cal ID ] = min ( p a rt R e s [ lo cal ID ] , break ; c a s e M A X _O P : p a rt R e s [ lo cal ID ] = max ( p art R e s [ lo cal ID ] , break ; }; idx += n u m Th r ; }

tmp ) ; tmp ) ;

70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102

_ _ s yn ct h re a d s ( ) ; // block−wide p a r t i a l r e s u l t c a l c u l a t i o n i n t st ep = 1 ; i n t o th e r Id x = lo cal ID | st ep ; w h i l e ( ( o th e r Id x < b lo ck D im . x ) && ( ( lo cal ID & st ep ) == 0 ) ) { i n t tmp = p a rt R e s [ o th e r Id x ] ; s wi t c h ( OP ) { c a s e SU M _O P : p a rt R e s [ lo cal ID ] += tmp ; break ; ca se M IN _O P : p a rt R e s [ lo cal ID ] = min ( p a rt R e s [ lo cal ID ] , tmp ) ; break ; c a s e M A X _O P : p art R e s [ lo cal ID ] = max ( p a rt R e s [ lo cal ID ] , tmp ) ; break ; }; step <<= 1 ; o th e r Id x = lo c al ID | st ep ; _ _ s yn ct h re a d s ( ) ; } // now the bl ock−wide p a r t i a l sum i s // add to g l o b a l co u n t e r i f ( lo cal ID == 0 ) s wi t c h ( OP ) { c a s e SU M _O P : ato m ic A d d ( res , p art R e s [ 0 ] ) ; break ;

in

part Res [ 0 ]

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

122

CHAPTER 6. GPU PROGRAMMING ca se M IN _O P : ato m ic M in ( res , p art R e s [ 0 ] ) ; break ; c a s e M A X _O P : ato m ic M ax ( res , p a rt R e s [ 0 ] ) ; break ; };

103 104 105 106 107 108 109

}

110 111

}

112 113 114 115 116 117

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t sh a re d Si z e ( i n t b ) { re t ur n b * s i z e o f ( i n t ) ; }

118 119

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

120 121 122 123

i n t main ( i n t argc , char ** argv ) { i n t N = a to i ( ar gv [ 1 ] ) ;

124

int

125

* ha ,

* hres , pointers

* da ,

* dres ;

// host ( h * ) and de v ic e

( d * ) →֓

126

ha = new i n t [ N ] ; hres = new i n t [ 1 ] ;

127 128 129

cu d a M a ll o c ( ( void cu d a M a ll o c ( ( v oid

130 131

**) **)

&da , s i z e o f ( i n t ) * N ) ; &dres , s i z e o f ( i n t ) * 1 ) ;

132

n u m b e r G e n ( N , M A X V AL U E , ha ) ;

133 134

cu d a M e m cp y ( da , ha , s i z e o f ( i n t ) * N , cu d a M e m cp y H o st T o D e v i ce ) ; cu d a M e m se t ( dres , 0 , s i z e o f ( i n t ) ) ;

135 136 137

i n t bl o ck Si ze , gr i d Si z e ; b lo ck S i ze = 2 5 6 ; gr i d Si z e = 1 6 ; r ed u ce <<< gri d Si ze , bl ock Si ze , b lo ck Si z e da , N , dres , M A X _O P ) ;

138 139 140 141

*

s i z e o f ( i n t ) >>> ( →֓

142

cu d a M e m cp y ( hres , dres , s i z e o f ( i n t ) , cu d a M e m cp y D e v ic e T o H o st ) ;

143 144

p rin tf ( ”%i \n” ,

145

* hres ) ;

146

cu d a Fr e e ( ( void * ) da ) ; cu d a Fre e ( ( v oid * ) dres ) ; d e l e t e [ ] ha ; d e l e t e [ ] hr es ; cu d a D e v ic e R e se t ( ) ;

147 148 149 150 151 152 153 154

}

re tu rn

0;

The code shown here is not nearly as optimum as it could be; it is a tradeoff between readability and performance. A more thorough derivation of an optimum CUDA reduction kernel is covered in http://developer. download.nvidia.com/assets/cuda/files/reduction.pdf, although the material in question targets a Compute Capability 1.0 device, which exhibits many more quirks than contemporary architectures.

4. Create CUDA implementations of the gsl stats mean() and gsl stats variance() functions offered by the GNU Scientific Library, that produce the mean and variance statistics of an array of type double data. Their signatures are: do u ble g sl _ st a t s_ m e a n ( co ns t data

do ubl e

DAT A [ ] , // Po i nt e r to inp ut →֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

123

Assuming that the STRIDE is 1, create a memory access pattern that utilizes coalescing. Suggest ways to deal with the problem if the STRIDE is not 1.

Answer The gsl stats mean and gsl stats variance functions are implemented as host front-ends, to allow for the hiding of the implementation details (such as memory management and grid/block design) from the application that calls them.

The solution is based on the reduction approach that is used in the two previous exercises. Two kernels are employed for calculating the sum of values and the sum of the square of the values respectively, in order for the variance to be calculated from the formula: Σ N −1 var =

i=0

N

vi2

ΣN −1 −

i=0

vi

!2

N

(6.1)

where N is the number of values.

In order to deal with a STRIDE parameter which is different from 1, the gsl stats mean and gsl stats variance front-end functions, copy the data to be actually processed in another array (lines 127-130 and 169172). The benefits are two-fold: A considerably smaller volume of data cross the PCIe bus, significantly reducing the communication cost.

The data items reside in contiguous memory locations allowing for memory coalescing to take place, at least to the extent allowed by the double data type.

The last major point of the following listing is the atomicAdd device function of lines 22-34, that is an operation for double-type operands that is missing from the arsenal of CUDA primitives.

1 2 3

#i n c l u d e <s t d i o . h> #i n c l u d e < s t d l i b . h> #i n c l u d e <cuda . h>

4 5 6 7 8 9 10 11 12 13 14

#d e f i n e MAXVALUE 10000 #d e f i n e min (A, B) ( ( A>B) ? B : A) //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void n u m b e r G e n ( i n t N , i n t max , doubl e * sto re ) { int i ; sra n d ( time ( 0 ) ) ; f o r ( i = 0 ; i < N ; i++) sto re [ i ] = i ; // rand ( ) % max ; }

15 16 17

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t sh a re d Si z e ( i n t b )

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

124

CHAPTER 6. GPU PROGRAMMING

18

{

19

re t ur n b * s i z e o f ( d ou bl e ) ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− _ _ d e vi ce _ _ double ato m ic A d d ( double * a ddress , double val ) { unsi g ne d l ong long i n t * a d d re ss_ a s_ u ll = ( u ns i g n e d long long * ) ad d re ss ; u n s i g n e d long long i n t old = * ad d re ss _ as _ u ll , as s u m e d ; do { as s u m e d = old ; old = at o m ic C A S ( ad d re ss _ as _ u ll , as s um e d , _ _ d o u b le _ a s_ lo n g lo n g ( val + _ _ lo n g lo n g _ a s_ d o u b le ( as s u m e d ) ) ) ; } w h i l e ( a ssu m e d != old ) ; re tu rn _ _ lo n g lo n g _ a s_ d o u b le ( old ) ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

_ _ glo b a l_ _ v oid sum ( doubl e * d , i n t N , double { e xt e rn _ _ sh a re d _ _ d ou bl e p a r t i a l Su m [ ] ;

i n t →֓

* to t al Su m )

40

i n t my ID = b lo ck Id x . x * b lo ck D im . x + th re ad Id x . x ; i n t n u m Th r = b lo ck D im . x * gri d D im . x ; i n t lo cal ID = th re ad Id x . x ; p a rt ia l Su m [ lo ca l ID ] = 0 ; i f ( my ID < N ) { p a r t i a l Su m [ lo cal ID ] = d [ my ID ] ; i n t idx = my ID + n u m Th r ; // p a r t i a l sum w hi l e ( idx < N ) { p a r t i a l Su m [ lo cal ID ] += d [ idx ] ; idx += n u m Th r ; }

41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

_ _ s yn ct h re a d s ( ) ;

58

// re d u c t i o n i n t st ep = 1 ; i n t o th e r Id x = lo cal ID | st ep ; w h i l e ( ( o th e r Id x < b lo ck D im . x ) && ( ( lo cal ID & st ep ) == 0 ) ) { p a r t i a l Su m [ lo cal ID ] += p a r t i a l Su m [ o th e r Id x ] ; step <<= 1 ; o th e r Id x = lo c al ID | st ep ; _ _ s yn ct h re a d s ( ) ; }

59 60 61 62 63 64 65 66 67 68 69 70 71 72 73

}

// now the block− wide p a r t i a l sum i s in par t ial Sum [ 0 ] // add to g l o b a l c o u n t e r i f ( lo cal ID == 0 ) ato m ic A d d ( total Sum , p a rti a l Su m [ 0 ] ) ;

74 75

}

76 77

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

78

_ _ glo b a l_ _ v oid sum 2 ( doub le * d , i n t N , double { e xt e rn _ _ sh a re d _ _ d ou bl e p a r t i a l Su m [ ] ;

79 80

* to t al Su m )

81 82 83 84 85 86 87 88 89 90

i n t my ID = b lo ck Id x . x * b lo ck D im . x + th re ad Id x . x ; i n t n u m Th r = b lo ck D im . x * gri d D im . x ; i n t lo cal ID = th re ad Id x . x ; p a rt ia l Su m [ lo ca l ID ] = 0 ; i f ( my ID < N ) { p a rti a l Su m [ lo cal ID ] = d [ my ID ] * d [ my ID ] ; i n t idx = my ID + n u m Th r ; // p a r t i a l sum

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

125 w hi l e ( idx < N ) { p a r t i a l Su m [ lo cal ID ] += d [ idx ] * d [ idx ] ; idx += n u m Th r ; }

91 92 93 94 95 96 97 98

_ _ s yn ct h re a d s ( ) ;

99

// re d u c t i o n i n t st ep = 1 ; i n t o th e r Id x = lo cal ID | st ep ; w h i l e ( ( o th e r Id x < b lo ck D im . x ) && ( ( lo cal ID & st ep ) == 0 ) ) { p a r t i a l Su m [ lo cal ID ] += p a r t i a l Su m [ o th e r Id x ] ; step <<= 1 ; o th e r Id x = lo c al ID | st ep ; _ _ s yn ct h re a d s ( ) ; }

100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115

}

}

// now the block− wide p a r t i a l sum i s in par t ial Sum [ 0 ] // add to g l o b a l c o u n t e r i f ( lo cal ID == 0 ) ato m ic A d d ( total Sum , p a rti a l Su m [ 0 ] ) ;

116 117 118

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

119

do u ble g sl _ st a t s_ m e a n ( co ns t d o ubl e DA T A [ ] , si z e _ t ST RIDE , s i z e _ t →֓ N) { d ou bl e * ha , * da , * dr es ; // hos t ( h * ) and de v i c e ( d * ) p o i n t e r s doubl e hres ;

120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142

i n t N e ffe c t i v e ; i f ( STR ID E ! =1 ) { N e ffe c t i v e = ( N −1)/ ST R ID E + 1 ; ha = new doubl e [ N e ffe c t i v e ] ; f o r ( i n t i=0 , j=0; i<N ; i+=STRIDE , j++) ha [ j ] = D ATA [ i ] ; } else { ha = ( do u ble * ) DA T A ; N e ffe c t i v e = N ; } cu d a M a ll o c ( ( v oid cu d a M a llo c ( ( v oid

* * ) &da , s i z e o f ( double ) * N e ffe c t i v e ) ; * * ) &dres , s i z e o f ( double ) * 1 ) ;

cu d a M e m cp y ( da , ha , s i z e o f ( dou bl e ) * N ef f ec t i v e , →֓ cu d a M e m cp y H o st T o D e v ic e ) ; cu d a M e m se t ( dres , 0 , s i z e o f ( doubl e ) ) ;

143 144 145 146

i n t bl o ck Si ze , gr i d Si z e ; cu d a O ccu p a n cy M a x P o te n tia l B lo ck S i z e V a ri a b le S M e m (& grid Siz e , &→֓ bl o ck Si ze , ( v oi d * ) sum , sh ared Si ze , N e ffe c t i v e ) ; gr i d Si z e = min ( gri d Si ze , N e ffe c t i v e / b lo ck Si z e ) ;

147 148

sum <<< grid Size , block Size , b lo ck Size * s i z e o f ( double ) >>> ( →֓ da , N e f f ec t iv e , dres ) ;

149 150

cu d a M e m cp y (& hres , dres , s i z e o f ( do u ble ) , →֓ cu d a M e m cp y D e v ic e T o H o st ) ;

151 152 153

i f ( STR ID E ! =1 ) d e l e t e [ ] ha ;

154 155 156 157 158 159

cu d a Fr e e ( ( void * ) da ) ; cu d a Fr e e ( ( v oi d * ) dr es ) ; re t ur n hr es / N e ffe c t i v e ;

} //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

126

CHAPTER 6. GPU PROGRAMMING

160 161 162 163 164 165

do u ble g sl _ s t a t s _ va ria n ce ( co nst do u ble DA T A [ ] , s i z e _t ST RIDE , →֓ si z e _ t N ) { d ou bl e * ha , * da , * dr es ; // hos t ( h * ) and de v i c e ( d * ) p o i n t e r s do ubl e hsum , h su m 2 ; i n t N e ffe c t i v e ; i f ( STR ID E ! =1 ) { N e ffe c t i v e = ( N −1)/ ST R ID E + 1 ; ha = new doubl e [ N e ffe c t i v e ] ; f o r ( i n t i=0 , j=0; i<N ; i+=STRIDE , j++) ha [ j ] = D ATA [ i ] ; } else { ha = ( d o ubl e * ) DAT A ; N e ffe c t i v e = N ; }

166 167 168 169 170 171 172 173 174 175 176 177 178 179

cu d a M a ll o c ( ( v oid cu d a M a llo c ( ( v oid

180 181 182

* * ) &da , s i z e o f ( double ) * N e ffe c t i v e ) ; * * ) &dres , s i z e o f ( double ) * 1 ) ;

cu d a M e m cp y ( da , ha , s i z e o f ( do ubl e ) * N e f f ec t iv e , →֓ cu d a M e m cp y H o st T o D e v ic e ) ; cu d a M e m se t ( dres , 0 , s i z e o f ( doubl e ) ) ;

183 184 185

// f i r s t c a l c u l a t e the sum o f v a l u e s i n t bloc k Siz e , gr id Siz e ; cu d a O ccu p a n cy M a x P o te n tia l B lo ck S i z e V a ri a b le S M e m (& grid Siz e , &→֓ bl o ck Si ze , ( v oi d * ) sum , sh ared Si ze , N e ffe c t i v e ) ;

186 187 188 189

gr i d Si z e = min ( gr i d Si ze , N e ffe c t i v e / b lo ck Si z e ) ; sum <<< grid Size , block Size , b lo ck Size * s i z e o f ( double ) >>> ( →֓ da , N e f f ec t iv e , dres ) ;

190 191 192

cu d a M e m cp y (& hsum , dres , s i z e o f ( d ou bl e ) , →֓ cu d a M e m cp y D e v ic e T o H o st ) ;

193 194

// then c a l c u l a t e the sum o f s q u a r e s cu d a O ccu p a n cy M a x P o te n tia l B lo ck S i z e V a ri a b le S M e m (& grid Siz e , &→֓ bl o ck Si ze , ( v oi d * ) sum 2 , sh are d Si ze , N e ffe c t i v e ) ; gr i d Si z e = min ( gr i d Si ze , N e ffe c t i v e / b lo ck Si z e ) ;

195 196 197 198

// r e s e t the r e s u l t v a r i a b l e cu d a M e m se t ( dres , 0 , s i z e o f ( doubl e ) ) ; sum 2 <<< grid Size , block Size , b lo ck Size * s i z e o f ( double ) >>> ( →֓ da , N e f f ec t iv e , dres ) ;

199 200 201 202

cu d a M e m cp y (& hs um 2 , dres , s i z e o f ( d ou bl e ) , →֓ cu d a M e m cp y D e v ic e T o H o st ) ;

203 204

i f ( STR ID E ! =1 ) d e l e t e [ ] ha ;

205 206 207 208 209 210 211 212

}

cu d a Fr e e ( ( void * ) da ) ; cu d a Fre e ( ( v oid * ) dres ) ; double avg = hsum / N e ffe c t i v e ; re t ur n h su m 2 / N e ffe c t i v e − avg * avg ;

213 214

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

215 216 217 218

i n t main ( i n t argc , char ** argv ) { i n t N = at o i ( a rgv [ 1 ] ) ;

219 220

double

* da ta ;

221 222

data = new doubl e [ N ] ;

223 224

n u m b e r G e n ( N , M A X V A L U E , data ) ;

225

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

127 p ri n tf ( ”Avg : % l f \n” , g sl _ st a t s _ m e a n ( data , 1 , N ) ) ; p r i n tf ( ” Var : % l f \n” , g sl _ sta ts_ va r i a n ce ( data , 1 , N ) ) ;

226 227 228

d e l e t e [ ] dat a ; cu d a D e v ic e R e se t ( ) ;

229 230 231 232 233

}

re tu rn

0;

5. Design and implement a CUDA program for calculating the histogram of a 24-bit color image. In this case, three separate histograms will be produced, one for each color component of the image.

Answer The histogram-calculating kernels of Section 6.7.3 can be used as a basis for the solution. The only thing that needs to be explicitly addressed, is the handling of the three color components. Sending the raw RGB data to the GPU could result in having to launch just one kernel only.

However, the memory layout in that case would be equivalent to an array of structures, which as discussed in Section 6.7.4 should be avoided.

The alternative, is to rearrange the pixel data in color planes, as shown in lines 84-110, packing four pixel values per integer (line 106). This arrangement also permits the use of the histogram kernel presented in Listing 6.22, without any modifications.

1 2 3 4 5 6 7

#i n c l u d e <s t d i o . h> #i n c l u d e < s t d l i b . h> #i n c l u d e < u n i s t d . h> #i n c l u d e <math . h> #i n c l u d e <cuda . h> #i n c l u d e <QImage> #i n c l u d e <QRgb>

8 9 10

co ns t i n t B I NS = 2 5 6 ; co ns t i n t B IN S 4 A LL = BI NS

*

32;

11 12

13 14 15 16 17 18 19 20

//→֓

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * →֓

_ _ glo b a l_ _ v oid G P U _ h i s t o g ra m _ a to m ic ( i n t * in , i n t N , i n t { i n t gl o ID = b lo ck Id x . x * b lo ck D im . x + th re ad Id x . x ; i n t lo c ID = th re ad Id x . x ; i n t G R ID S I Z E = gri d D im . x * b lo ck D im . x ; _ _ sh a re d _ _ i n t lo ca l H [ B IN S 4 A LL ] ; i n t b an k ID = lo c ID % w arp Si z e ; int i ;

*h )

21 22 23 24

// i n i t i a l i z e the l o c a l , shared−memory bins f o r ( i = lo c ID ; i < B IN S 4 A LL ; i += b lo ck D im . x ) lo cal H [ i ] = 0 ;

25 26 27

// wait f o r a l l warps to c o m p l et e the pr e vi o us st e p _ _ s yn ct h re a d s ( ) ;

28 29 30 31

32 33

// s t a r t p r o c e s s i n g the image data i n t * m y Sh a re d B a n k = lo c al H + b an k ID ; i f ( b lo ck D im . x > w arp Si z e ) // i f the b l o c k s i z e e xce e ds the →֓ warp Size , i t i s p o s s i b l e m u l t i p l e warps run at the same →֓ time f o r ( i = gl o ID ; i < N ; i += G R ID SI Z E ) {

34 35 36 37

i n t temp = in [ i ] ; i n t v = temp & 0 x FF ; i n t v2 = ( temp >> 8 ) & 0 x FF ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

128

CHAPTER 6. GPU PROGRAMMING i n t v3 = ( temp >> 16 ) & 0 x FF ; i n t v4 = ( temp >> 24 ) & 0 x FF ; ato m ic A d d ( m y Sh a re d B a n k + ( v << 5 ) , 1 ) ; ato m ic A d d ( m y Sh a re d B a n k + ( v2 << 5 ) , 1 ) ; ato m ic A d d ( m y Sh a re d B a n k + ( v3 << 5 ) , 1 ) ; ato m ic A d d ( m y Sh a re d B a n k + ( v4 << 5 ) , 1 ) ;

38 39 40 41 42 43 44 45 46 47 48

} else f o r ( i = glo ID ; i < N ; i += G R ID S IZ E ) {

49 50 51 52 53 54 55 56 57

}

58 59

i n t temp = in [ i ] ; i n t v = temp & 0 x FF ; i n t v2 = ( temp >> 8 ) & 0 x FF ; i n t v3 = ( temp >> 16 ) & 0 x FF ; i n t v4 = ( temp >> 24 ) & 0 x FF ; m y Sh a re d B a n k [ v << 5]++; // Optimized bankID + v * wa rp Size]++ m y Sh a re d B a n k [ v2 << 5]++; m y Sh a re d B a n k [ v3 << 5]++; m y Sh a re d B a n k [ v4 << 5]++;

v e r s i o n o f lo cal H [ →֓

60 61 62 63 64

m

65 66 67 68

* * →֓ 69

h

m R G B FE Q

Nx

w

a ge

70 71 72 73 74 75 76 77 78 79 80 81

d M d M

e

N e

82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

129 106 107 108 109 110 111 112

M m

113

d M m

M m e

114

m

115

m

116

M m

117

M m 118 119 120 121 122 123 124 125

* * →֓ 126 127 128 129 130 131 132

m G

133 134 135 136 137 138 139 140 141 142 143 144

The above program, which relies on Qt for image I/O, can be compiled with: $

nvcc −X co m p ile r −f PIC −m 64 −O 2 −D _ R E E N T R A N T \ −D Q T _ N O _ D E B U G −D Q T _ G U I_ LI B −D Q T _ C O R E _ LIB −I/ usr / lib / x 86 _ 64 −linux−gnu / qt 5 / m ksp e cs / linux−g++−64 −I . −I/ usr / in cl u d e / qt 5 −I/ usr / in cl u d e / qt 5 / Q t G u i −I/ usr / in cl u d e / qt 5 / Q t C o re −l Q t 5 C o re −l Q t 5 G u i −arch=sm _ 21 −o h i s t o g ra m _ a to m ic h i s t o g ra m _ a to m ic . cu

6. Create a variation of the program in Listing 6.24 to discover and plot the memory copy speed for host-to-device and device-to-host operations and for all the available types of host memory allocations : pageable, pinned and mapped. In order to test the last type of allocation, you will have to call a kernel that will try to access the memory, triggering the transfer. Compare your results with the ones returned by the $CUDA/samples/1 Utilities/bandwidthTest sample program.

Answer

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

130

CHAPTER 6. GPU PROGRAMMING This is an exercise that reveals a great deal about the underpinnings of the zero-copy memory mechanism. The program shown below transfers increasing amounts of data from the host to the device, using pageable, pinned and mapped memory allocation. In the last case, a dummy kernel is invoked to trigger the transfer of memory. The program parameters are: (a) The number of iterations for calculating averages. (b) The increment (in bytes) to use in testing increasing data chunks. (c) The maximum data size (in bytes) to test for.

The maximum device memory is allocated in one go, as dictated by the third command-line parameter, in lines 59-66. The for loop of lines 68 to 72, calls the timeCopyH2D function once for every type of device memory and for increasing data sizes.

The timeCopyH2D function uses a stream and events in the same fashion as Listing 6.24, to calculate the average time taken to transfer data from the host to the device. When a zero-copy memory arrangement is used, the dummyKernel function is also invoked (line 33) in order to trigger the memory transfer instead of the explicit call of line 36.

1 2

#i n c l u d e <s t d i o . h> #i n c l u d e <cuda . h>

3 4 5 6 7 8 9 10 11 12

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // a dummy k e r n e l j u s t f o r i n i t i a t i n g memory t r a n s f e r s when // mapped memory i s used _ _ glo b a l_ _ void d u m m y K e rn e l ( u n s i g n e d char * b , i n t N ) { i n t idx = b lo ck Id x . x * b lo ck D im . x + th re ad Id x . x ; i f ( idx < N ) b [ idx ] = idx % 2 5 6 ; }

13 14 15 16 17 18 19 20

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− f l o a t tim e C o p y H 2 D ( u nsi g n ed c har * h , u nsi gn e d c ha r * d , i n t N , i n t →֓ iter , bool ZE R O C O P Y = f a l s e ) { cu d a E ve n t_ t start T , end T ; cu d a Stre a m _ t str ; f l o a t d u ra ti o n ; int i ;

21 22 23 24

cu d a E ve n t C re a te (& s t a r t T ) ; cu d a E ve n t C re a te (& end T ) ; cu d a S t r e a m C re a te (& str ) ;

25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

cu d a E ve n t R e co rd ( start T , str ) ; f o r ( i = 0 ; i < i ter ; i++) { i f ( ZE R O C O P Y ) { i n t b lo ck = 102 4 ; i n t gri d = ( N − 1 ) / b lo ck + 1 ; d u m m y K e rn e l <<< 1 , 1 , 0 , str >>> ( h , N ) ; } else cu d a M e m cp y A sy n c ( d , h , N , cu d a M e m cp y H o st T o D e v ic e , str ) ; } cu d a E ve n t R e co rd ( end T , str ) ; cu d a E ve n t S yn ch ro n iz e ( end T ) ; cu d a E ve n t E la p se d T im e (& d ur at i o n , st art T , end T ) ;

41 42 43 44

cu d a S t r e a m D e s t r o y ( str ) ; cu d a E ve n t D e s t r o y ( s t ar t T ) ; cu d a E ve n t D e s tr o y ( end T ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

131 45 46 47 48 49 50 51 52

MAXDA A

53

[

]

d _d

54

h p

55

n d

56 57 58

m

59

M

60

M M

61 62

M

63

M

64 65

d Ho

66

A

p

M A X D A TA

p rin tf ( ” SIZE PAGEABLE PINNED ZERO−COPY\n” ) ; f o r ( i n t d a ta Si z e = 0 ; d at a Si z e <= M A X D A T A S IZ E ; d at a Si z e += →֓ step ) p r i n tf ( ”%i %f %f %f \n” , data Si z e , d at a Siz e / tim e C o p y H 2 D ( h_dat a , d_dat a , dat a Size , →֓ iter ) / (1<<20) , d at a Siz e / tim e C o p y H 2 D ( h _ p i n n e d , d_dat a , dat a Siz e →֓ , iter ) / (1<<20) , d at a Siz e / tim e C o p y H 2 D ( h _z er o co p y , d_dat a , →֓ dat a Si ze , iter , t r u e ) / (1<<20) ) ;

67 68 69 70 71 72 73 74 75 76 77 78 79 80

}

cu d a Fr e e H o st ( h _ p in n e d ) ; cu d a Fr e e H o st ( h _ ze ro co p y ) ; fr ee ( h _d at a ) ; cu d a Fre e ( d _d ata ) ; cu d a D e v ic e R e se t ( ) ; re tu rn 1 ;

A sample of the results (in MB/sec) as tested on a GTX 870M GPU using the CUDA 6.5.14 SDK, are shown below: m m 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000

8721 . 193359 9870 . 787109 8816 . 846680 8428 . 890625 8996 . 895508 9064 . 430664 9636 . 149414 9613 . 311523

11407 . 150391 11431 . 939453 11335 . 143555 10872 . 275391 10910 . 654297 11688 . 812500 11648 . 382812 11710 . 164062

2516 45 . 046875 4777 16 . 156250 7293 76 . 500000 9545 90 . 687500 1273 603 . 500000 1586 918 . 125000 1916 372 . 125000 2148 302 . 250000

The results for pageable and pinned host memory are consistent with the speeds reported from the $CUDA/samples/1 Utilities/bandwidthTest utility. However, the results for the zero-copy memory are clearly bogus!

It seems that the issue is that the single thread launched in line 33 does not trigger the transfer of the entire designated data block. Forcing the invocation of the kernel for all the tested memory arrangements, results in the following timeCopyH2D function: 1 2

f l o a t tim e C o p y H 2 D ( u nsi g n ed c ha r * h , u ns ig ne d c har iter , bool ZE R O C O P Y = f a l s e ) {

*d ,

i n t N , i n t →֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

132

CHAPTER 6. GPU PROGRAMMING cu d a E ve n t_ t start T , end T ; cu d a Stre a m _ t str ; f l o a t d u ra ti o n ; int i ;

3 4 5 6 7

cu d a E ve n t C re a te (& s t a r t T ) ; cu d a E ve n t C re a te (& end T ) ; cu d a S t r e a m C re a te (& str ) ;

8 9 10 11

cu d a E ve n t R e co rd ( start T , str ) ; f o r ( i = 0 ; i < i ter ; i++) { i n t b lo ck = 102 4 ; i n t gri d = ( N − 1 ) / b lo ck + 1 ; i f ( ! ZE R O C O P Y ) { cu d a M e m cp y A sy n c ( d , h , N , cu d a M e m cp y H o st T o D e v ic e , str ) ; d u m m y K e rn e l <<< grid , bl o ck , 0 , str >>> ( d , N ) ; } else d u m m y K e rn e l <<< grid , bl o ck , 0 , str >>> ( h , N ) ; } cu d a E ve n t R e co rd ( end T , str ) ; cu d a E ve n t S yn ch ro n iz e ( end T ) ; cu d a E ve n t E la p se d T im e (& d ur at i o n , st art T , end T ) ;

12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

}

cu d a S t r e a m D e s t r o y ( str ) ; cu d a E ve n t D e s t r o y ( s t ar t T ) ; cu d a E ve n t D e s tr o y ( end T ) ; re t ur n ( d u r a t i o n / iter ) / 1000 ;

The results of the modified program are shown below: m m 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000

w 7139 . 290039 7622 . 321777 7627 . 702148 6630 . 308105 7482 . 463379 7526 . 963379 7304 . 780273 7429 . 434570

7236 . 488770 7693 . 185547 8057 . 083984 7898 . 557129 8091 . 921387 8168 . 914062 8191 . 183105 8235 . 093750

4724 . 393066 5189 . 917480 5141 . 410156 4420 . 788086 4954 . 079102 5034 . 622070 4979 . 723633 4981 . 714844

The above clearly illustrate that zero-copy memory is a recommended solution only if just part of the host data are required for the computation.

Otherwise, even pageable memory is preferable in terms of performance.

The delays introduced by the suspension of the threads that access the memory accumulate to a substantial performance hit.

7. The Mandelbrot set calculators of Section 6.12.1 are limited to a maximum of 1024 iterations per pixel. However, the beauty of the Mandelbrot set is revealed for thousands or millions of iterations. Modify one or more of the solution of Section 6.12.1 so that up to 2 16—1 iterations can be performed for each pixel. Profile your program and analyze its performance. What is the grid/block arrangement that yields the best performance?

Answer The modifications required are minimum as they are concentrated on the memory management part of the code. Using the #1 version of the solution described in Section 6.12.1.1 as the basis of the answer, we need to modify only the kernel.cu file of Listing 6.29, so that the pixel values are

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

133 stored as short integers (instead of unsigned characters). The changes to Listing 6.29 are documented below: // L ine 5 , c h a n g i n g the upper i t e r a t i o n s t a t i c co nst i n t M A X IT ER = 0 x FFFF ;

limit

// Line 26 , chang i ng the s i g n a t u r e o f the k e r n e l to accommodate →֓ the d i f f e r e n t type o f p i x e l s t o r a g e _ _ glo b a l_ _ void m a n d e l K e rn e l ( u n s i g n e d s ho rt i n t * d_r es , d o ubl e →֓ upper X , d o ub le up per Y , d o ub le st ep X , do u bl e st ep Y , i n t resX , →֓ i n t resY , i n t p itc h ) // Li ne s 52 , 53 unsi g ne d s ho rt * h _res ; unsi g ne d s ho rt * d _res ; // L ine s 58 , 59 C U D A _ C H E C K _ R E T U R N ( cu d a M a llo c ( ( void * * ) &d_res , res X * r es Y * →֓ s i z e o f ( u ns i g n e d s ho rt i n t ) ) ) ; h _res = ( u nsi g ne d s ho rt i n t * ) m allo c ( res Y * p it ch * s i z e o f ( →֓ unsi g ne d s ho rt i n t ) ) ; // Line 71 , g e t t i n g the d i f f e r e n t −type r e s u l t s C U D A _ C H E C K _ R E T U R N ( cu d a M e m cp y ( h_ res , d_ res , res Y * p it ch s i z e o f ( u nsi g n e d s ho rt i n t ) , cu d a M e m cp y D e v ic e T o H o st ) ) ;

*

→֓

8. Examine the following modifications for improving the stand-alone CUDA AES implementations of Section ??: (a) Move the tables from constant to shared memory : the constant memory cache is faster than global memory, but it is still slower than shared memory. Will the speed improvement offset the need to copy the tables to constant memory for every block of threads? (b) Process multiple 16-byte blocks per thread : turning rijndaelGPUEncrypt() into a device function and introducing another global function as a front-end to it, to be called by rijndaelEncryptFE(), should require the smallest possible effort.

Modify the source code of the Version #2 program in order to introduce your chosen changes, and measure the performance improvement obtained. . Which of offers the biggest -if any- performance improvement?

Answer (a) Moving the tables from constant to shared memory The following prefix code is required in the rijndaelGPUEncrypt function contained in the rijndael device.cu file, to copy the five encoding tables residing in constant memory to shared memory: _ _ glo b a l_ _ voi d r i j n d a e l G P U E n cr y p t ( i n t nrounds , u 32 int N) { _ _ sh a re d _ _ u 32 s Te 0 [ 2 5 6 ] ; _ _ sh a re d _ _ u 32 s Te 1 [ 2 5 6 ] ; _ _ sh a re d _ _ u 32 s Te 2 [ 2 5 6 ] ; _ _ sh a re d _ _ u 32 s Te 3 [ 2 5 6 ] ; _ _ sh a re d _ _ u 32 s Te 4 [ 2 5 6 ] ;

*

data , →֓

f o r ( i n t i=th re ad Id x . x ; i< 2 5 6 ; i+= b lo ck D im . x ) { s Te 0 [ i ] = Te 0 [ i ] ; s Te 1 [ i ] = Te 1 [ i ] ; s Te 2 [ i ] = Te 2 [ i ] ; s Te 3 [ i ] = Te 3 [ i ] ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

134

CHAPTER 6. GPU PROGRAMMING s Te 4 [ i ] = Te 4 [ i ] ; } _ _ syn cth re a d s ( ) ; ...

(b) Processing multiple 16-byte blocks per thread This modification can be achieved with fewer changes, by reducing the size of the grid and introducing a loop in the rijndaelGPUEncrypt function. The resulting code, combining the changes of the previous step, is shown below:

m B LO C K SP E R T H R E A D ) ) ; w

m D

m D

m

w

my

m

e

U

In terms of performance, there is virtually no change as the encryption/decryption of a single block already constitutes a sufficiently

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

135 heavy computational load.

9. The MPI cluster AES implementations of Section 6.12.2 does not provide overlapping of communication and computation. This issue could be addressed if a new “work item” was downloaded by the worker nodes while the GPU was processing an already downloaded part. Modify the MPI solution to provide this functionality. Do you expect any problems with the load-balancing of the modified solution?

Answer A solution to this problem could be that each worker sets-up buffer space to hold three work items, and starts processing one while using a nonblocking send and a non-blocking receive operations to communicate the completed and next pending work items respectively.

The only modifications required, relate to the worker code as shown below for the case of a GPU worker. An identical arrangement is possible for a CPU worker. The communication/computation overlap is achieved by using immediate communication primitives to exchange data with the master (lines 31, 36) while computation is taking place (line 38). There is no need to use immediate functions in lines 29 and 34, as scalar primitive types can be buffered by MPI : 1 2 3 4 5

wo

6

m

[3] [4]

Pe B

7 8 9

w

10 11 12 13

w w w

w w w

m m m

14 15

MP

Reque

Rq

dRq

16 17

mp ne Bu

18 19

dBu

20 21

M

22

MP

23

m R P

CO

P

w UN

M

pos [ _ O R LD

Bu

NED_CHAR

] 0

TA G _ W OR K w TA G _D A TA

P

CO

_

O R LD

→ →֓

24 25 26 27 28 29 30

M

m m

31

m P I_ C

_

M R LD , &sen d R q ) ;

32 33

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

136

CHAPTER 6. GPU PROGRAMMING M P I_ R e cv (&( pos [ n e xt B u ff ] ) , 1 , MP I _I NT , 0 , TA G _ W OR K , →֓ M P I_ C O M M _ W O R LD , &stat ) ; i f ( pos [ n e xt B u ff ] > 0 ) M P I_ Ire cv ( w o rk B u ff [ n e xt B u ff ] , w o rk I tem Siz e , →֓ M P I_ U N SIG N E D _ C H A R , 0 , TA G _ D A TA , M P I_ C O M M _ W O R LD , →֓ &get R q ) ;

34 35 36

37

r i j n d a e l E n c r y p t F E ( rk , key bits , w o rk B u ff [ a c ti v e B u ff ] , →֓ w o rk B u ff [ a c t i v e B u ff ] , a ct u a l Si z e [ a ct i v e B u ff ] , →֓ th r P e r B lo ck ) ;

38

39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59

}

// make s ur e comm uni cati ons are c om pl et e b e f o r e up dati ng →֓ the i n d i c e s and comntinuing i f ( pos [ co m p le te d B u ff ] > 0 ) M P I_ W ai t (& send Rq , M P I_ S T A T U S _ IG N O R E ) ; i f ( pos [ n e xt B u ff ] > 0 ) { M P I_ W ait (& get Rq , &stat ) ; M P I_ G e t_ co u n t (& stat , M P I_ U N SI G N E D _ C H A R , &(→֓ a ct u a l Si z e [ n e xt B u ff ] ) ) ; to tal W o rk += a ct u a l Si z e [ n e xt B u ff ] ; } // update i n d i c e s co m p le te d B u ff = a c t i v e B u ff ; a c t i v e B u ff = n e xt B u ff ; n e xt B u ff = ( n e xt B u ff + 1 ) % 3 ; } ri j n d a e l S h u td o w n ( ) ; cout << ” Worker ” << rank << ” pro ce s s e d ” << to t al W o rk << →֓ endl ; d e l e t e [ ] w o rk B u ff [ 0 ] ; d e l e t e [ ] w o rk B u ff [ 1 ] ; d e l e t e [ ] w o rk B u ff [ 2 ] ;

The three work item buffers are addressed individually by the completedBuff, activeBuff and nextBuff indices. Each buffer is accompanied by a dedicated position (pos array) and length (actualSize array) variables. While one buffer is being processed (pointed to by activeBuff), another holding previous results (pointed to by completedBuff) is being send to the master node (line 31) and the third one (pointed to by nextBuff) is being filled with new input data (line 36).

In all the communication transactions, the pos array members guard against sending dummy data to the master (line 30) and against waiting for new data when the input is exhausted (lines 26 and 35).

A problem may arise with the load balancing of this solution if a very slow worker is participating in the proceedings. It is possible that it can delay the termination of the overall execution by receiving a pending work item that could be processed faster at another node.

10. Modify the MPI cluster AES implementations of Section 6.12.2 so that only two types of messages are needed for data exchange, instead of the current three. How can this be combined with the modification of the previous exercise?

Answer The messages tagged by the TAG WORK label can be eliminated from the communication protocol if the master node maintains a record of which part of the data is assigned to each worker node respectively. When a single work item is assigned to each worker, a simple integer array with

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

137 as many element as the number of workers, is sufficient for handling this requirement. i n t a s s i g n e d W o rk I te m [ co m m _ siz e ] ;

If more than a single work item is assigned at a time to a worker, then the assignedWorkItem array has to become an array of containers, such as a vector<int>: vec t or <int > a ss i g n e d W o rk I te m [ co m m _ siz e ] ;

The termination of the workers can be achieved by sending a work item of length zero.

There is also another alternative. The whole point of this exercise is to reduce the number of messages sent, while at the same time being able to identify the origin/destination in the input/output data of the data parts being exchanged. Towards this end, we could utilize the tag of the messages to carry this information (it is of type int). The only drawback of collapsing the two messages into one is that there is a need for an extra data copy from the generic message buffer used to collect the incoming results (line 33) to the output data repository (line 38): 1 2 3 4 5 6 7 8 9

i f ( ra nk == 0 ) { un si gn e d c ha r * io b u f ; un si gn e d c har * re cv b u f ; i f ( ( f = fo p en ( ar g v [ 1 ] , ” r ” ) ) == NULL ) { fp rin tf ( s t derr , ”Can ’ t open %s \n” , ar gv [ 1 ] ) ; exit ( E X IT _ FA I LU R E ) ; }

10 11

i n t w o rk I t e m Si z e = at oi ( ar gv [ 3 ] ) ;

12 13 14 15

fse ek ( f , 0 , SE E K _ E N D ) ; l Si ze = ft e ll ( f ) ; re w in d ( f ) ;

16 17 18 19 20 21 22

io b u f = new u n sig n ed c ha r [ l Siz e ] ; re cvb u f = new un s i g ne d char [ w o rk It e m Si z e ] ; a s s e r t ( io b u f != N ULL ) ; a s s e r t ( re cv b u f != N ULL ) ; fr ea d ( i ob uf , 1 , l Size , f ) ; fcl o se ( f ) ;

23 24 25 26

tim e val tim ; g e t t i m e o fd a y (& tim , NULL ) ; doubl e tm 2 = tim . t v _ s e c + ( tim . tv_ u se c / 1 0 0 0 0 0 0 . 0 ) ;

27 28 29 30 31 32 33

34 35 36 37 38 39

// master main loop i n t pos = 0 ; w h i l e ( pos < l Si ze ) { i n t ret Pos , re cv Siz e ; M P I_ R e cv ( r ecv b uf , w o rk I t e m S ize , M P I_ U N SI G N E D _ C H A R , →֓ M P I_ A N Y_ SO U R C E , M P I_A N Y_ TA G , M P I_ C O M M _ W O R LD , &stat →֓ ); M P I_ G e t_ co u n t (& stat , M P I_ U N SI G N E D _ C H A R , &re cv S iz e ) ; i f ( re cv Si z e > 0 ) { r et P o s = st at . M P I_ TA G ; m em cp y ( io b u f + ret Pos , r e cvb uf , re cv S i z e ) ; }

40 41 42

// a s s i g n next work item i n t a ctu a l Si z e = ( w o rk It e m Si z e < l Size − pos ) ? →֓ w o rk Ite m Si z e : ( l Size − pos ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

138

CHAPTER 6. GPU PROGRAMMING

43 44

}

45

M P I_ Se n d ( io b u f + pos , a ct u al Si ze , M P I_ U N SIG N E D _ C H A R , →֓ stat . M PI _SO U R C E , pos , M P I_ C O M M _ W O R LD ) ; pos += a ct u a l Si z e ;

46

// wait f o r l a s t r e s u l t s f o r ( i n t i = 1 ; i < co m m _ s i z e ; i++) { i n t ret Pos , re cv Siz e ; M P I_ R e cv ( r ecv b uf , w o rk I t em S ize , M P I_ U N SI G N E D _ C H A R , →֓ M P I_ A N Y_ SO U R C E , M P I_A N Y_ TA G , M P I_ C O M M _ W O R LD , &stat →֓ ); M P I_ G e t_ co u n t (& stat , M P I_ U N SI G N E D _ C H A R , &re cv Siz e ) ; i f ( re cv Si z e > 0 ) { r et P o s = st at . M P I_ TA G ; m em cp y ( io b u f + ret Pos , r e cvb uf , re cv S i z e ) ; }

47 48 49 50 51

52 53 54 55 56 57 58

// send te r m i n a t i o n s i g n a l M P I_ Se n d ( r ecv buf , 0 , M P I_ U N S IG N E D _ C H A R , stat . →֓ M PI_SO U R C E , 0 , M P I_ C O M M _ W O R LD ) ;

59 60

} g e t t i m e o fd a y (& tim , NULL ) ; doubl e tm 3 = tim . t v _ s e c + ( tim . tv_ u se c / 1 0 0 0 0 0 0 . 0 ) ;

61 62 63 64

FIL E * fo ut ; i f ( ( f out = fo p en ( argv [ 2 ] , ”w” ) ) == NULL ) { fp rin tf ( s t derr , ”Can ’ t open %s \n” , ar gv [ 2 ] ) ; exit ( E X IT _ FA I LU R E ) ; } fw rite ( i obuf , 1 , l Si ze , fout ) ; fcl o se ( f out ) ; d e l e t e [ ] io b u f ; d e l e t e [ ] re cvb u f ;

65 66 67 68 69 70 71 72 73 74 75

g e tti m e o fd a y (& ti me Mai n , NULL ) ; double tm 4 = tim e M ai n . tv _ s ec + ( tim e M ai n . tv _ u se c / →֓ 1000000 .0) ;

76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

101

} else {

// p r i nt −out some t i m i n g i nf o r m a t i o n p r i n tf ( ”%.9 l f \ t %.9 l f \n” , tm 4 − tm 1 , tm 3 − tm 2 ) ; // GPU worker i n t w o rk I t e m Si z e = at oi ( ar gv [ 3 ] ) ; i n t th r P e r B lo ck = atoi ( ar gv [ 4 ] ) ; i n t pos [ 3 ] = { 0 , 0 , 0 } ; i n t to t al W o rk = 0 ; unsi g ne d char * w o rk B u ff [ 3 ] ; w o rk B u ff [ 0 ] = new u ns i g n e d c har [ w o rk It e m Si z e ] ; w o rk B u ff [ 1 ] = new u ns i g n e d c har [ w o rk It e m Si z e ] ; w o rk B u ff [ 2 ] = new u ns i g n e d c har [ w o rk It e m Si z e ] ; i n t a ct u a l Si z e [ 3 ] = { 0 , 0 , 0 } ; M P I_ R e q u e st get Rq , s en d R q ; // i n d i c e s f o r work item b u f f e r

→֓

M P I_ Se n d ( w o rk B u ff [ co m p le te d B u ff ] , 0 , M P I_ U N S IG N E D _ C H A R , B O OR M w w

0 , →֓

i n t a c t i v e B u ff = 0 ; management i n t co m p le te d B u ff = 2 ; i n t n e xt B u ff = 1 ;

M

102 103 104 105 106

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

139 107

108 109

M

110

w P

UN

NED_CHAR

0

P

w N Y_ TA G

P

CO

_

O R LD →֓

111

dae En w

112

FE

wo

Bu

[a

Bu

]

113 114 115

M

116

W

117

pos [ n e xt B u ff ] = st at . M P I_ TA G ; to tal W o rk += a ct u a l Si z e [ n e xt B u ff ] ;

118 119 120

co m p le te d B u ff = a c t i v e B u ff ; a c t i v e B u ff = n e xt B u ff ; n e xt B u ff = ( n e xt B u ff + 1 ) % 3 ;

121 122 123 124 125 126 127 128 129 130

}

} ri j n d a e l S h u td o w n ( ) ; cout << ” Worker ” << rank << ” pro ce s s e d ” << to tal W o rk << →֓ endl ; d e l e t e [ ] w o rk B u ff [ 0 ] ; d e l e t e [ ] w o rk B u ff [ 1 ] ; d e l e t e [ ] w o rk B u ff [ 2 ] ;

In the above code the explicit communication of the buffer positions is eliminated. Termination and first job request conditions are detected by checking the length of the input (line 35) and output data (line 104) communicated respectively.

11. The whole point of the multi-core “adventure”, is to accelerate our programs. This should be our sole focus, beyond any mis- or pre-conceptions.

The evaluation of the different AES parallel implementations conducted in Section 6.12.2.4 considered only the encryption process, disregarding any I/O costs incurred. Perform your own experiment where the overall execution time is considered and not just the encryption time. Make sure that the file cache provided by the operating system is not utilized by: Either, calling the following from the command-line (root permissions are required): $

sync

; echo 3 > / proc / sys / vm / d ro p _ ca ch e s

Or, calling the posix fadvice() function from within your program, prior to any I/O: #i n c l u d e < u ni st d . h> #i n c l u d e < f c n t l . h> int ma i n ( int argc , cha r * ar gv [ ] ) { int fd ; fd = open ( argv [ 1 ] , O _ R D O N LY ) ; // Open the fil e h o ld in g the →֓ in p u t data fd at a s y n c ( fd ) ; p o si x _ fa d v i s e ( fd , 0 , 0 , P O S IX _ F A D V _ D O N T N E E D ) ; // cl ea r c ac h e cl o se ( fd ) ; . . . }

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

140

CHAPTER 6. GPU PROGRAMMING Analyze your findings.

Answer This is an activity that is to be carried out by the students. The following command line can be used for generating a random 32MB input file to be used in testing: $

dd i f =/dev / u ran d o m of=in 3 2 M B yte s bs=32M co u n t=1

12. Modify the program shown in Listing ??, so that the reduction phases are shortened to one, using the shuffling-down operation multiple times.

Answer 1 2 3

_ _ glo b a l_ _ voi d co u n t O d d s ( i n t * d , i n t N , i n t { e xte rn _ _ sh a re d _ _ i n t co u n t [ ] ;

* odds )

4

i n t my ID = b lo ck Id x . x * b lo ck D im . x + th re ad Id x . x ; i n t to t al T h r = b lo ck D im . x * gri d D im . x ; i n t lo cal ID = th re ad Id x . x ; i n t to ta l W a rp s = b lo ck D im . x / 32 + ( b lo ck D im . x % 32 != 0 ) ; i n t w a rp ID = lo cal ID / 3 2 ; co u n t [ lo cal ID ] = 0 ; f o r ( i n t i=my ID ; i < N ; i+= to tal T h r ) co u n t [ lo cal ID ] += ( d [ i ] % 2 ) ; _ _ s yn ct h re a d s ( ) ;

5 6 7 8 9 10 11 12 13 14

// re d u c t i o n phas e : sum up the warp r e s u l t s w h i l e ( to ta l W a rp s > 0 ) { u n s i g n e d mas k = 0 x f f f f f f f f ; i n t val = co u n t [ lo c al ID ] ; f o r ( i n t o f f s et = 1 6 ; o f f s et > 0 ; o f f s et /= 2 ) { val += _ _ sh fl _ d o w n _ syn c ( mask , val , o f fs et ) ; } co u n t [ lo cal ID ] = 0 ; to ta l W a rp s /= 3 2 ; _ _ s yn ct h re a d s ( ) ; i f ( lo cal ID % 32 == 0 ) // la n e 0 in a l l warps co u n t [ w arp ID ] = val ; _ _ sy n cth re a d s ( ) ; i f ( w arp ID==0) p ri n tf ( ”## %i %i %i \n” , war p ID , l o cal ID , co u n t [ lo cal ID ] ) ; }

15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47

}

i f ( w arp ID==0) { u n s i g n e d mas k = 0 x f f f f f f f f ; i n t val = co u n t [ lo c al ID ] ; f o r ( i n t o f f s et = 1 6 ; o f f s et > 0 ; o f f s et /= 2 ) { val += _ _ sh fl _ d o w n _ syn c ( mask , val , o f fs et ) ; } co u n t [ 0 ] = val ; } // add to g l o b a l c o u n t e r i f ( lo cal ID == 0 ) ato m ic A d d ( odds , co u n t [ 0 ] ) ;

13. Listing ?? presents a kernel that uses grid synchronization for filtering an array of numbers. The problem is that we can only launch one block per SM, and given that each thread processes one data element, this limits the maximum problem size we can handle. Write a modified version of the kernel in Listing ?? that processes multiple elements per thread. Are there any shortcomings that you will have to deal with?

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

141 Answer The following kernel processes multiple elements per thread. The shortcoming is that the filtered data are reordered in the destination array relative to the source array. By sacrificing coalesced memory access, this can be overcome.

1 2 3 4 5 6 7

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− _ _ glo b a l_ _ v oid f il t e r O d d s ( i n t * src , i n t * dest , i n t N , o d d B lo ck C o u n t ) { i n t my ID = b lo ck Id x . x * b lo ck D im . x + th re ad Id x . x ; i n t to t al T h r = b lo ck D im . x * gri d D im . x ; int myCount = 0; // t hr e a d s p e c i f i c i n t m y D e st O f f se t ; // t hre a d s p e c i f i c

int

*→֓

8 9 10 11 12

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // Step 1 : p r o c e s s the pa rt o f the inp ut thread f o r ( i n t i = my ID ; i < N ; i += to t al T h r ) m y C o u n t += src [ i ] % 2 ;

as s i g n e d to the →֓

13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // Step 2 : f i n d the number o f odds as s i g n e d to each warp co a le sce d _ g ro u p cg = co a le sce d _ th re a d s ( ) ; i n t val = m y C o u n t ; f o r ( i n t o f f s et = 1 ; o f f s e t < 3 2 ; o f f s et *= 2 ) { i n t tmp = cg . sh fl _ u p ( val , o f f s et ) ; i f ( cg . th re a d _ ra n k ( ) >= o f f s et ) // so u r c e e x i s t s ? val += tmp ; } m y D e st O f f s e t = val ; i n t gr o u p O f f s e t = 0 ; i f ( cg . th re a d _ ra n k ( ) == cg . s ize ( ) − 1 ) // l a s t th rea d →֓ has the t o t a l count f o r the group gr o u p O ff s e t = at o m ic A d d ( o d d B lo ck C o u n t + b lo ck Id x . x , val ) ;

28 29 30

// pr o p a g at e to the whole group / warp gro u p O ff s e t = cg . shfl ( g ro u p O ff s et , cg . siz e ( ) − 1 ) ;

31 32 33 34 35 36 37

// ” S h i f t ” warp o f f s e t s up by one p o s i t i o n val = cg . sh fl_ u p ( m y D est O f f s et , 1 ) ; i f ( cg . th re a d _ ra n k ( ) == 0 ) m y D e st O f fs e t = 0 ; else m y D e st O f f s e t = val ;

38 39 40 41

// a d j u s t o f f s e t s f o r the b l o c k i f ( cg . th re a d _ ra n k ( ) > 0 ) m y D e st O f fs e t += gro u p O f f s e t ;

42 43 44 45

// wait f o r a l l blo cks to r eac h gri d _ gr o u p g = th is _ gri d ( ) ; g . s ync ( ) ;

t h i s po int

46 47 48 49

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // Step 3 : f i n d the bl ock−s p e c i f i c o f f s e t s i n t * accu m m = o d d B lo ck C o u n t + g r id D im . x ;

50 51 52 53 54 55 56 57

i f ( my ID < grid D im . x ) { i n t tmp = ato m ic A d d ( accum m , o d d B lo ck C o u n t [ my ID ] ) ; o d d B lo ck C o u n t [ my ID ] = tmp ; } // wait f o r a l l the bl o ck o f f s e t s to be updated g . s ync ( ) ;

58 59 60

// a d j u s t the thr ea d o f f s e t s f o r the g r i d m y D e st O f f s e t += o d d B lo ck C o u n t [ b lo ck Id x . x ] ;

61 62 63 64 65

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // Step 4 : move the data f o r ( i n t i = my ID ; i < N ; i += to t al T h r ) {

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

142

CHAPTER 6. GPU PROGRAMMING

66 67 68 69 70

}

}

i n t tmp = src [ i ] ; i f ( tmp % 2 ) dest [ m y D e st O f f s e t++] = tmp ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Chapter 7

GPU and Accelerator Programming : OpenCL Exercises 1. Write a device query program that will report all the OpenCL devices present in your computer, along with the version of OpenCL they support, the number of compute units, the size of the global memory and the size of the local memory available in each one.

Answer 1 2 3 4 5 6 7 8 9 10

MAXNUMD V

11 12 13

N m m

14 15 16

M

17

M

m

18 19 20 21 22

Num

e e e G P Num C

e

e

SU C C

numP

e

e e umP

23 24 25 26 27 28 29

m

30 31

m

32

C

P

FO R M V R

N

Na me

am

n

33 34 35

143

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

144CHAPTER 7. GPU AND ACCELERATOR PROGRAMMING : OPENCL err N u m = cl G e t D e vi ce ID s ( p l a tf o rm s [ p ID ] , C L_ D E V IC E _ T Y P E _ A LL →֓ , M A X N UM D EV , dev ID , &n u m D ev ) ; i f ( er r N u m != C L_ SU C C E SS | | n u m D ev <= 0 ) { cer r << ” F a i l e d to f i n d any OpenCL d e v i c e s . ” << en d l ; re t ur n 2 ; }

36 37 38 39 40 41 42

f o r ( i n t i = 0 ; i < n u m D ev ; i++) { err N u m = cl G e t D e v i c e In fo ( d ev ID [ i ] , C L_ D EV IC E_ N A M E , →֓ 100 , ( voi d * ) dev Na me , &n am e Le n ) ; i f ( err N u m == C L_ SU C C E SS ) cout << ”\ t De vic e ” << i << ” i s ” << d e v N am e << en dl →֓ ;

43 44 45 46 47 48

cl_ u in t cu ; err N u m = cl G e t D e v i c e In fo ( d ev ID [ i ] , →֓ C L_ D E V IC E _ M A X _ C O M P U T E _ U N IT S , s i z e o f ( cl _ u in t ) , ( →֓ void * ) &cu , NULL ) ; i f ( err N u m == C L_ SU C C E SS ) cout << ”\ t De vic e ” << i << ” has ” << cu << ” →֓ compute u n i t s \n” ;

49 50

51 52 53

cl _ u lo n g glo b al M e m ; err N u m = cl G e t D e v i c e In fo ( d ev ID [ i ] , →֓ C L_ D E V IC E _ G LO B A L_ M E M _ SIZ E , s i z e o f ( cl_ u lo n g ) , ( →֓ void * ) &gl oba l Mem , NULL ) ; i f ( err N u m == C L_ SU C C E SS ) cout << ”\ t De vi ce ” << i << ” has ” << glo b al M e m << ”→֓ byte s g l o b a l memory\n” ;

54 55

56 57 58 59 60

61 62 63

}

64

}

cl_ u lo n g lo c a ll M e m ; err N u m = cl G e t D e v i c e In fo ( d ev ID [ i ] , →֓ C L_ D E V IC E _ LO C A L_ M E M _ SIZ E , s i z e o f ( cl _ u lo n g ) , ( voi d →֓ * ) &locall Mem , N ULL ) ; i f ( err N u m == C L_ SU C C E SS ) cout << ”\ t De vice ” << i << ” has ” << lo c a l l M e m << ”→֓ byt e s l o c a l memory\n” ;

65 66 67

}

re tu rn

0;

2. Implement the trapezoid rule integration technique in OpenMP for an arbitrary function f (). As explained in Section ??, the definite integral between [a, b] of f () can be approximated by the formula

h

! n−1 f (a) + f (b) Σ + f (xi) 2 i=1

where n is the number of divisions and h = b−na . Explore the following possibilities : (a) have the device calculate the target function’s f (xi) values and the host accumulate the sum. (b) Have the device perform all the calculations, including the reduction of the final result. For the latter approach employ work group functions.

Answer Host program: 1 2 3 4

#i n c l u d e < s t d l i b . h> #i n c l u d e <i os t r e am > #i n c l u d e <s t d i o . h> #i n c l u d e <atomic>

5

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

145

1

mm

1

m

1 1

O

1 1 1 1 1 1 2 2 2 2 2

R

2

P

m

2 2 2 2 3 3

O

3

M

3 3

M mO

3 3 3 3 3 4 4 4 4

N m

4 4

m

4 4 4 4 5 5 5 5

Nu

Ge De

dev

C L_ D E V

E_

AX_CO

P UTE_UN

S

→֓

N m

5 5 5 5 5

!

5

p

mAnd

6 6

m

6 6 6 6 6 6 6 6 7 7 7

O

7

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

146CHAPTER 7. GPU AND ACCELERATOR PROGRAMMING : OPENCL O

74 75 76 77 78 79 80 81 82 83 84 85 86 87

O

88 89

e

90

Nu

e Ke

e A

Num

Se K e

e A

V

Po

0

e

91

0

eo

em

92 93

Num

Se K e

e A

eo

94

Num

K

A

e

95

N m

o

96 97 98 99 100 101 102 103 104 105 106

N m

107

ND

108

{

109 110 111

}

112 113

cl e an U p ( ) ; re tu rn 1 ;

if

( ! O C L 2 su p p o rt ) er r N u m = cl E n q u e u e R e a d B u ffe r ( q , d_dat a , C L _T RU E , 0 , s i z e o f ( →֓ f l o a t ) , data , 0 , NULL , N ULL ) ; else { //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // wait f o r the k e r n e l to f i n i s h cl_ e ve n t e vl i s t [ ] = { ke rn C o m p le te } ; err N u m = cl W a it Fo r E ve n ts ( 1 , e v l i s t ) ; i f ( is E rro r ( err Num , ” F a i l e d to wait f o r ev ent ” ) ) { cl e an U p ( ) ; re tu rn 1 ; } }

114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132

// clean− up a l l o c a t e d memory f l o a t res=* data ; cl e an U p ( ) ; re t ur n res ;

133 134

}

135 136

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

137

i n t main ( i n t argc , char {

138

* ar gv [ ] )

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

147 f l o a t st , en ; i n t divisio n ;

139 140 141

st = at of ( ar gv [ 1 ] ) ; en = at of ( ar gv [ 2 ] ) ; d i v i s i o n = a toi ( a rgv [ 3 ] ) ;

142 143 144 145

cout << ” Re s ult

146 147 148 149

}

re tu rn

i s : ” << h o st FE ( st , en , d i v i s i o n ) << endl ;

0;

Device code: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

float f( float x) { re t u r n x * x + 2 }

*

sin ( x ) ;

void a to m ic _ flo a t_ a d d i ti o n ( v o l a t i l e a to m ic_ flo a t * ptr , f l o a t a ) { f l o a t oldv=a to m i c_ lo a d ( ptr ) ; f l o a t newv=oldv+a ; w h i l e ( ! a to m i c _ co m p a re _ e x ch a n g e _ s t r o n g ( ptr , &oldv , ne wv ) ) newv=oldv+a ; } kern el voi d in te g r at e ( glo b al a to m ic_ flo a t * res , f l o a t st , f l o a t →֓ en , i n t div ) { si z e _ t ID = g e t_ g lo b a l_ id ( 0 ) ; s i z e_ t lo cal ID = g e t_ lo ca l_ id ( 0 ) ; i n t to t al W I = g e t_ g lo b a l_ s iz e ( 0 ) ; // t o t a l work items

19

float float float float

20 21 22 23

lo cal R e s = 0 ; st ep = ( en − st ) / div ; s t r i d e = to t al W I * step ; x = st + ( ID +1)* step ; // s kip

f ( s t ) by adding 1 to ID

24

f o r ( i n t i = ID ; i < div −1; i+=to t al W I ) { lo cal R e s += f ( x ) ; x += s t r i d e ; } // f i r s t work item in the whole index range i f ( ID==0) lo cal R e s += ( f ( st ) + f ( en ) ) / 2 ;

25 26 27 28 29 30 31 32 33 34 35

lo cal R e s *= step ;

36

// re du ce the work group l o c a l r e s u l t f l o a t w o rk G ro u p Su m = w o r k _ g ro u p _ re d u ce _ a d d ( lo cal R e s ) ;

37 38

i f ( lo cal ID==0) p r i n tf ( ”WG % l i

39 40 41 42 43 44 45

}

: %f \n” , g e t_ g ro u p _ id ( 0 ) , w o rk G ro u p Su m ) ;

// f i r s t work item in grpup add the l o c a l sum i f ( lo cal ID==0) a to m ic_ f lo a t_ a d d i t i o n ( res , w o rk G ro u p Su m ) ;

3. Use the solution of the previous exercise to calculate π. It is known that ∫1 π = 4 0 x12+1dx. Test your program with single and double floating point precision arithmetic. How does this change influence the accuracy and performance? Answer To accommodate the double-precision calculation, one needs to change the float type variables to double. OpenCL requires an extension for handling 64-bit atomics, which can be enabled by entering in the device code the following lines: 1

#pragma OPENCL EXTENSION c l k h r i n t 6 4 b a s e a t o m i c s : e nable

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

148CHAPTER 7. GPU AND ACCELERATOR PROGRAMMING : OPENCL 2

#pragma OPENCL EXTENSION c l k h r i n t 6 4 e x t e n d e d a t o m i c s : e nable

4. Write an OpenCL program to calculate the outer product of two vectors.

If we have two vectors U = (u1, u2, . . . , un) and V = (v1, v2, . . . , vm), their outer product is a matrix nxm:

U⊗V =

u1v1 u2v1

u1v2 u2v2

. unv1

. u1v2

. . . u1vm . . . u2vm .. . . . . . unvm

(7.1)

Make your program accept as command-line parameters the n and m sizes, so that you can experiment with different workloads.

Answer Host code: 1 2 3 4 5 6 7 8 9 10

mm

11

m

12 13 14 15 16 17 18 19 20 21 22 23 24 25

R

P

m

26 27 28 29 30 31 32 33

Re

34

a

M mO M mO e Ob

d_ou

o

n

35 36 37 38 39 40 41 42

ho

u

o

m

o

ou

43 44

N m

45 46

m

47 48

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

149 49 50 51 52

mA

53 54 55

m

56 57 58 59 60 61 62

M M

AD ON

M M R

D ON

N m 63 64 65 66 67

C

68

B

C

m

N m 69 70 71 72 73 74

N m 75 76 77 78 79 80 81

N m

W

82

N m

W

m

83

err N u m |= cl Se t K e rn e l A rg d_u ) ; err N u m |= cl Se t K e rn e l A rg err N u m |= cl Se t K e rn e l A rg d_v ) ; err N u m |= cl Se t K e rn e l A rg err N u m |= cl Se t K e rn e l A rg d _ o u te r ) ;

84 85 86 87 88

*)

&→֓

( kern ,

0,

s i z e o f ( cl _ m em ) , ( v oi d

( kern , ( kern ,

1, 2,

s i z e o f ( i n t ) , ( v oi d * ) &n ) ; s i z e o f ( cl _ m em ) , ( v oi d * ) &→֓

( kern , ( kern ,

3, 4,

s i z e o f ( i n t ) , ( void * ) &m ) ; s i z e o f ( cl _ m em ) , ( v oi d * ) &→֓

89

i f ( is E rro r ( err Num , ” F a i l e d to s e t k e r n e l p a ra m et er s ” ) ) { cl e an U p ( ) ; re tu rn ; }

90 91 92 93 94 95

si z e _ t di m X = ( n + 15 ) / 16 * 1 6 ; si z e _ t di m Y = ( m + 15 ) / 16 * 1 6 ; cout << dim X << ” ” << dim Y << endl ; si z e _ t id x Sp ace [ ] = { dim X , di m Y } ; si z e _ t lo ca l W o rk Si z e [ ] = { 16 , 16 } ;

96 97 98 99 100 101

// lau nc h GPU k e r n e l err N u m = cl E n q u e u e N D R a n g e K e rn e l ( q , kern , 2 , NULL , idx Spac e , →֓ lo ca l W o rk Si z e , 0 , NULL , &ke rn C o m p le te ) ; i f ( is E rro r ( err Num , ” F a i l e d to la unc h k e r n e l ” ) ) { cl e an U p ( ) ; re tu rn ; }

102 103 104 105 106 107 108 109 110 111

}

err N u m = cl E n q u e u e R e a d B u f f e r ( q , d _o ut er , C L_T R UE , 0 , n * m s i z e o f ( f l o a t ) , outer , 0 , NULL , NULL ) ;

*

→֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

150CHAPTER 7. GPU AND ACCELERATOR PROGRAMMING : OPENCL 112 113

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

114 115 116 117

i n t main ( i n t argc , char { int n , m ;

* ar gv [ ] )

118

n = at of ( ar gv [ 1 ] ) ; m = at of ( ar gv [ 2 ] ) ;

119 120 121

u n iq u e _ p tr < f l o a t [ ] > u = m a ke _ u n iq u e < f l o a t [ ] > ( n ) ; u n iq u e _ p tr < f l o a t [ ] > v = m a ke _ u n iq u e < f l o a t [ ] > ( m ) ; u n iq u e _ p tr < f l o a t [ ] > o u t er = m a ke _ u n iq u e < f l o a t [ ] > ( n

122 123 124

*

m) ;

125

f o r ( i n t i = 0 ; i < n ; i++) u[i] = i ; f o r ( i n t i = 0 ; i < m ; i++) v[i] = i;

126 127 128 129 130

h o st FE ( u . get ( ) , n , v . get ( ) , m ,

131

o u ter . get ( ) ) ;

132

// pr i nt −out the r e s u l t f o r ( i n t j = 0 ; j < m ; j++) { f o r ( i n t i = 0 ; i < n ; i++) cout << o u ter [ j * n + i ] << ” ” ; cout << endl ; }

133 134 135 136 137 138 139 140 141 142 143

}

cl e an U p ( ) ; re tu rn 0 ;

Device code: 1 2 3 4 5

kern el v oid o u te r P ro d u ct ( glo b al f l o a t i n t m , gl o b al f l o a t * o u ter ) { si z e _ t x ID = g e t_ g lo b a l_ id ( 0 ) ; si z e _ t y ID = g e t_ g lo b a l_ id ( 1 ) ;

6 7 8 9 10

}

*u ,

i n t n , g lo b al f l o a t

* v , →֓

i f ( x ID < n && y ID < m ) { o u ter [ y ID * n + x ID ] = u [ x ID ] * v [ y ID ] ; }

5. Write an OpenCL program to calculate the inner product of two vectors.

If we have two vectors U = (u1, u2, . . . , un) and V = (v1, v2, . . . , vn), their Σn inner or dot product is a scalar defined as the sum: U · V = i=1 ui · vi.

Experiment with two kernels: one that assigns one work item per product ui ·vi, and one that assigns multiple products per work item. Which of the two kernels performs best?

Also try running your program with different work group sizes, e.g. 32, 64, 128, 256, 512. How is the execution time affected?

Answer Host code: 1 2 3 4 5 6 7 8 9

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

151 10 11 12 13 14 15 16 17 18 19 20 21 22 23

s t a t i c cl _ co n te xt cont = 0 ; s t a t i c cl _ co m m a n d _ q u e u e q = 0 ; s t a t i c cl_ p ro gra m pr = 0 ; s t a t i c cl_ ke rn e l ker n = 0 ; s t a t i c cl _m em d_u = 0 ; s t a t i c cl_m em d_v = 0 ; s t a t i c cl _m em d _ in n e r = 0 ;

// i n i t i a l i z e

f o r c l e a n u p check

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * void cl e an U p ( ) { i f ( kern != 0 ) cl R e le a se K e rn e l ( kern ) ;

24

if

( pr != 0 ) cl R e le a se P ro g ra m ( pr ) ;

if

( q != 0 ) cl R e le a se C o m m a n d Q u e u e ( q ) ;

25 26 27 28 29

i f ( c ont != 0 ) { cl R e le a se M e m O b je ct ( d_u ) ; cl R e le a se M e m O b je ct ( d_v ) ; cl R e le a se M e m O b je ct ( d _ in n e r ) ; cl R e le a se C o n te xt ( cont ) ; }

30 31 32 33 34 35 36 37 38

}

39

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // Host f r o nt −end f u n c t i o n t hat a l l o c a t e s the memory and l a u n c h e s →֓ the GPU k e r n e l void h o st FE ( f l o a t * u , i n t n , f l o a t * v , f l o a t &inner , i n t →֓ gro u p S i z e ) { cl _i n t err N u m ; cl_ d e v ic e _ id d ev ID ; cl _ e ve n t ke rn C o m p le te ;

40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77

// co nte xt and queue setup se tu p D e v ic e ( cont , q , NULL , dev ID , C L_ D E V IC E _ T Y P E _ A LL ) ; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // r ead program so ur c e and b u i l d i t i f ( ! se tu p P ro g ra m A n d K e rn e l ( dev ID , cont , ” i n n e r . c l ” , pr , ”→֓ inne r P r o d u c t ” , ker n ) ) { i f ( pr != 0 ) p r i n tf ( ” E rror : %s \n” , g e t C o m p ila t i o n E rro r ( pr , d ev ID ) ) ; cl e an U p ( ) ; re tu rn ; } d_u = cl C re a te B u ffe r ( cont , C L_ M E M _ R E A D _ O N LY , n * s i z e o f ( f l o a t →֓ ) , NULL , &err N u m ) ; i f ( is Erro r ( err Num , ” F a i l e d to a l l o c a t e memory” ) ) { cl e an U p ( ) ; re tu rn ; } d_v = cl C re a te B u ffe r ( cont , C L_ M E M _ R E A D _ O N LY , n * s i z e o f ( f l o a t →֓ ) , NULL , &err N u m ) ; i f ( is E rro r ( err Num , ” F a i l e d to a l l o c a t e memory” ) ) { cl e an U p ( ) ; re tu rn ; } d _ in n e r = cl C re a te B u ffe r ( cont , C L_ M E M _ R E A D _ W R IT E , s i z e o f ( →֓ f l o a t ) , NULL , &err N u m ) ; i f ( is E rro r ( err Num , ” F a i l e d to a l l o c a t e memory” ) ) { cl e an U p ( ) ; re tu rn ; }

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

152CHAPTER 7. GPU AND ACCELERATOR PROGRAMMING : OPENCL err N u m = cl E n q u e u e W ri t e B u ff e r ( q , d_u , C L_T R UE , 0 , n * s i z e o f ( →֓ f l o a t ) , u , 0 , NULL , NULL ) ; err N u m = cl E n q u e u e W ri t e B u ff e r ( q , d_v , C L_T R UE , 0 , n * s i z e o f ( →֓ f l o a t ) , v , 0 , NULL , N ULL ) ; f l o a t p at t e rn =0; cl E n q u e u e F ill B u ff e r ( q , d_inner , ( void * )&pattern , s i z e o f ( f l o a t →֓ ) , 0 , s i z e o f ( f l o a t ) , 0 , NULL , NULL ) ;

78 79 80 81 82

err N u m |= cl Se t K e rn e l A rg d_u ) ; err N u m |= cl Se t K e rn e l A rg err N u m |= cl Se t K e rn e l A rg d_v ) ; err N u m |= cl Se t K e rn e l A rg d _ in n e r ) ;

83 84 85 86

*)

&→֓

( kern ,

0,

s i z e o f ( cl _ m em ) , ( v oi d

( kern , ( kern ,

1, 2,

s i z e o f ( i n t ) , ( v oi d * ) &n ) ; s i z e o f ( cl _ m em ) , ( v oi d * ) &→֓

( kern ,

3,

s i z e o f ( cl _ m em ) , ( v oi d

*)

&→֓

87

i f ( is E rro r ( err Num , ” F a i l e d to s e t k e r n e l p a ra m et er s ” ) ) { cl e an U p ( ) ; re tu rn ; }

88 89 90 91 92 93

si z e _ t di m X = ( n + gr ou p Si ze −1) / gr o u p Si z e * gr o u p Si z e ; si z e _ t id x Sp ac e [ ] = { di m X } ; si z e _ t lo ca l W o rk Si z e [ ] = { ( si z e _ t ) gr o u p Si z e } ;

94 95 96 97

// lau nc h GPU k e r n e l err N u m = cl E n q u e u e N D R a n g e K e rn e l ( q , kern , 1 , NULL , idx Spac e , lo ca l W o rk Si z e , 0 , NULL , &ke rn C o m p le te ) ; i f ( is Er r o r ( err Num , ” F a i l e d to lau nc h k e r n e l ” ) ) { cl e an U p ( ) ; re tu rn ; }

98 99 100 101 102 103 104 105 106

err N u m = cl E n q u e u e R e a d B u f f e r f l o a t ) , &inner , 0 , NULL ,

( q , d _in n er , C L_T R UE , 0 , s i z e o f NULL ) ;

107 108

}

109 110

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

111

i n t main ( i n t argc , char { i n t n , gro u p Siz e ;

112 113 114

( →֓

* ar gv [ ] )

n = at of ( ar gv [ 1 ] ) ; gr o u p Si z e = at of ( ar gv [ 2 ] ) ;

115 116 117

u n iq u e _ p tr < f l o a t [ ] > u = m a ke _ u n iq u e < f l o a t [ ] > ( n ) ; u n iq u e _ p tr < f l o a t [ ] > v = m a ke _ u n iq u e < f l o a t [ ] > ( n ) ; f l o a t in n er ;

118 119 120 121

for ( int u[i ] = for ( int v[i] =

122 123 124 125 126

i = 0 ; i < n ; i++) 1; i = 0 ; i < n ; i++) 1;

127 128

h o st FE ( u . get ( ) , n , v . get ( ) ,

129

// pr i nt −out the r e s u l t cout << in n er << endl ;

130 131 132 133 134

→֓

}

inner , gro u p Siz e ) ;

cl e an U p ( ) ; re tu rn 0 ;

Device code: 1 2 3 4 5 6

void a to m ic _ flo a t_ a d d i ti o n ( v o l a t i l e a to m ic_ flo a t * ptr , f l o a t a ) { f l o a t oldv=a to m ic _ lo a d ( ptr ) ; f l o a t newv=oldv+a ; w h i l e ( ! a to m i c_ co m p a re _ e x c h a n g e _ s t r o n g ( ptr , &oldv , ne wv ) ) newv=oldv+a ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

153 7

}

8 9 10 11 12 13

kern el v oid in n e r P ro d u ct ( glo b al f l o a t gl o b al v o l a t i l e f l o a t * o u t er ) { si z e _ t x ID = g e t_ g lo b a l_ id ( 0 ) ; s i z e_ t lo cal ID = g e t_ lo ca l_ id ( 0 ) ;

*u ,

i n t n , glo b al f l o a t

14 15

f l o a t lo cal R e s =0;

16

// perf or m c a l c u l a t i o n onl y i f co rr e s po n d i n g data e x i s t i f ( x ID < n ) lo ca l R e s = u [ x ID ] * v [ x ID ] ;

17 18 19

// re du ce the work group l o c a l r e s u l t f l o a t w o rk G ro u p Su m = w o r k _ g ro u p _ re d u ce _ a d d ( lo cal R e s ) ;

20 21 22 23 24 25 26

* v , →֓

}

// f i r s t work item in group add the l o c a l sum i f ( lo cal ID==0) a to m i c_ flo a t_ a d d i ti o n ( out e r , w o rk G ro u p Su m ) ;

6. Write an OpenCL program for calculating the histogram of a grayscale image. Use an OpenCL image object to represent the input data. Use one work item per pixel.

Answer Host code: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

mm p m ke e em d _h em d _

g

P

m

0 0 0 0

17 18 19 20 21 22 23 24

R

25 26 27 28 29 30

Re

a

e

Ob

d_

g

31 32 33

M mO

34 35 36 37 38 39 40 41

M

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

154CHAPTER 7. GPU AND ACCELERATOR PROGRAMMING : OPENCL 42 43 44 45 46

{ cl _i n t err N u m ; cl_ d e v ic e _ id d ev ID ; cl _ e ve n t ke rn C o m p le te ;

47 48

i n t n Col = img . n u m _ co lo rs +1;

49

// co nte xt and queue setup se tu p D e v ic e ( cont , q , NULL , dev ID , C L_ D E V IC E _ T Y P E _ A LL ) ;

50 51 52 53 54 55 56 57 58 59 60 61

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // r ead program so ur c e and b u i l d i t i f ( ! se tu p P ro g ra m A n d K e rn e l ( dev ID , cont , ” h is t o gr a m . c l ” , pr , ”→֓ h is t o g r a m ” , ker n ) ) { i f ( pr != 0 ) p r i n tf ( ” E rror : %s \n” , g e t C o m p ila t i o n E rro r ( pr , d ev ID ) ) ; cl e an U p ( ) ; re tu rn ; } // image s p e c i f i c a t i o n

62 63

D gDe gDe gDe

64 65 66 67 68 69 70

ge C a ge _ w age_he h age_ w_p m m m m m m

OB x_d y_d 0

C

pe

C

AG

D

;

71 72 73

Fm

74

e_

nne

d

_

UN

NED_

T8

M M_R AD_ON

|

75 76 77

d_

C M M Num

e O

C

HO

78 79 80 81 82 83 84 85 86

m

m N

W

m

87 88 89 90 91 92

M M

93

AD ON

N m 94 95 96 97 98 99 100 101 102

Num

Se K e

e A

0

eo

103

Num

Se K e

e A

1

eo

104

Num

Se K e

e A

2

eo

105

Num

Se K e

e A

3

eo

em

em

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

155 d _h i st ) ; err N u m |= cl Se t K e rn e l A rg ( kern , 4 , s i z e o f ( i n t ) , ( v oid * ) &n Col →֓ ); err N u m |= cl Se t K e rn e l A rg ( kern , 5 , s i z e o f ( cl_ sa m p le r ) , ( void →֓ * ) &sam p le r ) ;

106 107 108

i f ( is E rro r ( err Num , ” F a i l e d to s e t { cl e an U p ( ) ; re tu rn ; }

109 110 111 112 113

k e r n e l p ara m et ers ” ) )

114 115

s i z e_ t dim X = ( img . x _d im + 15 ) / 16 * 1 6 ; s i z e_ t dim Y = ( img . y _d im + 15 ) / 16 * 1 6 ; si z e _ t id x Sp ace [ ] = { dim X , di m Y } ; si z e _ t lo ca l W o rk Si z e [ ] = { 16 , 16 } ;

116 117 118 119 120

// lau nc h GPU k e r n e l err N u m = cl E n q u e u e N D R a n g e K e rn e l ( q , kern , 2 , NULL , idx S pace , lo ca l W o rk Si z e , 0 , NULL , &ke rn C o m p le te ) ; i f ( is E rro r ( err Num , ” F a i l e d to la unc h k e r n e l ” ) ) { cl e an U p ( ) ; re tu rn ; }

121 122 123 124 125 126 127 128

→֓

N m

129 130 131 132 133 134 135

GM

136 137

m

138

m

139 140 141 142 143 144 145 146 147 148 149

Device code: 1 2 3 4 5 6

kern el void h i s to g ra m ( re ad _ o n ly im age 2 d _ t in , i n t w , i n t h , →֓ gl o b al i n t * hist , i n t n Col , sam p le r_ t s ) { si z e _ t x ID = g e t_ g lo b a l_ id ( 0 ) ; si z e _ t y ID = g e t_ g lo b a l_ id ( 1 ) ; si z e _ t x T h re ad s = g e t_ g lo b a l_ si z e ( 0 ) ; si z e _ t y T h re ad s = g e t_ g lo b a l_ si z e ( 1 ) ;

7 8 9 10 11 12 13 14 15

}

i f ( x ID < w && y ID < h ) { int 2 co o rd = { xID , y ID } ; int 4 co lo r = ( int 4 ) re a d _ im a ge i ( in , s , co o rd ) ; i n t gra y = co lo r . x ; a to m ic _ a d d ( hist+gray , 1 ) ; }

7. Modify the program of the previous exercise so that you process multiple pixels per work item.

Answer

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

156CHAPTER 7. GPU AND ACCELERATOR PROGRAMMING : OPENCL The only changes in the host code, affect the index space generation. The code below basically partitions the image across the y-axis amongst the CUs. 1 2 3 4 5 6 7 8 9 10 11

. . . void h o st FE ( P G M Im a ge &img , i n t * hist ) { // d i s c o v e r the number o f CUs cl_ u in t cu ; err N u m = cl G e t D e vi c e In fo ( dev ID , C L_ D E V IC E _ M A X _ C O M P U T E _ U N IT S , →֓ s i z e o f ( c l _ u in t ) , ( voi d * ) &cu , NULL ) ; i f ( er r N u m != C L_ SU C C E SS ) { cl e an U p ( ) ; re tu rn ; }

12

si z e _ t si z e _ t si z e _ t si z e _ t

13 14 15 16

di m X = 1 6 ; di m Y = cu * 1 6 ; id x Sp ace [ ] = { dim X , di m Y } ; lo ca l W o rk Si z e [ ] = { 16 , 16 } ;

17 18 19 20

// lau nc h GPU k e r n e l err N u m = cl E n q u e u e N D R a n g e K e rn e l ( q , kern , 2 , NULL , idx Spac e , →֓ lo ca l W o rk Si z e , 0 , NULL , &ke rn C o m p le te ) ; . . .

In the device code, loops allow the iteration over the whole image: 1 2 3 4 5 6 7 8

kern el void h i s to g ra m ( re ad _ o n ly im age 2 d _ t in , i n t w , i n t h , →֓ gl o b al i n t * hist , i n t n Col , sam p le r_ t s ) { si z e _ t x ID = g e t_ g lo b a l_ id ( 0 ) ; si z e _ t y ID = g e t_ g lo b a l_ id ( 1 ) ; si z e _ t x T h re ad s = g e t_ g lo b a l_ si z e ( 0 ) ; si z e _ t y T h re ad s = g e t_ g lo b a l_ si z e ( 1 ) ; si z e _ t b lo ck ID = g e t_ lo ca l_ l i n e a r_ id ( ) ;

9 10 11 12 13 14 15 16 17

}

f o r ( i n t i=x ID ; i<w ; i+= x T h re ad s ) f o r ( i n t j=y ID ; j<h ; j+= y T h re ad s ) { int 2 co o rd = { i , j } ; int 4 co lo r = ( int 4 ) re a d _ im a ge i ( in , s , co o rd ) ; i n t gr ay = co lo r . x ; a to m ic _ a d d ( hist+gray , 1 ) ; }

8. Modify the histogram calculation of the previous exercise so that your kernel uses local memory to speed up the process. How does your new version compare with the previous one in terms of execution time?

Answer The only changes in the host code, affect the addition of one more kernel parameter, for the dynamically allocated local memory: 1 2 3 4 5 6

. . . void h o st FE ( P G M Im a ge &img , i n t * hist ) { . . . err N u m |= cl Se t K e rn e l A rg ( kern , 6 , s i z e o f ( i n t ) * n Col , NULL ) ; . . .

In the device code, the local memory has to be initialized before the image can be processed. Fences separate the blocks of code that need to complete before the next one:

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

157 w

1 2

{ si z e_t x ID = g e t_ g lo b a l_ id ( 0 ) ; si z e_t y ID = g e t_ g lo b a l_ id ( 1 ) ; si z e_t x T h re ad s = g e t_ g lo b a l_ si z e ( 0 ) ; si z e_t y T h re ad s = g e t_ g lo b a l_ si z e ( 1 ) ; si z e_t lo cal ID = g e t_ lo ca l_ li n e a r_ id ( ) ; si z e_t gr o u p Si z e = g e t_ lo ca l_ s i z e ( 0 ) * g e t_ lo ca l_ s i z e ( 1 ) ;

3 4 5 6 7 8 9

// l o c a l memory i n i t f o r ( i n t i=lo cal ID ; i<n Col ; i+=gro u p Si z e ) lo c H ist [ i ] = 0 ; w o rk_ g ro u p _ b a r ri e r ( C LK _ LO C A L_ M E M _ F E N C E ) ;

10 11 12 13 14

// image p r o c e s s . f o r ( i n t i=x ID ; i<w ; i+= x T h re ad s ) f o r ( i n t j=y ID ; j<h ; j+= y T h re ad s ) { int 2 co o rd = { i , j } ; int 4 co lo r = ( int 4 ) re a d _ im a ge i ( in , s , co o rd ) ; i n t gr ay = co lo r . x ; a to m ic_ a d d ( lo c H ist+gray , 1 ) ; }

15 16 17 18 19 20 21 22 23 24 25 26

w o rk_ g ro u p _ b a r ri e r ( C LK _ LO C A L_ M E M _ F E N C E ) ;

27

// adding the l o c a l hi s t o g r a m to the g l o b a l one f o r ( i n t i=lo cal ID ; i<n Col ; i+=gro u p Si z e ) a to m ic_ a d d ( hist+i , lo c H ist [ i ] ) ;

28 29 30 31

}

9. Create OpenCL implementations of the gsl stats mean() and gsl stats variance() functions offered by the GNU Scientific Library, that produce the mean and variance statistics of a supplied array. To avoid issues with 64-bit extension support, you can settle for the following signatures: f l o a t g sl _ st a ts _ m e a n ( co nst f l o a t DATA [ ] , // Po i nt e r to inp ut →֓ data s i z e _ t S TRIDE , // Step used to read the →֓ i nput . Normally t h i s s h oul d be s e t to →֓ 1. si z e _ t N ) ; // S i z e o f DATA array f l o a t g sl _ s t a t s _ va ri a n ce ( co ns t f l o a t DA T A [ ] , // Same as above . s iz e _t STRI DE , si z e _ t N ) ;

Answer The host code filters out the data that need to be send to the device, if STRIDE is different than 1, by copying the data to be used in a new array. This reduces the amount of data that need to be transferred. The host code is factorized to reduce the number of lines required. This is a substandard solution for execution efficiency, as the data are transferred twice: 1 2 3 4 5 6 7 8 9 10 11 12

mm

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

158CHAPTER 7. GPU AND ACCELERATOR PROGRAMMING : OPENCL 13 14 15 16 17 18 19 20 21 22 23

s t a t i c cl_ p ro gra m pr = 0 ; s t a t i c cl_ ke rn e l ker n = 0 ; s t a t i c cl _m em d _d ata = 0 ; s t a t i c cl _m em d _res = 0 ; co ns t i n t G R O U P S IZ E =256; // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * void cl e an U p ( ) { i f ( kern != 0 ) cl R e le a se K e rn e l ( kern ) ;

24

if

( pr != 0 ) cl R e le a se P ro g ra m ( pr ) ;

if

( q != 0 ) cl R e le a se C o m m a n d Q u e u e ( q ) ;

if

( d _d at a != 0 ) cl R e le a se M e m O b je ct

25 26 27 28 29 30 31 32

if

33 34 35 36 37 38 39

}

( d _d at a ) ;

( d _r es != 0 ) cl R e le a se M e m O b je ct ( d _r es ) ;

i f ( c ont != 0 ) cl R e le a se C o n te xt ( cont ) ;

40 41 42 43 44 45 46 47 48 49 50 51

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * f l o a t h o st FE ( co nst f l o a t DATA [ ] , si z e _t STRI DE , s iz e _ t N , char ke rn N am e ) { cl _i n t err N u m ; cl_ d e v ic e _ id d ev ID ; cl _ e ve n t ke rn C o m p le te ; // p r e p r o c e s s host data in case STRIDE i s not 1 f l o a t * ha ; // hos t ( h * ) and de v i c e ( d * ) p o i n t e r s f l o a t hr es ;

52 53

u n iq u e_p tr <f l o a t [] >

54

i n t N e ffe c t i v e ; i f ( STR ID E ! =1 ) { N e ffe c t i v e = ( N −1)/ ST R ID E + 1 ; tm p D ata = m a k e_ u n iq u e <f l o a t [ ] > ( N e ff e c t i v e ) ; f o r ( i n t i=0 , j=0; i<N ; i+=STRIDE , j++) tm p D ata [ j ] = D ATA [ i ] ; ha = tm p D ata . get ( ) ; } else { ha = ( f l o a t * ) D ATA ; N e ffe c t i v e = N ; }

55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83

*→֓

tm p D ata=n u ll p tr ;

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // co nte xt and queue setup se tu p D e v ic e ( cont , q , NULL , dev ID , C L_ D E V IC E _ T Y P E _ A LL ) ; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // r ead program so ur c e and b u i l d i t i f ( ! se tu p P ro g ra m A n d K e rn e l ( dev ID , cont , ” s t a t i s t i c s . c l ” , pr , →֓ k er n Name , k er n ) ) { i f ( pr != 0 ) p r i n tf ( ” E rror : %s \n” , g e t C o m p ila t i o n E rro r ( pr , d ev ID ) ) ; cl e an U p ( ) ; re tu rn 0 ; } B

M M R AD ON

N

N m

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

159 N m

84

W

85

{

86 87 88

}

89 90

cl e an U p ( ) ; re tu rn 0 ;

d _res = cl C re a te B u ffe r ( cont , C L_ M E M _ R E A D _ W R IT E , s i z e o f ( f l o a t ) →֓ , NULL , &err N u m ) ; f l o a t p att e rn =0; cl E n q u e u e F ill B u ffe r ( q , d_res , ( void * )&pattern , s i z e o f ( f l o a t ) , →֓ 0 , s i z e o f ( f l o a t ) , 0 , NULL , N ULL ) ;

91 92 93 94

err N u m |= cl Se t K e rn e l A rg ( kern , 0 , s i z e o f ( cl_m em ) , ( void * ) &→֓ d _d at a ) ; err N u m |= cl Se t K e rn e l A rg ( kern , 1 , s i z e o f ( i n t ) , ( void * ) &→֓ N e ffe c t i v e ) ; err N u m |= cl Se t K e rn e l A rg ( kern , 2 , s i z e o f ( cl_m em ) , ( void * ) &→֓ d _res ) ; err N u m |= cl Se t K e rn e l A rg ( kern , 3 , s i z e o f ( f l o a t ) * G R O UP S I ZE , →֓ NULL ) ;

95 96 97 98 99

i f ( is E rro r ( err Num , ” F a i l e d to s e t k e r n e l p a ra m et er s ” ) ) { cl e an U p ( ) ; re tu rn 0 ; }

100 101 102 103 104 105 106

// d i s c o v e r the number o f CUs cl_ u in t cu ; err N u m = cl G e t D e vi c e In fo ( dev ID , C L_ D E V IC E _ M A X _ C O M P U T E _ U N IT S , →֓ s i z e o f ( c l _ u in t ) , ( void * ) &cu , NULL ) ; i f ( er r N u m != C L_ SU C C E SS ) { cl e an U p ( ) ; re tu rn 0 ; }

107 108 109 110 111 112 113 114 115

// l i m i t the number o f work gr o ups i f t hey work i f ( cu * G R O U P S I Z E > N ) cu = ( N +GRO UP SIZE −1)/ G R O U P S I ZE ; si z e _ t id x Sp a ce [ ] = { G R O U P SI Z E * cu } ; si z e _ t lo ca l W o rk Si z e [ ] = { G R O U P SI Z E } ;

116 117 118 119 120

w i l l not have any →֓

121

// lau nc h GPU k e r n e l err N u m = cl E n q u e u e N D R a n g e K e rn e l ( q , kern , 1 , NULL , idx Spac e , →֓ lo ca l W o rk Si z e , 0 , NULL , &ke rn C o m p le te ) ; i f ( is E rro r ( err Num , ” F a i l e d to la unc h k e r n e l ” ) ) { cl e an U p ( ) ; re tu rn 0 ; }

122 123 124 125 126 127 128 129 130 131 132

err N u m = cl E n q u e u e R e a d B u f f e r ( q , d_r es , C L _T RUE , 0 , s i z e o f ( →֓ f l o a t ) , &hres , 0 , NULL , NU LL ) ; cl e an U p ( ) ; re t urn hres / N e ff e c t iv e ;

133 134

}

135

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * f l o a t g sl _ sta t s_ m e a n ( co ns t f l o a t DA T A [ ] , si z e _ t ST RIDE , si z e _ t N→֓ ) { re t ur n h o st FE ( DATA , ST RIDE , N , ”sum” ) ; } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * f l o a t g sl _ sta ts_ va ria n ce ( co nst f l o a t DAT A [ ] , s i z e _t STRI DE , →֓ si z e _ t N ) { f l o a t s , s2 ; s = h o st FE ( DATA , ST RIDE , N , ”sum” ) ;

136 137 138 139 140 141 142 143 144

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

160CHAPTER 7. GPU AND ACCELERATOR PROGRAMMING : OPENCL 145 146 147 148 149 150 151

s2 = h o st FE ( DATA , ST RIDE , N , ”sum2” ) ; i n t N e ffe c t i v e ; i f ( STR ID E ! =1 ) N e ffe c t i v e = ( N −1)/ ST R ID E + 1 ; else N e ffe c t i v e = N ; re t ur n ( s2 − s * s ) / N e ffe c t i v e ;

152 153

}

154 155

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

156

i n t main ( i n t argc , char { f l o a t st , en ; int N ;

157 158 159 160

st = at of ( ar gv [ 1 ] ) ; en = at of ( ar gv [ 2 ] ) ; N = at oi ( ar gv [ 3 ] ) ;

161 162 163 164

u n iq u e_p tr <f l o a t [] > data=m ak e _ u n iq u e <f l o a t [ ] > ( N ) ; f l o a t scal e= n u m e ri c _ lim it s <int > : : max ( ) / ( en−st ) ; f o r ( i n t i=0; i<N ; i++) dat a [ i ] = r and ( ) / s cale+ st ;

165 166 167 168 169

p ri n tf ( ”Avg : %l f \n” , g sl _ s t a t s _ m e a n ( dat a . get ( ) , 1 , N ) ) ; p r i n tf ( ” Var : %l f \n” , g sl _ sta ts_ va ri a n ce ( data . get ( ) , 1 , N ) ) ;

170 171 172 173 174

* ar gv [ ] )

}

re tu rn

0;

Device code: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

void a to m ic _ flo a t_ a d d i ti o n ( v o l a t i l e a to m ic_ flo a t * ptr , f l o a t a ) { f l o a t oldv=a to m i c_ lo a d ( ptr ) ; f l o a t newv=oldv+a ; w h i l e ( ! a to m i c_ co m p a re _ e x c h a n g e _ s t r o n g ( ptr , &oldv , ne wv ) ) newv=oldv+a ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− k er n el void sum ( glo b al f l o a t * d , i n t N , glo b al f l o a t lo cal f l o a t * lo cal Su m ) { si z e _ t ID = g e t_ g lo b a l_ id ( 0 ) ; si z e _ t n u m T h re a d s = g e t_ g lo b a l_ si z e ( 0 ) ; s i z e _ t lo cal ID = g e t_ lo ca l_ l i n e a r_ id ( ) ;

16 17

lo ca l Su m [ lo c al ID ] = 0 ;

18

f o r ( i n t i=ID ; i<N ; i+=n u m T h re a d s ) lo cal Su m [ lo cal ID ] += d [ i ] ;

19 20

* to tal Sum ,

21 22

f l o a t g ro u p T o ta l=w o rk_ g ro u p _ re d u ce _ a d d ( lo cal Su m [ lo cal ID ] ) ;

23

i f ( lo cal ID == 0 ) a to m i c _ fl o a t_ a d d i t i o n ( t o t a l Sum , gr o u p T o ta l ) ;

24 25 26

}

27

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− k er n el void sum 2 ( glo b al f l o a t * d , i n t N , glo b al f l o a t lo cal f l o a t * lo cal Su m ) { si z e_t ID = g e t_ g lo b a l_ id ( 0 ) ; si z e_t n u m T h re a d s = g e t_ g lo b a l_ si z e ( 0 ) ; si z e_t lo cal ID = g e t_ lo ca l_ li n e a r_ id ( ) ;

28 29 30 31 32 33 34

→֓

* to tal Sum , →֓

lo ca l Su m [ lo c al ID ] = 0 ;

35 36 37 38 39

f o r ( i n t i=ID ; i<N ; i+=n u m T h re a d s ) { f l o a t tmp=d [ i ] ; lo ca l Su m [ lo ca l ID ] += tmp * tmp ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

161 }

40 41

f l o a t gro u p T o ta l=w o rk_ g ro u p _ re d u ce _ a d d ( lo cal Su m [ lo cal ID ] ) ;

42 43 44 45 46

}

i f ( lo cal ID == 0 ) a to m i c _ fl o a t_ a d d i t i o n ( t o t a l Sum , gr o u p T o ta l ) ;

10. In the filtering example of Section ??, two kernels are employed in order to get the filtered data in order in the output array. Solve the problem using one kernel, by relaxing the keeping-the-order constraint.

Answer The only thing that needs to be modified, is that the kernel should atomically increment a global counter to get the next free position in the filtered data array: Host code: 1 2 3 4 5 6 7 8 9

mm

10

m

11 12 13 14 15 16 17 18 19 20 21 22 23 24

R

P

m

25 26 27 28 29 30 31 32

M

33 34 35

M

D

36 37 38

M

39 40 41 42 43 44 45 46 47

N m

48 49

m

m

m

50 51 52 53

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

162CHAPTER 7. GPU AND ACCELERATOR PROGRAMMING : OPENCL 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96

re tu rn 0 ; } i n t N = a to i ( ar gv [ 1 ] ) ; // co nte xt and queue setup se tu p D e v ic e ( cont , q , NULL , d ev ID ) ; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // r ead program so ur c e and b u i l d i t i f ( ! se tu p P ro g ra m A n d K e rn e l ( dev ID , cont , ” f i l t e r . c l ” , pr , ”→֓ f i l t e r O d d s ” , ker n ) ) { i f ( pr != 0 ) p r i n tf ( ” E rror : %s \n” , g e t C o m p ila t i o n E rro r ( pr , d ev ID ) ) ; cl e an U p ( ) ; re tu rn 1 ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // SVM memory a l l o c a t i o n i n t * data = ( i n t * ) cl SV M A ll o c ( cont , C L_ M E M _ R E A D _ W R IT E , N s i z e o f ( i n t ) , 0) ; i f ( data == N ULL ) { cl e an U p ( ) ; re tu rn 1 ; } f o r ( i n t i = 0 ; i < N ; i++) da ta [ i ] = i ;

*

→֓

i n t * f il t e re d D a ta = ( i n t * ) cl SV M A llo c ( cont , C L_ M E M _ R E A D _ W R IT E →֓ , N * s i z e o f ( i nt ) , 0) ; i f ( fi lt e re d D a ta == N ULL ) { cl e an U p ( ) ; re tu rn 1 ; } i n t n u m W o rk G ro u p s = ( N + 255 ) / 2 5 6 ; i n t * f i l t C o u n t = ( i n t * ) cl SV M A llo c ( cont , C L_ M E M _ R E A D _ W R IT E , →֓ s i z e o f ( i n t ) , 0) ; i f ( f i l t C o u n t == NULL ) { cl e an U p ( ) ; re tu rn 1 ; } * f i l t C o u n t =0;

97 98 99 100 101 102

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // s p e c i f y the p ara m ete rs that w i l l be used f o r the e xe c u t i o n // s et u p the work item index s pace and group s i z e si z e _ t id x Sp ace [ ] = { N } ; si z e _ t lo ca l W o rk Si z e [ ] = { 256 } ;

k e r n e l →֓

103 104 105 106 107 108 109 110 111 112 113 114 115

// enqueue k e r n e l e xe c ut i o n re q u e s t err N u m = cl S e t K e rn e l A rg S V M P o in te r ( kern , 0 , data ) ; err N u m |= cl Se t K e rn e l A rg ( kern , 1 , s i z e o f ( i n t ) , ( void * ) &N ) ; err N u m |= cl S e t K e rn e l A rg S V M P o in te r ( kern , 2 , f i l t e re d D a ta ) ; err N u m |= cl S e t K e rn e l A rg S V M P o in te r ( kern , 3 , f i l t C o u n t ) ; i f ( is E rro r ( err Num , ” F a i l e d to s e t ke r n e l 1 pa ra me t ers ” ) ) { cl e an U p ( ) ; re tu rn 1 ; } N m

ND

116 117 118 119 120 121

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

163 //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // wait f o r the k e r n e l to f i n i s h cl_ e ve n t e vl i s t [ ] = { co u n t K e rn C o m p le te } ; err N u m = cl W a it Fo r E ve n ts ( 1 , e v l i s t ) ; i f ( is E rro r ( err Num , ” F a i l e d to wait f o r ev ent ” ) ) { cl e an U p ( ) ; re tu rn 1 ; }

122 123 124 125 126 127 128 129 130 131 132 133

cout << ” F i l t e r e d data : ” << * f i l t C o u n t << endl ;

134

f o r ( i n t i = 0 ; i < * f il t C o u n t ; i++) cout << f i lt e re d D a ta [ i ] << ” ” ; cout << endl ;

135 136 137 138 139 140 141

}

// r e l e a s e r e s o u r c e s cl e an U p ( ) ; re tu rn 0 ;

Device code: 1 2 3 4

i n l i n e bool p re d i c a t e ( i n t i ) { re t ur n i % 2 ; }

5 6 7 8 9 10 11 12 13 14 15 16

kern el v oid f il t e r O d d s ( glo b al i n t * src , i n t N , glo b al i n t gl o b al v o l a t i l e i n t * f i l t C o u n t ) { i n t ID = g e t_ g lo b a l_ id ( 0 ) ; i f ( ID < N ) { i n t h a ve O d d = p re d i c a t e ( src [ ID ] ) ; i f ( h av e O d d ) { i n t o ff s e t = a to m ic _ fe tch _ a d d ( fil t C ount , 1 ) ; dest [ o f fs et ] = src [ ID ] ;

17 18 19

}

}

* dest , →֓

}

11. Conduct an experiment to find out how changing the image resolution and/or the values of the symbolic constants DEVICEALLOC and CPUALLOC influences the distribution of the load in the hybrid Mandelbrot set calculation program of Section ??.

Answer N/A 12. The hybrid Mandelbrot calculator of Section ?? partitions the target image row-wise. Modify the code so that the partitioning is done column-wise.

As there are more columns than rows in an image, do you think this will allow for better load balancing and faster execution?

Answer The device code remains identical. The changes affect mainly the LoadMonitor class and the way the computed data are moved to the QImage object: 1 2 3 4 5 6

. . . // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * s t r u c t B lo ck D e scr { d ou bl e upper X , upper Y , step X , s t ep Y ; i n t widt h , h ei g h t ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

164CHAPTER 7. GPU AND ACCELERATOR PROGRAMMING : OPENCL 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

i n t start C o l ; }; // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * co nst i n t D E V IC E A LLO C = 6 4 ; c l a s s Lo a d M o n ito r { private : m u tex l ; i n t w idth , h eigh t ; do u ble upper X , u p p er Y ; i n t cu rre n t C o l = 0 ; doubl e x Step , y Step ; public : Lo a d M o n it o r ( double uX , double uY , double lX , doubl e lY , i n t i n t h ) : u p p er X ( u X ) , u p p er Y ( u Y ) , w id th ( w ) , h eigh t ( h ) { x Step = ( lX − u p p er X ) / w id th ; y Step = ( u p p er Y − lY ) / h eigh t ; } bool g e t N e xt B lo ck ( B lo ck D e scr & d , bool is D e vi c e = f a l s e ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // r e t u r n s t r u e i f the d s t r u c t u r e i s p o p u l a t e d . F a l s e i f work done bool Lo a d M o n it o r : : g e t N e xt B lo ck ( B lo ck D e scr & d , bool is D e vi c e ) { lo ck_ gu a rd < m u t ex > lg ( l ) ;

w , →֓

i s →֓

33

i f ( cu rre n t C o l == w id th ) re t ur n f a l s e ;

34 35 36 37 38 39 40 41 42 43 44 45 46 47

}

d . u p p er Y = u p p er Y ; d . u p p er X = u p p er X + x St ep * cu r r e n t C o l ; d . step X = x S tep ; d . step Y = y Step ; d . h ei gh t = h ei gh t ; d . s t a r t C o l = cu rre n t C o l ; i n t to A llo c = ( w id th − cu r r e n t C o l >= D E V IC E A LLO C ) ? D E V IC E A LLO C →֓ : w id th − cu rre n t C o l ; cu r r e n t C o l+=to A ll o c ; d . w id th = to A ll o c ; re t ur n t r u e ;

48 49 50 51 52 53 54 55 56 57 58 59 60 61 62

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * c l a s s H o st FE { private : cl _ co n te xt cont = 0 ; // i n i t i a l i z e f o r c l e a n u p check cl _ co m m a n d _ q u e u e q = 0 ; cl_ p ro gra m pr = 0 ; cl_ ke rn e l ker n = 0 ; bool O C L 2 su p p o rt = t r u e ; un si gn e d char * da ta = 0 ; // image data cl _m em d _d ata = 0 ; // image data i f OpenCL 2 . 0 i s not →֓ supported sh a re d _ p tr < Q Im age > img ; sh a re d _ p tr < Lo a d M o n ito r > mon ; cl_ d e v ic e _ ty p e t ;

63 64 65 66 67 68 69 70 71 72 73 74

public : i n t co ls P ro ce ss e d =0; H o st FE ( sh a re d _ p tr < Q Im age > img , sh a re d _ p tr < Lo a d M o n it o r > →֓ mon , cl _ d e v i c e _ t y p e t ) { t hi s −>i mg=img ; t hi s −>m on=mon ; t hi s −>t=t ; } void o pe ra to r ( ) ( ) ; void cl e an U p ( ) ; };

75

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

165 76 77 78 79 80 81

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void H o st FE : : cl e an U p ( ) { i f ( kern != 0 ) cl R e le a se K e rn e l ( kern ) ;

82

if

( pr != 0 ) cl R e le a se P ro g ra m ( pr ) ;

if

( q != 0 ) cl R e le a se C o m m a n d Q u e u e ( q ) ;

83 84 85 86 87 88 89 90 91 92 93 94 95

i f ( c ont != 0 ) { i f ( O C L 2 su p p o rt ) cl SV M F r e e ( cont , data ) ; else cl R e le a se M e m O b je ct ( d _d ata ) ; cl R e le a se C o n te xt ( cont ) ; }

96 97

}

98

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // Host f r o nt −end f u n c t i o n t hat a l l o c a t e s the memory and l a u n c h e s →֓ the GPU k e r n e l void H o st FE : : o pe r at o r ( ) ( ) { B lo ck D e scr w or k ; cl _i n t err N u m ; cl_ d e v ic e _ id d ev ID ; cl _ e ve n t ke rn C o m p le te ;

99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119

// co nte xt and queue setup se tu p D e v i c e ( cont , q , NULL , dev ID , t ) ; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // r ead program so ur c e and b u i l d i t i f ( ! se tu p P ro g ra m A n d K e rn e l ( dev ID , cont , ” k e r n e l . c l ” , pr , ”→֓ mandelKer nel ” , ker n ) ) { i f ( pr != 0 ) p r i n tf ( ” E rror : %s \n” , g e t C o m p ila t i o n E rro r ( pr , d ev ID ) ) ; cl e an U p ( ) ; re tu rn ; }

120 121 122 123 124 125

O

126 127 128 129 130 131 132 133 134 135 136 137 138 139

O SV M A

140 141 142

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

166CHAPTER 7. GPU AND ACCELERATOR PROGRAMMING : OPENCL wor k . w id th * wor k . h ei gh t * s i z e o f ( u ns i g n e d →֓ char ) , NULL , &err N u m ) ; data = new u nsi g n ed char [ w or k . w id th * w or k . h eig h t ] ;

143 144 145 146 147

148 149 150

}

} th re ad X = ( i n t ) ceil ( work . w id th * 1 . 0 / T H R _ B LK _ X ) ; th re ad Y = ( i n t ) ceil ( work . h eig h t * 1 . 0 / T H R _ B LK _ Y ) ; th re ad X = ( ( th re ad X + B LO C K _ SI D E − 1 ) / B LO C K _ SI D E ) * →֓ B LO C K _ SID E ; // make s u r e that a work item →֓ d i v i d e s the idx S pace e ve nly th re ad Y = ( ( th re ad Y + B LO C K _ SI D E − 1 ) / B LO C K _ SI D E ) * →֓ B LO C K _ SID E ;

151 152 153 154 155

i f ( O C L 2 su p p o rt ) err N u m = cl S e t K e rn e l A rg S V M P o in te r ( kern , 0 , data ) ; else err N u m = cl Se t K e rn e l A rg ( kern , 0 , s i z e o f ( cl_ m em ) , ( void →֓ * ) &d _d ata ) ;

156 157 158 159 160 161 162 163 164 165 166 167

err N u m |= cl Se t K e rn e l A rg ( kern , 1 , s i z e o f * ) &work . u p p er X ) ; err N u m |= cl Se t K e rn e l A rg ( kern , 2 , s i z e o f * ) &work . u p p er Y ) ; err N u m |= cl Se t K e rn e l A rg ( kern , 3 , s i z e o f * ) &work . st ep X ) ; err N u m |= cl Se t K e rn e l A rg ( kern , 4 , s i z e o f * ) &work . s t ep Y ) ; err N u m |= cl Se t K e rn e l A rg ( kern , 5 , s i z e o f work . w id th ) ; err N u m |= cl Se t K e rn e l A rg ( kern , 6 , s i z e o f work . h eig h t ) ; i f ( is E rro r ( err Num , ” F a i l e d to s e t k e r n e l { cl e an U p ( ) ; re t ur n ; }

( do ubl e ) ,

( void

→֓

( do ubl e ) ,

( void

→֓

( do ubl e ) ,

( void

→֓

( do ubl e ) ,

( void

→֓

( i n t ) , ( vo id

*)

&→֓

( i n t ) , ( void

*)

&→֓

p a ra m et er s ” ) )

168 169 170

s i ze _ t id x Sp ace [ ] = { t hr e ad X , th re ad Y } ; si z e _ t lo ca l W o rk Si z e [ ] = { B LO C K_S ID E , B LO C K _ SID E } ;

171 172 173 174 175 176 177 178

// lau nc h GPU k e r n e l err N u m = cl E n q u e u e N D R a n g e K e rn e l ( q , kern , 2 , NULL , idx Spac e →֓ , lo cal W o rk Si ze , 0 , NULL , &ke rn C o m p le te ) ; i f ( is E rro r ( err Num , ” F a i l e d to la unc h k e r n e l ” ) ) { cl e an U p ( ) ; re tu rn ; }

179 180 181

182 183 184 185 186 187 188 189 190 191 192 193

i f ( ! O C L 2 su p p o rt ) err N u m = cl E n q u e u e R e a d B u ffe r ( q , d_dat a , C L _T R UE , 0 , w or k →֓ . w id th * w or k . h eigh t * s i z e o f ( u ns ig ne d char ) , data , →֓ 0 , NULL , NULL ) ; else { //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // wait f o r the k e r n e l to f i n i s h cl_ e ve n t e vl i s t [ ] = { ke rn C o m p le te } ; err N u m = cl W a it Fo r E ve n ts ( 1 , e v l i s t ) ; i f ( is E rro r ( err Num , ” F a i l e d to wait f o r ev ent ” ) ) { cl e an U p ( ) ; re tu rn ; } }

194 195 196 197 198 199 200 201

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // copy r e s u l t s i n t o QImage o b j e c t f o r ( i n t j = 0 ; j < work . h ei gh t ; j++) f o r ( i n t i = 0 ; i < work . w id th ; i++) { i n t co lo r = data [ j * work . w id th + i ] ; img−>s e t P ixe l ( i + work . s tart Col , j , q Rgb ( 256 − →֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

167

}

202 203 204 205 206 207

}

color , 256 − color , 256 − co lo r ) ) ;

} // clean− up a l l o c a t e d memory cerr << t << ” P r oc e sse d ” << co ls P ro ce sse d << ” rows\n” ; cl e an U p ( ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

168CHAPTER 7. GPU AND ACCELERATOR PROGRAMMING : OPENCL

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Chapter 8

Shared-memory programming : OpenMP Exercises 1. Modify the program in Listing 4.1, so that printf is used instead of cout.

Do you see any difference in the output compared to the one reported in Section 4.2? Can you explain it?

Answer 1 2 3 4 5 6 7 8

m

9 10 11

p

omp_ e

d_num

12 13 14

The output does not appear mangled, as the printf strings are build and output as a unit, while the cout object goes through several method invocations.

2. In the matrix multiplication example of Section 4.4.2 we could get three perfectly nested loops, if we initialize the C matrix outside: do ubl e A [ K ] [ L ] ; double B [ L ] [ M ] ; do ubl e C [ K ] [ M ] ; . . . #pragma omp p a r a l l e l f o r c o l l a p s e ( 3 ) f o r ( i n t i = 0 ; i < K ; i++) f o r ( i n t j = 0 ; j < M ; j++) f o r ( i n t k = 0 ; k < L ; k++) C [ i ] [ j ] += A [ i ] [ k ] * B [ k ] [ j ] ;

Is the above code correct? If not, what kind of modification is required for fixing it?

169

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

170

CHAPTER 8. SHARED-MEMORY PROGRAMMING : OPENMP

j:0,M−1 0

1

2

3

4

5

i:1,K−2

1

2

3

4

Figure 8.1: Dependency graph for Exercise 3.

Answer The code is not correct, because for the inner loop C[i][j] is effectively a reduction variable. To eliminate the race condition, a critical section is needed: #pragma omp p a r a l l e l f o r c o l l a p s e ( 3 ) f o r ( i n t i = 0 ; i < K ; i++) f o r ( i n t j = 0 ; j < M ; j++) f o r ( i n t k = 0 ; k < L ; k++) #pragma omp c r i t i c a l C [ i ] [ j ] += A [ i ] [ k ] * B [ k ] [ j ] ;

In terms of performance, this will effectively make all threads execute sequentially, because unnamed critical constructs use the same mutex.

A better alternative is to make sure that each execution of the inner loop is assigned to a single thread, by using a schedule clause, which is equivalent to converting the collapse(3) clause to collapse(2): #pragma omp p a r a l l e l f o r c o l l a p s e ( 3 ) s c h e d u l e ( s t a t i c , L) f o r ( i n t i = 0 ; i < K ; i++) f o r ( i n t j = 0 ; j < M ; j++) f o r ( i n t k = 0 ; k < L ; k++) C [ i ] [ j ] += A [ i ] [ k ] * B [ k ] [ j ] ;

3. Draw the iteration space dependency graph for the following program: f o r ( i n t i = 1 ; i < K −1; i++) f o r ( i n t j = 0 ; j < M ; j++) a [ i ] [ j ] = a [ i −1][ j ] + a [ i +1][ j ] ;

What kind of dependencies exist? How can you eliminate them?

Answer The ISDG is shown in Figure 8.1.

We have a flow dependence from a[i-1][j] and an anti-dependence for a[i+1][j]. The anti-dependence can be eliminated by creating a copy a2 of the original matrix a. The flow dependence can be honored (not removed) by parallelizing only the inner loop, resulting in the following code:

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

171 f o r ( i n t i = 0 ; i < K −1; i++) f o r ( i n t j = 0 ; j < M ; j++) a2 [ i ] [ j ] = a [ i + 1][ j ] ; f o r ( i n t i = 1 ; i < K −1; i++) #pragma omp p a r a l l e l f o r f o r ( i n t j = 0 ; j < M ; j++) a [ i ] [ j ] = a [ i −1 ][ j ] + a2 [ i ] [ j ] ;

4. Create a C++ program for visualizing the thread iteration assignment performed by a parallel for directive, for different schedule schemes.

Your program should have a per-thread vector that accumulates the loop control variable values assigned to each thread. Use this program, and appropriate schedule settings, to experiment with multiple schemes, without recompiling your program. This is a sample of the output you should be able to get for a loop of 100 iterations: $

$

e xport O M P _ SC H E D U LE=” s t a t i c , 5 ” . / so lu tio n Th read 0 : 0 1 2 3 4 40 41 42 43 44 80 81 82 83 84 Th read 1 : 5 6 7 8 9 45 46 47 48 49 85 86 87 88 89 Th read 2 : 10 11 12 13 14 50 51 52 53 54 90 91 92 93 94 Th read 3 : 15 16 17 18 19 55 56 57 58 59 95 96 97 98 99 Th read 4 : 20 21 22 23 24 60 61 62 63 64 Th read 5 : 25 26 27 28 29 65 66 67 68 69 Th read 6 : 30 31 32 33 34 70 71 72 73 74 Th read 7 : 35 36 37 38 39 75 76 77 78 79

Answer The solution is based on having a vector per thread (line 11), to store the values of the loop control variable (line 18): 1 2 3 4

#i n c l u d e < ios t r ea m > #i n c l u d e < s t d l i b . h> #i n c l u d e <ve ct or > #i n c l u d e <omp . h>

5 6

u s i n g namespace std ;

7 8 9 10 11

i n t main ( ) { i n t N = o m p _ g e t_ m a x_ th re a d s ( ) ; v e ct o r < i n t >* st o re = new v e ct o r < i n t >[ N ] ;

12 13 14 15 16 17 18 19

#pragma omp p a r a l l e l f o r s c h e d u l e ( r unt ime ) f o r ( i n t k = 0 ; k < 1 0 0 ; k++) { i n t idx = o m p _ g e t_ th re a d _ n u m ( ) ; sto re [ idx ] . p u sh _ b ack ( k ) ; }

20

// output f o r ( i n t i = 0 ; i < N ; i++) { cout << ” Thread ” << i << ” : ” ; f o r ( i n t j = 0 ; j < sto re [ i ] . size ( ) ; j++) cout << sto re [ i ] [ j ] << ” ” ; cout << endl ; }

21 22 23 24 25 26 27 28 29 30 31

}

re tu rn

0;

5. In Listing 4.19 we examined the issue of processing a linked-list concurrently. Is there a way to improve this program for a doubly-linked list?

Write the corresponding program.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

172

CHAPTER 8. SHARED-MEMORY PROGRAMMING : OPENMP Answer In the following code, the list is traversed in two opposite directions, allowing the generation of two tasks per iteration of the while loop of lines 61-75.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

m

m

43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

173 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83

6. Write a program for traversing and processing the elements of a binary tree in parallel, using a pre-order, in-order or post-order traversal. Use the task construct to that effect.

Answer In-order traversal cannot be parallelized as in enforces a total order between the tree nodes.

1 2 3 4 5 6 7 8 9 10 11 12 13

//−−−−−−−−−−−−−−−−−−−−−−−− t e m pl at e < typename T > s t r u c t T re e N o d e { T valu e ; T re e N o d e < T > * left ; T re e N o d e < T > * ri gh t ; }; //−−−−−−−−−−−−−−−−−−−−−−−− // empty p l a c e h o l d e r f o r a p r o c e s s i n g f u n c t i o n te mpl ate < typename T > void p ro ce ss S tu b ( T n ) { }

14 15

//−−−−−−−−−−−−−−−−−−−−−−−−

16

te mpl ate < typename T > voi d p re O rd e r ( T re e N o d e < T > * n , voi d p ro ce ss ( T n ) ) { i f ( n == NULL ) re tu rn ;

17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

p ro ce ss ( n−>v alu e ) ; #pragma omp p a r a l l e l { #pragma omp s i n g l e { #pragma omp task p re O rd e r ( n−>le ft , p ro ce ss ) ; // keep u s i n g t h i s t hr e a d f o r the o t he r c h i l d p re O rd e r ( n−>r ight , p ro ce ss ) ; #pragma omp t as k w a it } } }

36 37

//−−−−−−−−−−−−−−−−−−−−−−−−

38

t e m pl at e < typename T > voi d p ro ce ss ( T n ) ) { i f ( n == NULL )

39 40

*→֓

p o st O rd e r ( T re e N o d e < T >

*n ,

v oi d

*→֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

174

CHAPTER 8. SHARED-MEMORY PROGRAMMING : OPENMP re tu rn ;

41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

#pragma omp p a r a l l e l { #pragma omp s i n g l e { #pragma omp task p o st O rd e r ( n−>le ft , p ro ce ss ) ; // keep u s i n g t h i s t h r e a d f o r the o t he r c h i l d p o st O rd e r ( n−>r ight , p ro ce ss ) ; #pragma omp t as k w a it } } p ro ce ss ( n−>v alu e ) ; }

58 59

//−−−−−−−−−−−−−−−−−−−−−−−−

60

// in O rder cannot be p a r a l l e l i z e d as in e n f o r c e s a t o t a l o rde r →֓ between the t r e e nodes . // post Or der and pre Or der only have a p a r t i a l o r d e r i n g te mpl ate < typename T > voi d in O rd e r ( T re e N o d e < T > * n , void *→֓ p ro ce ss ( T n ) ) { i f ( n == NULL ) re tu rn ;

61 62 63 64 65 66

in O rd e r ( n−>le ft , p ro ce ss ) ; p ro ce ss ( n−>v alu e ) ; in O rd e r ( n−>r ight , p ro ce ss ) ;

67 68 69 70 71

}

7. Modify the program of the previous exercise, so that undeferred tasks are generated after the traversal has moved beyond the fifth level of the tree (assume that the root sits at level 0).

Answer The required modifications relative to the answer of the previous question, are shown below. The solution is based on the introduction of a function parameter for counting the depth of the recursion/depth of the tree nodes examined. The default value of “0” means that the caller does not need to be aware of this parameter.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

te mpl ate < typename T > voi d p re O rd e r ( T re e N o d e < T > * n , voi d p ro ce ss ( T n ) , i n t l evel =0) { i f ( n == NULL ) re tu rn ;

*→֓

p ro ce ss ( n−>v alu e ) ; #pragma omp p a r a l l e l { #pragma omp s i n g l e { #pragma omp t as k i f ( l e v e l >=5) p re O rd e r ( n−>le ft , proc ess , l evel +1) ; // keep u s i n g t h i s t h r e a d f o r the o t he r c h i l d p re O rd e r ( n−>r ight , pr oc e s s , l evel +1) ; #pragma omp t as k w a it } } }

21 22

//−−−−−−−−−−−−−−−−−−−−−−−−

23

t e m pl at e < typename T > voi d p o st O rd e r ( T re e N o d e < T > p ro ce ss ( T n ) , i n t l evel =0)

*n ,

v oi d

*→֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

175 24 25 26 27 28 29 30 31 32 33

{ if

( n == N ULL ) re tu rn ;

#pragma omp p a r a l l e l { #pragma omp s i n g l e { #pragma omp ta sk i f ( l e v e l >=5) p o st O rd e r ( n−>le ft , proc es s , l evel +1) ;

34 35 36 37 38 39 40 41

// keep u s i n g t h i s t hr e a d f o r the o t he r c h i l d p o st O rd e r ( n−>r ight , pr oc es s , l evel +1) ; #pragma omp t as k w a it } } p ro ce ss ( n−>v alu e ) ; }

8. Modify the Fibonacci sequence-calculating program of Listing 4.20, so that it counts the number of child tasks generated.

Answer Use of an atomic pragma is required for properly incrementing the shared counter. Only changes relative to Listing 4.20 are shown.

1

i n t n u m T as ks = 0 ;

2 3 4 5 6 7 8 9 10 11 12 13 14

i n t fib ( i n t i ) { i n t t1 , t2 ; i f ( i == 0 | | i == 1 ) re t ur n 1 ; else { #pragma omp ta sk sh a re d ( t 1 ) i f ( i >25) mergea ble { #pragma omp atomic n u m T asks++; t1 = fib ( i − 1 ) ;

15 16 17 18 19 20 21 22

} // keep t hr e a d f o r c a l l i n g f u n c t i o n again t2 = fib ( i − 2 ) ; #pragma omp t a s k w a i t re t u r n t1 + t2 ; } }

9. Use the task directive to create a solution to the single producer, multiple consumers problem, with a variable number of consumers, as specified in the command-line.

Answer The following code solves the integration example that is presented in Section 4.5.1.1 and solved with a fixed number of consumers in Listing 4.18. To minimize clutter, we show only the main function that generates the consumer threads (via a for loop in lines 21-25) and executes the producer thread code (lines 28-52).

1 2 3 4 5 6

i n t main ( i n t argc , c ha r ** a rgv ) { i f ( ar gc == 1 ) { cer r << ” Usage ” << ar gv [ 0 ] << ” #t hr e ads #jo bs \n” ; exit ( 1 ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

176

CHAPTER 8. SHARED-MEMORY PROGRAMMING : OPENMP

7 8 9 10 11 12 13 14

Q m pho QM

bu

BU

15 16

17 18 19 20 21 22 23 24 25 26 27

W

28

W

29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63

10. Use the task directive and its depend clause, to model the dependency graph of Figure 4.12. You can use stub functions to represent each of the tasks in the figure.

Answer 1 2 3 4

#i n c l u d e < ios t r ea m > #i n c l u d e < s t d l i b . h> #i n c l u d e <omp . h> #i n c l u d e < u ni st d . h>

5

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

177

T1

T3

T2

T6

T5

T4

Figure 8.2: A dependency graph consisting of six tasks.

6

u si ng namespace std ;

7 8 9 10

i n t main ( i n t argc , c ha r { int x , y , z ;

** a rgv )

11 12 13 14 15 16 17 18 19

#pragma omp p a r a l l e l { #pragma omp s i n g l e { #pragma omp task s h ar e d ( x ) depend ( out : x ) { // T1 cout << ”T1\n” ; }

20 21 22 23 24

#pragma omp t as k s h ar e d ( x ) depend ( out : y , z ) { // T3 cout << ”T3\n” ; }

25 26 27 28 29 30

#pragma omp task sha re d ( x ) depend ( ino ut : x , y ) { // T2 sleep ( 1 ) ; cout << ”T2\n” ; }

31 32 33 34 35

#pragma omp ta sk sha re d ( x ) depend ( in : y ) { // T4 cout << ”T4\n” ; }

36 37 38 39 40 41

#pragma omp task sha re d ( x ) depend ( in : y ) { // T5 sleep ( 1 ) ; cout << ”T5\n” ; }

42 43 44 45 46

#pragma omp ta sk sh a re d ( x ) depend ( in : z ) { // T6 cout << ”T6\n” ; }

47 48 49 50

#pragma omp t as k w a it } }

51 52 53

re tu rn }

0;

11. Finding the odd integers in a array can be accomplished by the following

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

178

CHAPTER 8. SHARED-MEMORY PROGRAMMING : OPENMP OpenMP code: i n t dat a [ N ] ; i n t o d d C o u n t =0; #pragma omp p a r a l l e l f o r f o r ( i n t i = 0 ; i < N ; i++) i f ( dat a [ i ] % 2 ) #pragma omp atomic o d d C o u n t ++;

Modify the above code, so that there is no need for a critical or atomic directive, by introducing a counter array, with one element per thread.

Compare the performance of your version with and without cache false sharing. If we were to use a reduction variable, what false-sharing elimination technique, does this correspond to?

Answer The following listing contains four different versions of the code snippet in question, encapsulated in appropriately named functions: 1 2 3 4 5 6 7 8 9 10 11

// O r i g i n a l code wih atomic i n t co u n t_ a to m ic ( i n t * data , i n t N ) { i nt oddCount = 0; #pragma omp p a r a l l e l f o r f o r ( i n t i = 0 ; i < N ; i++) i f ( dat a [ i ] % 2 ) #pragma omp atomic o d d C o u n t++; re t ur n o d d C o u n t ; }

12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // Array o f c o u n t e rs , but t he re i s f a l s e s h a r i n g i n t co u n t_ fa ls e _ sh a r in g ( i n t * data , i n t N ) { i n t n u m Th r = o m p _ g e t_ m a x_ th re a d s ( ) ; i n t o d d C o u n t [ n u m Th r ] ; m em set ( odd C ount , 0 , n u m Th r * s i z e o f ( i n t ) ) ; i n t to ta l C o u n t = 0 ; #pragma omp p a r a l l e l f o r f o r ( i n t i = 0 ; i < N ; i++) { i f ( data [ i ] % 2 ) o d d C o u n t [ o m p _ g e t_ th re a d _ n u m ( ) ]++; }

27 28 29 30 31

}

f o r ( i n t i = 0 ; i < n u m Th r ; i++) to ta l C o u n t += o d d C o u n t [ i ] ; re tu rn to ta l C o u n t ;

32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− // E l i m i n a t e s f a l s e s h a r i n g with padding i n t co u n t_ w i t h o u t_ fa l se _ sh a r i n g ( i n t * data , i n t N ) { i n t n u m Th r = o m p _ g e t_ m a x_ th re a d s ( ) ; i n t o d d C o u n t [ n u m Th r * 8 ] ; m em set ( odd C ount , 0 , n u m Th r * 8 * s i z e o f ( i n t ) ) ; i n t to ta l C o u n t = 0 ; #pragma omp p a r a l l e l f o r f o r ( i n t i = 0 ; i < N ; i++) { i f ( data [ i ] % 2 ) o d d C o u n t [ o m p _ g e t_ th re a d _ n u m ( ) * 8]++; }

47 48 49 50 51

}

f o r ( i n t i = 0 ; i < n u m Th r ; i++) to ta l C o u n t += o d d C o u n t [ i * 8 ] ; re tu rn to ta l C o u n t ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

179 52 53 54 55 56 57 58 59 60 61 62 63 64

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t co u n t_ w it h _ re d u c t i o n ( i n t * data , i n t N ) { int oddCount = 0; #pragma omp p a r a l l e l f o r re d u c t i o n (+: oddCount ) f o r ( i n t i = 0 ; i < N ; i++) { i f ( data [ i ] % 2 ) o d d C o u n t++; } re t ur n o d d C o u n t ; }

The use of a reduction variable is equivalent to using private variables for eliminating false sharing.

12. Quicksort rightfully holds the place of the top-performer amongst generic sorting algorithms. It’s design can be considered a reflection of mergesort, although both algorithms employ a divide-and-conquer design. Write a sequential implementation of quicksort and proceed to parallelize it using OpenMP constructs. Measure the speedup that can be achieved, and compare it to the performance that can be obtained from the bottom-up mergesort of Section 4.8.1.

Answer The solution involves the use of task constructs in lines and . The code incorporates several quicksort optimizations such as median-of-three pivot element selection (lines 54-62) and a switch to insertion sort once the size of the data fall below a fixed threshold (lines 92-96). To reduce the generation of tasks, a similar heuristic to the one described in Section 4.8.2 is used ( thresh variable in line 121).

In terms of performance, the program more closely matches the top-down mergesort, failing to fully exploit a multicore CPU because of its recursive nature. The bottom-up mergesort was about twice as fast as the quicksort on an i7 3770K CPU. 1 2 3 4 5 6

#i n c l u d e < s t d l i b . h> #i n c l u d e <s t d i o . h> #i n c l u d e < u n i s t d . h> #i n c l u d e <omp . h> #i n c l u d e <QTime> #i n c l u d e < ios t r ea m >

7 8

u si ng namespace std ;

9 10 11 12

co nst i n t IN S E R T IO N _ T H R E S H O LD =50; co nst i n t m ax T as ks =256; i n t _ th re sh _ ; // used f o r the i f c l a u s e s

13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * void in se rti o n So rt ( i n t * data , i n t N ) { f o r ( i n t i = 1 ; i < N ; i++) { i n t tmp = data [ i ] ; i n t loc=i −1; w hi l e ( loc >=0 && da ta [ loc ] > tmp ) { dat a [ loc +1] = dat a [ loc ] ; loc −−; } dat a [ loc +1] = tmp ; }

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

180 28

CHAPTER 8. SHARED-MEMORY PROGRAMMING : OPENMP }

29 30 31 32 33 34 35 36 37

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * void n u m b e r G e n ( i n t N , i n t max , i n t * sto re ) { int i ; sra n d ( time ( 0 ) ) ; f o r ( i = 0 ; i < N ; i++) sto re [ i ] = r and ( ) % max ; }

38 39 40 41 42 43 44 45

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * void s w ap ( i n t * data , i n t x , i n t y ) { i n t temp = data [ x ] ; data [ x ] = data [ y ] ; data [ y ] = temp ; }

46 47 48 49 50

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t p a r t i t i o n ( i n t * data , i n t N ) { int i = 0 , j = N ;

51

// median o f 3 i n t a=data [ 0 ] , b=data [ N >> 1 ] , c=data [ N − 1 ] ; i f ( ( a > b && b >c ) | | ( c > b && b >a ) ) { swap ( data , 0 , N >>1) ; } e l s e i f ( ( a > c && c >b ) | | ( b > c && c>a ) ) { sw ap ( data , 0 , N −1) ; }

52 53 54 55 56 57 58 59 60 61 62

i n t p ivo t = data [ 0 ] ;

63 64

do {

65 66

do {

67 68

i++; } w h i l e ( p ivo t > data [ i ] && i < N ) ;

69 70 71 72

do {

73 74

j−−; } w h i l e ( p ivo t < data [ j ] && j > 0 ) ; sw ap ( data , i , j ) ;

75 76 77 78

} while ( i < j ) ;

79 80 81

sw ap ( data , i , j ) ;

82 83 84 85 86

}

sw ap ( data , 0 , j ) ; re t u r n j ;

87 88 89 90 91 92 93 94 95 96 97

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * void Q So rt ( i n t * data , i n t N ) { i f ( N <= 1 ) re tu rn ; e l s e i f ( N<IN S E R T IO N _ T H R E S H O LD ) { in se r t i o n So rt ( data , N ) ; re tu rn ; }

98 99 100 101

i n t p ivo t P o s = p a r t i t i o n ( data , N ) ; #pragma omp p a r a l l e l {

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

181 102

#pragma

103 104

#pragma

105 106

#pragma

107 108

#pragma

109 110 111

}

}

omp s i n g l e { omp task i f (N > t h r e s h ) merg eabl e Q So rt ( data , p iv o t P o s ) ; omp task i f (N > t h r e s h ) merg eabl e Q So rt (&( dat a [ p i vo t P o s + 1 ] ) , N − p i vo t P o s − 1 ) ; omp t as k w a it }

112 113 114 115 116 117 118 119 120 121 122 123

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t m ain ( i n t argc , char * ar gv [ ] ) { i f ( ar gc == 1 ) { fp ri n tf ( st der r , ”%s N\n” , ar gv [ 0 ] ) ; exit ( 0 ) ; } i n t N = a to i ( ar gv [ 1 ] ) ; _ th re sh _ = 2 . 0 * N / ( m ax T a s ks + 1 ) ; i n t * data = new i n t [ N ] ; n u m b e r G e n ( N , 1000 , dat a ) ; Q Tim e t ; t . s tart ( ) ;

124 125 126 127 128 129 130

Q So rt ( data , N ) ;

131 132

cout << ” S or t e d in ” << t . e l a p se d ( ) / 100 0 . 0 << ” s e c \n” ;

133

delete [ ] re tu rn 0 ;

134 135

}

dat a ;

#pragma omp t a r g e t teams #pragma omp p a r a l l e l f o r c o l l a p s e ( 2 ) f o r ( i n t i=0; i<N ; i++) f o r ( i n t j=0; j<N ; j++) f o r ( i n t k =0; k<N ; k++) c [ i ] [ j]+= a [ i ] [ k ] * b [ k ] [ j ] ;

What would be the outcome of this change?

Answer This change cannot be done, as it will force the execution of the parallel for by all the teams. If effect, the computation will be duplicated as many times as the number of teams.

13. Assuming we have a dual socket platform with 32 cores per CPU, each supporting two hardware threads, write the appropriate environment variable so that the following places are defined: (a) Each socket is a place. (b) Each set of four cores constitutes a place. (c) Each set of 16 hardware threads is a place.

Answer (a) OMP PLACES=“sockets” or OMP PLACES=“{0:32}:2:32” (b) OMP PLACES=“{0:4}:16:4” (c) OMP PLACES=“{0:16}:2:16”

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

182

CHAPTER 8. SHARED-MEMORY PROGRAMMING : OPENMP

14. The AMD Epyc 7742 CPU comes with 64 cores, each supporting two hardware threads. The cores are organized in the form of CCD chiplets, each hosting eight cores and their dedicated 32MiB L3 cache. The L3 cache is split between two Core Complexes (CCXs). Explain what will happen for each of the following settings in terms of thread placement: (a) OMP PLACES=“ll caches”, OMP PROC BIND=“close”, for 64 threads and the master thread at p0 (b) OMP PLACES=“{0:4}:16:4”, OMP PROC BIND=“master”, for 256 threads and the master thread at p1 (c) OMP PLACES=“{0:8}:8:8”, OMP PROC BIND=“spread”, for 16 threads and the master thread at p2 (d) OMP PLACES=“{0:8}:8:8”, OMP PROC BIND=“spread”, for 256 threads and the master thread at p3 (e) OMP PLACES=“cores”, OMP PROC BIND=“spread”, for 64 threads and the master thread at p4 (f) OMP PLACES=“cores”, OMP PROC BIND=“spread”, for 16 threads and the master thread at p5 Answer (a) There will be 16 places, one for each CCX. The threads will be placed as follows: 0-3 at p0, 4-7 at p1, etc. (b) There will be 16 places defined, each made of 4 hardware threads. All the threads will be placed at p1. (c) There will be 8 places. The threads will be placed as follows: 0-1 at p2, 2-3 at p3, etc. (d) There will be 8 places. The threads will be placed as follows: 0-31 at p3, 32-63 at p4, etc. (e) There will be 64 places. The threads will be placed as follows: 0 at p4, 1 at p5, etc. (f) There will be 64 places. The places will be grouped into 16 new groups p0′ = {p0, p1, . . . , p3}, etc. The threads will be placed as follows: 0 at {p4, p5, p6, p7}, 1 at {p8, p9, p10, p11}, etc.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Chapter 9

The Thrust Template Library Exercises 1. Develop a Thrust program for calculating the inner product of two vectors by using the thrust::transform algorithm and an appropriate functor.

Answer The thrust::transform algorithm can be used to multiple the elements of two vectors, but it has to be accompanied by the thrust::reduce algorithm in order to sum-up the partial products. In the following code, two randomly-initialized vectors, of user-supplied size (as specified by a single command-line parameter), are multiplied accordingly. The result is computed on both the CPU and GPU in order to compare the results:

183

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

184

CHAPTER 9.

}

THE THRUST TEMPLATE LIBRARY

h_y [ i ] = rand ( ) % 1000 ;

// copy data to de vice d_x = h_x ; d_y = h_y ; // de vice − based c a l c u l a t i o n th ru st : : t r a n s f o rm ( d_x . b egin ( ) , d_x . end ( ) , d_y . b egin ( ) , →֓ d _p ro d . b egin ( ) , in n e r P ro d Fu n ct ( ) ) ; f l o a t res = th ru st : : r ed u ce ( d _p ro d . b egin ( ) , d _p ro d . end ( ) ) ; cout << ” I nne r produc t on GPU i s ” << res << endl ; // hos t c a l c u l a t i o n th ru st : : t r a n s f o rm ( h_x . b egin ( ) , h_x . end ( ) , h_y . b egin h _p ro d . b egin ( ) , in n e r P ro d Fu n ct ( ) ) ; res = th ru st : : r ed u ce ( h _p ro d . b egin ( ) , h _p ro d . end ( ) ) ; cout << ” I n ne r pr o d uct on CPU i s ” << res << en dl ; }

re tu rn

( ) , →֓

0;

2. Calculate the inner product of two vectors by using the thrust::inner product algorithm. Compare the performance of this version with the performance of a CUDA-based solution.

Answer The use of the thrust::inner product can replace the sequence of thrust::transform and thrust::reduce of the previous exercise. The resulting program does not require a functor for the calculation to take place:

mu

3. Use Thrust algorithms to find the absolute maximum of an array of values.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

185 Answer The solution can be based on the thrust::transform reduce algorithm, which permits kernel fusion. A custom unary functor (lines 8-14) can be used to calculate the absolute values during the transformation phase, with the built-in thrust::maximum functor used in the reduction phase.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

m

32

um

o

33 34 35 36 37 38 39 40 41 42 43 44 45

max = 0 ; f o r ( i n t i = 0 ; i < N ; i++) { f l o a t temp = fabs ( h_x [ i ] ) ; i f ( temp > max ) max = t emp ; } cout << ”Max by CPU: ” << max << endl ; re tu rn

0;

46

4. Write a Thrust program for calculating the sum of two matrices.

Answer The solution can be based on the thrust::transform algorithm, combined with the built-in thrust::plus functor. The matrices can be simply treated as vectors, as their dimensions have no influence on the computation (as they do in multiplication). In the following code the user supplies the dimensions of the matrices to be randomly generated for testing the code.

A thrust::host vector is used for initializing the two matrices (lines 39-40 and 43-44) before passing them to the device (lines 41 and 45) and for retrieving and printing out the result (lines 49-50).

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

186

CHAPTER 9.

THE THRUST TEMPLATE LIBRARY

1 2 3 4 5 6 7 8 9

M

10

N

M

11 12 13 14

M

15 16

m

17

M

18 19 20 21 22 23 24 25 26 27

de

28

u

ndom_eng

e

0

29 30 31 32

N M

33 34 35 36 37 38 39 40 41 42

M

M

M

M

M

M

43 44 45 46 47 48 49 50 51 52 53

5. Write a Thrust program for calculating the definite integral of a function f (x) over a range [a, b], using the trapezoidal rule. Consider a solution that avoids the need to create a vector for all the x-values for which a trapezoid is calculated. The details of the trapezoidal rule can be found in Section 3.5.2.

Answer The answer is to employ a thrust::counting iterator to generate a sequence of indices, and a functor which scales them to generate x values for the calculation of the desired function.

The functor shown in the listing below (lines 13-28) is initialized with the appropriate offset (data member a) and scale (data member h) to convert

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

187 an index i to an appropriate xi value (line 25), as dictated by the equation that calculates the integral of a function f () in the range [a, b] : h

! n−1 f (a) + f (b) Σ + f (xi) 2 i=1

where x0 = a, xn = b, xi = a + i · h and h = b−na .

Line 26 calculates the value of the function. The thrust::transform reduce algorithm of lines 39-42, sums-up the produced function values, effectively calculating the summation term of the above equation.

By having the initial reduction value set to ( f(0.0f) + f(N * 1.0f) ) /2, (f(0.0f) evaluates to f(a) and f(N * 1.0f) to f(b))the reduction calculates the complete parenthesis expression. The multiplication by h in line 43 completes the result.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

#i n c l u d e < ios t r ea m > #i n c l u d e < s t d l i b . h> #i n c l u d e <math . h> #i n c l u d e <t h r u s t / i t e r a t o r / c o u n t i n g i t e r a t o r . h> #i n c l u d e <t h r u s t / t r a n s fo rm re d u c e . h> #i n c l u d e <t h r u s t / f u n c t i o n a l . h> co nst f l o a t LO W E R LIM IT = 0 ; // s ta n d s f o r rang e l i m i t co nst f l o a t U P P E R LIM IT = 1 0 ; // s ta n d s f o r rang e l i m i t u si ng namespace std ; // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * s t r u c t in te gr Fu n ct { float h , a ;

17

{

18 19 20

in te gr Fu n ct ( f l o a t st , f l o a t w id th ) a = st ; h = w id th ;

21 22

}

23

_ _ h o st_ _ _ _ d e v i c e _ _ f l o a t o pe rat o r ( ) ( f l o a t i ) { float x = a + h * i ; re t u r n fa bs ( sin ( x ) ) ; }

24 25 26 27 28 29 30 31 32 33

a b

}; // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t main ( i n t argc , c ha r ** a rgv ) { i n t N = a to i ( ar gv [ 1 ] ) ; f l o a t h = ( U P P E R L IM IT − LO W E R LIM IT ) / N ; ; in te gr Fu n ct f ( LO W ER LI M IT , h ) ;

34 35 36 37 38

th ru st : : co u n t i n g _ it e ra to r < f l o a t >x ( 1 ) ;

39

f l o a t res = th ru st : : tra n sf o rm _ re d u ce ( x , x + N − 1 , f, ( f ( 0 . 0 f ) + f ( N * 1 . 0 f ) ) →֓ /2 , th ru st : : plus < f l o a t >() ) ; res *= h ; cout << ” D e f i n i t e i n t e g r a l i s ” << res << e ndl ;

40 41 42 43 44 45 46 47

}

re tu rn

0;

6. Use Thrust to calculate the mean and variance of a data set X of cardinality N . For convenience, the corresponding formulas are:

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

188

CHAPTER 9.

THE THRUST TEMPLATE LIBRARY ΣN −1

E[X] =

i=0

xi

(9.1)

N

σ2 = E[X2] − (E[X])2

(9.2)

Answer This is a straight-forward application of the thrust::reduce algorithm for calculating the sum of the data set values, and the thrust::transform reduce algorithm for calculating the sum of the square data set values. The remaining calculations are very simple.

de

u

ndom_eng

e

0

7. Measure the performance of the thrust::sort algorithm in Thrust by sorting varying volumes of data. Compare the achieved times with the STL version running on the host. For this purpose create a big array (make sure it is not too big to fit in the GPU’s memory) and populate it with random data.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

189 Answer The following listing employs the POSIX high-resolution timing functions (see Appendix C.2) for reporting the time it takes to sort a variablesize array of integers using STL on the host and Thrust on the device respectively. The program requires three command-line parameters corresponding to the minimum (Nmin) and maximum array size (Nmax), and the iteration step used in testing (Nstep).

Once the arrays are initialized and placed in their respective memory spaces, the loop of lines 40-52 proceeds to sort an increasing part of the arrays and report the time and processing rate in million integers/sec units.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51

52 53 54

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

190 55

CHAPTER 9.

THE THRUST TEMPLATE LIBRARY

}

The results of a sample run on a 3.5GHz i7-3770K CPU and a GTX 560Ti GPU are shown below: $

$

nvcc −O 2 so rt T h ru st . cu − o so rt T h ru st . / so rt T h ru st 1000000 5000000 500000 1000000 Tim es CPU , GPU ( sec ) : 0 . 0 5 4 8 1 86 0 . 00260711 m i ll io n in te ge rs / sec ) : 18 . 242 383 . 567 1500000 Tim es CPU , GPU ( sec ) : 0 . 0 6 4 9 8 69 0 . 00337315 m i ll io n in te ge rs / sec ) : 23 . 081 6 444 . 689 2000000 Tim es CPU , GPU ( sec ) : 0 . 0 7 9 1 8 55 0 . 00413704 m i ll io n in te ge rs / sec ) : 25 . 257 2 483 . 438 2500000 Tim es CPU , GPU ( sec ) : 0 . 0 9 5 3 4 43 0 . 00491595 m i ll io n in te ge rs / sec ) : 26 . 220 8 508 . 548 3000000 Tim es CPU , GPU ( sec ) : 0 . 1 0 3 8 9 6 0 . 00588679 m i ll io n in te ge rs / sec ) : 28 . 875 1 509 . 615 3500000 Tim es CPU , GPU ( sec ) : 0 . 1 2 4 5 2 1 0 . 0065577 m i ll io n in te ge rs / sec ) : 28 . 107 7 533 . 723 4000000 Tim es CPU , GPU ( sec ) : 0 . 1 2 7 9 6 6 0 . 00732589 m i ll io n in te ge rs / sec ) : 31 . 258 3 546 . 009 4500000 Tim es CPU , GPU ( sec ) : 0 . 1 3 6 4 8 2 0 . 00801635 m i ll io n in te ge rs / sec ) : 32 . 971 3 561 . 353 5000000 Tim es CPU , GPU ( sec ) : 0 . 1 5 1 1 9 4 0 . 00879908 m i ll io n in te ge rs / sec ) : 33 . 070 1 568 . 241

R ates

( →֓

R ates

( →֓

R ates

( →֓

R ates

( →֓

R ates

( →֓

R ates

( →֓

R ates

( →֓

R ates

( →֓

R ates

( →֓

8. Measure the performance hit that an Array-of-Structures design approach will have in device-based sorting of 108 randomly generated instances of the following structure: s t r u c t Te st Str { i n t key ; f l o a t valu e ; _ _ h o st _ _ _ _ d e vi c e _ _ bool ope r at or <( co nst Te st S tr &o ) co nst { re tu rn t h i s . key < o . key ; } };

To measure the performance deterioration, you will have to implement and time a Structure-of-Arrays alternative.

Answer The answer is given in the listing below. While the thrust::sort algorithm can used for sorting the array of structures, the thrust::sort by key algorithm needs to be employed for the structure-of-arrays alternative.

The data are randomly generated and are common in both arrangements.

The POSIX high-resolution timing functions (see Appendix C.2) are used for reporting the time.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

191

}

tim e sp e c ts ; clo c k _ g e t t i m e ( C LO C K _ R EA LT IM E , &ts ) ; doubl e aux = ts . t v _ s ec + ts . tv_ n se c / 1 0 0 0 0 0 0 0 0 0 . 0 ; re tu rn aux ;

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * s t r u c t Te st Str { i n t key ; f l o a t valu e ; _ _ h o s t _ _ _ _ d e vi c e _ _ bool ope r at or < ( co nst { re tu rn t hi s −>k ey < o . key ; }

Te st St r & o )

co nst

}; // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t main ( i n t argc , c ha r ** a rgv ) { i n t N = a to i ( ar gv [ 1 ] ) ; th ru st : : h o st_ ve cto r < Te st Str > h _str ( N ) ; th ru st : : d e v ic e _ ve cto r < Te st Str > d _str ; th ru st : : h o s t _ ve ct o r < i n t >h _ k e y s ( N ) ; th ru st : : h o s t _ ve ct o r < f l o a t >h _ va l u e s ( N ) ; th ru st : : d e vi c e _ ve ct o r < i n t >d _ k e y s ( N ) ; th ru st : : d e v i c e _ ve ct o r < f l o a t >d _ v a l u e s ( N ) ; sra n d ( time ( 0 ) ) ; f o r ( i n t i = 0 ; i < N ; i++) { h _str [ i ] . key = rand ( ) % 1 00 0 0 00 ; h _st r [ i ] . valu e = rand ( ) % 10 00 0 00 ;

}

h _ k e ys [ i ] = h _s tr [ i ] . key ; h _ v a l u e s [ i ] = h _str [ i ] . valu e ;

d _str = h _str ; d _k ey s = h _ ke ys ; d_valu es = d _valu e s ; d ou bl e t0 , t1 , t2 ; t0 = h rcl o ck_ se c ( ) ; th ru st : : sort ( d _str . b egin ( ) , d _str . end ( ) ) ; t1 = h rcl o ck_ se c ( ) ; th ru st : : so r t _ b y_ ke y ( d _ k e y s . b egin ( ) , d _ k e ys . end ( ) , d _ v a l u e s . →֓ b egin ( ) ) ; t2 = h rcl o ck_ se c ( ) ; cout << N << ” ” << t1 − t0 << ” ” << t2 − t1 << en dl ; }

re tu rn

0;

9. Create a Thrust program for computing the Mandelbrot set. Section 3.8.2 covers the mathematical details of how the set is calculated.

Answer Thrust has the limitation of not providing a 2D counting iterator, that would have been handy in this problem. We can still use the thrust::counting iterator facility as long as we can couple it with a transformation that maps each generated index to a coordinate on the complex plane. In the functor of lines 19-57 in the listing that follows, this is exactly what is taking place.

The functor is initialized with all the information necessary to map what is essentially a pixel counter running from top-to-bottom and left-to-right,

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

192

CHAPTER 9.

THE THRUST TEMPLATE LIBRARY

into a pair of complex plane coordinates. The conversion is taking place in lines 40-43. The remaining lines in the operator() method is just a copy of the MandelCompute::diverge method shown in Listing 3.29 in page 127.

The thrust::transform call in line 85 calculates the number of iterations required for each examined complex plane point to diverge. These data are subsequently copied to the host (line 88), prior to using them for painting the pixels of a QImage object (lines 91-96).

1 2

3 4 5 6 7 8 9 10 11 12 13 14 15

MAX

16 17 18 19 20 21 22 23 24 25

m

w

26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

MAX

47 48 49 50 51 52 53 54 55 56

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

193 57

};

58 59 60 61 62 63 64 65

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t main ( i n t argc , c ha r ** a rgv ) { i n t img X = 1024 , img Y = 7 6 8 ; do u ble u p p er C o rn er X , u p p e r C o rn e r Y ; doubl e lo w er C o rn er X , lo w e r C o rn e r Y ; doubl e x Step , y Step ;

66

u p p e r C o rn e r X = atof ( argv [ 1 ] ) ; u p p e r C o rn e r Y = atof ( argv [ 2 ] ) ; lo w e r C o rn e r X = atof ( argv [ 3 ] ) ; lo w e r C o rn e r Y = a tof ( argv [ 4 ] ) ;

67 68 69 70 71

x Step = ( lo w e r C o rn e r X − u p p e r C o rn e r X ) / img X ; y Step = ( u p p e r C o rn e r Y − lo w e r C o rn e r Y ) / img Y ;

72 73 74

Q Im ag e * img = new Q Im ag e ( img X , img Y , Q Im ag e : : Fo rm a t_ R G B 3 2 ) ;

75 76

m a n d e l Fu n ct

77

f ( u p p er C o rn er X ,

u p p er C o rn er Y ,

x Step ,

y S tep ,

img X ) ;

78

th ru st : : h o st_ ve c to r < i n t >h _ ite r ; th ru st : : d e v i c e _ ve ct o r < i n t >d _i t er ( i mg X

79 80

*

i mg Y ) ;

81

th ru st : : co u n t i n g _ it e ra to r < f l o a t >x ( 0 ) ;

82 83

// c a l c u l a t e the p i x e l v a l u e s on the de vice th ru st : : tra n sfo rm ( x , x + i mg X * img Y , d _i ter . b egin ( ) , f ) ;

84 85 86

// copy r e s u l t s to host h _i t er = d _i ter ;

87 88 89 90 91 92 93 94 95 96 97 98 99

}

// draw the image p i x e l s a c c o r d i n g l y f o r ( i n t j = 0 ; j < img Y ; j++) f o r ( i n t i = 0 ; i < img X ; i++) { i n t co lo r = h _i t er [ img X * j + i ] ; img−>s e t P ixe l ( i , j , q Rgb ( 256 − colo r , 256 − colo r , 256 →֓ — co lo r ) ) ; } img−>s ave ( ” mandel . png” , ”PNG” , 0 ) ; re t u r n 0 ;

10. A problem related to the furthest-distance point solved in Section 7.5, is the problem of finding the pair of points which are the furthest apart from each other. Create a brute-force solution to this problem, i.e. by examining all pairs of points, using Thrust. Consider the case where the number of points is too big to allow storage of all the calculated distances.

You should avoid duplicate calculations since the distance from point A to B is the same as the distance from B to A. Answer Not being able to store the distances creates both opportunities and difficulties: The number N of points that can be used as input is not limited by 2 the need to allocate Θ( N −N 2 ) storage for the pair distances.

The distances might have to be calculated multiple times.

Each pair of points can be identified by the index of the corresponding cell in the distances table. We can compare the “indices” of two pairs of points, by calculating and comparing the corresponding pair-wise distances. In order to avoid duplicate calculations, we can limit the indices to either of

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

194

CHAPTER 9.

THE THRUST TEMPLATE LIBRARY

Figure 9.1: Illustration of how a pair of points and their distance can be identified by the index of the corresponding cell in the distances matrix. The indices are calculated by numbering the cells in a top-to-bottom and left-to-right fashion, skipping the gray cells. The matrix is not actually allocated. the two table halves, as split by the main diagonal. In the following code we use the lower half. The indices are calculated as illustrated in Figure 9.1, i.e. to identify the pair of point (i, j), with i > j, we use the formula: Σ i−1

idx(i,j) =

k+j =

i(i −1)

k=1

2

+j

(9.3)

The opposite mapping, i.e. from an index to a pair of points can be also performed. Given an initial approximation for i :

idx(i,j) =

i2 — i) 2

2

+ j ⇒ i = 2 · idx(i,j) − 2j + i ⇒ i ≈

q

2 · idx(i,j) (9.4)

we can calculate an exact value for i and j by utilizing the knowledge that i > j: i = ( i n t ) flo o r ( s qrt ( 2 . 0 * idx ) ) ; w h i l e ( ( j = idx − i * ( i − 1 ) / 2 ) >= i ) i++;

The solution given below is based on the application of a thrust::reduce algorithm on a thrust::counting iterator that enumerates all the possible cell indices/point pairs. Upon comparing two indices, the supplied functor (lines 20-49) reverses the mapping to convert the two pair indices into two pairs of points, pair A and pair B. The point indices Ai, Aj and Bi, Bj are subsequently used to calculate and compare the square of the −−−→ 2 −−→ distances − AiAj 2 BiBj (indices used liberally here to represent the actual points).

The only restriction of the following solution is that the type thrust::counting iterator<int> iterator is appropriate for up to N < 46340. For bigger inputs, a switch to thrust::counting iterator<long> would be necessary, as a 32-bit signed integer is not sufficient for holding the generated indices.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

195 1 2 3 4 5 6 7 8

#i n c l u d e < ios t r ea m > #i n c l u d e <t h r u s t / h o s t v e c t o r . h> #i n c l u d e <t h r u s t / d e v i c e v e c t o r . h> #i n c l u d e <t h r u s t / i t e r a t o r / c o u n t i n g i t e r a t o r . h> #i n c l u d e <t h r u s t / r e d u c e . h> #i n c l u d e <t h r u s t /random . h> #i n c l u d e <math . h>

9 10

u si ng namespace std ;

11

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * _ _ h o st _ _ _ _ d e v i c e _ _ v oi d re ve r se M a p ( i n t idx , i n t &i , i n t &j ) { i = ( i n t ) flo o r ( s qrt ( 2 . 0 * idx ) ) ; w h i l e ( ( j = idx − i * ( i − 1 ) / 2 ) >= i ) i++; }

12 13 14 15 16 17 18 19 20 21 22 23

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * s t r u c t m ax Fu n ct { th ru st : : d e v ic e _ p tr < i n t >x , y , z ; // comparing d i s t a n c e s o f p a i r s o f p o i n t s A and B as ind ex ed idxA , idxB _ _ h o st_ _ _ _ d e vi c e _ _ i n t o pe ra t o r ( ) ( i n t idxA , i n t idx B ) { i n t Ai , Aj , Bi , Bj ; i n t dist A , d ist B ;

24 25 26 27 28 29

re ve rs e M a p ( idxA , Ai , Aj ) ; re ve rs e M a p ( idxB , Bi , Bj ) ;

30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47

}

48

i n t x Diff , y Diff , z D iff ; x D iff = x [ Ai ] − x [ Aj ] ; y D iff = y [ Ai ] − y [ Aj ] ; z D iff = z [ Ai ] − z [ Aj ] ; d ist A = x D iff * x D iff + y D iff

*

y D iff + z D iff

*

z D iff ;

x D iff = x [ Bi ] − x [ Bj ] ; y D iff = y [ Bi ] − y [ Bj ] ; z D iff = z [ Bi ] − z [ Bj ] ; d ist B = x D iff * x D iff + y D iff

*

y D iff + z D iff

*

z D iff ;

i f ( d ist A > d ist B ) re t ur n idx A ; else re t ur n idx B ;

49 50

};

51

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t main ( i n t argc , c ha r ** a rgv ) { // i n i t i a l i z e the RNG th ru st : : d e fa u l t _ ra n d o m _ e n g in e rng ( time ( 0 ) ) ; th ru st : : u n ifo rm _ in t_ d i s t r i b u t io n < i n t >u n i D is t r ( −10000 , →֓ 10000 ) ;

52 53 54 55 56

by→֓

57 58 59

i n t N = a to i ( ar gv [ 1 ] ) ;

60

// ge ne r at e the data on the hos t and move them to the de vice th ru st : : d e v i c e _ ve cto r < i n t >d_x ( N ) ; th ru st : : d e v i c e _ ve cto r < i n t >d_y ( N ) ; th ru st : : d e v i c e _ ve cto r < i n t >d_z ( N ) ; th ru st : : h o st _ ve ct o r < i n t >h_x ( N ) ; th ru st : : h o s t _ ve ct o r < i n t >h_y ( N ) ; th ru st : : h o st_ ve c to r < i n t >h_z ( N ) ; f o r ( i n t i = 0 ; i < N ; i++) h_x [ i ] = u n i D istr ( rng ) ; d_x = h_x ; f o r ( i n t i = 0 ; i < N ; i++) h_y [ i ] = u n i D istr ( rng ) ; d_y = h_y ; f o r ( i n t i = 0 ; i < N ; i++) h_z [ i ] = u n i D istr ( rng ) ; d_z = h_z ;

61 62 63 64 65 66 67 68 69 70 71 72

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

196

CHAPTER 9.

THE THRUST TEMPLATE LIBRARY

73

// i n i t i a l i z e t he f u n c t o r t hat w i l l f i n d the maximum d i s t a n c e , →֓ so t hat i t has a c c e s s to the c o o rd in a te ar r ay s m ax Fu n ct f ; f . x = d_x . data ( ) ; f . y = d_y . data ( ) ; f . z = d_z . data ( ) ;

74 75 76 77 78 79

// re du ce the index o f the most d i s t a n t po int i n t fu r t h e st = th ru st : : r ed u ce ( th ru st : : co u n t in g _ it e ra to r < i n t →֓ >(0) , th ru st : : co u n t i n g _ i t e ra to r < i n t →֓ >(( N * N − N ) / 2 ) , 0, f) ;

80 81 82 83 84 85

cout << ”The most d i s t a n t p a i r i s ” << fu rth e st << e ndl ; i n t Ai , Aj ; re ve rs e M a p ( f ur th es t , Ai , Aj ) ; cout << ” F i r s t po int i s ” << Ai << ” ( ” << h_x [ Ai ] << ” ” << h_y →֓ [ Ai ] << ” ” << h_z [ Ai ] << ” ) ” << endl ; cout << ” Second po int i s ” << Aj << ” ( ” << h_x [ Aj ] << ” ” << →֓ h_y [ Aj ] << ” ” << h_z [ Aj ] << ” ) ” << endl ;

86 87 88 89 90 91

// CPU v e r i f i c a t i o n doubl e m ax_ d i st = 0 , maxI , max J ; f o r ( i n t i = 0 ; i < N − 1 ; i++) f o r ( i n t j = i + 1 ; j < N ; j++) { i n t x Diff , y Diff , z D iff ; x D iff = h_x [ i ] − h_x [ j ] ; y D iff = h_y [ i ] − h_y [ j ] ; z D iff = h_z [ i ] − h_z [ j ] ; i n t dist = x D iff * x D iff + y D iff * y D iff + z D iff * z D iff ; i f ( dist > m a x_ d ist ) { m ax_ d is t = dist ; max I = i ; max J = j ; } } cout << endl ; cout << ” D i s t a n c e by CPU : ” << m ax _ d is t << en dl ; cout << ” F i r s t po int i s ” << max I << ” ( ” << h_x [ max I ] << ” ” <<→֓ h_y [ max I ] << ” ” << h_z [ max I ] << ” ) ” << e ndl ; cout << ” Second po int i s ” << max J << ” ( ” << h_x [ max J ] << ” ” →֓ << h_y [ max J ] << ” ” << h_z [ max J ] << ” ) ” << endl ;

92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115

}

re tu rn

0;

11. The all-pairs-shortest paths graph problem can be solved by the dynamic programming algorithm by Floyd and Warshall. Assuming that the graph is described by a V xV adjacency matrix where V is the number of vertices, then the pseudocode of this algorithm below, shows how the solution is obtain in V stages, where each stage involves the update of the distance matrix via the involvement of an intermediate vertex: // i n i t i a l i z a t i o n phase A llo c a t e a Vx V d i s t a n ce m at ri x , and set all e le m e n ts to in f in ity f o r each v e r t ex v d i st a n ce [ v ] [ v ] <− 0 f o r each edge ( u , v ) d i s t a n ce [ u ] [ v ] <− adj ( u , v ) // the w ei ght o f the edge ( u , v ) // comput ation phase f o r k fr om 1 to V // f o r each k i n t e rm e d i a t e ve rt e x f o r i fr om 1 to V // r e c o n s i d e r each p a i r o f v e r t i c e s f o r j fr om 1 to V i f d i s t a n ce [ i ] [ j ] > d i st a n ce [ i ] [ k ] + d i st a n ce [ k ] [ j ] d i st a n ce [ i ] [ j ] <− d i st a n ce [ i ] [ k ] + d i s t a n ce [ k ] [ j ]

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

197 end i f re t ur n d i st a n ce

Create a Thrust program that could perform the V individual stages of the algorithm in parallel, i.e. parallelize the two inner loops of the algorithm.

Answer Similarly to the previous exercise for finding the pair of most distant points in a set, we can use a thrust::counting iterator to go over all the elements of the adjacency matrix. Fortunately, as all elements are to be considered1, the mapping between an index and the adjacency matrix coordinates is straightforward as shown in lines 31,32.

The functor that is used to update the distance of two vertices (lines 2241), is setup so that it has access to the adjacency matrix data and the number of vertices V (lines 64,65). Before each phase of the algorithm executed by the for loop of lines 68-76, the index of the vertex to be used as an intermediate, is also set (line 71).

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

1Actually the kth column and row are unchanged during stage k but we do not exclude them from the computation as this would complicate the code. Not only do we need to change the mapping from the thrust::counting iterator to the table cells, but also the results of each stage’s computation cannot be stored in-place.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

198 43 44 45 46 47 48

CHAPTER 9.

THE THRUST TEMPLATE LIBRARY

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t main ( i n t argc , char ** argv ) { // i n i t i a l i z e the RNG th ru st : : d e fa u l t _ ra n d o m _ e n g in e rng ( time ( 0 ) ) ; th ru st : : u n ifo rm _ in t_ d i s t r i b u t io n < i n t >u n i D is t r ( 1 , 100 ) ;

49

i n t V = at o i

50

( ar gv [ 1 ] ) ;

51

// ge ne r at e the data on the hos t and move them to the de vice th ru st : : d e v i c e _ ve ct o r < i n t >d _ad j ( V * V ) ; th ru st : : h o st _ ve ct o r < i n t >h _ad j ( V * V ) ; f o r ( i n t i = 0 ; i < V * V ; i++) h _a d j [ i ] = u n i D ist r ( rng ) ; f o r ( i n t i = 0 ; i < V ; i++) h _ad j [ i * V + i ] = 0 ;

52 53 54 55 56 57 58 59

d _ad j = h _ad j ;

60 61

// i n i t i a l i z e the f u n c t o r so t hat c o o rd in a te arra ys fl o yd Fu n ct f ; f.V = V; f . adj = d _ad j . da ta ( ) ;

62 63 64 65 66

p ri n t A d j ( h_a dj , V ) ; f o r ( i n t k = 0 ; k < V ; k++) { // s e t the f u n c t o r to use the k−th node as i n t e rm e d ia t e f.k = k; th ru st : : t r a n s f o rm ( th ru st : : co u n t i n g _ i t e ra to r <int >(0) , th ru st : : co u n t in g _ it e ra to r <int >(V * V ) , d _ad j . b egi n ( ) , f) ; }

67 68 69 70 71 72 73 74 75 76 77

p rin tf ( ”−−−−−−−−−−−−−−−−\n” ) ; h _ad j = d _ad j ; p ri n t A d j ( h_a dj , V ) ;

78 79 80 81 82 83

i t has a c c e s s to the →֓

}

re tu rn

0;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Chapter 10

High-level multi-threaded programming with the Qt library Exercises 1. Use the QtConcurrent functionality to implement a prime number checker.

Compare it in terms of speed, efficiency and programming effort to your QThread-based attempt of the previous exercise.

Answer The solution involves a NumberRange class for representing parts of the number range to scan for prime numbers. When an instance of this class is processed by the static NumberRange::process method, the prime numbers in that range are deposited in the designated repository as referenced by the repo pointer.

The main method uses the user-supplied desired length of the NumberRange instances (variable PARTLEN), to populate a vector of them (lines 80-90).

This vector is subsequently processed by using the “mapping” functionality of the QtConcurrent namespace (line 92). The blockingMap function applies the NumberRange::process method on all elements of the input vector, effectively calculating the desired results.

1 2 3 4 5 6 7

#i n c l u d e <QtConcurr ent / QtConcurrentMap> #i n c l u d e <QMutex> #i n c l u d e <QMutexLocker> #i n c l u d e <math . h> #i n c l u d e < s t d l i b . h> #i n c l u d e <i os t r e am > #i n c l u d e <ve ct or >

8 9

u si ng namespace std ;

10 11 12 13 14 15 16 17

//=========================================== // used f o r s t o r i n g the r e s u l t s in a t hr ead− s a f e manner c l a s s R e p o si t o ry { private : vector <int > resu lt ; Q M u tex l ;

199

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

200CHAPTER 10. HIGH-LEVEL MULTI-THREADED PROGRAMMING WITH THE QT LIBRAR 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76

public : v oi d st o re ( i n t i ) ; vect or <int > * ge t R e su lt ( ) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void R e p o s i t o ry : : sto re ( i n t i ) { Q M u te x Lo cke r ml(&l ) ; resu lt . p u sh _ b ack ( i ) ; } //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− vector <int > * R e p o si t o ry : : ge t R e su lt ( ) { re t ur n &resu lt ; } //=========================================== c l a s s N u m b e r R a n ge { private : i n t first , l as t ; R e p o sito ry * repo ; public : N u m b e r R a n ge ( ) {} N u m b e r R a n ge ( i n t a , i n t b , R e p o s it o ry * r ) : firs t ( a ) , last ( b ) , →֓ repo ( r ) {} voi d se t F i r st ( i n t f ) { first=f ; } voi d se t La st ( i n t l ) { last=l ; } v oi d se t R e p o ( R e p o si t o ry * r ) { repo=r ; } s t a t i c void p ro ce ss ( N u m b e r R a n ge &) ; }; //−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void N u m b e r R a n ge : : p ro ce ss ( N u m b e r R a n ge &n ) { f o r ( i n t i=n . fi rs t ; i<=n . l ast ; i+=2) { bool is P r im e=tr ue ; i n t j=3; i n t lim it = s qrt ( i ) ; whi l e ( j<= li m it && is P ri m e ) { i f ( i % j == 0 ) is P rim e=f a l s e ; j+=2; } i f ( is P rim e ) n . repo −>s to re ( i ) ; } } //=========================================== i n t main ( i n t argc , char * ar gv [ ] ) { int a , b ; a=atoi ( a rgv [ 1 ] ) ; b=atoi ( ar gv [ 2 ] ) ; i n t P A R TLEN =atoi ( arg v [ 3 ] ) ; // make s ur e a and b ar e odd a = ( a % 2 == 0 ) ? a+1 : a ; b = ( b % 2 == 0 ) ? b−1 : b ; P A R TL E N = ( P A R TL E N % 2 == 0 ) ? P A R TL E N : P A R TL E N +1; PARTLEN s h o u l d be even

// →֓

77 78 79 80 81 82 83 84 85 86 87 88 89

R e p o si t o ry rep ; vector <N u m b er R an ge > data ; i n t n u m P a rt s = ( i n t ) ceil ( ( b−a ) * 1 . 0 / P A R T LE N ) ; da ta . r e si z e ( n u m P a rt s ) ; i n t f , l=a −2; f o r ( i n t i=0; i<n u m P a rts ; i++) { f=l+2; l=f+P A R T LE N ; i f ( l>b ) l=b ; da ta . at ( i ) . se t F i r s t ( f ) ; da ta . at ( i ) . se t La st ( l ) ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

201 90

}

91

dat a . at ( i ) . se t R e p o (& rep ) ;

92

Q t C o n cu rre n t : : b lo ck i n g M a p ( data , N u m b e r R a n ge : : p ro ce ss ) ;

93 94 95 96 97

}

cout << ” T ot al pr i mes f ound ” << rep . ge t R e su lt ( )−>s ize ( ) << →֓ endl ; re t ur n 0 ;

To run the above program in order to find all the prime numbers in the range [103, 108], one has to use the following command-line: $

p ri m e C h e cke r_ V 3 1000 100000000

100000

The last parameter concerns the value of PARTLEN. In terms of performance, an identical set of tests as in the previous exercise, results in an average execution time of 0.42 sec, for PARTLEN = 10000, which is on par with the monitor-based solution. In terms of development effort, the QtConcurrent based solution is marginally shorter.

2. Create a big array of randomly generated 2D coordinates (x, y). Each of the coordinates should be a number in the range [-1000, 1000]. Use appropriate QtConcurrent functions to find the points that are in a ring of distances between 100 and 200 from the point of origin. Compare the performance of your solution against a sequential implementation.

Answer In the following program a Point structure is used to hold randomly generated data. A vector<Point> container holds the input data, instead of a vector<Point*>, in order to simplify and speedup memory management.

Upon resizing the vector container in line 57, all the memory required for holding the Point instances is reserved.

Also, in order to avoid the use of the costly sqrt function, the filtering is performed on the squares of the distances.

1 2 3 4 5 6 7 8

#i n c l u d e <QtConcurr ent / QtConcurrentMap> #i n c l u d e <Qt Concurr ent / Qt C o n c u r r e nt F ilt e r > #i n c l u d e <QThreadPool> #i n c l u d e <QTime> #i n c l u d e <b oos t / bind . hpp> #i n c l u d e <ve ct or > #i n c l u d e < ios t r ea m > #i n c l u d e < s t d l i b . h>

9 10

u s i n g namespace std ;

11 12 13 14 15 16 17 18

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− s t r u c t Po in t { int x , y ; i n t d2 ; // d i s t a n c e s quar e Po in t ( ) {}; };

19 20 21 22 23 24

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− void d 2 C a l c ( Po in t & p ) { p . d2 = p . x * p . x + p . y * p . y ; }

25 26 27

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− bool in R in g ( Po in t p , i n t min Dist , i n t m ax D ist )

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

202CHAPTER 10. HIGH-LEVEL MULTI-THREADED PROGRAMMING WITH THE QT LIBRAR 28

{

29 30 31 32 33

}

i n t d = p . d2 ; i f ( d >= m in D is t && d <= m ax D is t ) re tu rn tr ue ; re tu rn f a l s e ;

34 35 36 37 38 39 40 41 42 43

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t main ( i n t argc , char * ar gv [ ] ) { i n t n u m Th r ; i n t n u m P o in ts ; v e ct o r < Po in t > dat a ; double tio , tfu ll ; Q Tim e t ; i f ( argc < 3 ) { cerr << arg v [ 0 ] << ” numThreads numPoints\n” ; exit ( 1 ) ; } // f o r t iming t . s ta rt ( ) ;

44 45 46 47 48 49 50 51

n u m Th r = a to i ( argv [ 1 ] ) ; n u m P o in ts = atoi ( arg v [ 2 ] ) ;

52 53 54

// i n i t i a l i z e data array sra n d ( time ( 0 ) ) ; data . r e s i z e ( n u m P o in ts ) ; // one memory a l l o c a t i o n only f o r ( i n t i = 0 ; i < n u m P o in ts ; i++) { data [ i ] . x = rand ( ) % 2000 − 1000 ; data [ i ] . y = rand ( ) % 2000 − 1000 ; }

55 56 57 58 59 60 61 62 63

tio = t . e lap se d ( ) / 1 0 0 0 . 0 ; // c o n t r o l the number o f t hre ads Q T h re a d P o o l : : g lo b a l In sta n ce ( )−>s e t M a x T h re a d C o u n t ( n u m Th r ) ;

64 65 66 67

// c a l c u l a t e d i s t a n c e s f i r s t Q t C o n cu rre n t : : b lo c k in g M a p ( data , d 2 C a l c ) ;

68 69 70

// f i l t e r p o i n t e s bas ed on d i s t a n c e next Q t C o n cu r r e n t : : b lo c ki n g F i l te r ( data , b o o st : : bind ( in Ring , _1 , →֓ 100 * 100 , 200 * 200 ) ) ;

71 72 73 74 75 76 77 78

}

tfu ll = t . e l a p se d ( ) / 1 0 0 0 . 0 ; cout << ” Tota l : ” << tfu ll << ” Comp. time : ” << tfu ll − tio << →֓ endl ; cout << da ta . si ze ( ) << e ndl ; re tu rn 0 ;

3. Use the QtConcurrent functionality to implement a parallel bucketsort.

Does the number of buckets play a significant role in your implementation’s performance?.

Answer In the following Listing, the bucketsort template function implements a variation of the algorithm, whereas (i) the input is scanned and distributed to buckets (lines 43-48), (ii) the buckets are sorted separately via STL’s sort function (lines 51-54), and (iii) the resulting partial sorted arrays are copied back to the original data repository (lines 57-62).

The number of buckets matches the number of threads used by QThreadPool, and for this reason the number of buckets is invariably connected with the level of concurrency.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

203 To avoid expensive synchronization logic, such as using mutices to control access to the buckets (STL’s vector is not thread safe), each thread is responsible for a dedicated bucket. The bucket array address and bucket index for which a thread is responsible for, are passed as parameters to the filterOp template function that is invoked by QtConcurrent::run in line 45. The minimum value of the input range and the value range per bucket are also passed to enable filterOp to determine the bucket that an item belongs to.

Finally, the sorting is performed via the unary front-end vecSort function that is “mapped” on all the buckets. To this end, a vector of vector<T> pointers is populated in lines 52-53 prior to the application of QtConcurrent::blockingMap. 1 2 3 4 5 6 7 8 9 10 11

#i n c l u d e <QtConcurr ent / QtConcurrentMap> #i n c l u d e <QtConcurr ent / QtConcurrentRun> #i n c l u d e <QFuture> #i n c l u d e <QThreadPool> #i n c l u d e <QTime> #i n c l u d e <b oos t / bind . hpp> #i n c l u d e <ve ct or > #i n c l u d e < a l g o r it h m > #i n c l u d e < ios t r ea m > #i n c l u d e < s t d l i b . h>

12 13

u si ng namespace std ;

14

co nst i n t MI NV = −1000; co nst i n t MAX V = 1000 ;

15 16 17 18 19 20 21 22 23 24 25 26 27

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− tem pla te <typename T> i n l i n e v oid fi l t e r O p ( v ector <T> * v , targ e t B u cke t Id x , vector <T> * b , T m , T b W id th ) { f o r ( i n t i=0; i< v−>s ize ( ) ; i++) { T tmp = v−>a t ( i ) ; i n t b u cke t Id x = ( tmp − m ) / b W id th ; i f ( b u cke t Id x == ta rg e t B u cke t Id x ) b−>p u sh _ b ack ( tmp ) ; } }

i n t →֓

28 29 30 31 32 33

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− t e m p l at e <typename T> i n l i n e void ve c So rt ( vect or <T> * v ) { sort ( v−>b egin ( ) , v−>e nd ( ) ) ; }

34 35 36 37 38

39 40

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− t e m p l at e <typename T> void b u cke t So rt ( vect or <T> &data , co ns t T &→֓ min V , co nst T &max V ) { i n t n u m B u cke ts = Q T h re a d P o o l : : g lo b a l In st a n ce ( )−>→֓ m a x T h re a d C o u n t ( ) ; // match buc k ets with the number o f →֓ t h r e ads vec t or <T> b u c k et [ n u m B u c k e ts ] ; T b w id th = ( max V − min V ) / n u m B u ck e ts ; // rang e per bucket

41 42 43 44 45

// laun ch as many r e q u e s t s as the number o f b u ck e t s / t h r e ads QFut ur e <void> f [ n u m B u c k e ts ] ; f o r ( i n t i=0; i<n u m B u cke ts ; i++) f [ i ] = Q t C o n cu rr e n t : : run ( b o o st : : bind ( f i l t er Op <T >, &data , →֓ i , b u cket + i , min V , b w id th ) ) ;

46 47 48

f o r ( i n t i=0; i<n u m B u cke ts ; i++) f [ i ] . w a it Fo r Fi n i sh e d ( ) ;

49 50 51

// s o r t each o f the b uc k et s vector < v ec tor <T> *> vv ;

separately

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

204CHAPTER 10. HIGH-LEVEL MULTI-THREADED PROGRAMMING WITH THE QT LIBRAR f o r ( i n t i=0; i<n u m B u cke ts ; i++) vv . p u sh _ b ack (&( b u ck et [ i ] ) ) ; Q t C o n cu rre n t : : b lo cki n g M a p ( vv , vec Sort <T >) ;

52 53 54 55

// copy back to o r i g i n a l ve ct o r i n t loc =0; f o r ( i n t i=0; i<n u m B u cke ts ; i++) { f o r ( i n t j=0; j<b u cket [ i ] . si ze ( ) ; j++) data [ loc++] = b u ck et [ i ] . at ( j ) ; }

56 57 58 59 60 61 62 63 64

}

65

//−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− i n t main ( i n t argc , char * ar gv [ ] ) { i n t n u m Th r ; i n t n u m It e m s ; v ec t o r < i n t > data ; double tio , tfu ll ; Q Tim e t ;

66 67 68 69 70 71 72 73

i f ( argc < 3 ) { cerr << arg v [ 0 ] << ” numThreads exit ( 1 ) ; } // f o r t iming t . s ta rt ( ) ;

74 75 76 77 78 79 80 81

numItems\n” ;

n u m Th r = a to i ( argv [ 1 ] ) ; n u m It e m s = a to i ( arg v [ 2 ] ) ;

82 83 84

// i n i t i a l i z e data arr ay sra n d ( time ( 0 ) ) ; data . r e s i z e ( n u m It e m s ) ; // one memory a l l o c a t i o n f o r ( i n t i = 0 ; i < n u m Ite m s ; i++) { dat a [ i ] = ( r and ( ) % ( MA X V − MINV ) ) + MINV ; }

85 86 87 88 89 90 91 92

only

tio = t . e lap se d ( ) / 1 0 0 0 . 0 ; // c o n t r o l the number o f t hre ads Q T h re a d P o o l : : g lo b a l In sta n ce ( )−>s e t M a x T h re a d C o u n t ( n u m Th r ) ;

93 94 95 96 97 98

b u cket So rt <int >( data , MINV , MAX V ) ;

99

tfu ll = t . e l a p se d ( ) / 1 0 0 0 . 0 ; cout << ” Tota l : ” << tfu ll << ” Comp. time : ” << tfu ll − tio →֓ << endl ; re tu rn 0 ;

100 101 102

}

4. Create a program that utilizes the QtConccurrent functionality, for calculating the definite integral of a function between a user-supplied range.

Answer

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Chapter 11

Load Balancing Exercises 1. Design a brute-force/exhaustive algorithm for determining the optimum subset of N nodes/processors in the case of static load balancing. What is the complexity of your algorithm?

Answer In general, finding the optimum subset of nodes requires enumerating all the possible subsets, i.e. 2N —1 if we ignore the empty set. The enumeration can be implemented in a variety of ways, the following recursive algorithm being a generic approach that does not impose any restrictions on N .

The following description is based on the assumption that a function to evaluate the execution time on a subset S exists (symbolic name eval() used to represent this function).

The recursive auxiliary function OptimumSetAux, calls itself by including (line 15) or excluding (line 16) the i-th element of the processor set P , until the parameter i becomes equal to N (base case of line 8), which indicates that a subset is ready to be evaluated.

1: procedure OPTIMUMSET(P ) 2: bestS → ∅ 3: bestT → inf 4: OptimumSetAux(P , ∅, 0, bestS, bestT ) 5: Return bestS 6: end procedure 7: procedure OPTIMUMSETAUX(P , S, i, bestS, bestT ) 8: if i = N then 9: T → eval(S) 10: if T < bestT then 11: bestS → S 12: bestT → T 13: end if 14: else 15: OptimumSetAux(P , S ∪ Pi, i + 1, bestS, bestT ) 205

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

206

CHAPTER 11. LOAD BALANCING 16: OptimumSetAux(P , S, i + 1, bestS, bestT ) 17: end if 18: end procedure

The complexity of this algorithm depends on the cost of the eval() function. Given that there are 2N subsets to be evaluated, we can conclude that the complexity ∈ Ω(2N ).

2. Design a greedy algorithm for determining the optimum subset of N nodes/processors in the case of static load balancing. Will you always get the same solution as the one provided by an exhaustive algorithm?

Answer A greedy algorithm can be employed that initially sorts the nodes in descending order of their computational power, i.e. ascending of their pi parameter as identified in Table 8.1.

The following algorithm iterates over the sorted nodes, adding one node at a time for as long as the execution time decreases (while loop terminates otherwise via line 14) or until all nodes are examined (line 6). The complex condition of line 9 ensures that sets leading to invalid solutions (i.e. with one or more partk ≤ 0) are not considered.

Actually, the condition in line 9 can be simply “if T < bestT then”, as a negative part to a node enlarges the parts assigned to other nodes, inevitable making the execution time higher.

1: procedure GREEDYSET(P ) 2: P ′ → sort(P ) 3: bestS → P 0′ 4: bestT → eval(bestS) 5: i→1 6: while i < N do 7: S → bestS ∪ P i′ 8: T → eval(S) 9: if T < bestT AND 6Pk ∈ S, partk > 0 then 10: bestS → S 11: bestT → T 12: i→i+1 13: else 14: Break 15: end if 16: end while 17: Return bestS 18: end procedure A heuristic algorithm cannot produce an identical solution to an exhaustive algorithm, unless this is done by chance, or special circumstances/problem settings allow the generation of an optimum solution via a greedy approach.

3. The closed-form solutions in Section 8.3.3.1 for the N-port communication setup, are based on the assumption that the master participates in the processing of the load. Derive the equations that would govern the solution

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

207 if the master abstained from this task, facilitating only I/O functionality instead.

Answer The answer to this question is already provided in Section 8.3.3.1, albeit in a condensed form, without a length explanation. This exercise serves the purpose of testing student understanding and their ability to follow the proper steps towards the derivation of a solution.

Once we remove the master node from computation duties, Equation 8.9 becomes: (i)

(i)

(i)

(j)

(j)

(j)

tdistr + tcomp + tcoll = tdistr + tcomp + tcoll ⇒ li(a partiL + b) + pi(partiL + ei) + li(c partiL + d) = lj(a partjL + b) + pj(partjL + ej) + lj(c partjL + d)

(11.1)

To solve the problem, we need to find one of the assigned parts, e.g. part1. So from the above equation we can express all parti with i /= 1 as a function of part1: part = part l1(a + c) + p1 + p1e1 − piei + (l1 − li)(b + d) i 1 li(a + c) + pi L(li(a + c) + pi)

(11.2)

Then, the normalization equation allows us to get a solution for part1: N −1

Σ

partj = 1 ⇒

j=1

part + part l1(a + c) + p1 + p1e1 − p2e2 + (l1 − l2)(b + d) + . . .

1 1 l2(a + c) + p2 L(l2(a + c) + p2) l p 1(a + c) + p1 1e1 − pN−1eN−1 + (l1 − lN−1)(b + d) +part + =1⇒ 1 lN−1(a + c) + pN−1 L(lN−1(a + c) + pN−1) , N −1 N −1 Σ l1(a + c) + p1 Σ pjej — p1e1 + (lj — l1)(b + d) ⇒ part1 1 + = 1+ L(l j(a + c) + pj ) lj (a + c) + pj

j=2

part1

N −1 Σ

j=2

l1(a + c) + p1 =1+ l (a + c) + pj j=1 j

N −1 Σ

pjej − p1e1 + (lj − l1)(b + d) ⇒ L(lj(a + c) + pj) j=2

Σ −1 pj ej −p1e1+(lj −l1)(b+d) 1 + L−1 N lj (a+c)+pj j=2 part 1 = ΣN −1 (l1 (a + c) + p1 ) j=1 (l j(a + c) + p j)−1

(11.3)

4. Write a program that calculates part0 from Equations 8.11 and 8.39 in a linear number of steps.

Answer The following listing contains two functions that calculate part0 and subsequently all parti, for the two problem settings that Equations 8.11 and 8.39 correspond to. The “trick” (especially for Equation 8.39 that involves double summations) is to pre-compute and reuse terms, such as e.g. the

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

208

CHAPTER 11. LOAD BALANCING Σk

j=1 bj term of Equation. 8.39, via lines 51-53 in OneportBlockSingleInst.

The two functions also return the predicted total execution time.

It is important to note that OneportBlockSingleInst just implements Equations 8.38 and 8.39, without ordering the nodes as mandated by Theorem 8.3.3.

1 2 3 4 5 6

7 8

double double double

9 10 11

su m Te rm [ N ] ; to tal T im e ; nomin , d en o m ;

12

f o r ( i n t i = 0 ; i < N ; i++) lacp [ i ] = l [ i ] * ( a + c ) + p [ i ] ;

13 14 15

f o r ( i n t i = 1 ; i < N ; i++) su m Te rm [ i ] = p [ 0 ] * e [ 0 ] − p [ i ]

16 17 18

*

e[i] − l[i]

*

(b + d) ;

n o m in = 0 ; f o r ( i n t i = 1 ; i < N ; i++) n o m in += su m Te rm [ i ] / lacp [ i ] ; n o m in = 1 − n o m in / L ;

19 20 21 22 23

d en o m = 1 ; f o r ( i n t i = 1 ; i < N ; i++) d en o m += 1 / lac p [ i ] ; d en o m *= p [ 0 ] ;

24 25 26 27 28 29 30

part [ 0 ] = n o m in / d en o m ;

31

f o r ( i n t i = 1 ; i < N ; i++) pa rt [ i ] = part [ 0 ] * p [ 0 ] / lac p [ i ] + su m Te rm [ i ] ]) ;

32

/ (L

*

lacp [ i→֓

33 34 35

to t al T im e = p [ 0 ] * ( pa rt [ 0 ] re t ur n to t al T im e ;

*

L + e[0]) ;

36 37

}

38

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // S t o r e s the computed p a r t i in arra y part and the r e t u r n s t o t a l e xe c u t i o n time doubl e O n e p o rt B lo ck S in g le In st ( doubl e * p , doubl e * e , double double * b , i n t N , long L , double * part ) { d ou bl e pl [ N ] ; // s h o u l d be r e p l a c e d by dynamic a l l o c a t i o n i s big do ubl e b Sum [ N ] ;

39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58

the →֓ l , →֓ i f N →֓

doubl e to tal T im e ; do ubl e nom i n , d en o m ; f o r ( i n t i = 0 ; i < N ; i++) pl [ i ] = p [ i ] + l ; b Sum [ 0 ] = 0 ; f o r ( i n t i = 1 ; i < N ; i++) b Sum [ i ] = b [ i ] + b Sum [ i − 1 ] ; n o m in = 0 ; f o r ( i n t i = 1 ; i < N ; i++) n o m in += l / pl [ i ] * b Sum [ i ] ; n o m in /= L ;

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

209 n o m in++; f o r ( i n t i = 1 ; i < N ; i++) n o m in += l / pl [ i ] ;

59 60 61 62

d en o m = 0 ; f o r ( i n t i = 0 ; i < N ; i++) d en o m += 1 / pl [ i ] ; d en o m *= pl [ 0 ] ;

63 64 65 66 67 68 69

part [ 0 ] = n o m in / d en o m ;

70

f o r ( i n t i = 1 ; i < N ; i++) pa rt [ i ] = part [ 0 ] * pl [ 0 ] / pl [ i ] − l / ( L — l / pl [ i ] ;

71

*

pl [ i ] )

*

b Sum [ i ] →֓

72 73 74

to t al T im e = p [ 0 ] * part [ 0 ] re t ur n to t al T im e ;

*

L;

75 76

}

77

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t ma i n ( ) { // s ma ll t e s t e r data double p [ 4 ] = { 1 , 2 , 3 , 4 } ; double e [ 4 ] = { 1 , 1 , 1 , 1 } ; double b [ 4 ] = { 1000 , 1000 , 1000 , 1000 } ; double l [ 4 ] = { 1 , 1 , 1 , 1 } ; double part [ 4 ] ;

78 79 80 81 82 83 84 85 86

cout << ” t : ” << N p o rt B lo ck S in g le In st ( p , e , l , 1 , 0 , 1 , 0 , 4 , →֓ 1000000 , part ) << endl ; f o r ( i n t i = 0 ; i < 4 ; i++) cout << par t [ i ] << ” ” ; cout << endl ;

87 88 89 90 91

cout << ” t : ” << O n e p o rt B lo ck S in g le In st ( p , e , 1 , b , 4 , →֓ 1000 00 0 , par t ) << endl ; f o r ( i n t i = 0 ; i < 4 ; i++) cout << par t [ i ] << ” ” ; cout << endl ;

92 93 94 95 96 97 98

}

re tu rn

0;

5. Solve the example at the end of Section 8.3.3.2, by reversing the computing power of the nodes: p0 = 4·6.6·10−8, p1 = 3·(6.6·10−8), p2 = 2·(6.6·10−8) and p3 = (6.6 · 10−8).

Answer Before applying the equations, the nodes (excluding the master) should be sorted according to their bi(pi + l) property. This presents a challenge, as while we can switch the place of a node, the bi overheads are node order specific and not node specific. Hence. changing the order of a node, potentially causes a change in its ei parameter. Fortunately, the closed-form solutions of Section 11.3.3.2 allow the easy calculation of the predicted time for the different permutations of the nodes (surely not advisable for large N , but quite doable for the particular problem).

The results show that load should be distributed in the order P3, P2, P1. The corresponding parts are part0 = 0.156456, part3 = 0.444968, part2 = 0.237034 and part1 = 0.161541, with a total execution time of 1.028sec.

6. An alternative to having the master node distribute the load, is to have all the nodes access a network filesystem and retrieve the data from there.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

210

CHAPTER 11. LOAD BALANCING What kind of communication configuration would correspond to such an arrangement? Calculate the partitioning that would be produced in this case, for the same problem setting as the previous question.

Answer Such a design corresponds to a N-port communication configuration. If we assume that all nodes (including P0 which was previously considered as having the load locally) perform the same sequence of getting the load, processing it and sending the results back to the filesystem server, then the minimum execution time is achieved by having all nodes finish at the same time instance, i.e. for every pair of Pi and Pj, we have:

(i)

(i)

(i)

(j)

(j)

(j)

tdistr + tcomp + tcoll = tdistr + tcomp + tcoll ⇒ l(partiL+bi)+pi partiL+l partiL = l(partjL+bj)+pj partjL+l partjL ⇒ partiL(pi + 2 l) = partjL(pj + 2 l) + l(bj − bi) ⇒ part = part pj + 2 l + l(bj − bi) i j pi + 2 l L(pi + 2 l)

(11.4)

Thus, we can express all parti as a function of part0: part = part p0 + 2 l + l(b0 − bi) i 0 pi + 2 l L(pi + 2 l)

(11.5)

The normalization equation can then yield a closed-form solution for part0: N −1

Σ

parti = 1 ⇒

i=0 N −1

part0

N −1 Σ p0 + 2 l l Σ b0 − bi = 1 ⇒ + L i=1 pi + 2 l pi + 2 l i=0 l Σ N −1 bi−b0 1+ L i=1 pi+2 l part = Σ N −1 0 (p0 + 2 l) i=0 (pi + 2 l)−1

(11.6)

If we assume that the nodes are ordered as specified by the problem statement, then we have b1 = b2 = 2 · 3 · 2160B and b0 = b3 = 3 · 2160B. Using the above equations produces part0 = 0.125605, part1 = 0.165952, part2 = 0.24447 and part1 = 0.463997, with a total execution time of 2.097sec.

7. The partitioning performed using DLT in the kernel convolution example, returns the percent of the input bytes that should be assigned to each node? How can we convert this number to pixels? One may argue that we should instead use image rows. How can we convert the result of our analysis to rows?

Answer As long as there is a linear relationship between the data volume and the desired load unit, there is no need for a conversion. The parti results are

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

211 percentages and can be used without conversion to split the number of rows as well as the number of pixels.

8. Use the tiobench utility available at http://sourceforge.net/projects/ tiobench/ to measure the performance you can get by concurrent access to a network filesystem (e.g. a NFS volume). Calculate the collective throughput of the server versus the number of threads used.

Answer No model answer is available for this exercise. The students should experiment with the tiobench utility and plot the results. A plot similar to Figure 8.6 should be produced.

9. One way that can be used to improve the distribution cost of input data is to compress them. What kind of compression algorithms could be used in this case? How could we adapt the cost models to reflect this change?

Answer There are two distinct possibilities: (a) The input data are already in compressed form. This imposed restrictions on the compression algorithm and data encoding format, as they should permit random access to the input data. This immediately rules out dynamic compression schemes (e.g. dictionary based) and enforces some sort of partitioning of the compressed data, in the form of packets or blocks. A static Huffman/arithmetic code compression algorithm would be suitable for the task. (b) The data are compressed before the distribution phase. Any compression algorithm could be used in this case, as long as the compressiondecompression overhead does not exceed the gains from the reduced communication time.

Let’s assume that a node decompresses the input upon receiving it, and compresses the results prior to sending them back, spending compri time originalDataSize per load unit. If on average, the compression ratio is cr = compressedDataSize , then, the cost models would have to be modified as follows:

(i)

tcomp = pi(partiL + ei) + compripartiL = (p + compr )(part L +

piei

(11.7)

L + b) = l ( a part L + b)

(11.8)

i

i

(i) = l (a part tdistr i i

(i) = l (c part tcoll i i

) pi + compri

i

i

cr

i

cr

L + d) = l ( c part L + d) cr

i

cr

i

(11.9)

Equations 11.7-11.9 adhere to the formulation of the book’s Equations 8.5, 8.7 and 8.8(see pages 583, 585 and 586). This is a clear indication that existing analysis results could be applicable for such a design, by adapting the model parameters.

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

212

CHAPTER 11. LOAD BALANCING

10. Use the DLTlib library to calculate the optimum partitioning for a “query processing” application, where the communication cost is independent of the workload, consisting of a query during distribution and the result during the collection. You can assume that b = d = 1, the workload consists of L = 106 items, and the parallel platform is made of 10 compute nodes connected in a single level tree, with pi = (i +1) 0.01sec/item · 6i ∈ [0, 9] and l = 0.001sec/item.

What would happen if the communication was 10 times slower?

Answer The following listing illustrates how the appropriate DLTlib function can be called and the results printed-out: 1 2 3 4 5 6 7 8 9

m

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

213 // S i n g l e i n s t a l l m e n t c as e . . . cout << h−>n ame << ”\ t ” << h−>p art << endl ; h = h−>n ext _n ;

50 51 52 53 54 55 56 57 58

}

} } else cout << ” S o l u t i o n co uld not be f ound \ n” ; re t u r n 0 ;

A noteworthy addition to the sample provided in the book, is line 10 that changes the maximum number of child nodes of a machine. The default of the library is only four, far less than the problem requirement of nine, which if left unchanged would cause the program to fail during execution with a protection fault.

This program can be compiled and run as follows:

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9

0 . 341 418 0 . 170 709 0 . 113 806 0 . 0853543 0 . 0682834 0 . 0569028 0 . 0487738 0 . 042 677 0 . 0379351

Finally, testing for a 10 times slower communication medium is as simple as changing the l variable value of line 17 to 0.01. The reported results are almost identical to the previous ones, as the communication cost is negligible in comparison to the computation: P re d icte d e xe cu t io n time : 3414 . 22 So lu tio n in term s of load p e rce n t : P0 0 . 341 422 P1 0 . 17071 P2 0 . 113 806 P3 0 . 085 354 P4 0 . 0682828 P5 0 . 056 902 P6 0 . 048 7729 P7 0 . 042 676 P8 0 . 037 934 P9 0 . 0341404

11. Derive the equivalent of Equations 8.47 and 8.48 for arbitrary a and c, and implement them as part of a partitioning function.

Answer The question is essentially to solve the N-port, block-type, single-installment distribution problem (i.e. what Section 8.3.3.1 covers) for the case when P0 is not the load originating node. As such, P0 has to incur communication cost both for the distribution and the result collection phases.

The solution is identical to the one in Exercise 3. The only difference is that while in Exercise 3 we assume that P0 is not participating in the load processing, in this exercise we assume that P0 does not hold the load originally, i.e. there is another master node that is only distributing the

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

214

CHAPTER 11. LOAD BALANCING data. We can reuse the solution of Exercise 3, with a simple change of the indices, so that all parti are expressed as functions of part0, i.e.: part = part l0(a + c) + p0 + p0e0 − piei + (l0 − li)(b + d) i 0 li(a + c) + pi L(li(a + c) + pi) and part = 0

1 + L−1

(11.10)

ΣN −1 pj ej −p0e0+(lj −l0)(b+d) lj (a+c)+pj

j=1

(11.11)

ΣN −1 (l0 (a + c) + p0 ) j=0 (l j(a + c) + p j)−1

The following C++ code implements these two Equations, with function NportBlockSingleInst returning the parti and total execution time: 1 2 3 4 5

6 7

double su m Te rm [ N ] ; double to tal T im e ; double nomin , d en o m ;

8 9 10 11

f o r ( i n t i = 0 ; i < N ; i++) lacp [ i ] = l [ i ] * ( a + c ) + p [ i ] ;

12 13 14

f o r ( i n t i = 1 ; i < N ; i++) su m Te rm [ i ] = p [ i ] * e [ i ] − p [ 0 ] d) ;

15 16

*

e [0] + (l[ i] − l [0])

( b +→֓

*

17

n o m in = 0 ; f o r ( i n t i = 1 ; i < N ; i++) n o m in += su m Te rm [ i ] / lacp [ i ] ; n o m in = 1 + n o m in / L ;

18 19 20 21 22

d en o m = 0 ; f o r ( i n t i = 0 ; i < N ; i++) d en o m += ( 1 / la cp [ i ] ) ; d en o m *= lac p [ 0 ] ;

23 24 25 26 27 28 29

part [ 0 ] = n o m in / d en o m ;

30

f o r ( i n t i = 1 ; i < N ; i++) part [ i ] = part [ 0 ] * la cp [ 0 ] / la cp [ i ] − su m Te rm [ i ] / ( L lacp [ i ] ) ;

31

*

→֓

32 33 34 35 36

}

to t al T im e = p [ 0 ] * ( pa rt [ 0 ] * L + e [ 0 ] ) + l [ 0 ] L + b ) + l [ 0 ] * ( c * part [ 0 ] * L + d ) ; re t ur n to t al T im e ;

*

(a

*

pa rt [ 0 ]

*→֓

37 38 39 40 41 42 43 44 45

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t main ( ) { double p [ 4 ] = { 1 , 2 , 3 , 4 } ; double e [ 4 ] = { 1 , 1 , 1 , 1 } ; double b [ 4 ] = { 1000 , 1000 , 1000 , 1000 } ; double l [ 4 ] = { 1 , 1 , 1 , 1 } ; double part [ 4 ] ;

46 47 48

cout << ” t : ” << N p o rt B lo ck S in g le In st 1000 00 0 , par t ) << endl ; f o r ( i n t i = 0 ; i < 4 ; i++)

(p , e , l ,

1 , 0 , 1 , 0 , 4 , →֓

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

215 cout << par t [ i ] << ” ” ; cout << endl ;

49 50 51 52 53

}

re tu rn

0;

12. The DLT examples presented in Sections 8.3.3.1 and 8.3.3.2, do not address an aspect of the problem which is significant : what is the optimum subset of nodes to use to process a load? Implement the heuristic algorithm you came up with in Exercise 2, to derive such a set.

Hint: nodes that should not be part of this set, get a negative load assignment parti.

Answer The effects of node choice and order are more significant when a one-port configuration is used. When N-port configurations are used, one needs to deal with the node selection problem, only when the communication cost to a node can exceed the total execution time on the other nodes.

In the following listing we implement the heuristic outlined in Exercise 2.

The program tests for a maximum number of nodes, as specified in the command line, and reports the optimum number of nodes to be used and associated partitioning. The nodes are sorted (actually they are generated in this fashion) in descending order of their computing power.

1 2

#i n c l u d e < ios t r ea m > #i n c l u d e < s t d l i b . h>

3 4

u s i n g namespace std ;

5 6 7 8 9 10

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * doubl e O n e p o rt B lo ck S in g le In st ( doubl e * p , doubl e * e , doubl e l , →֓ double * b , i n t N , long L , double * part ) { d ou bl e pl [ N ] ; // s h o u l d be r e p l a c e d by dynamic a l l o c a t i o n i f N →֓ i s big double b Sum [ N ] ;

11 12 13

do u ble to t al T im e ; do u ble nomin , d en o m ;

14 15 16

f o r ( i n t i = 0 ; i < N ; i++) pl [ i ] = p [ i ] + l ;

17 18 19 20

b Sum [ 0 ] = 0 ; f o r ( i n t i = 1 ; i < N ; i++) b Sum [ i ] = b [ i ] + b Sum [ i − 1 ] ;

21 22 23 24 25 26 27 28

n o m in = 0 ; f o r ( i n t i = 1 ; i < N ; i++) n o m in += l / pl [ i ] * b Sum [ i ] ; n o m in /= L ; n o m in++; f o r ( i n t i = 1 ; i < N ; i++) n o m in += l / pl [ i ] ;

29 30 31 32 33 34

d en o m = 0 ; f o r ( i n t i = 0 ; i < N ; i++) d en o m += 1 / pl [ i ] ; d en o m *= pl [ 0 ] ;

35 36

part [ 0 ] = n o m in / d en o m ;

37 38

f o r ( i n t i = 1 ; i < N ; i++)

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

216

CHAPTER 11. LOAD BALANCING pa rt [ i ] = part [ 0 ] — l / pl [ i ] ;

39

*

pl [ 0 ] / pl [ i ] − l / ( L

*

pl [ i ] )

*

b Sum [ i ] →֓

40 41 42

to t al T im e = p [ 0 ] * part [ 0 ] re t ur n to t al T im e ;

*

L;

43 44

}

45

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * i n t main ( i n t argc , char ** argv ) { i n t N = a to i ( ar gv [ 1 ] ) ;

46 47 48 49

do ubl e p [ N ] ; do ubl e e [ N ] ; do ubl e b [ N ] ; double l = . 1 ; double part [ N ] ; long L = 1 0 0 ;

50 51 52 53 54 55 56

f o r ( i n t i = 0 ; i < N ; i++) { p [ i ] = ( i + 1) * 1 ; e[i ] = 0.1; b [ i ] = 1; }

57 58 59 60 61 62 63

doubl e b est T = p [ 0 ] * L + e [ 0 ] , b est N = 1 ; f o r ( i n t i = 1 ; i < N ; i++) { doubl e t = O n e p o rt B lo ck S in g le In st ( p , e , 1 , b , i + 1 , L , →֓ pa rt ) ; bool va l i d = t r u e ; f o r ( i n t j = 0 ; j < i + 1 && val i d ; j++) i f ( par t [ j ] < 0 ) va l i d = f a l s e ; cout << i + 1 << ” ” << t << ” ” << va l i d << endl ; →֓ // d eb u gg in g output i f ( t < b est T ) // v a l i d does not need to be checked as per →֓ the comments in E xe r c i s e 2 { b est N = i + 1 ; b est T = t ; } }

64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79

cout << ” Best time ” << b est T << ” , be st N ” << b es t N << endl ; O n e p o rt B lo ck S in g le In st ( p , e , 1 , b , best N , L , part ) ; // s o l v e →֓ aga in to o b t a i n p a r t i f o r ( i n t i = 0 ; i < b est N ; i++) cout << par t [ i ] << ” ” ; cout << endl ;

80 81 82 83 84 85 86 87

}

re tu rn

0;

A sample of the generated output is shown below: . / h e u r i s t i c s D LT 50 2 80.2 1 3 73 . 4615 1 4 70 . 039 1 5 67 . 9655 1 6 66 . 5785 1 7 65 . 5904 1 8 64 . 8559 1 9 64 . 2931 1 10 63 . 8524 1 ...

23 62 . 1486 1 24 62 . 1394 1 25 62 . 1375 1 26 62 . 1422 0 27 62 . 1526 0 28 62 . 1683 0 $

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

Solutions Manual for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas

217 ...

50 63 . 2093 0 B es t t ime 6 2 . 1 3 7 5 , bes t N 25 0 . 6 2 1 3 7 5 0 . 0 77 5 8 3 5 0 . 0 5 56 8 7 6 0 . 0 42 5 5 0 1 0 . 0 3 37 9 1 7 0 . 0 2 7 53 5 8 →֓ 0 . 0 2 2 8 4 38 0 . 0 1 9 1 9 4 5 0 . 0 1 6 2 7 5 0 . 0 1 3 8 86 4 0 . 0 1 1 8 9 59 0 . 0 1 0 21 1 6 →֓ 0 . 0 0 8 7 6 78 9 0 . 0 0 7 5 16 7 0 . 0 0 6 4 2 19 0 . 0 0 5 4 5 59 1 0 . 00 4 5 9 7 2 5 →֓ 0 . 0 0 3 8 2 89 7 0 . 0 0 3 1 37 5 2 0 . 0 0 2 5 11 9 3 0 . 0 0 1 9 4 32 0 . 0 0 1 4 2 3 9 3 →֓ 0 . 0 0 0 9 4 7 9 3 4 0 . 0 0 0 5 1 0 0 1 7 0 . 000 10 578 6

SOLUTIONS MANUAL for Multicore and GPU Programming An Integrated Approach 2nd Edition by Gerassimos Barlas