CSE 6220 Intro to High Performance Computing - CSE 6220 (OMS CS): Intro to HPCFinal Exam J. _ Georgi

downloaded from

Scholarfriends

https://scholarfriends.com Downloaded by: writerforacademic@gmail.com | https://scholarfriends.com/singlePaper/315052/cse-6220-intro-to-high-performancecomputing-cse-6220-oms-cs-intro-to-hpcfinal-exam-j-georgi Distribution of this document is illegal.

CSE 6220 (OMS CS): Intro to HPC

Final Exam J

Apr 27-Apr 30 (AOE)

Question 1 (10 points) TopoZoo: Binomial Heap This problem concerns a binomial tree network topology, defined as follows. Let Bh denote a binomial tree network of height h. If h = 0, then B0 is a single processing node. Otherwise, it is constructed by taking two binomial tree networks of height h − 1 and making the root of one tree as the child of the root of the other tree, as shown below.

Five different binomial tree networks. Figure taken from book.huihoo.com. (a) (6 points) Suppose every node of a given Bh network holds a unique random integer.

Give an efficient parallel algorithm to heapify the values as a minHeap. That is, after your parallel-heapify, the integer on each node should be smaller than all integers on its children. Your description must be concise and clear. There is no need to deal with corner cases or to provide an exact algorithm. However, you need to argue convincingly that the parallel running time of your scheme is O(h).

To simplify your notation, let Bh [k, l] represent the node at the l-th level and the k-th column of the network. (b) (4 points) Describe an injective, dilation-1 embedding to map a Bh to a Hypercube of dimension h. Your description must be concise and clear. You don’t need to deal with corner cases or provide an exact algorithm.

CSE 6220 (OMS CS)

Final Exam J, Page 2 of 6

Spring 2019

Question 2 (10 points) Dense Matrix Algorithms (a) (5 points) Consider the matrix-vector multiplication operation, y = Ax, where A is an n×n matrix P and x and y are n×1 vectors. Each output y vector element may be computed as y[i] = nj=0 A[i, j] · x[j] Suppose that the matrix is partitioned with 1D column-wise partitioning, such that each processor stores np complete columns of the matrix A and has matching np elements of the vectors x and y, i.e., if a processor holds column j (i.e., A[∗, j]), then it also owns x[j] and y[j]. Design an efficient distributed memory parallel algorithm to compute matrixvector multiplication in parallel. Analyze the run-time of this algorithm and specify the computation and communication time. (b) (5 points) Let A be an n × n matrix. The transpose of A, denoted AT is defined as AT [i, j] = A[j, i]. Design a parallel algorithm to compute AT from A using an n × n virtual torus topology. During the execution of the algorithm, each processor can hold at most a constant number of matrix elements at a time (can be greater than one). Specify the total runtime for your algorithm.

CSE 6220 (OMS CS)

Final Exam J, Page 3 of 6

Spring 2019

Question 3 (10 points) Hypercube Embedding Let Pi and Pj be two processors in a d-dimensional hypercube and let the bit representations of i and j differ in h bits. Two paths connecting the processors Pi and Pj are said to be parallel if they do not have any processors (and thus no communication links) in common except Pi and Pj . Show that there are exactly d parallel paths between Pi and Pj .

Also show that h of these paths have length h and that the remaining d − h paths have length h + 2.

CSE 6220 (OMS CS)

Final Exam J, Page 4 of 6

Spring 2019

Question 4 (10 points) Build your own collective. One of the nice features of MPI is that it includes a lot of primitives that we can use. But what happens if we find one that isn’t working correctly? In particular, suppose you’re trying to implement an algorithm and find out that MPI Alltoallv() no longer works.

For this problem, assuming the following pieces of information.

1. The number of nodes is P .

2. The initial buffer for each node is stored in id.

3. The resulting buffer is stored in od.

4. The number of items to send are stored in sc[P ].

5. The number of items to receive are stored in rc[P ].

6. The send displacements are stored in si[P ].

7. The receive displacements are stored in ri[P ].

8. The MPI Datatype is M P I GEN ERIC. 9. The communicator is comm. (a) (8 points) Give pseudocode for MPI Alltoallv() using point-to-point operations. Your pseudocode should be reasonably detailed, that is, close to being “compileable.” (b) (2 points) Would your solution change if run on a 1-D ring topology? What about a fully connected network? Why?

CSE 6220 (OMS CS)

Final Exam J, Page 5 of 6

Spring 2019

Question 5 (10 points) Like-minded users. Suppose you are given an n × m matrix, A, where entry ai,j = 1 if user i likes movie j, and 0 otherwise. Two users i and i0 are identical if their rows A(i, :) and A(i0 , :) are the same. Give a cache-efficient algorithm to determine which users are identical. Analyze the work and cache-complexity of your algorithm for a machine with a cache of size Z and line size L = 1. Assume the cache is tall and that the data are “large,” meaning n, m > Z. That is, you cannot fit an entire user (row) or movie (column) in cache.

CSE 6220 (OMS CS)

Final Exam J, Page 6 of 6

Spring 2019

Question 6 (10 points) TopoZoo: Array Tower. An ArrayTower (AT) of height h is defined as follows: it has h levels indexed from 0 to h−1. At each level l, it has an array of length 2l . Let AT (k, l) be the k-th node at the l-th level in an ArrayTower. It has two child nodes, AT (2k, l + 1) and AT (2k + 1, l + 1).

Array Tower of height 3.

Suppose you wish to embed an ArrayTower of height h into a Hypercube of dimension 2h − 2. Hint: how do you embed an 2-D Torus or Mesh into a Hypercube and what are those additional links after embedding? Recall that an h-dimensional Hypercube has 2h nodes.

The notation AT (·, ·) indicates a coordinate of a node in an AT rather than in a Hypercube.

Similarly, H(·, ·, ...) indicates a coordinate in a Hypercube. Critically, your embedding must satisfy the following requirements. There will be a significant deduction if you miss any.

1. The embedding should be injective. That means you cannot map multiple nodes from the AT to one node in the Hypercube.

2. The dilation should be at most two. That is, if two nodes are connected by one direct link in the AT, then, after embedding, they can be connected by a path consisting of at most 2 links in the Hypercube. In other words, each link in AT should be mapped to a path of length at most 2 (two end nodes plus an intermediate transmission node) in the Hypercube.

3. If a node in the Hypercube is in one path mentioned in item 2 above, i.e., the node is an end node or an intermediate transmission node of a path, then this node cannot be used as an intermediate transmission node in other paths.

Now, given an AT of height h and a Hypercube of dimension 2h − 2, show how to embed each node from an AT into an Hypercube, and then show how to choose the path in the Hypercube for each link in the AT. (a) (3 points) Describe your idea for mapping each node AT (k, l) from the AT into the Hypercube. Your description should be clear, non-ambiguous and easy to follow. You do not necessarily need to explicitly write the exact coordinate conversion algorithm or given specific equations. (b) (3 points) Describe how to map the link between AT (k, l) and AT (2k, l + 1) in the AT to a path in the Hypercube. Similarly, do this to the link between AT (k, l) and AT (2k+1, l+1).

Your description should be clear and easy to follow and you do not need to explicitly write the exact coordinate conversion algorithm or equations. (c) (4 points) Similarly, map the horizontal link between AT (2k, l) and AT (2k+1, l) to a path in the Hypercube. Then, do the same to the link between AT (2k − 1, l) and AT (2k, l)