Multicore & GPU Programming : An Integrated Approach, 2e Instructor’s Manual Gerassimos Barlas June 15, 2022
2
Contents Contents
2
1 Introduction
5
2 Multicore and Parallel Program Design
9
3 Threads and Concurrency in standard C++
13
4 Parallel data structures
57
5 Distributed memory programming
61
6 GPU Programming
117
7 GPU and Accelerator Programming : OpenCL
143
8 Shared-memory programming : OpenMP
169
9 The Thrust Template Library
183
10 High-level multi-threaded programming with the Qt library
199
11 Load Balancing
205
3
4
CONTENTS
Chapter 1
Introduction Exercises 1. Study one of the top 10 most powerful supercomputers in the world. Discover:
What kind of operating system does it run? How many CPUs/GPUs is it made of? What is its total memory capacity? What kind of software tools can be used to program it? Answer Students should research the answer by visiting the Top 500 site and -if available- the site of one of the reported systems. 2. How many cores are inside the top GPU offerings from NVidia and AMD? What is the GFlop rating of these chips? Answer N/A. 3. The performance of the most powerful supercomputers in the world is usually reported as two numbers Rpeak and Rmax, both in TFlops (tera floating point operations per second) units. Why is this done? What are the factors reducing performance from Rpeak to Rmax? Would it be possible to ever achieve Rpeak? Answer This is done because the peak performance is unattainable. Sustained, measured performance on specific benchmarks, is a better indicator of the true machine potential. The reason these are different is communication overhead. Rpeak and Rmax could never be equal. Extremely compute-heavy applications, that have no inter-node communications, could asymptotically approach Rpeak if they were to run for a very long time. A very long execution time is required to diminish the influence of the start-up costs. 5