3.1 Parallel Computing Concepts
Source repo: sdsc-summer-institute-2025 | Branch:
main| Last synced: 2026-04-24 10:27:17.425 UTC
SDSC Summer Institute 2025
Session 3.1 Parallel Computing Concepts
Date: Tuesday, August 5, 2025
Summary: Advanced cyberinfrastructure users, whether they develop their own software or run 3rd party applications, should understand fundamental parallel computing concepts. Here we cover supercomputer architectures, the differences between threads and processes, implementations of parallelism (e.g., OpenMP and MPI), strong and weak scaling, limitations on scalability (Amdahl’s and Gustafson’s Laws) and benchmarking. We also discuss how to choose the appropriate number of cores, nodes or GPUs when running your applications and, when appropriate, the best balance between threads and processes. This session does not assume any programming experience.
Presented by: Bob Sinkovits (rssinkovits @ucsd.edu)
Reading and Presentations:
Jupyter notebooks used to create scaling plots and illustrate Amdahl's law are available at COMPLECS Parallel-computing-concepts