Parallel Programming: Core Concurrency Patterns

Parallel Programming in Practice (Theory)

Parallel programming means doing more work at the same time by leveraging multiple cores/threads and/or accelerators.

Two main forms:

• Task parallelism: run independent tasks concurrently
• Data parallelism: split a dataset and process partitions concurrently

Beginner design goals:

1. Keep shared state minimal
2. Prefer “embarrassingly parallel” computations
3. Use synchronization only when necessary

Code Example 1 — Data Parallel “Map” in Python (ThreadPool)

from concurrent.futures import ThreadPoolExecutor

def work(x: int) -> int:
    # CPU-bound example; threads may not speed up due to GIL,
    # but the pattern is the same for I/O-bound work.
    return x * x

def map_parallel(values, max_workers=4):
    with ThreadPoolExecutor(max_workers=max_workers) as ex:
        return list(ex.map(work, values))

if __name__ == "__main__":
    values = list(range(10))
    squares = map_parallel(values, max_workers=4)
    print(squares)

Code Example 2 — Task Parallel “Run Independent Jobs” in C++ (std::async)

#include <future>
#include <iostream>

int job(int x) {
  return x * 2;
}

int main() {
  auto f1 = std::async(std::launch::async, job, 10);
  auto f2 = std::async(std::launch::async, job, 20);

  std::cout << "f1=" << f1.get() << "\n";
  std::cout << "f2=" << f2.get() << "\n";
  return 0;
}

Common Gotchas

• Relying on execution order: parallel tasks can finish in any order.
• Over-sharing mutable state: sharing without locks/atomics causes nondeterministic bugs.
• Silent partial failures: always handle exceptions from workers.

Quick Checklist

• Identify if your problem is task-parallel or data-parallel
• Partition data/work; avoid shared writes
• Use futures/promises or structured concurrency to wait for completion
• Validate correctness with small inputs before scaling up