Concurrency in C++

Since C++11 the language has a memory model and a portable threading library. The hard part isn’t starting threads — it’s sharing data between them. A data race (two threads touching the same memory, at least one writing, with no synchronization) is undefined behavior, and the examples below show what that looks like and how each tool prevents it. Every example that prints from several threads synchronizes on a real condition; none relies on sleep.

std::thread — Creating and Joining Threads

std::thread runs a function on a new OS thread. Give each thread its own output slot and there’s nothing to synchronize except the final join():

#include <iostream>
#include <numeric>
#include <thread>
#include <vector>

void partial_sum(const std::vector<int>& data, std::size_t begin, std::size_t end, long long& out) {
    out = std::accumulate(data.begin() + begin, data.begin() + end, 0LL);
}

int main() {
    std::vector<int> data(1'000'000);
    std::iota(data.begin(), data.end(), 1);                // 1..1,000,000

    const std::size_t n = 4, chunk = data.size() / n;
    std::vector<long long> results(n);                     // one slot per thread: no sharing
    std::vector<std::thread> threads;
    for (std::size_t i = 0; i < n; ++i)
        threads.emplace_back(partial_sum, std::cref(data), i * chunk, (i + 1) * chunk,
                             std::ref(results[i]));        // std::ref: pass by reference
    for (auto& t : threads) t.join();                      // wait for all four

    for (std::size_t i = 0; i < n; ++i) std::cout << "chunk " << i << ": " << results[i] << "\n";
    std::cout << "total: " << std::accumulate(results.begin(), results.end(), 0LL) << "\n";
}

$ g++ -std=c++20 -Wall -Wextra -O2 -o threads threads.cpp
$ ./threads
chunk 0: 31250125000
chunk 1: 93750125000
chunk 2: 156250125000
chunk 3: 218750125000
total: 500000500000

Each thread summed a quarter of the range into its own results[i], so no two threads wrote the same memory. Arguments to std::thread are copied into the thread; std::ref/std::cref pass references instead — without them, data would be copied four times and results[i] would be written into a copy nobody reads. join() blocks until the thread finishes; only after all four joins is reading results safe. (With glibc 2.34+ no -pthread flag is needed; older systems require it.)

A std::thread that is still joinable when destroyed ends the program:

#include <iostream>
#include <thread>

int main() {
    std::thread t([] { /* some work */ });
    std::cout << "leaving main without join()" << std::endl;
}   // ~thread on a joinable thread calls std::terminate

$ g++ -std=c++20 -Wall -Wextra -o no_join no_join.cpp
$ ./no_join; echo "exit code: $?"
leaving main without join()
terminate called without an active exception
Aborted
exit code: 134

Data Races

Eight threads increment one plain int 100,000 times each:

#include <iostream>
#include <thread>
#include <vector>

int counter = 0;                                   // shared, unprotected

int main() {
    std::vector<std::thread> threads;
    for (int i = 0; i < 8; ++i)
        threads.emplace_back([] { for (int j = 0; j < 100'000; ++j) ++counter; });
    for (auto& t : threads) t.join();
    std::cout << "counter = " << counter << " (expected 800000)\n";
}

$ g++ -std=c++20 -Wall -Wextra -o race race.cpp
$ ./race
counter = 129730 (expected 800000)
$ ./race
counter = 376826 (expected 800000)

Lost updates: ++counter is a load, an add and a store, and when two threads interleave those steps, one increment overwrites the other. The result changes from run to run. Build the same code with -O2 and it can look fine:

$ g++ -std=c++20 -Wall -Wextra -O2 -o race race.cpp
$ ./race
counter = 800000 (expected 800000)

The optimizer turned each thread’s loop into a single counter += 100000, which shrinks the window for collision — here to nothing, on this run. The race is still there and still undefined; it’s just hiding. ThreadSanitizer finds it regardless of luck:

$ g++ -std=c++20 -g -fsanitize=thread -o race race.cpp
$ ./race 2>&1 | grep -E 'WARNING|SUMMARY|counter ='
WARNING: ThreadSanitizer: data race (pid=7079)
SUMMARY: ThreadSanitizer: data race /tmp/demo/race.cpp:10 in operator()
WARNING: ThreadSanitizer: data race (pid=7079)
SUMMARY: ThreadSanitizer: data race /tmp/demo/race.cpp:10 in operator()
counter = 618533 (expected 800000)

TSan reports two races, both on line 10 — the ++counter — and the full report (without the grep) names the threads, the conflicting read and write, and the global counter. It also slows the program enough that updates were lost again. Run concurrent code under -fsanitize=thread in CI.

Mutexes, Atomics, and Not Sharing at All

Three correct ways to count to 8,000,000 with eight threads, timed:

#include <atomic>
#include <chrono>
#include <iostream>
#include <mutex>
#include <thread>
#include <vector>

template <typename Body>
void timed(const char* name, Body body) {                 // run body on 8 threads
    auto t0 = std::chrono::steady_clock::now();
    std::vector<std::thread> threads;
    for (int i = 0; i < 8; ++i) threads.emplace_back(body);
    for (auto& t : threads) t.join();
    auto ms = std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - t0);
    std::cout << name << ms.count() << " ms, ";
}

int main() {
    constexpr int N = 1'000'000;
    std::mutex m;
    long long locked = 0, batched = 0;
    std::atomic<long long> atomic_count{0};

    timed("mutex per increment:  ", [&] { for (int i = 0; i < N; ++i) { std::lock_guard g(m); ++locked; } });
    std::cout << "count " << locked << "\n";
    timed("atomic per increment: ", [&] { for (int i = 0; i < N; ++i) atomic_count.fetch_add(1, std::memory_order_relaxed); });
    std::cout << "count " << atomic_count << "\n";
    timed("local sum, one lock:  ", [&] {
        long long local = 0;
        for (int i = 0; i < N; ++i) ++local;
        std::lock_guard g(m);
        batched += local;
    });
    std::cout << "count " << batched << "\n";
}

$ g++ -std=c++20 -Wall -Wextra -O2 -o counters counters.cpp
$ ./counters
mutex per increment:  443.349 ms, count 8000000
atomic per increment: 219.484 ms, count 8000000
local sum, one lock:  0.615325 ms, count 8000000

All three give the right count; the costs differ enormously. std::lock_guard locks a std::mutex in its constructor and unlocks in its destructor — even if the body throws — and every increment here pays for a lock handoff between cores. A relaxed std::atomic fetch_add is a single locked CPU instruction, cheaper but still contended on one cache line. Summing into a thread-local variable and taking the lock once per thread avoids nearly all contention; the optimizer even folded each local loop into one addition, which is why it takes well under a millisecond. Timings vary from run to run, but across repeated runs here the ordering never changed: mutex slowest, atomic roughly half to two-thirds of that, local sums in well under a millisecond. Share as little as possible.

Condition Variables — Waiting for a Condition

A condition variable lets a thread sleep until another thread changes some state. This bounded queue blocks producers when it’s full and consumers when it’s empty:

#include <condition_variable>
#include <cstddef>
#include <iostream>
#include <mutex>
#include <optional>
#include <queue>
#include <thread>

template <typename T>
class BoundedQueue {
    std::queue<T> q_;
    std::mutex m_;
    std::condition_variable not_empty_, not_full_;
    std::size_t cap_;
public:
    explicit BoundedQueue(std::size_t cap) : cap_(cap) {}
    void push(T v) {
        std::unique_lock lock(m_);
        not_full_.wait(lock, [&] { return q_.size() < cap_; });   // sleeps, re-checks on wake
        q_.push(std::move(v));
        not_empty_.notify_one();
    }
    T pop() {
        std::unique_lock lock(m_);
        not_empty_.wait(lock, [&] { return !q_.empty(); });
        T v = std::move(q_.front());
        q_.pop();
        not_full_.notify_one();
        return v;
    }
};

int main() {
    BoundedQueue<std::optional<int>> q(3);                 // tiny: the producer must block
    std::thread producer([&] {
        for (int i = 1; i <= 10; ++i) q.push(i * i);
        q.push(std::nullopt);                              // sentinel: no more items
    });
    int received = 0;
    long long sum = 0;
    while (auto item = q.pop()) { ++received; sum += *item; }
    producer.join();
    std::cout << "received " << received << " items, sum of squares " << sum << "\n";
}

$ g++ -std=c++20 -Wall -Wextra -o queue queue.cpp
$ ./queue
received 10 items, sum of squares 385

With capacity 3, the producer blocks after three pushes until the consumer pops, so the two threads take turns. wait(lock, predicate) releases the mutex while sleeping and re-checks the predicate each time it wakes, which handles both spurious wakeups and a notify that arrived before the wait began. std::unique_lock (not lock_guard) is required because wait must unlock and relock it. An empty std::optional works as an end-of-stream sentinel, and FIFO order means the consumer sums all ten squares: 385.

Atomics: Compare-and-Swap and Memory Order

compare_exchange_strong changes a value only if it still holds what you expected — the building block of lock-free code. Release/acquire ordering publishes ordinary data along with an atomic flag:

#include <atomic>
#include <cassert>
#include <iostream>
#include <thread>
#include <vector>

bool claim_once(std::atomic<bool>& flag) {
    bool expected = false;
    return flag.compare_exchange_strong(expected, true);   // only one thread can flip it
}

int main() {
    std::atomic<bool> claimed{false};
    std::atomic<int> winners{0};
    std::vector<std::thread> threads;
    for (int i = 0; i < 8; ++i)
        threads.emplace_back([&] { if (claim_once(claimed)) ++winners; });
    for (auto& t : threads) t.join();
    std::cout << "threads that won the claim: " << winners << "\n";

    int published = 0;
    for (int round = 0; round < 10'000; ++round) {
        std::atomic<bool> ready{false};
        int data = 0;
        std::thread producer([&] {
            data = round;                                   // plain write...
            ready.store(true, std::memory_order_release);   // ...published by the release
            ready.notify_one();
        });
        ready.wait(false, std::memory_order_acquire);       // blocks until true; acquire pairs with release
        assert(data == round);                              // guaranteed visible
        ++published;
        producer.join();
    }
    std::cout << "release/acquire rounds that saw the data: " << published << "\n";
}

$ g++ -std=c++20 -Wall -Wextra -O2 -o atomics atomics.cpp
$ ./atomics
threads that won the claim: 1
release/acquire rounds that saw the data: 10000

Eight threads raced to claim the flag; CAS guarantees exactly one saw false and swapped in true. In the second part, the producer writes a plain int and then stores ready with memory_order_release; the consumer’s wait(false, memory_order_acquire) returns once it sees true, and is then guaranteed to also see data. C++20’s wait/notify_one blocks instead of spinning in a while (!ready.load()) loop, which on a single-core machine can starve the producer it is waiting for. Ten thousand rounds, zero failures of the assert — but that is a guarantee from the memory model, not a statistic: with relaxed on both sides, the program would have a data race on data even if it happened to pass.

Futures, Promises, and std::async

For “run this and give me the result later”, futures beat hand-rolled threads:

#include <future>
#include <iostream>
#include <numeric>
#include <stdexcept>
#include <thread>
#include <vector>

int main() {
    std::future<double> mean = std::async(std::launch::async, [] {
        std::vector<int> v(1'000'000);
        std::iota(v.begin(), v.end(), 1);
        return std::accumulate(v.begin(), v.end(), 0LL) / double(v.size());
    });
    std::cout << "mean = " << mean.get() << "\n";           // blocks until ready

    std::promise<int> prom;
    std::future<int> answer = prom.get_future();
    std::thread t([&prom] { prom.set_value(42); });         // fulfils the future
    std::cout << "promise delivered " << answer.get() << "\n";
    t.join();

    auto failing = std::async(std::launch::async, []() -> int {
        throw std::runtime_error("worker failed");
    });
    try { failing.get(); }                                  // the exception crosses threads
    catch (const std::exception& e) { std::cout << "caught from worker: " << e.what() << "\n"; }
}

$ g++ -std=c++20 -Wall -Wextra -O2 -o futures futures.cpp
$ ./futures
mean = 500000
promise delivered 42
caught from worker: worker failed

std::async(std::launch::async, ...) runs the lambda on a new thread and returns a std::future; get() blocks until the value is ready. Without std::launch::async the implementation may defer the work until get(). A std::promise is the manual version: some thread calls set_value, and the matching future wakes up. The last case is the one hand-rolled threads can’t do easily — an exception thrown in the worker is stored in the future and rethrown by get() in the calling thread.

std::jthread and Stop Tokens (C++20)

std::jthread joins in its destructor and hands the thread a std::stop_token. std::condition_variable_any can wait on both a predicate and a stop request:

#include <condition_variable>
#include <iostream>
#include <mutex>
#include <queue>
#include <stop_token>
#include <thread>

int main() {
    std::mutex m;
    std::condition_variable_any cv;
    std::queue<int> jobs;
    int done = 0;

    std::jthread worker([&](std::stop_token st) {
        std::unique_lock lock(m);
        while (true) {
            cv.wait(lock, st, [&] { return !jobs.empty(); });   // wakes on work OR stop
            if (jobs.empty()) break;                            // woken by stop request
            int job = jobs.front(); jobs.pop();
            std::cout << "worker: job " << job << "\n";
            ++done;
            cv.notify_all();
        }
        std::cout << "worker: stop requested, exiting cleanly\n";
    });

    for (int i = 1; i <= 3; ++i) { std::lock_guard g(m); jobs.push(i); cv.notify_all(); }
    {
        std::unique_lock lock(m);
        cv.wait(lock, [&] { return done == 3; });               // real condition, no sleep
    }
    std::cout << "main: all jobs done, requesting stop\n";
    worker.request_stop();                                      // wakes the waiting worker
}   // ~jthread joins

$ g++ -std=c++20 -Wall -Wextra -O2 -o jthread_cv jthread_cv.cpp
$ ./jthread_cv
worker: job 1
worker: job 2
worker: job 3
main: all jobs done, requesting stop
worker: stop requested, exiting cleanly

The worker sleeps inside cv.wait(lock, st, pred), which returns when there’s a job or when stop is requested — no polling loop, no sleep_for. Main waits for the real condition done == 3 rather than guessing how long three jobs take, then request_stop() wakes the worker, which sees an empty queue and exits. The jthread destructor joins. Every line prints in the same order on every run because each step waits on the previous one’s state.

A Thread Pool

Creating a thread per task is expensive. A pool starts a fixed set of workers once and feeds them from a queue; std::packaged_task connects each task to a future:

#include <condition_variable>
#include <functional>
#include <future>
#include <iostream>
#include <mutex>
#include <queue>
#include <thread>
#include <vector>

class ThreadPool {
    std::vector<std::thread> workers_;
    std::queue<std::function<void()>> tasks_;
    std::mutex m_;
    std::condition_variable cv_;
    bool stop_ = false;
public:
    explicit ThreadPool(std::size_t n) {
        for (std::size_t i = 0; i < n; ++i)
            workers_.emplace_back([this] {
                for (;;) {
                    std::function<void()> task;
                    {
                        std::unique_lock lock(m_);
                        cv_.wait(lock, [this] { return stop_ || !tasks_.empty(); });
                        if (stop_ && tasks_.empty()) return;          // drain, then exit
                        task = std::move(tasks_.front());
                        tasks_.pop();
                    }
                    task();                                           // run outside the lock
                }
            });
    }
    template <typename F>
    auto submit(F f) -> std::future<decltype(f())> {
        auto task = std::make_shared<std::packaged_task<decltype(f())()>>(std::move(f));
        auto fut = task->get_future();
        { std::lock_guard g(m_); tasks_.emplace([task] { (*task)(); }); }
        cv_.notify_one();
        return fut;
    }
    ~ThreadPool() {
        { std::lock_guard g(m_); stop_ = true; }
        cv_.notify_all();
        for (auto& w : workers_) w.join();
    }
};

int main() {
    ThreadPool pool(4);
    std::vector<std::future<int>> results;
    for (int i = 1; i <= 8; ++i) results.push_back(pool.submit([i] { return i * i; }));
    for (auto& f : results) std::cout << f.get() << " ";       // in submission order
    std::cout << "\n";
}

$ g++ -std=c++20 -Wall -Wextra -O2 -o pool pool.cpp
$ ./pool
1 4 9 16 25 36 49 64 

Four workers ran eight tasks in whatever order the scheduler chose, but the results print in submission order because main reads the futures in order — each get() waits for that particular task. Tasks run outside the lock so workers don’t serialize on the queue. The destructor sets stop_, wakes everyone, and joins; the stop_ && tasks_.empty() check lets queued work drain before the workers exit.

Thread-Local Storage

thread_local gives each thread its own copy of a variable, so no lock is needed:

#include <iostream>
#include <thread>
#include <vector>

thread_local int calls = 0;                    // one independent copy per thread
int shared_calls = 0;                          // touched only by main below

void work(int times, int& report) {
    for (int i = 0; i < times; ++i) ++calls;   // no lock needed: nobody else sees it
    report = calls;
}

int main() {
    std::vector<int> reports(3);
    std::vector<std::thread> threads;
    for (int i = 0; i < 3; ++i) threads.emplace_back(work, (i + 1) * 100, std::ref(reports[i]));
    for (auto& t : threads) t.join();
    for (int i = 0; i < 3; ++i) std::cout << "thread " << i << " counted " << reports[i] << "\n";
    ++shared_calls;
    std::cout << "main's own calls: " << calls << ", shared_calls: " << shared_calls << "\n";
}

$ g++ -std=c++20 -Wall -Wextra -O2 -o tls tls.cpp
$ ./tls
thread 0 counted 100
thread 1 counted 200
thread 2 counted 300
main's own calls: 0, shared_calls: 1

Each thread’s calls started at 0 and counted only its own work: 100, 200, 300. Main’s copy is still 0, untouched by all of it. Typical uses are per-thread caches, random-number generators and scratch buffers.

Deadlock and std::scoped_lock

Two threads that lock the same two mutexes in opposite orders can each hold one and wait forever for the other. std::scoped_lock (C++17) locks several mutexes with a deadlock-avoidance algorithm:

#include <iostream>
#include <mutex>
#include <thread>

struct Account { std::mutex m; int balance = 1000; };

void transfer(Account& from, Account& to, int amount) {
    std::scoped_lock lock(from.m, to.m);       // locks both with a deadlock-avoidance algorithm
    from.balance -= amount;
    to.balance += amount;
}

int main() {
    Account a, b;
    std::thread t1([&] { for (int i = 0; i < 100'000; ++i) transfer(a, b, 1); });   // locks a, b
    std::thread t2([&] { for (int i = 0; i < 100'000; ++i) transfer(b, a, 1); });   // locks b, a
    t1.join();
    t2.join();
    std::cout << "a = " << a.balance << ", b = " << b.balance
              << ", total = " << a.balance + b.balance << "\n";
}

$ g++ -std=c++20 -Wall -Wextra -O2 -o scoped_lock scoped_lock.cpp
$ ./scoped_lock
a = 1000, b = 1000, total = 2000

t1 transfers a→b and t2 b→a, 100,000 times each — the classic deadlock setup, since each call names the mutexes in the opposite order. With two separate lock_guards this can hang; scoped_lock acquires both without a lock-order cycle, and the program finishes with the total conserved at 2000.

False Sharing

Cores exchange memory in 64-byte cache lines. Two threads writing different variables on the same line still fight over it:

#include <atomic>
#include <chrono>
#include <iostream>
#include <thread>

struct Packed { std::atomic<long long> a{0}, b{0}; };                          // one cache line
struct Padded { alignas(64) std::atomic<long long> a{0}; alignas(64) std::atomic<long long> b{0}; };

template <typename Counters>
double time_it(Counters& c) {
    auto t0 = std::chrono::steady_clock::now();
    std::thread t1([&] { for (int i = 0; i < 20'000'000; ++i) c.a.fetch_add(1, std::memory_order_relaxed); });
    std::thread t2([&] { for (int i = 0; i < 20'000'000; ++i) c.b.fetch_add(1, std::memory_order_relaxed); });
    t1.join();
    t2.join();
    return std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - t0).count();
}

int main() {
    Packed p;
    Padded q;
    std::cout << "sizeof(Packed) = " << sizeof(Packed) << ", sizeof(Padded) = " << sizeof(Padded) << "\n";
    std::cout << "same cache line:      " << time_it(p) << " ms\n";
    std::cout << "separate cache lines: " << time_it(q) << " ms\n";
    std::cout << "counts: " << p.a + p.b << " " << q.a + q.b << "\n";
}

$ g++ -std=c++20 -Wall -Wextra -O2 -o false_sharing false_sharing.cpp
$ ./false_sharing
sizeof(Packed) = 16, sizeof(Padded) = 128
same cache line:      910.665 ms
separate cache lines: 134.076 ms
counts: 40000000 40000000

The two counters are independent — no data is shared — yet packed into one 16-byte struct they took several times longer than when alignas(64) placed each on its own cache line (about 6.8× in the run shown, and up to roughly 8× across repeated runs here). Padding costs memory (128 bytes instead of 16), so apply it only to hot, independently-written data such as per-thread counters.

Practice

1. Fix race.cpp with a std::atomic<int> and confirm it prints 800000 under both -O0 and ThreadSanitizer.

   Solution

   std::atomic<int> counter{0};
   // loop body unchanged: ++counter is now an atomic fetch_add

   Every build prints counter = 800000, and TSan reports nothing. ++ on an atomic is a single indivisible read-modify-write.

2. In threads.cpp, replace the threads with std::async futures and sum their get() results.

   Solution

   std::vector<std::future<long long>> parts;
   for (std::size_t i = 0; i < n; ++i)
       parts.push_back(std::async(std::launch::async, [&data, i, chunk] {
           return std::accumulate(data.begin() + i * chunk, data.begin() + (i + 1) * chunk, 0LL);
       }));
   long long total = 0;
   for (auto& f : parts) total += f.get();   // 500000500000

   The futures replace both the results vector and the join() loop.

3. Add a second consumer thread to queue.cpp. How must the sentinel logic change?

   Solution

   Push one std::nullopt per consumer — two sentinels — because each consumer stops at the first one it pops. Each consumer keeps its own received/sum and main adds them after joining; the totals are still 10 and 385, but the split between consumers varies from run to run.

4. Change the ready store and load in atomics.cpp to memory_order_relaxed and run it under -fsanitize=thread. What does TSan report?

   Solution

   A data race on data: with relaxed ordering the flag no longer orders the plain write before the consumer’s read, so the two accesses are unsynchronized. The program may still pass the assert on x86, whose hardware ordering is strong — which is exactly why you need a tool, not a test run, to find this.

5. Give ThreadPool a wait_idle() method that blocks until the queue is empty and no task is running.

   Solution

   Track int active_ = 0 under the mutex: increment it when a worker pops a task, decrement it (and notify_all a second condition variable idle_) after the task finishes. Then:

   void wait_idle() {
       std::unique_lock lock(m_);
       idle_.wait(lock, [this] { return tasks_.empty() && active_ == 0; });
   }

   Checking only tasks_.empty() is not enough — the last task may still be running after it leaves the queue.