GPU Programming

CUDA C++ Tutorial with Examples.

CUDA C++ tutorial for GPU kernel development: thread hierarchies, shared memory, warp primitives and performance tuning.

1 Beginner 1 Intermediate 2 Advanced +2 From other topics

Beginner

Start here — no prior experience needed

1 lesson

cuda-cpp beginner

5 min read

CUDA C++ Kernel Anatomy

Write and launch CUDA C++ kernels: global thread indexing, bounds guards, launch errors, __device__ vs __global__, and 2D grids, run on a Tesla T4.

Read Tutorial

Intermediate

For developers with core concepts down

1 lesson

cuda-cpp intermediate

5 min read

Shared Memory & Warp-Level Programming in CUDA C++

Tile a matrix multiply in shared memory, see a missing __syncthreads() break it, fix bank conflicts with padding, and sum a warp with shuffles on a T4.

Read Tutorial

Advanced

Production-grade patterns for experienced engineers

2 lessons

cuda-cpp advanced

5 min read

CUDA Atomics & Parallel Reductions

See a data race lose updates, fix it with atomicAdd, measure atomic contention, and build fast block and warp reductions in CUDA C++ on a Tesla T4.

Read Tutorial

cuda-cpp advanced

5 min read

CUDA C++: Streams, Async Copies and Overlap

Overlap copies and kernels with CUDA streams on a Tesla T4: pinned vs pageable memory, a 4-stream pipeline in nsys, the default-stream trap, events.

Read Tutorial

Also part of CUDA C++

Taught in other topics. Each link opens the lesson in its home topic.

2 lessons

gpu-fundamentals intermediate

5 min read

GPU Fundamentals: Coalescing & Occupancy

Measure how coalesced memory access and occupancy change GPU bandwidth, with CUDA programs timed on a real Tesla T4: strides, layouts and block sizes.

Read Tutorial

cuda advanced

5 min read

CUDA Performance Tuning & Profiling

Profile CUDA on a Tesla T4: time launch overhead, read nsys API and kernel summaries, use the occupancy API, and measure achieved occupancy with ncu.

Read Tutorial