GPU Programming

CUDA Tutorial with Examples.

CUDA tutorial: GPU-accelerated computing with NVIDIA CUDA, covering parallel kernels, the memory hierarchy and deep learning acceleration.

1 Beginner 1 Intermediate 1 Advanced +4 From other topics

Beginner

Start here — no prior experience needed

1 lesson

cuda beginner

5 min read

CUDA Programming Basics

Write your first CUDA kernels on a Tesla T4: vector add, thread and block indexing, unified memory, launch error checks and a 2D grid, with real output.

Read Tutorial

Intermediate

For developers with core concepts down

1 lesson

cuda intermediate

5 min read

CUDA Memory Hierarchy & Optimization

Measure CUDA memory on a Tesla T4: coalesced vs scattered reads, a shared-memory stencil that loses to the cache, ncu load counts and ptxas reports.

Read Tutorial

Advanced

Production-grade patterns for experienced engineers

1 lesson

cuda advanced

5 min read

CUDA Performance Tuning & Profiling

Profile CUDA on a Tesla T4: time launch overhead, read nsys API and kernel summaries, use the occupancy API, and measure achieved occupancy with ncu.

Read Tutorial

Also part of CUDA

Taught in other topics. Each link opens the lesson in its home topic.

4 lessons

gpu-fundamentals intermediate

5 min read

GPU Fundamentals: Coalescing & Occupancy

Measure how coalesced memory access and occupancy change GPU bandwidth, with CUDA programs timed on a real Tesla T4: strides, layouts and block sizes.

Read Tutorial

cuda-cpp intermediate

5 min read

Shared Memory & Warp-Level Programming in CUDA C++

Tile a matrix multiply in shared memory, see a missing __syncthreads() break it, fix bank conflicts with padding, and sum a warp with shuffles on a T4.

Read Tutorial

cuda-cpp advanced

5 min read

CUDA Atomics & Parallel Reductions

See a data race lose updates, fix it with atomicAdd, measure atomic contention, and build fast block and warp reductions in CUDA C++ on a Tesla T4.

Read Tutorial

cuda-python advanced

5 min read

CUDA Python: Streams, Overlap and Multi-GPU Patterns

Use CUDA streams from Python to overlap copies with kernels, synchronise safely, and split work across GPUs, with real Tesla T4 timings in Numba.

Read Tutorial