GPU Programming

CUDA Python Tutorial with Examples.

CUDA Python tutorial: GPU programming from Python using CuPy, Numba CUDA and PyCUDA for accelerated numerical and ML workloads.

1 Beginner 2 Intermediate 1 Advanced +2 From other topics

Beginner

Start here — no prior experience needed

1 lesson

cuda-python beginner

GPU Kernels in Python with Numba and CuPy

Write and launch your first GPU kernels from Python with Numba and CuPy: grid sizing, the bounds guard and host vs device memory, run on a real T4.

Intermediate

For developers with core concepts down

2 lessons

cuda-python intermediate

CUDA Python: Memory Transfers and Benchmarking

Benchmark GPU code from Python correctly: synchronise, warm up, use CUDA events, and measure host-device transfers and pinned memory on a real T4.

cuda-python intermediate

CUDA Python: Shared Memory and Atomics with Numba

Use shared memory and atomics in Numba CUDA kernels: a syncthreads race, a block reduction, histograms and compare-and-swap, measured on a real T4.

Advanced

Production-grade patterns for experienced engineers

1 lesson

cuda-python advanced

CUDA Python: Streams, Overlap and Multi-GPU Patterns

Use CUDA streams from Python to overlap copies with kernels, synchronise safely, and split work across GPUs, with real Tesla T4 timings in Numba.

Also part of CUDA Python

Taught in other topics. Each link opens the lesson in its home topic.

2 lessons

gpu-fundamentals beginner

GPU Fundamentals: SIMT & Memory Hierarchy

Learn SIMT execution, warps, branch divergence and the GPU memory hierarchy (registers, shared, L2, DRAM) from CUDA programs run on a real T4.

gpu-fundamentals intermediate

GPU Fundamentals: Coalescing & Occupancy

Measure how coalesced memory access and occupancy change GPU bandwidth, with CUDA programs timed on a real Tesla T4: strides, layouts and block sizes.