Apache Spark.
Core Apache Spark architecture — RDDs, DAG execution, shuffles, Spark SQL, and cluster tuning.
Beginner
Start here — no prior experience needed
Spark Architecture: Driver, Executors, and Cluster Managers
Submit a job and watch the driver plan it, executors run it, and the cluster manager allocate the resources — reading each component's output as it happens.
RDDs: The API Underneath DataFrames
Work directly with RDDs, measure how much slower they are than the equivalent DataFrame, and identify the few cases where you still need them.
Spark SQL and the Catalyst Optimizer
Register views, query them with SQL, then read Catalyst's four plan stages to see exactly which rewrites it applied to your query.
Intermediate
For developers with core concepts down
Deploying Spark Jobs with spark-submit
Package an application, submit it to a cluster, and work through the failures that only appear outside the shell — missing jars, wrong Python, and containers killed for memory.
Reading the Spark UI to Diagnose a Slow Job
Work through the Jobs, Stages, SQL, and Executors tabs on a deliberately broken job, and learn which number to look at first for each kind of slowness.
Advanced
Production-grade patterns for experienced engineers