CI/CD Pipeline Patterns: Sequential, Parallel, and Staged

Two pipelines, same three stages, run two different ways — one nine seconds slower, one silently broken.

Same stages, sequential vs. parallel

Three independent CI stages — lint, unit tests, integration tests — each simulated with a sleep to stand in for real work:

# .scratch/lint.py
import time
time.sleep(2)
print(f"lint stage: 2.0s")

# .scratch/run_sequential.py
import subprocess
import time

PY = "py"
stages = [".scratch/lint.py", ".scratch/unit.py", ".scratch/integration.py"]

start = time.time()
for stage in stages:
    subprocess.run([PY, stage], check=True)
print(f"\nTotal sequential pipeline time: {time.time() - start:.1f}s")

$ py run_sequential.py
lint stage: 2.0s
unit stage: 3.0s
integration stage: 4.0s

Total sequential pipeline time: 10.0s

Now launch all three at once with Popen instead of waiting on each subprocess.run in turn:

# .scratch/run_parallel.py
import subprocess
import time

PY = "py"
stages = [".scratch/lint.py", ".scratch/unit.py", ".scratch/integration.py"]

start = time.time()
procs = [subprocess.Popen([PY, stage], stdout=subprocess.PIPE, text=True) for stage in stages]
for p in procs:
    out, _ = p.communicate()
    print(out.strip())
print(f"\nTotal parallel pipeline time: {time.time() - start:.1f}s")

$ py run_parallel.py
lint stage: 2.0s
unit stage: 3.0s
integration stage: 4.0s

Total parallel pipeline time: 4.2s

Sequential pays for every stage: 2+3+4 = 9s of work plus overhead, landing at 10.0s. Parallel only pays for the slowest one — integration’s 4s — plus a little process-launch overhead, landing at 4.2s. This is the entire case for parallelizing independent CI stages: total time collapses toward max(stage times) instead of sum(stage times).

The bug parallelism introduces

Lint, unit, and integration don’t share any state, so running them together is safe. But a real pipeline usually has a stage that produces something another stage consumes — a build step writing an artifact that a test step then reads:

# .scratch/build.py
import time
time.sleep(2)
with open(".scratch/artifact.txt", "w") as f:
    f.write("built-artifact-v1")
print("build stage: wrote artifact.txt")

# .scratch/test_stage.py
with open(".scratch/artifact.txt") as f:
    contents = f.read()
print(f"test stage: read artifact contents = {contents!r}")

Launch both the same naive way — Popen for each, no ordering:

# .scratch/run_naive_parallel.py
import subprocess

PY = "py"
stages = [".scratch/build.py", ".scratch/test_stage.py"]
procs = [subprocess.Popen([PY, s], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True) for s in stages]
for p, s in zip(procs, stages):
    out, err = p.communicate()
    print(f"--- {s} (exit {p.returncode}) ---")
    if out:
        print(out.strip())
    if err:
        print(err.strip())

$ py run_naive_parallel.py
--- .scratch/build.py (exit 0) ---
build stage: wrote artifact.txt
--- .scratch/test_stage.py (exit 1) ---
Traceback (most recent call last):
  File "run_naive_parallel.py", line 1, in <module>
    with open(".scratch/artifact.txt") as f:
         ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '.scratch/artifact.txt'

The test stage started at the same instant as the build stage, tried to open a file the build stage’s 2-second sleep hadn’t written yet, and crashed. Both processes launched in parallel successfully — the bug isn’t a crash in either script on its own, it’s the ordering assumption that parallel launch throws away. This is exactly why “just parallelize everything” is unsafe advice for CI pipelines: it only works between stages with no shared state.

The fix: a build gate, then a parallel fan-out

Run the build stage to completion first — a gate — and only start the dependent stages once it succeeds. Stages that depend on nothing but the gate’s output can still run in parallel with each other:

# .scratch/run_staged_pipeline.py
import os
import subprocess
import time

PY = "py"

if os.path.exists(".scratch/artifact.txt"):
    os.remove(".scratch/artifact.txt")

start = time.time()

# Gate: build must finish before any test stage starts
subprocess.run([PY, ".scratch/build.py"], check=True)

# Fan-out: test stages only need the artifact, not each other
procs = [subprocess.Popen([PY, ".scratch/test_stage.py"], stdout=subprocess.PIPE, text=True) for _ in range(3)]
for p in procs:
    out, _ = p.communicate()
    print(out.strip())

print(f"\nTotal staged pipeline time: {time.time() - start:.1f}s")

$ py run_staged_pipeline.py
build stage: wrote artifact.txt
test stage: read artifact contents = 'built-artifact-v1'
test stage: read artifact contents = 'built-artifact-v1'
test stage: read artifact contents = 'built-artifact-v1'

Total staged pipeline time: 2.2s

All three test-stage copies now read the real artifact instead of racing ahead of it, and the pipeline still isn’t sequential end-to-end — the three consumers of the artifact still run concurrently with each other, they just all wait on the one thing they actually depend on. This is what needs: in GitHub Actions and stages: in GitLab CI are for: expressing which stages must gate which, so the scheduler can parallelize everything else.

Practice

# .scratch/security.py
import time
time.sleep(1)
print("security stage: 1.0s")