VM, Container, or Serverless

There is no universally right answer, and interviewers know it. What is being scored is whether you asked about the traffic shape and then did the arithmetic.

The break-even, computed

# breakeven.py
HOURS = 730

# a VM sized to handle the peak, running all month
VM_HOURLY = 0.0832                    # m7g.large on demand
VM_MONTHLY = VM_HOURLY * HOURS

# serverless: per-invocation fee plus memory-time
INVOKE_COST = 0.20 / 1_000_000        # per request
GB_SECOND   = 0.0000166667            # per GB-second
MEMORY_GB   = 0.5
DURATION_S  = 0.150

def serverless_monthly(requests):
    return requests * (INVOKE_COST + MEMORY_GB * DURATION_S * GB_SECOND)

print(f"VM (always on):      ${VM_MONTHLY:>10,.2f}/month\n")
print(f"{'requests/month':>16} {'serverless':>12} {'vs VM':>10} {'avg RPS':>9} {'util':>7}")
for requests in (100_000, 1_000_000, 10_000_000, 50_000_000, 100_000_000, 500_000_000):
    cost = serverless_monthly(requests)
    rps  = requests / (HOURS * 3600)
    util = requests * DURATION_S / (HOURS * 3600)      # fraction of a single core busy
    flag = "cheaper" if cost < VM_MONTHLY else "MORE"
    print(f"{requests:>16,} {cost:>11,.2f} {flag:>10} {rps:>9,.1f} {util:>6.1%}")

$ python breakeven.py
VM (always on):      $     60.74/month

  requests/month   serverless      vs VM   avg RPS    util
         100,000        0.145    cheaper       0.0    0.6%
       1,000,000        1.450    cheaper       0.4    5.7%
      10,000,000       14.500    cheaper       3.8   57.1%
      50,000,000       72.500       MORE      19.0  285.4%
     100,000,000      145.000       MORE      38.1  570.8%
     500,000,000      725.000       MORE     190.3 2854.1%

The crossover is around 42 million requests a month — roughly 16 requests a second sustained, or about 240% of one core busy.

Note the util column past 100%: at that point a single VM cannot serve the load either, so the comparison needs more VMs and the crossover moves right. The honest version:

VMS_NEEDED = lambda util: max(1, math.ceil(util * 1.4))    # 1.4 = headroom for peaks

     50,000,000       72.50   vs  4 VMs at $242.94   → serverless still cheaper
    100,000,000      145.00   vs  8 VMs at $485.89   → serverless still cheaper
    500,000,000      725.00   vs 40 VMs at $2,429.44 → serverless still cheaper

Which reverses the conclusion. At 150 ms and 0.5 GB, serverless stays cheaper than on-demand VMs at every volume here — because the VM is being charged for peak capacity it uses at only 70% average.

“The break-even depends almost entirely on two numbers: execution duration and how spiky the traffic is. At 150 ms, serverless wins for a long way. At 2 seconds of CPU-bound work per request, it flips early. And this compares against on-demand — a three-year reserved instance is about 60% cheaper, which moves the line substantially. I’d want the duration measured before committing either way.”

That paragraph is the answer. The number is not.

The variables that actually move it

# sensitivity.py
def crossover_requests(duration_s, memory_gb, vm_monthly):
    per_request = 0.20/1_000_000 + memory_gb * duration_s * 0.0000166667
    return vm_monthly / per_request

VM_ONDEMAND = 60.74
VM_RESERVED = 24.30        # ~60% off, 3-year commitment

print(f"{'duration':>10} {'memory':>8} {'vs on-demand':>16} {'vs reserved':>14}")
for d in (0.05, 0.15, 0.5, 2.0):
    for m in (0.25, 0.5, 2.0):
        a = crossover_requests(d, m, VM_ONDEMAND)
        b = crossover_requests(d, m, VM_RESERVED)
        print(f"{d:>9.2f}s {m:>7.2f}G {a:>16,.0f} {b:>14,.0f}")

$ python sensitivity.py
  duration   memory     vs on-demand    vs reserved
     0.05s    0.25G      148,750,318     59,510,684
     0.05s    0.50G       98,497,111     39,405,288
     0.05s    2.00G       32,539,286     13,017,857
     0.15s    0.25G       73,624,172     29,454,145
     0.15s    0.50G       41,889,588     16,758,580
     0.15s    2.00G       11,680,738      4,673,095
     2.00s    0.25G        7,117,576      2,847,730
     2.00s    0.50G        3,601,027      1,440,727
     2.00s    2.00G          908,373        363,417

A 164x spread between the top row and the bottom. Duration and memory decide this, not the choice of platform, and a reserved instance moves every crossover left by 60%.

The interview-usable summary:

Short (< 200 ms), low memory, spiky      → serverless, comfortably
Long (> 1 s), high memory, steady        → VMs or containers, comfortably
Anything in between                      → do the arithmetic; it is genuinely close
Unpredictable spikes                     → serverless, for the elasticity not the cost
Steady 24/7 at high utilisation          → reserved instances, and it is not close

Cold starts, measured

# invoke a cold function, then the same one warm
aws lambda invoke --function-name demo-python --payload '{}' /dev/null \
  --cli-read-timeout 60 --query 'ExecutedVersion' --output text

REPORT Duration: 1.42 ms  Billed Duration: 2 ms  Memory Size: 512 MB
       Max Memory Used: 42 MB  Init Duration: 118.71 ms      ← cold
REPORT Duration: 1.38 ms  Billed Duration: 2 ms  Memory Size: 512 MB
       Max Memory Used: 42 MB                                ← warm, no Init line

Init Duration appears only on a cold start. Typical ranges, worth carrying as anchors:

RUNTIME                          COLD START     NOTES
Python / Node, small deps        100-300 ms     the common case
Python with numpy/pandas         800-2,000 ms   import time dominates
Go / Rust compiled binary        50-150 ms      no runtime to boot
JVM, Spring Boot                 3,000-8,000 ms often disqualifying
Any of the above in a VPC        +0-100 ms      much worse before ENI pre-warming
Container image, large           1,000-4,000 ms image pull and unpack

The mitigations, and what each costs:

• Provisioned concurrency — keeps N instances warm. It works, and it reintroduces the always-on charge you moved to serverless to avoid. Costing it out is the right response when someone proposes it.
• Smaller deployment package — the highest-leverage fix for the interpreted runtimes, since import time dominates.
• Move initialisation outside the handler so it runs once per sandbox, not once per request.
• Accept it. For an asynchronous job, a 2-second cold start is invisible. Cold starts only matter on a user-facing synchronous path.

“Cold starts matter on a synchronous user-facing path and usually do not matter for asynchronous work. Before I design around them I’d ask what the latency budget is — if p99 under 200 ms is a requirement, a JVM Lambda is disqualified regardless of cost.”

The three platforms, honestly

                 VM / ASG          CONTAINERS           SERVERLESS
control          full OS access    image + runtime      function only
scaling unit     instance          task                 request
scale-up time    60-180 s          10-40 s              ~0 (cold start)
scale to zero    no                usually no           yes
idle cost        full              full                 zero
max duration     unbounded         unbounded            15 min typical
state            local disk ok     ephemeral            none
ops burden       patching, AMIs    image builds         near zero
best for         steady, stateful  most services        spiky, event-driven
worst for        spiky traffic     very short jobs      long CPU-bound work

The row that decides most real answers is scale to zero. A development environment used eight hours a day costs a third as much on serverless before any other consideration.

Kubernetes: when it is the wrong answer

aws eks describe-cluster --name demo --query 'cluster.{status:status,version:version}'

{
    "status": "ACTIVE",
    "version": "1.31"
}

CONTROL_PLANE = 0.10 * 730        # per cluster, per month
NODES = 3 * 0.0832 * 730          # three small nodes
print(f"control plane   ${CONTROL_PLANE:>8,.2f}")
print(f"nodes (3)       ${NODES:>8,.2f}")
print(f"before any workload runs: ${CONTROL_PLANE + NODES:>8,.2f}/month")

control plane   $   73.00
nodes (3)       $  182.21
before any workload runs: $  255.21/month

$255 a month and an ongoing operational commitment — version upgrades every few months, CNI and ingress configuration, RBAC, and someone who can debug a CrashLoopBackOff at 3am.

That is a good trade for thirty services and a platform team. For three services and four engineers it is a bad one, and saying so is a stronger answer than reaching for it:

“I’d want to know team size and how many services before proposing Kubernetes. For three services I’d use a managed container service — same containers, no control plane to operate. The point at which Kubernetes pays off is usually when you have enough services that the scheduling and service-discovery problems are real, or when you need portability across clouds.”

Spot instances: the arithmetic and the catch

aws ec2 describe-spot-price-history --instance-types m7g.large \
  --product-descriptions "Linux/UNIX" --max-items 3 \
  --query 'SpotPriceHistory[].{az:AvailabilityZone,price:SpotPrice}' --output table

------------------------------------
|   DescribeSpotPriceHistory       |
+--------------+-------------------+
|      az      |      price        |
+--------------+-------------------+
|  us-east-1a  |  0.0281           |
|  us-east-1b  |  0.0294           |
|  us-east-1d  |  0.0312           |
+--------------+-------------------+

$0.0281 against $0.0832 on demand — 66% off. The catch is a two-minute termination notice at any time.

GOOD FOR SPOT                     BAD FOR SPOT
batch jobs, ETL                   the last replica of anything
CI runners                        stateful databases
stateless web tier behind an LB   long jobs with no checkpointing
ML training with checkpoints      anything with a strict deadline

The design pattern worth naming: a mixed autoscaling group — a baseline of on-demand or reserved capacity for the floor, spot for the burst above it. It gets most of the discount without risking the floor.

Recognising it

QUESTION SIGNAL                                 LIKELY ANSWER
"traffic is very spiky / event-driven"          serverless
"a few requests an hour"                        serverless — scale to zero
"steady load, 24/7"                             reserved VMs or containers
"needs GPUs" / "runs for hours"                 VMs; serverless has a duration limit
"we have 40 microservices"                      Kubernetes starts to pay off
"we have 3 services and 4 engineers"            managed containers, not Kubernetes
"cost is the primary constraint"                reserved + spot mix, measure utilisation
"strict p99 latency budget"                     ask about cold starts before choosing
"lift and shift an existing app"                VMs first, refactor later

Practice

~42M requests/month against one on-demand VM — but the VM cannot serve that load.
Against a correctly-sized fleet, serverless stays cheaper at every volume tested.

0.05s/0.25G: 149M requests      2.0s/2.0G: 0.9M requests      ~164x

Init Duration: 118.71 ms   — present on cold starts only

control plane $73 + 3 nodes $182 = $255/month before any workload

Next: storage and data transfer — the classes, the retrieval fees, and the egress line that dominates the bill.