Skip to main content
Terraform · lesson 15 of 17

Beginner

7

Intermediate

7

Advanced

3

Also part of Terraform

4
Open the Terraform page →
Terraform advanced Lesson 15 of 17

Provisioners: A Last Resort, and Why HashiCorp Calls It That

local-exec runs a command Terraform can't track: a provisioner that silently goes stale, a justified validation step, and a failure that taints it.

By SkillByExample · Updated

terraform provisioners local-exec null_resource terraform_data

A provisioner runs a shell command as part of creating or destroying a resource. HashiCorp’s documentation calls it a last resort, because a provisioner opts out of most of what makes the rest of Terraform reliable. Examples use Terraform 1.16.5, hashicorp/null 3.3.2 and hashicorp/local 2.9.1 on Linux, with this terraform block:

terraform {
  required_providers {
    null  = { source = "hashicorp/null", version = "~> 3.3" }
    local = { source = "hashicorp/local", version = "~> 2.9" }
  }
}

The smell: a provisioner doing a resource’s job

resource "null_resource" "write_config_the_hard_way" {
  provisioner "local-exec" {
    command = "echo version=1.0 > smell-config.txt"
  }
}
$ terraform apply -auto-approve -no-color

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # null_resource.write_config_the_hard_way will be created
  + resource "null_resource" "write_config_the_hard_way" {
      + id = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.
null_resource.write_config_the_hard_way: Creating...
null_resource.write_config_the_hard_way: Provisioning with 'local-exec'...
null_resource.write_config_the_hard_way (local-exec): Executing: ["/bin/sh" "-c" "echo version=1.0 > smell-config.txt"]
null_resource.write_config_the_hard_way: Creation complete after 0s [id=8886080396148157605]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The plan shows a resource with only an id. The file it writes appears nowhere in the plan. Now change the command to version=2.0:

$ terraform plan -no-color
null_resource.write_config_the_hard_way: Refreshing state... [id=8886080396148157605]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.
$ cat smell-config.txt
version=1.0

The command changed, the file is out of date, and Terraform reports nothing. A provisioner only runs at creation, and nothing about this resource changed. The local_file resource used throughout this track would show the difference as a content replacement. If a provider has a resource for the job, use it.

The justified case: a check no resource can do

Here the JSON is hand-written in a heredoc, so a typo is possible. No resource validates arbitrary JSON, so an external tool (here Python’s json.tool) does it, and triggers re-runs the check whenever the file’s content changes:

resource "local_file" "app_json_config" {
  filename = "${path.module}/config.json"
  content  = <<-EOT
    {"name": "orders-api", "replicas": 3}
  EOT
}

resource "null_resource" "validate_json" {
  triggers = {
    config_hash = local_file.app_json_config.content_sha256
  }

  provisioner "local-exec" {
    command = "python3 -m json.tool ${local_file.app_json_config.filename} > /dev/null && echo valid JSON"
  }
}
$ terraform apply -auto-approve -no-color | sed -n '/Plan:/,$p'
Plan: 2 to add, 0 to change, 0 to destroy.
local_file.app_json_config: Creating...
local_file.app_json_config: Creation complete after 0s [id=6c7e33ead78eafe3314ca8956649848ad15bdb48]
null_resource.validate_json: Creating...
null_resource.validate_json: Provisioning with 'local-exec'...
null_resource.validate_json (local-exec): Executing: ["/bin/sh" "-c" "python3 -m json.tool ./config.json > /dev/null && echo valid JSON"]
null_resource.validate_json (local-exec): valid JSON
null_resource.validate_json: Creation complete after 0s [id=113417273972497890]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

This is the legitimate case. It isn’t a workaround for a missing resource, and triggers ties the provisioner to the thing it checks, which the smell example lacked.

A failing provisioner taints its resource

Add a trailing comma, "replicas": 3,}, and apply. These messages come from Python 3.13+. Older Pythons word the JSON error differently.

$ terraform apply -auto-approve -no-color 2>&1 | sed -n "/Plan:/,\$p"
Plan: 2 to add, 0 to change, 2 to destroy.
null_resource.validate_json: Destroying... [id=113417273972497890]
null_resource.validate_json: Destruction complete after 0s
local_file.app_json_config: Destroying... [id=6c7e33ead78eafe3314ca8956649848ad15bdb48]
local_file.app_json_config: Destruction complete after 0s
local_file.app_json_config: Creating...
local_file.app_json_config: Creation complete after 0s [id=87c6d03efef050eef44ed6284b73f7ef9686536e]
null_resource.validate_json: Creating...
null_resource.validate_json: Provisioning with 'local-exec'...
null_resource.validate_json (local-exec): Executing: ["/bin/sh" "-c" "python3 -m json.tool ./config.json > /dev/null && echo valid JSON"]
null_resource.validate_json (local-exec): Illegal trailing comma before end of object: line 1 column 37 (char 36)

Error: local-exec provisioner error

  with null_resource.validate_json,
  on main.tf line 20, in resource "null_resource" "validate_json":
  20:   provisioner "local-exec" {

Error running command 'python3 -m json.tool ./config.json > /dev/null && echo
valid JSON': exit status 1. Output: Illegal trailing comma before end of
object: line 1 column 37 (char 36)

$ terraform state list
local_file.app_json_config
null_resource.validate_json
$ terraform plan -no-color | grep -E "^  #|Plan:"
  # null_resource.validate_json is tainted, so must be replaced
Plan: 1 to add, 0 to change, 1 to destroy.

The apply failed, but the broken config.json was still written, because the file resource succeeded before the check ran. Nothing is rolled back. The null_resource is in state but tainted, so the next apply replaces it and runs the check again. Until the JSON is fixed, every apply fails here. That makes this a gate, but one that runs after the bad file already exists. A precondition or a validation block catches problems before anything is written, whenever Terraform itself can check the condition.

Practice

1. Remove the trailing comma and apply. What happens to the tainted resource?

Solution 
$ terraform apply -auto-approve -no-color | grep -E 'valid|Apply'
null_resource.validate_json: Refreshing state... [id=7170083869391036938]
  # null_resource.validate_json is tainted, so must be replaced
-/+ resource "null_resource" "validate_json" {
null_resource.validate_json: Destroying... [id=7170083869391036938]
null_resource.validate_json: Destruction complete after 0s
null_resource.validate_json: Creating...
null_resource.validate_json: Provisioning with 'local-exec'...
null_resource.validate_json (local-exec): Executing: ["/bin/sh" "-c" "python3 -m json.tool ./config.json > /dev/null && echo valid JSON"]
null_resource.validate_json (local-exec): valid JSON
null_resource.validate_json: Creation complete after 0s [id=5381329228105744022]
Apply complete! Resources: 2 added, 0 changed, 2 destroyed.

The fixed file gets a new hash, and the tainted check is replaced, so the provisioner runs again and passes. The grep matches valid inside validate_json too, which is why most lines appear.

2. Add a destroy-time provisioner that logs which config hash it is cleaning up, then run terraform destroy.

Solution 
  provisioner "local-exec" {
    when    = destroy
    command = "echo removing validation for ${self.triggers.config_hash}"
  }
$ terraform destroy -auto-approve -no-color | sed -n '/Plan:/,$p'
Plan: 0 to add, 0 to change, 2 to destroy.
null_resource.validate_json: Destroying... [id=5381329228105744022]
null_resource.validate_json: Provisioning with 'local-exec'...
null_resource.validate_json (local-exec): Executing: ["/bin/sh" "-c" "echo removing validation for a8cdb14c12f50457e34534cc66bdcd72b9efe85ba37edb0a08201e5fa64d1324"]
null_resource.validate_json (local-exec): removing validation for a8cdb14c12f50457e34534cc66bdcd72b9efe85ba37edb0a08201e5fa64d1324
null_resource.validate_json: Destruction complete after 0s
local_file.app_json_config: Destroying... [id=6c7e33ead78eafe3314ca8956649848ad15bdb48]
local_file.app_json_config: Destruction complete after 0s

Destroy complete! Resources: 2 destroyed.

A destroy-time provisioner can only refer to self, not to other resources, because those may already be gone. Only the when = destroy provisioner ran. The create-time one didn’t.

3. Put the trailing comma back but add on_failure = continue to the check. Does the apply still fail?

Solution 
  provisioner "local-exec" {
    command    = "python3 -m json.tool ${local_file.app_json_config.filename} > /dev/null && echo valid JSON"
    on_failure = continue
  }
$ terraform apply -auto-approve -no-color | sed -n '/Plan:/,$p'
Plan: 2 to add, 0 to change, 0 to destroy.
local_file.app_json_config: Creating...
local_file.app_json_config: Creation complete after 0s [id=87c6d03efef050eef44ed6284b73f7ef9686536e]
null_resource.validate_json: Creating...
null_resource.validate_json: Provisioning with 'local-exec'...
null_resource.validate_json (local-exec): Executing: ["/bin/sh" "-c" "python3 -m json.tool ./config.json > /dev/null && echo valid JSON"]
null_resource.validate_json (local-exec): Illegal trailing comma before end of object: line 1 column 37 (char 36)
null_resource.validate_json: Creation complete after 0s [id=255971373448764852]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

It succeeds, and the resource isn’t tainted, even though the error is right there in the log. on_failure = continue turns a check into a comment. Use it only for best-effort side effects, never for validation.

4. Replace null_resource with the built-in terraform_data, so the check needs no extra provider.

Solution 
resource "terraform_data" "validate_json" {
  triggers_replace = local_file.app_json_config.content_sha256

  provisioner "local-exec" {
    command = "python3 -m json.tool ${local_file.app_json_config.filename} > /dev/null && echo valid JSON"
  }
}
$ terraform apply -auto-approve -no-color | sed -n '/Plan:/,$p'
Plan: 2 to add, 0 to change, 0 to destroy.
local_file.app_json_config: Creating...
local_file.app_json_config: Creation complete after 0s [id=6c7e33ead78eafe3314ca8956649848ad15bdb48]
terraform_data.validate_json: Creating...
terraform_data.validate_json: Provisioning with 'local-exec'...
terraform_data.validate_json (local-exec): Executing: ["/bin/sh" "-c" "python3 -m json.tool ./config.json > /dev/null && echo valid JSON"]
terraform_data.validate_json (local-exec): valid JSON
terraform_data.validate_json: Creation complete after 0s [id=de9e7691-6664-e8af-d2f0-071b0a1438be]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

The behaviour is the same, but the resource now comes from Terraform core (terraform_data needs Terraform 1.4+), and triggers_replace takes the hash directly instead of a map.

Next: terraform import, which brings something that already exists under Terraform’s management.

Frequently Asked Questions

Why does HashiCorp call provisioners a last resort?
A provisioner's command runs once, when the resource is created (or destroyed), and Terraform has no idea what it did or whether it is still true later. A provider resource gets drift detection, typed attributes and in-place updates; a provisioner gets none of them, so a change to its command is never noticed and its effects are never checked.
What's the difference between local-exec and remote-exec?
local-exec runs on the machine running Terraform. remote-exec connects to the resource being created over SSH or WinRM and runs commands there, so it only applies to resources such as virtual machines; cloud-init or a configuration-management tool is usually a better fit for that job.
Does a provisioner run again if I change its command?
No. Provisioners run only when their resource is created or destroyed. Changing the command text alone produces 'No changes' in plan. To re-run on purpose, tie the resource to the thing that matters with triggers (null_resource) or triggers_replace (terraform_data) so a change replaces it.
What happens when a provisioner fails?
The apply stops with 'local-exec provisioner error', and the resource stays in state marked tainted: it exists, but Terraform does not trust it. The next plan says it 'is tainted, so must be replaced', and the next apply recreates it and runs the provisioner again. Setting on_failure = continue ignores the failure instead.
Should I use null_resource or terraform_data?
terraform_data, on Terraform 1.4 or later. It is built into Terraform core, so it needs no provider download, and its triggers_replace argument does what null_resource's triggers map did. null_resource from the hashicorp/null provider still works and appears in a lot of existing code.