Skip to main content
Terraform · lesson 12 of 17
Terraform intermediate Lesson 12 of 17

Remote State Safety (locking, drift, and recovery)

A second Terraform run blocked by the state lock, -lock-timeout waiting it out, and a hand-made hotfix that plan would silently revert, all from real runs.

By SkillByExample · Updated

terraform state locking drift

Shared state needs two protections. Locking stops two runs writing the state at once, and drift review stops a plan from quietly undoing someone’s manual change. Both are demonstrated here with the local backend, which locks the same way remote backends do. Examples use Terraform 1.16.5, hashicorp/time 0.14.2 and hashicorp/local 2.9.1.

Two runs against one state

time_sleep makes an apply take 20 seconds, standing in for a slow cloud resource:

terraform {
  required_providers {
    time = {
      source  = "hashicorp/time"
      version = "~> 0.14"
    }
  }
}

# Stands in for a slow resource, like a database that takes minutes to create.
resource "time_sleep" "slow" {
  create_duration = "20s"
}

Start terraform apply -auto-approve -no-color in one terminal and, while it runs, plan in a second:

$ terraform plan -no-color

Error: Error acquiring the state lock

Error message: resource temporarily unavailable
Lock Info:
  ID:        8b5a533f-426e-64c5-8f30-9ac867d34c04
  Path:      terraform.tfstate
  Operation: OperationTypeApply
  Who:       root@vm
  Version:   1.16.5
  Created:   2026-10-07 11:44:47.638336309 +0000 UTC
  Info:      


Terraform acquires a state lock to protect the state from being written
by multiple users at the same time. Please resolve the issue above and try
again. For most commands, you can disable locking with the "-lock=false"
flag, but this is not recommended.

Even a read-only plan takes the lock, because it reads state and can refresh it. The lock info says who holds it, what they’re doing and since when. That’s what you need to decide whether to wait or to ask a colleague. While the apply runs, .terraform.tfstate.lock.info sits next to the state file. Remote backends store the same information in a lock object.

Waiting instead of failing

$ time terraform plan -lock-timeout=60s -no-color
Acquiring state lock. This may take a few moments...
time_sleep.slow: Refreshing state... [id=2026-10-07T11:45:07Z]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.

real	0m31.273s
user	0m0.184s
sys	0m0.108s

Meanwhile the first terminal finished normally:

$ terraform apply -auto-approve -no-color

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # time_sleep.slow will be created
  + resource "time_sleep" "slow" {
      + create_duration = "20s"
      + id              = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.
time_sleep.slow: Creating...
time_sleep.slow: Still creating... [00m10s elapsed]
time_sleep.slow: Still creating... [00m20s elapsed]
time_sleep.slow: Creation complete after 20s [id=2026-10-07T11:45:07Z]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The second plan waited for the lock, then saw the apply’s result. That’s why it reports no changes and refreshes the time_sleep the first run created. It took about 31 seconds, longer than the apply, because Terraform retries the lock with increasing back-off rather than continuously. Your timing will differ. Use -lock-timeout in CI so queued pipelines wait, and never use -lock=false against shared state.

For a team, the same lock lives in the remote backend, for example S3 with use_lockfile = true, shown as configuration in Terraform state.

Drift: a hotfix Terraform would undo

resource "local_file" "nginx_conf" {
  filename        = "${path.module}/nginx.conf"
  content         = "worker_processes 4;\n"
  file_permission = "0644"
}

After applying, someone hot-fixes the file by hand:

$ echo "worker_processes 16;" > nginx.conf   # a hotfix made by hand
$ terraform plan -refresh-only -no-color
local_file.nginx_conf: Refreshing state... [id=d23eb5de8376327f87a8ccc41359c13f8bfe5c96]

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:

  # local_file.nginx_conf has been deleted
  - resource "local_file" "nginx_conf" {
      - content              = <<-EOT
            worker_processes 4;
        EOT -> null
      - content_base64sha256 = "ceFYS0KuuPoLs9FU3lVO6q2aEuzf9muiqXLwv2wX6EI=" -> null
      - content_base64sha512 = "H5J3xGg+YGQE6U7jpPawD1zCkMf6/gaRubGr61/naVWjDXrcsTTwDWRmriwmyKydJIJCaJOrcggl/IQsly+/lA==" -> null
      - content_md5          = "7e2dab2c74ea66ccab185462ae644be6" -> null
      - content_sha1         = "d23eb5de8376327f87a8ccc41359c13f8bfe5c96" -> null
      - content_sha256       = "71e1584b42aeb8fa0bb3d154de554eeaad9a12ecdff66ba2a972f0bf6c17e842" -> null
      - content_sha512       = "1f9277c4683e606404e94ee3a4f6b00f5cc290c7fafe0691b9b1abeb5fe76955a30d7adcb134f00d6466ae2c26c8ac9d2482426893ab720825fc842c972fbf94" -> null
      - directory_permission = "0777" -> null
      - file_permission      = "0644" -> null
      - filename             = "./nginx.conf" -> null
      - id                   = "d23eb5de8376327f87a8ccc41359c13f8bfe5c96" -> null
    }


This is a refresh-only plan, so Terraform will not take any actions to undo
these. If you were expecting these changes then you can apply this plan to
record the updated values in the Terraform state without changing any remote
objects.

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.
$ terraform plan -no-color | sed -n '1,/^Plan/p'
local_file.nginx_conf: Refreshing state... [id=d23eb5de8376327f87a8ccc41359c13f8bfe5c96]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # local_file.nginx_conf will be created
  + resource "local_file" "nginx_conf" {
      + content              = <<-EOT
            worker_processes 4;
        EOT
      + content_base64sha256 = (known after apply)
      + content_base64sha512 = (known after apply)
      + content_md5          = (known after apply)
      + content_sha1         = (known after apply)
      + content_sha256       = (known after apply)
      + content_sha512       = (known after apply)
      + directory_permission = "0777"
      + file_permission      = "0644"
      + filename             = "./nginx.conf"
      + id                   = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

-refresh-only shows only what changed outside Terraform. The local_file provider identifies a file by its content hash, so an edited file counts as “deleted”. A cloud resource would usually show a ~ attribute change instead. The ordinary plan is the dangerous one. It says “will be created” with worker_processes 4; and never mentions 16. Applying it would quietly revert the hotfix. Read the refresh-only plan first, then make a choice: bring the hotfix into the config, or apply to put it back.

Practice

1. Make drift detection scriptable: run the plan so only its exit code matters.

Solution
$ terraform plan -detailed-exitcode -no-color > /dev/null; echo "exit=$?"
exit=2

-detailed-exitcode returns 0 for no changes, 2 for changes and 1 for errors. A nightly CI job can alert on 2 without parsing any text.

2. Accept what reality looks like into state without changing the file, using a refresh-only apply. What does state contain afterwards?

Solution
$ terraform apply -refresh-only -auto-approve -no-color | tail -3
objects.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
$ terraform state list
$ cat nginx.conf
worker_processes 16;

Nothing. The provider reported the file as deleted, so recording reality removed it from state. The hotfixed file stays on disk, now unmanaged. A refresh-only apply updates state to match reality and never touches the real infrastructure.

3. Keep the hotfix properly: put worker_processes 16; in the config, apply, and confirm the drift check is clean.

Solution
$ terraform apply -auto-approve -no-color | tail -3
local_file.nginx_conf: Creation complete after 0s [id=0e9b4e54ea836a9e8b9472b0aa1da8b03c527339]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
$ terraform plan -detailed-exitcode -no-color > /dev/null; echo "exit=$?"
exit=0

The file is under management again with the hotfix value, and exit code 0 means config, state and reality all agree. Making the config match reality is usually better than reverting a fix someone made for a reason.

Next: workspaces and environments, which keep separate state for dev and prod.

Frequently Asked Questions

What is state locking in Terraform?
Before any operation that could write state, Terraform takes a lock on it; a second run against the same state fails with 'Error acquiring the state lock' and shows who holds the lock and since when. The local backend locks the state file; remote backends such as S3 (use_lockfile), GCS, azurerm and HCP Terraform lock the shared state.
What does -lock-timeout do?
It makes Terraform retry acquiring the lock for up to the given duration instead of failing immediately. In CI, -lock-timeout=5m lets a pipeline queue behind another run rather than failing; the default of 0s fails at once, which is better for an interactive user who should know someone else is working.
What is drift in Terraform?
Drift is any difference between real infrastructure and what Terraform last recorded, usually from a manual change in a console or a hotfix on a server. terraform plan -refresh-only shows the drift alone, and a normal plan proposes changes to put reality back to the configuration, which would undo the manual change.
How do I detect drift automatically?
Run terraform plan -detailed-exitcode on a schedule. It exits 0 when there are no changes, 2 when there are changes, and 1 on an error, so a scheduled CI job can alert on exit code 2 without parsing plan text.
When is terraform force-unlock appropriate?
Only when you are certain the run that took the lock is dead, for example a CI runner killed mid-apply against a remote backend, and you have the lock ID from the error message. Unlocking while that run is still alive lets two runs write state at once, which is exactly what the lock exists to prevent.