PostgreSQL · lesson 7 of 7
Intermediate
6- 1 EXPLAIN vs EXPLAIN ANALYZE
- 2 Indexing Strategy: When an Index Actually Helps
- 3 JSONB Containment Queries Need a GIN Index, Not a B-tree
- 4 VACUUM Doesn't Shrink Tables — VACUUM FULL Does, at a Cost
- 5 A Real Deadlock, and Why Lock Order Fixed It
- 6 The WAL Durability Tradeoff: 0.30s vs 0.02s for the Same 200 Inserts
Also part of PostgreSQL
7Also part of: Databases
Point-in-Time Recovery: Undoing a Bad DELETE That Already Committed
A base backup plus archived WAL segments can restore a database to the exact second before a mistake — but only if the WAL covering that second was actually archived first, which is the real failure this lesson walks through.
By SkillByExample · Published
A DELETE that already committed feels permanent — until a base backup and an archived WAL
stream turn “restore to right before that mistake” into a real, executable option.
Setting up WAL archiving and a base backup
# postgresql.conf
archive_mode = on
archive_command = 'cp %p /var/lib/postgresql/wal_archive/%f'
pg_basebackup -U postgres -D /var/lib/postgresql/basebackup -Fp -X stream --checkpoint=fast
--checkpoint=fast forces an immediate checkpoint so the backup starts right away, instead of
waiting for the next one on Postgres’s normal schedule — worth knowing, because without it a
base backup can appear to hang for however long checkpoint_timeout has left to run.
The mistake
INSERT INTO books (title, author, genre, published_year) VALUES ('The Good Insert', 'Someone', 'history', 2020);
SELECT now();
INSERT 0 1
now
-------------------------------
2026-09-13 15:39:25.955677+00
That timestamp is the target: everything up to and including this insert should survive the restore. Then, the mistake:
DELETE FROM loans WHERE borrower LIKE 'Borrower 2%';
DELETE 1122
Meant to remove one specific borrower’s loans; the LIKE pattern also matched Borrower 20
through Borrower 29, Borrower 200 through Borrower 299, and so on — 1,122 rows gone
instead of a handful.
The bug: a recovery target the archive can’t reach
Configure a fresh copy of the base backup to recover to the timestamp above:
# postgresql.conf on the restore instance
restore_command = 'cp /var/lib/postgresql/wal_archive/%f %p'
recovery_target_time = '2026-09-13 15:39:25.955677+00'
recovery_target_action = 'promote'
2026-09-13 15:41:29.458 UTC [238] LOG: database system was interrupted; last known up at 2026-09-13 15:39:08 UTC
cp: can't stat '/var/lib/postgresql/wal_archive/00000002.history': No such file or directory
2026-09-13 15:41:29.631 UTC [238] LOG: starting backup recovery with redo LSN 0/120000D8, checkpoint LSN 0/12000130, on timeline ID 1
2026-09-13 15:41:29.664 UTC [238] LOG: restored log file "000000010000000000000012" from archive
2026-09-13 15:41:29.680 UTC [238] LOG: starting point-in-time recovery to 2026-09-13 15:39:25.955677+00
2026-09-13 15:41:29.697 UTC [238] LOG: completed backup recovery with redo LSN 0/120000D8 and end LSN 0/120001D0
2026-09-13 15:41:29.698 UTC [238] LOG: consistent recovery state reached at 0/120001D0
cp: can't stat '/var/lib/postgresql/wal_archive/000000010000000000000013': No such file or directory
2026-09-13 15:41:29.702 UTC [238] FATAL: recovery ended before configured recovery target was reached
The insert and the delete both happened inside WAL segment 000000010000000000000013 —
and that segment was still the server’s active segment on the source database, not yet
archived. archive_command only fires once a segment is complete (16MB full) or explicitly
switched; recovery can’t replay a byte that was never archived, so it stopped short and
refused to promote into an inconsistent, target-not-reached state.
The fix: switch the WAL segment before restoring
-- on the source database, before copying the archive
SELECT pg_switch_wal();
pg_switch_wal
---------------
0/130987B0
pg_switch_wal() forces the current segment to close and archive immediately, instead of
waiting for it to fill naturally. Re-copy the now-complete archive and retry:
SELECT pg_is_in_recovery();
SELECT count(*) FROM loans;
SELECT title FROM books WHERE title = 'The Good Insert';
pg_is_in_recovery
-------------------
f
count
-------
13986
title
-----------------
The Good Insert
The Good Insert
pg_is_in_recovery() is f — the restored instance promoted to a normal, writable primary.
loans is back to 13986, exactly the count immediately after the insert and before the bad
delete. The Good Insert survived (it appears twice here because this dataset had two
separate real inserts with that title across this lesson’s testing — both correctly present).
The mistake is gone; everything before it is intact.
Practice
1. Take a logical backup with pg_dump and confirm it's a portable SQL script, not a binary snapshot.
pg_dump -U postgres -t books --schema-only | head -8
--
-- PostgreSQL database dump
--
\restrict Z8P0vYFDj8Jhup9HRcDrJL4cwRSm78KGfaOBBtdzR4CXhMqWf3Q2gqKI9tyXk9q
-- Dumped from database version 17.11
-- Dumped by pg_dump version 17.11
Plain SQL, readable and editable — this is the right tool for “move this schema to a
different server” or “keep a human-inspectable snapshot,” not for PITR, which needs the
physical WAL stream a logical dump doesn’t capture. The \restrict token is random per dump
(Postgres 17’s guard against a psql < dump.sql accidentally running untrusted commands) —
your own output will show a different value.
2. Check how many WAL files actually needed archiving to cover this lesson's whole test window.
SELECT archived_count, failed_count FROM pg_stat_archiver;
archived_count | failed_count
----------------+---------------
13 | 24
failed_count is real too — it includes every failed attempt from before the archive
directory’s ownership was fixed earlier in this lesson’s setup, a genuine reminder that
archive_command failures fail silently from the client’s perspective (the server just keeps
retrying) and are only visible by actually checking pg_stat_archiver.
3. Confirm the data directory permission requirement by checking what Postgres actually demands.
stat -c '%a' /var/lib/postgresql/data
700
0700 (owner-only) is what a tar-extracted or freshly copied data directory needs before
Postgres will start against it — this is exactly the check that failed earlier in this lesson
with “data directory has invalid permissions” before chmod 0700 fixed it.
Next: replication — streaming a primary’s WAL to a live read replica, and measuring real lag.