Primary/Standby Architecture, WAL Sender/Receiver, Replay, and Replication Slots

Learning outcomes

ServiceHub has reached the point where a single database process is not an acceptable availability boundary. The first instinct is often “add a replica,” but a physical PostgreSQL standby is not a second independent database that periodically copies rows. It is another PostgreSQL cluster that receives the primary's Write-Ahead Log (WAL) byte stream and replays the same physical changes against a base copy of the same cluster.

This lesson builds a disposable local pair so you can observe that pipeline. The primary runs on port 55436; the standby runs on 55437. Both use the same PostgreSQL 18 major version and free upstream utilities. These ports and directories are intentionally separate from the long-lived servicehub_lab used earlier in the course.

1. The physical replication pipeline

A committing backend on the primary generates WAL. A WAL sender (walsender) process streams requested WAL to the standby. The standby's WAL receiver (walreceiver) receives and writes that stream. The standby's startup/recovery process then replays the WAL records into its local data files. These stages can be at different Log Sequence Numbers (LSNs), which is why “replication lag” is not one number.

SELECT pg_current_wal_insert_lsn() AS insert_lsn,       pg_current_wal_lsn()        AS write_lsn,       pg_current_wal_flush_lsn()  AS flush_lsn;SELECT pid, backend_type, state, wait_event_type, wait_eventFROM pg_stat_activityWHERE backend_type IN ('walsender','walreceiver','startup')ORDER BY backend_type, pid;

On the primary you normally see a walsender once the standby connects. On the standby you see a walreceiver plus recovery/startup activity. Do not infer end-to-end replication health from process existence alone: the processes can be connected while replay is delayed.

2. Prepare a disposable primary

The lab requires PostgreSQL server utilities (initdb, pg_ctl, pg_basebackup, psql) from the same PostgreSQL 18 installation. Exact binary paths and service wrappers differ by Windows, Linux, macOS, and packages, so the examples use direct upstream-style commands. Run them only against disposable directories.

initdb -D ./ch14_primarypg_ctl -D ./ch14_primary -o "-p 55436" -l ./ch14_primary.log start

listen_addresses = 'localhost'port = 55436wal_level = replicamax_wal_senders = 10max_replication_slots = 10hot_standby = on# Optional retention guard for the lab; size according to measured WAL in real systems.max_slot_wal_keep_size = '1GB'

wal_level=replica is sufficient for physical streaming. max_wal_senders and max_replication_slots are capacity limits, not targets to maximize. Some settings require restart; check pg_settings.context and pending_restart instead of guessing.

SELECT name, setting, unit, context, source, pending_restartFROM pg_settingsWHERE name IN ('wal_level','max_wal_senders','max_replication_slots',               'max_slot_wal_keep_size','hot_standby')ORDER BY name;

3. Create a least-privilege replication identity

Physical streaming uses the replication protocol. Create a login with the REPLICATION attribute rather than reusing a superuser. For a local lab, allow only loopback replication traffic in pg_hba.conf. Do not put the password on a process command line.

CREATE ROLE ch14_repl LOGIN REPLICATION;\password ch14_repl

host    replication    ch14_repl    127.0.0.1/32    scram-sha-256host    replication    ch14_repl    ::1/128         scram-sha-256

Reload host-based authentication after editing it. A replication rule uses the special database field replication. Authentication succeeds or fails before any standby can receive WAL, so connection errors should be diagnosed separately from replay lag.

4. Bootstrap the standby from a physical base backup

A physical standby needs a physically compatible starting copy. pg_basebackup can create that copy while also arranging streaming recovery. -R writes standby.signal and connection settings. -C -S creates and names a physical slot on the primary. Use an interactive password prompt or a protected password file; the example intentionally omits a secret.

pg_basebackup \  -h localhost -p 55436 -U ch14_repl \  -D ./ch14_standby \  -Fp -Xs -P -R \  -C -S ch14_standby_slot -W

pg_ctl -D ./ch14_standby -o "-p 55437" -l ./ch14_standby.log start

-Xs streams WAL needed for the base backup. The standby remains read-only while in recovery. The physical base copy and WAL stream must correspond to the same cluster history; this is not a cross-major logical migration technique.

5. Observe sender, receiver, write, flush, and replay

SELECT pid, usename, application_name, client_addr, state,       sent_lsn, write_lsn, flush_lsn, replay_lsn,       write_lag, flush_lag, replay_lag,       sync_state, reply_timeFROM pg_stat_replication;

sent_lsn is how far this sender has transmitted. write_lsn is how far the standby reports writing to its operating system, flush_lsn is how far it reports flushing durably, and replay_lsn is how far recovery has applied changes. The lag-time columns describe recent acknowledgement delay; they are not forecasts of how long a badly lagged standby will need to catch up.

SELECT status, receive_start_lsn, written_lsn, flushed_lsn,       latest_end_lsn, latest_end_time, slot_name, sender_host, sender_portFROM pg_stat_wal_receiver;SELECT pg_is_in_recovery() AS standby,       pg_last_wal_receive_lsn() AS received,       pg_last_wal_replay_lsn() AS replayed,       pg_last_xact_replay_timestamp() AS last_replayed_commit;

written_lsn in pg_stat_wal_receiver is explicitly not a durability proof; flushed_lsn is the durable-receipt boundary reported by the receiver. Replay can still trail flush because CPU, I/O, conflicts, or long WAL records delay application.

6. Generate an observable ServiceHub change

CREATE DATABASE servicehub_ha_lab;\c servicehub_ha_labCREATE SCHEMA app;CREATE TABLE app.ch14_work_orders (  work_order_id bigint PRIMARY KEY,  status text NOT NULL,  changed_at timestamptz NOT NULL DEFAULT clock_timestamp());INSERT INTO app.ch14_work_orders VALUES (14001,'queued',clock_timestamp());SELECT pg_current_wal_flush_lsn() AS committed_through;

\c servicehub_ha_labSELECT pg_is_in_recovery(), * FROM app.ch14_work_orders;-- This should fail on a hot standby:INSERT INTO app.ch14_work_orders VALUES (14002,'should_fail',clock_timestamp());

The expected write failure is a read-only/recovery restriction, not evidence that replication is unhealthy. Physical replay has already reconstructed the primary's catalog and table state on the standby.

7. Physical slots solve one risk by creating another

A slot prevents the primary from recycling WAL that its consumer still needs. That protects a disconnected standby from immediately losing its place—but an abandoned or inactive slot can retain enormous WAL. PostgreSQL 18 exposes status and safety fields specifically so operators can monitor that risk.

SELECT slot_name, slot_type, active, active_pid,       restart_lsn, wal_status, safe_wal_size,       inactive_since, invalidation_reasonFROM pg_replication_slotsWHERE slot_name = 'ch14_standby_slot';SELECT pg_size_pretty(         pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)       ) AS retained_distanceFROM pg_replication_slotsWHERE slot_name = 'ch14_standby_slot'  AND restart_lsn IS NOT NULL;

8. Verification and cleanup

Before calling the pair healthy, verify the standby is streaming, that byte distance is converging under a quiet workload, that a committed ServiceHub marker appears on the standby, and that the slot is active. Save logs and LSN evidence. Then stop and remove only these disposable clusters.

pg_ctl -D ./ch14_standby stop -m fastpg_ctl -D ./ch14_primary stop -m fast# Delete ./ch14_standby and ./ch14_primary only after confirming they are the lab directories.

Authoritative references

Replication and HA behavior is topology-, version-, privilege-, and operating-system-sensitive. These primary PostgreSQL sources define the mechanisms used in this lesson.

• PostgreSQL 18 — Log-Shipping Standby Servers
• PostgreSQL 18 — Monitoring Statistics
• PostgreSQL 18 — pg_replication_slots
• PostgreSQL 18 — Replication Configuration
• PostgreSQL 18 — pg_basebackup