Rolling Upgrade Considerations for Replication and High-Availability Topologies

Learning outcomes

A standalone upgrade can use downtime; a replicated or highly available deployment adds ordering and mixed-version constraints. ServiceHub must remain correct while nodes are drained, upgraded, validated, and rejoined. A reachable node is not automatically safe: it may be behind, divergent, read-only by compatibility policy, or using an unsupported version combination.

Asynchronous replication: validate upgraded redundancy before the writer

A common rolling scheme upgrades replicas while the source remains the known writer, validates each replica, then performs the final source upgrade or a controlled promotion according to the exact supported path. The plan must be based on current source/replica compatibility documentation, not an assumption that any newer replica can replicate from any older source.

SELECT @@version,@@server_uuid,@@server_id,@@gtid_mode;SELECT @@global.gtid_executed,@@global.gtid_purged;SHOW REPLICA STATUS\GSELECT CHANNEL_NAME,SERVICE_STATE,LAST_ERROR_NUMBER,LAST_ERROR_MESSAGEFROM performance_schema.replication_connection_status;SELECT CHANNEL_NAME,SERVICE_STATE,LAST_ERROR_NUMBER,LAST_ERROR_MESSAGEFROM performance_schema.replication_applier_status;

Record GTID and business-state evidence before and after each node. A replica can be connected while its applier is stopped or its data is wrong. “Seconds behind” is useful only as one signal.

Group Replication member-version policy affects upgrade order

For online upgrades in single-primary Group Replication, Oracle recommends upgrading secondaries and then the primary. During a mixed-version window, compatibility policies constrain what upgraded members can do. Newer members can remain read-only while older members are still present. In multi-primary mode that reduces the number of writable members until the group reaches a compatible version state.

SELECT MEMBER_ID,MEMBER_HOST,MEMBER_PORT,       MEMBER_STATE,MEMBER_ROLE,MEMBER_VERSIONFROM performance_schema.replication_group_membersORDER BY MEMBER_HOST,MEMBER_PORT;SELECT @@version,@@super_read_only,@@group_replication_start_on_boot;SELECT @@global.gtid_executed;

For raw Group Replication, a member is deliberately taken offline, prevented from prematurely rejoining while the server upgrade is incomplete, upgraded, validated, then rejoined and allowed to catch up. In an InnoDB Cluster, use current MySQL Shell/AdminAPI procedures so cluster metadata remains authoritative.

Every node passes the same drain/rejoin gates

Failure scenario: never skip replication errors to save the schedule

If an upgraded replica fails while applying a transaction, manually skipping the event may make status look healthy while silently creating data drift. Pause the rollout, preserve the error and GTID evidence, determine whether the problem is data-, version-, or infrastructure-specific, and rebuild/reprovision when correctness cannot be proved.

SELECT @@version,@@server_uuid,@@global.gtid_executed;SHOW REPLICA STATUS\G-- Written decision questions:-- Is the old writer fenced if promotion is considered?-- Is the candidate caught up and GTID-compatible?-- Do business invariants match?-- Is the failure reproducible on staging?-- Repair, rebuild, abort rollout, or promote? Why?-- Do not fabricate GTIDs or skip an unexplained transaction.

Availability and correctness can conflict during maintenance. It is safer to leave a suspect node out of service than to force it green without understanding the failure.

Shell, Router, connector, and application behavior are part of the topology

InnoDB Cluster operations depend on MySQL Shell/AdminAPI, and applications may reach the topology through MySQL Router. Test selected Shell and Router versions against the server target. Test real application connectors through the same endpoint and TLS/authentication policy used in production. A database server that upgrades cleanly can still be an application outage if the client stack is incompatible.

Before upgrading the next node verify:- member/replica state stable- GTID catch-up complete- no receiver/applier errors- expected primary/source role- Router endpoint healthy (if used)- real connector TLS/authentication test passes- read/write business invariants match- error rate and latency stay inside accepted baseline bands- backup/PITR continuity is preserved- abort/rebuild plan remains executable

GTID set comparison makes catch-up decisions explicit

Global Transaction Identifiers (GTIDs) let operators compare transaction history without relying only on binary-log file names. During a rolling upgrade, compare the candidate’s executed set with the source or group’s expected set. A missing subset can mean the node still needs to catch up; an unexpected extra set can be an errant transaction and should block promotion until explained.

SELECT @@global.gtid_executed AS local_executed;-- Paste captured sets from source/candidate into a controlled diagnostic:SELECT GTID_SUBSET(@candidate_set,@source_set) AS candidate_has_no_extra_gtids,       GTID_SUBTRACT(@source_set,@candidate_set) AS source_transactions_missing_on_candidate,       GTID_SUBTRACT(@candidate_set,@source_set) AS candidate_errant_transactions;

Do not fabricate GTIDs to force these expressions to look clean. If a candidate has unexpected history, rebuild or reconcile it according to the documented recovery plan.

Rolling upgrade is a timeline, not a loop over hostnames

Each node passes through drain, offline/upgrade, startup validation, replication/group rejoin, catch-up, canary, and observation. The next node is not touched until the current node has passed all gates. This prevents a latent incompatibility from being replicated across the entire redundancy set before anyone notices.

In a single-primary Group Replication group, upgrading secondaries first preserves the existing primary while newer members are validated. When the primary is eventually upgraded, primary election and member-version compatibility policies must be understood in advance. In multi-primary mode, newer upgraded members can be read-only while older versions remain, which temporarily reduces writable capacity. Capacity planning during the maintenance window must account for that.

Application draining and Router behavior need their own acceptance test

A database node can be “offline for maintenance” while application pools still hold established connections. Before shutting a node down, stop new routing to it and allow or terminate existing sessions according to the application’s transaction policy. Through MySQL Router or another proxy, verify that the intended endpoint no longer selects the node. Then test a transaction/reconnect path from the real application connector.

For a primary/source transition, define how ambiguous commits are handled. A connection can fail after the server committed but before the client received acknowledgment. The application must use idempotency/reconciliation rules from Chapter 19 rather than blindly retrying every write after failover.

Abort conditions protect correctness during a mixed-version window

A low-downtime upgrade is successful when the system remains correct and recoverable throughout, not merely when all nodes eventually report the new version.

Authoritative references

• MySQL 8.4 — Upgrading MySQL
• MySQL 8.4 — Upgrade Paths
• MySQL 8.4 — Upgrade Best Practices
• MySQL Shell 8.4 — Upgrade Checker Utility
• MySQL 8.4 — InnoDB and Online DDL
• MySQL 8.4 — Upgrading Group Replication
• MySQL 8.4 — Downgrading MySQL
• MySQL 8.4 — Native Authentication Plugin
• MySQL Community Server 8.4 Downloads
• MySQL Shell 8.4 Downloads