Galera Architecture, Write Sets, Certification, Quorum, and Cluster Membership

Learning outcomes

ServiceHub has outgrown a single MariaDB server. The team wants a database topology that can survive one node failure without waiting for an asynchronous replica to catch up, and it wants more than one node capable of accepting writes. Calling that requirement “multi-master” is not enough. MariaDB Galera Cluster changes the commit path itself: transactions are converted into write sets, globally ordered by the Galera group-communication layer, checked by certification, and then applied on every cluster member.

Galera is the write-set replication provider integrated with MariaDB through the wsrep API (Write Set REPlication API). A Primary Component is the cluster membership group that has quorum and is allowed to process replicated updates. Quorum is a majority decision over the configured membership, not “the node with the newest timestamp.” Virtually synchronous means commit ordering/certification is coordinated across the active component, while remote row application can complete shortly after the originating node returns success.

1. Galera is inside the commit path, not a background binlog copier

Chapter 14's asynchronous replica receives binary-log events after the primary creates them. Galera is different. A transaction executed on node n1 accumulates changes locally. At commit, MariaDB exposes a write set containing the transaction's relevant keys/changes to the wsrep provider. The provider participates in global ordering and certification. If the transaction conflicts with a concurrently ordered write set, the local transaction can be rolled back even though its SQL statements executed successfully before COMMIT.

A cluster that is healthy at the wsrep layer does not automatically expose a single stable client endpoint. That separation becomes central in Lesson 4.

2. Certification is optimistic concurrency across nodes

Galera does not take a distributed lock before every update. Two transactions can execute independently on different nodes. When they reach replication, the ordered write sets are compared against the certification index. If both modify keys that cannot coexist in the same serial order, one transaction loses certification and the client sees a retryable transactional error.

This is why “all nodes accept writes” does not mean “writes scale linearly.” Workloads with independent rows can parallelize well; hot-row workloads can spend increasing time aborting/retrying. The application must treat commit failure as a normal distributed-database outcome.

3. Quorum creates one writable component

Membership changes create components. The component with quorum becomes Primary; a minority partition becomes Non-Primary and will normally reject replicated application work rather than independently accepting divergent writes. In an ordinary equally weighted three-node cluster, two mutually connected nodes retain quorum if the third is isolated.

4. Reproducible three-node Community lab

Save the following as compose.yaml. The official MariaDB 12.3.2 image currently installs the Galera-capable server package and exposes the provider at /usr/lib/galera/libgalera_smm.so. The lab uses rsync for State Snapshot Transfer (SST) because it avoids introducing SST database credentials before Lesson 2. It is intentionally a disposable local topology.

services:  n1:    image: mariadb:12.3.2    container_name: mdb15-n1    environment:      MARIADB_ROOT_PASSWORD: labroot    command:      - --bind-address=0.0.0.0      - --binlog-format=ROW      - --default-storage-engine=InnoDB      - --innodb-autoinc-lock-mode=2      - --wsrep-on=ON      - --wsrep-provider=/usr/lib/galera/libgalera_smm.so      - --wsrep-cluster-name=servicehub15      - --wsrep-cluster-address=gcomm://n2,n3      - --wsrep-node-name=n1      - --wsrep-sst-method=rsync      - --wsrep-new-cluster    ports:      - "33151:3306"  n2:    image: mariadb:12.3.2    container_name: mdb15-n2    environment:      MARIADB_ROOT_PASSWORD: labroot    command:      - --bind-address=0.0.0.0      - --binlog-format=ROW      - --default-storage-engine=InnoDB      - --innodb-autoinc-lock-mode=2      - --wsrep-on=ON      - --wsrep-provider=/usr/lib/galera/libgalera_smm.so      - --wsrep-cluster-name=servicehub15      - --wsrep-cluster-address=gcomm://n1      - --wsrep-node-name=n2      - --wsrep-sst-method=rsync    ports:      - "33152:3306"  n3:    image: mariadb:12.3.2    container_name: mdb15-n3    environment:      MARIADB_ROOT_PASSWORD: labroot    command:      - --bind-address=0.0.0.0      - --binlog-format=ROW      - --default-storage-engine=InnoDB      - --innodb-autoinc-lock-mode=2      - --wsrep-on=ON      - --wsrep-provider=/usr/lib/galera/libgalera_smm.so      - --wsrep-cluster-name=servicehub15      - --wsrep-cluster-address=gcomm://n1      - --wsrep-node-name=n3      - --wsrep-sst-method=rsync    ports:      - "33153:3306"

docker compose down -v --remove-orphans# Bootstrap exactly one node for a brand-new disposable cluster.docker compose up -d n1# Wait until n1 reports wsrep_ready=ON, then join the other nodes.docker compose up -d n2 n3# Verify all three nodes before creating application data.docker exec mdb15-n1 mariadb -uroot -plabroot -e "SHOW GLOBAL STATUS LIKE 'wsrep_cluster_size'; SHOW GLOBAL STATUS LIKE 'wsrep_cluster_status';"docker exec mdb15-n2 mariadb -uroot -plabroot -e "SHOW GLOBAL STATUS LIKE 'wsrep_local_state_comment';"docker exec mdb15-n3 mariadb -uroot -plabroot -e "SHOW GLOBAL STATUS LIKE 'wsrep_local_state_comment';"

Now connect to each node and run the same status query.

SHOW GLOBAL STATUS WHERE Variable_name IN (  'wsrep_cluster_status','wsrep_cluster_size','wsrep_cluster_state_uuid',  'wsrep_connected','wsrep_ready','wsrep_local_state_comment',  'wsrep_provider_name','wsrep_provider_version','wsrep_last_committed');SHOW GLOBAL VARIABLES WHERE Variable_name IN (  'wsrep_on','wsrep_cluster_name','wsrep_cluster_address',  'wsrep_sst_method','binlog_format','innodb_autoinc_lock_mode');

5. Prove a write is replicated, then prove that routing is separate

CREATE DATABASE IF NOT EXISTS servicehub_galera_lab;CREATE TABLE IF NOT EXISTS servicehub_galera_lab.tickets (  ticket_id BIGINT PRIMARY KEY AUTO_INCREMENT,  customer_id BIGINT NOT NULL,  state VARCHAR(24) NOT NULL,  priority INT NOT NULL DEFAULT 0,  updated_at TIMESTAMP(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6)             ON UPDATE CURRENT_TIMESTAMP(6)) ENGINE=InnoDB;CREATE TABLE IF NOT EXISTS servicehub_galera_lab.counters (  counter_name VARCHAR(64) PRIMARY KEY,  counter_value BIGINT NOT NULL) ENGINE=InnoDB;INSERT INTO servicehub_galera_lab.counters(counter_name,counter_value)VALUES ('dispatch',0)ON DUPLICATE KEY UPDATE counter_value=VALUES(counter_value);INSERT INTO servicehub_galera_lab.tickets(customer_id,state,priority) VALUES (101,'open',2);

SELECT ticket_id,customer_id,state,priorityFROM servicehub_galera_lab.ticketsORDER BY ticket_id;

INSERT INTO servicehub_galera_lab.tickets(customer_id,state,priority)VALUES (202,'assigned',4);SELECT LAST_INSERT_ID();

SELECT ticket_id,customer_id,state,priorityFROM servicehub_galera_lab.ticketsORDER BY ticket_id;

The replication evidence proves cluster data flow. It does not create an application endpoint. Your client still connected explicitly to n1/n2/n3. A proxy, service discovery layer, DNS policy, connection pool, or application retry/routing strategy must decide where new connections go.

6. Wrong approach: “two nodes are enough because data is duplicated”

A two-node cluster has two copies but awkward quorum behavior. If communication between the nodes is lost, neither side can safely assume the other is dead and independently continue as the only writer. Operators sometimes “fix” this by manually bootstrapping one side. That is dangerous because the other side may still be serving clients or may later return with divergent state.

For the lab, do not force a second bootstrap. Instead, stop n3 gracefully and observe the membership view on n1/n2, then start n3 and verify it returns to Synced. The lesson is that redundancy, quorum, and routing are different requirements.

docker stop mdb15-n3docker exec mdb15-n1 mariadb -uroot -plabroot -e "SHOW GLOBAL STATUS WHERE Variable_name IN ('wsrep_cluster_size','wsrep_cluster_status');"docker start mdb15-n3

After n3 catches up, verify all three nodes again. A temporary size of 2 is expected after a graceful leave; do not infer that a permanently designed two-node cluster has the same failure tolerance as three independent voting members.

7. What “virtually synchronous” does and does not promise

• Successful commits are globally ordered/certified with the active Primary Component; this is much stronger than “ship the binlog later.”
• Remote nodes can still have receive/apply queues; successful certification does not mean every remote storage page changed before the client received success.
• A node that is not ready/synced must not be treated as a normal application target.
• Network latency becomes part of write latency because commit requires cluster communication.
• Galera is not a backup. Replicated mistakes and logical corruption can reach all nodes quickly.

8. Production judgment

Use Galera when you need a tightly coupled, quorum-based multi-primary cluster and your workload tolerates certification semantics and inter-node latency. Prefer an odd number of voting members placed so a majority can survive the failures you care about. Keep InnoDB tables keyed, use row binlogging, verify the exact server/provider combination, and design application retry behavior before enabling writes on several nodes.

Do not choose Galera merely because “three copies are safer.” Backups, disaster recovery, client routing, security, schema-change policy, capacity matching, and operational drills remain separate disciplines.

docker compose down -v --remove-orphans