Data Guard Broker Configuration, Health, Switchover, Failover, and Reinstate Workflows

Learning outcomes

ServiceHub has a healthy physical standby, but the role-change runbook contains 40 manual SQL commands, parameter edits and service steps. A planned maintenance window becomes risky because humans must coordinate two databases precisely. Oracle Data Guard Broker centralizes configuration state and role transitions; DGMGRL is the Broker command-line interface. Broker simplifies operations, but it does not erase prerequisite failures or data-loss risk.

1. Broker is a control plane over Data Guard state

When DG_BROKER_START=TRUE, Broker background processes manage/monitor configuration metadata. A broker configuration contains the primary and standby/far-sync members. DGMGRL changes broker-managed properties and orchestrates role transitions so transport/apply/service state is updated consistently.

SHOW PARAMETER dg_broker_startALTER SYSTEM SET dg_broker_start=TRUE  SCOPE=BOTH;

This does not create a standby. It only starts the broker infrastructure on a database that must already meet Data Guard licensing/topology prerequisites.

2. Create and enable a broker configuration

CONNECT sys@servicehub_priCREATE CONFIGURATION 'ServiceHubDR'  AS PRIMARY DATABASE IS 'servicehub_pri'  CONNECT IDENTIFIER IS servicehub_pri;ADD DATABASE 'servicehub_stby'  AS CONNECT IDENTIFIER IS servicehub_stby  MAINTAINED AS PHYSICAL;ENABLE CONFIGURATION;SHOW CONFIGURATION;

DB_UNIQUE_NAME, connect identifiers, listener/service registration, password-file/SYSDG/SYSDBA authentication and network reachability must already be correct. Broker cannot fix an ambiguous/incorrect Oracle Net architecture.

3. Validate before changing roles

SHOW CONFIGURATION VERBOSE;SHOW DATABASE VERBOSE 'servicehub_pri';SHOW DATABASE VERBOSE 'servicehub_stby';VALIDATE DATABASE 'servicehub_pri';VALIDATE DATABASE 'servicehub_stby';

Validation checks role-transition readiness and reports warnings/errors. Treat warning output as work to investigate, not as decorative text. Also confirm apply/transport lag, archived-log gaps, SRLs, services and application connection behavior outside Broker.

4. Switchover is planned role reversal with no data loss

A switchover transitions the current primary to standby while promoting the target standby. Broker coordinates redo transport/apply and role properties. Oracle defines switchover as a no-data-loss planned operation, normally used for maintenance and DR testing.

VALIDATE DATABASE 'servicehub_stby';SWITCHOVER TO 'servicehub_stby';SHOW CONFIGURATION;SHOW DATABASE 'servicehub_stby';SHOW DATABASE 'servicehub_pri';

“No data loss” is database-role semantics, not “no user-visible outage.” Connections must drain/reconnect and services must start at the new primary. Measure the application interruption separately.

5. Failover is an emergency promotion and can lose data

A failover promotes a standby after the primary is failed/unreachable. Data loss depends on the redo already transported/applied and the protection/transport mode. Broker can reduce manual errors, but it cannot recover redo that never arrived at the target.

SHOW DATABASE 'servicehub_stby';VALIDATE DATABASE 'servicehub_stby';FAILOVER TO 'servicehub_stby';SHOW CONFIGURATION;

Do not issue failover merely because one client cannot connect. Verify primary database/site status, observer/quorum state, transport/apply point, network partitions and fencing. An incorrect failover decision can produce unnecessary data loss and split-brain risk.

6. Reinstate can turn the former primary into a standby

After failover, Broker can sometimes reinstate the old primary instead of rebuilding it. Reinstatement uses Flashback Database to rewind divergent changes to a point compatible with the new primary, then resumes redo transport/apply. The former primary needs a usable flashback history from before failover.

REINSTATE DATABASE 'servicehub_pri';SHOW CONFIGURATION;SHOW DATABASE 'servicehub_pri';

If required flashback history is unavailable, storage/database integrity is questionable, or the failed site has been rebuilt, create a fresh standby from a known-good primary backup/duplicate instead of forcing reinstate.

7. Free design-validation lab: write a role-transition evidence pack

Free cannot run the Data Guard configuration, but it can exercise the evidence discipline that prevents unsafe role changes.

SELECT  db_unique_name,  database_role,  open_mode,  log_mode,  protection_mode,  protection_level,  switchover_status,  flashback_onFROM v$database;SELECT  SYS_CONTEXT('USERENV','CON_NAME') AS con_name,  SYS_CONTEXT('USERENV','SERVICE_NAME') AS service_nameFROM dual;SELECT name,pdb,network_nameFROM v$servicesORDER BY name;

CREATE TABLE servicehub_dr_runbook (  step_no      NUMBER PRIMARY KEY,  evidence     VARCHAR2(200) NOT NULL,  pass_rule    VARCHAR2(500) NOT NULL,  owner_role   VARCHAR2(40) NOT NULL);INSERT INTO servicehub_dr_runbook VALUES (10,'Broker configuration health','SUCCESS with no unresolved role-blocking warnings','DBA');INSERT INTO servicehub_dr_runbook VALUES (20,'Transport/apply freshness','Lag/gap within documented transition policy','DBA');INSERT INTO servicehub_dr_runbook VALUES (30,'Target service readiness','Role-based service can start on target','Platform');INSERT INTO servicehub_dr_runbook VALUES (40,'Client reconnect test','Pool reconnects only to verified new primary','App');INSERT INTO servicehub_dr_runbook VALUES (50,'Former-primary fencing','Old primary cannot accept writes','Platform');SELECT * FROM servicehub_dr_runbook ORDER BY step_no;DROP TABLE servicehub_dr_runbook PURGE;

The table is a local discipline simulator, not a Broker replacement. On the real topology each row must be backed by actual DGMGRL/database/network evidence.

8. Deliberately wrong: force failover to silence a transient network alert

If the primary is still processing writes but the operator promotes a lagging standby during a network partition, clients can be split between two writable sites. A manual failover must therefore be coupled with incident authority and primary-site fencing/isolation. Fast-Start Failover in Lesson 5 automates this only after quorum/health rules are engineered.

9. Production judgment

Use Broker/DGMGRL as the authoritative Data Guard role-control interface when possible. Validate before switchovers, rehearse services/clients, and document the maximum allowed lag/data-loss behavior before a failover decision. Preserve Flashback Database on both role partners when reinstate/FSFO operations rely on it.

DGMGRL/observer software can run on separate systems without a separate observer-host license, but the primary/standby databases still require the applicable Data Guard/Active Data Guard entitlements. Lesson 3 now quantifies why transport choice matters: SYNC changes the commit path; ASYNC changes exposure and bandwidth buffering.

Authoritative references

• Oracle Data Guard Broker — broker architecture and DGMGRL
• Overview of Switchover and Failover — planned versus emergency role transitions
• DGMGRL Command-Line Interface — SHOW/VALIDATE/SWITCHOVER operations
• Fast-Start Failover — reinstate and flashback dependencies
• Licensing Information — Data Guard broker observer licensing note and feature entitlements