Chapter 22Lesson 03180–240 min

Listeners, PreRunModifiers, Visitors, and Programmatic Execution: Configuration, Design Patterns, and Trade-Offs

Choose listeners, modifiers, visitors, CLI selection, and programmatic execution according to whether you need observation, mutation, post-processing, embedding, portability, or isolation.

Observation vs mutationCLI vs APIVisitor modelsListener versionsCI trade-offs

Current compatibility baseline — verified 2026-08-31. Robot Framework 7.4.2 is the stable course baseline and requires Python 3.8+. Listener API version 3 is the default from Robot Framework 7.0 and is generally recommended; version 2 remains supported for compatibility. Optional ListenerV2/ListenerV3 base classes are public APIs from 6.1. Pre-run modifiers normally extend robot.api.SuiteVisitor and are applied before ordinary selection such as --include/--exclude. Result processing uses robot.api.ExecutionResult and ResultVisitor. robot.run() returns an integer status code, not an ExecutionResult; run_cli(..., exit=False) returns the code instead of exiting. The mandatory labs require no third-party library beyond Robot Framework itself.

1. Choose by lifecycle phase, not by power

The question is not “Which extension API can do this?” but “At what phase should this concern be visible and owned?” A simpler phase boundary usually produces better diagnostics and fewer hidden side effects.

2. Decision table

Need	Prefer	Reason	Main risk
Observe status during run	Listener v3	Event-driven cross-cutting observation.	Sensitive logging or accidental mutation.
Transform tags/tests before run	Pre-run modifier	Explicit pre-execution mutation; normal selection follows.	Silent removal or policy drift.
Inspect executable model offline	TestSuiteBuilder + SuiteVisitor	No execution required.	Confusing model phases.
Inspect output.xml	ExecutionResult + ResultVisitor	Public structured result API.	Overwriting original evidence.
Embed Robot in Python	robot.run	Direct Python options/objects.	Ignored rc and shared-process coupling.
Forward CLI arguments	run_cli(..., exit=False)	Preserves CLI grammar.	Unexpected SystemExit if exit=True.
Select existing tags/names	--include/--exclude/--test	No custom code.	Overengineering if replaced by modifier.

3. Listener versus ordinary keyword

Keywords are visible domain actions called from the suite. Listeners are enabled outside the suite and should stay cross-cutting: IDs, status, policy telemetry, lightweight metadata. If a listener starts performing business transactions, retries, browser steps, or database changes, it has become hidden automation.

4. Observation versus mutation

Listener v3 can mutate Robot's live model objects. Capability does not mean it should. Treat listeners as read-only by default. Put deterministic before-run changes in a pre-run modifier and record a mutation manifest.

For new listeners, prefer snake_case callback names such as start_test and end_test. CamelCase callback names are retained for backward compatibility with older/Jython-era integrations, but the Robot Framework 7.4.2 User Guide does not recommend them for new listener code.

Rule: if an operator should be able to answer “what will run?” before the first test starts, prefer a transparent pre-run modifier over runtime listener mutation.

5. Listener interface versions

Characteristic	v2	v3
Supported in 7.4.2	Yes	Yes
Default if unspecified	No	Yes (since 7.0)
Callback data	Strings + attribute dictionaries	Running/result model objects
Direct model mutation	No	Yes
Best use	Compatibility integrations	New listener development

6. Pre-run modifier versus CLI selection

Use native selection when source names/tags already express the rule. Use a modifier when the rule must transform or generate model metadata. Because Robot applies modifiers before ordinary selection, a modifier can add a tag and --include can then consume it. Document that ordering.

7. SuiteVisitor versus ResultVisitor

SuiteVisitor is the general traversal abstraction used for suite structures and pre-run modification. ResultVisitor traverses full execution results, including result-specific structures. Result mutation should create a derivative; the first output.xml remains immutable evidence.

8. robot.run versus subprocess CLI

Concern	robot.run	Subprocess robot
Process isolation	Shared process	Separate process
Python extension objects	Can pass directly	Import by name/path
--pythonpath support	Not supported by run()	Supported
Status	Integer rc	Child process exit code
Crash containment	Lower	Higher
Best fit	Python platform integration	Shell/CI orchestration

9. One extension versus many chained plugins

Robot can load multiple listeners and modifiers. Every hook introduces ordering, performance, state, and failure-isolation questions. Keep components narrow, version the complete extension set, document listener priority if used, and avoid hidden shared mutable state.

10. Public versus internal API

from robot import run, run_cli
from robot.api import ExecutionResult, ResultVisitor, SuiteVisitor, TestSuiteBuilder
from robot.api.interfaces import ListenerV3

Prefer documented exports. Reachable internal modules are not automatically stable extension contracts. If an unavoidable internal dependency exists, isolate it behind your own adapter and pin/test the exact Robot version.

11. Worked scenario

Requirement	Choice	Evidence
Record test ID/status	Read-only listener v3	JSONL events, no arguments/secrets.
Add compliance tag before selection	Pre-run modifier	Before/after mutation manifest.
Publish result summary	ResultVisitor	Derivative JSON from preserved output.xml.
Run in Python orchestrator	robot.run + rc check	return-code.txt + process status.
Run in generic CI shell	CLI/subprocess	Native process exit code.

12. Knowledge check

If --include already selects the right tests, why avoid a modifier?

Can listener v3 mutate data?

Why may subprocess execution be preferable in CI?

When should a result visitor create a new output path?

13. Summary and bridge

Extension architecture is now tied to phase, ownership, and evidence. Lesson 4 engineers common failure modes and diagnoses them without hiding the cause.

Next lesson

Listeners, PreRunModifiers, Visitors, and Programmatic Execution: Diagnostics, Failure Modes, and Production Practices

Continue with Listeners, PreRunModifiers, Visitors, and Programmatic Execution: Diagnostics, Failure Modes, and Production Practices. It builds directly on the state, evidence, and operating assumptions established here, so carry those constraints forward rather than treating the next page as an isolated topic.

References and version anchors

Listener interface — v2/v3 behavior
Model visitor API — visitor semantics
Public robot.api — stable external API surface

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20

0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0 Send only Ethereum/ERC-20 compatible assets to this address.