Multibranch Pipelines, Branch Sources, Pull Request Discovery, Jenkinsfile Trust, and Branch Indexing: Diagnostics, Failure Modes, Security, and Performance

1. Diagnostic sequence for Multibranch incidents

1. Preserve parent full name, indexing/event log, child job name, queue/build IDs and exact source SHA.
2. Confirm Jenkins core/Java plus Branch API, SCM API and provider branch-source versions/advisories.
3. Confirm repository identity, discovery traits, Jenkinsfile path and branch/change-request head.
4. Determine candidate revision, trusted revision and trust policy where the provider supports them.
5. Inspect whether indexing created/updated the expected child and what caused the child build.
6. Inspect queue/label/executor eligibility and agent/workspace/tool state.
7. Inspect credential availability, status-feedback permissions and any protected external capability.
8. Apply the smallest safe configuration/code correction, then rescan/rebuild only the required scope.

2. Intentionally broken example: the secret guard lives in untrusted code

Suppose a fork Jenkinsfile contains:

if (env.CHANGE_ID) {
  echo 'PR build: skip deployment'
} else {
  withCredentials([string(credentialsId: 'production-token', variable: 'TOKEN')]) {
    sh './deploy.sh'
  }
}

This looks cautious but is not a security boundary. A fork author who controls the Jenkinsfile can delete the if. The repair is not a stronger string comparison. Remove protected credentials/agents from the untrusted control plane and use provider trust or a separate trusted promotion job whose code the contributor cannot rewrite.

3. Duplicate builds from scan + webhook/event

Do not immediately disable one trigger. Preserve:

• both build numbers and causes;
• the exact source SHA;
• parent indexing logs around the timestamps;
• provider event delivery/request ID where available.

If the same commit was independently scheduled twice, decide whether webhook and periodic-scan policies overlap unnecessarily. If one event only rescanned without creating a second build, do not misdiagnose normal indexing as duplicate execution.

4. Indexing storm and SCM API pressure

Symptoms include frequent scans, controller CPU/IO load, provider API throttling and a large queue of branch jobs. Causal layers can include organization-wide discovery, too-frequent periodic scans, a webhook loop, excessive branch/PR traits, or repository explosion. Preserve event-routing/indexing logs before changing credentials or increasing API limits.

The safer fix is to reduce redundant discovery work, narrow traits, use provider events appropriately and bound branch retention. A broader provider token hides the symptom while increasing blast radius.

5. Mutable trusted library widening trust

If a privileged Shared Library is loaded from a mutable main ref, its behavior can change across builds even when the branch Jenkinsfile/source SHA is unchanged. For incident evidence record both the application SHA and library revision. Pin or review privileged library versions according to your release model.

6. Stale branch jobs that never disappear

First determine whether the branch still exists according to the SCM source. Then inspect the latest index and orphaned-item strategy. A child retained by policy is not “stuck.” If deletion is expected but absent, examine branch-source/plugin logs before manually deleting the job.

7. Causal separation table

8. Performance and scale

Multibranch scale is multiplicative. Hundreds of repositories, heads and matrix cells can create controller indexing load and agent queue pressure. Track index duration, discovered-head count, child-job count, scan frequency, queue time and storage growth. Do not “solve” scale by unbounded executors or agents; Chapter 21’s capacity discipline still applies.

9. Actions that require explicit authorization

Treat provider credential changes, webhook creation/deletion, trust-trait changes, Script Console use, plugin upgrades, child-job deletion and organization discovery as security-sensitive/admin operations. Use only disposable/synthetic resources in this course. Never disable CSRF/TLS/authorization/Script Security or SSH host-key checks to make discovery work.

Knowledge check

Answer before revealing the explanation.

Official references and version notes

• Jenkins LTS changelog — chapter baseline Jenkins 2.568.3 LTS, released 2026-09-02 and tested with Java 21 and 25; labs use Java 21.
• Jenkins: Branches and Pull Requests — Multibranch discovery, branch child jobs, indexing and change-request environment variables.
• Pipeline: Multibranch — version 841.vec5b_9e1806ec, requiring Jenkins 2.504.3.
• Branch API — version 2.1280.v0d4e5b_b_460ef; documents Multibranch event/indexing log locations and routing.
• SCM API — version 728.vc30dcf7a_0df5.
• Git plugin — version 5.10.1.
• Git Client plugin — version 6.6.1; current releases include fixes for earlier command-injection issues on agents.
• GitHub Branch Source — provider-specific optional reference, version 1983.vfa_27ed961853, requiring Jenkins 2.541.1.
• Pipeline: Multibranch step/reference — documents provider pull-request trust strategies and trusted-file behavior.
• Credentials API — version 1511.v2e3cb_0008ef0; credentials remain a separate authorization boundary from SCM discovery.

Version note — 2026-09-17: executable local examples assume Jenkins 2.568.3 LTS, Java 21, Pipeline: Multibranch 841.vec5b_9e1806ec, Branch API 2.1280.v0d4e5b_b_460ef, SCM API 728.vc30dcf7a_0df5, Git 5.10.1, and Git Client 6.6.1. GitHub Branch Source 1983.vfa_27ed961853 is used only to explain a real provider fork/PR trust model; the mandatory local lab does not require an external SCM account or real SCM credential. Re-check provider plugin advisories and trust semantics before production adoption.