Source Control Integration, Git Plugin, Credentials, Polling, Webhooks, and Multirepository Checkout: Diagnostics, Failure Modes, Security, and Performance

1. Diagnose source-control failures by layer

SCM incidents often become expensive because engineers retry before identifying the failing layer. Preserve the original build number, queue/build cause, polling log, console log, controller/plugin baseline, agent/tool versions, workspace path, remote URL, and any resolved SHA before cleanup. Then move through the source-control chain in order.

1. Trigger/cause: why did Jenkins evaluate this job?
2. Job SCM configuration: which URL, branch specifier/refspec, credential ID, and extensions were configured?
3. Network/trust: can the relevant controller/agent reach and authenticate the Git server?
4. Fetch/resolution: which refs did the Git client receive?
5. Checkout: which immutable SHA is actually at HEAD?
6. Workspace: was an old or second checkout allowed to overlap?
7. Downstream build: did scripts use the source Jenkins actually checked out?
8. Changelog/artifact/external state: what evidence was retained?

2. Failure mode: Jenkins built a different commit than expected

A user says “I pushed commit X to main, but Jenkins built something else.” Do not start by rerunning. Record the push SHA, build cause, build number, configured repository URL, branch specifier, and workspace HEAD.

printf 'remote=%s\n' "$(git remote get-url origin)"
printf 'head=%s\n' "$(git rev-parse HEAD)"
printf 'branch=%s\n' "$(git symbolic-ref --short -q HEAD || printf detached)"
git log -1 --decorate --oneline
git reflog -n 5 || true

Possible causes include a second push after the first event, an unexpected refspec, a merge/ref produced by a provider integration, a stale workspace combined with custom checkout logic, or a second repository being mistaken for the primary checkout. The immutable workspace SHA resolves the debate; the task is then to explain how Jenkins selected it.

3. Failure mode: broad credential attached to untrusted source

Imagine a test job that only needs read access to repository A but is configured with a credential that also writes to repositories B through Z. Even if the checkout succeeds, the security design has failed because job configuration or executable source may be able to misuse the broader identity.

Repair by creating or selecting a narrower credential, constraining it to the intended folder/trust domain, and testing representative checkout operations. Preserve the old credential ID in the incident/change record—never its secret value—so reviewers can understand the exposure that existed.

4. Failure mode: webhook notification plus polling causes apparent duplicates

A repository may have a frequent Poll SCM schedule and also send notifications. Depending on timing and plugin/provider behavior, operators can observe closely spaced polling activity or builds and assume Jenkins is randomly duplicating work.

Preserve provider delivery IDs/timestamps where available, Jenkins polling logs, queue/build causes, and the resolved SHA for each build. Then decide which mechanism is authoritative. Usually a secure notification path plus polling-on-notification is sufficient; a frequent independent schedule may be unnecessary. Do not delete builds before correlation.

5. Intentionally broken example: SSH host-key verification failure

A disposable job uses git@scm-lab.example:team/app.git and fails with a host-key verification error after the Git server was rebuilt. The unsafe reaction is to switch Git Host Key Verification to “No verification.” The correct diagnostic path is to preserve the failure, inspect the hostname and presented key through trusted administration channels, compare with the expected server key, and then update the chosen verification source if the rotation is legitimate.

Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

This error occurs before repository authorization can be proven. A valid private key does not fix a server-identity mismatch. Conversely, accepting a new host key blindly because “the build used to work” defeats the protection entirely.

6. Failure mode: two repositories collide in one workspace

A Pipeline checks out app source into the workspace root, then checks out an infrastructure repository into the same root without cleaning or using dir(). Files disappear or are silently replaced; the final git rev-parse HEAD now represents only the last checkout. The build artifact may contain a mixture of untracked leftovers from both repositories.

Repair by using bounded directories, cleaning only the exact lab paths you own, and capturing each repository's URL/SHA immediately after checkout. If shared generated files are needed, copy them into a separate build directory instead of making repository roots overlap.

7. Performance: clone cost, polling cost, and controller load

Git performance problems can come from different layers: network latency, repository size, large histories, submodules, LFS, many refs, slow credentials/provider APIs, too-frequent polling, or too many parallel checkouts. Measure before adding shallow clone, sparse checkout, reference repositories, caches, or custom refspecs.

8. Evidence-first command set

# Run only inside the affected disposable workspace after preserving logs.
printf 'pwd=%s\n' "$PWD"
git --version
git remote -v
git status --short --branch
git rev-parse HEAD
git show -s --format='commit=%H%nparents=%P%ncommit_time=%cI%nsubject=%s' HEAD
git config --get remote.origin.url || true

# List fetched refs without printing credential helpers or all config.
git show-ref | sed -n '1,40p'

These commands deliberately avoid git config --list and environment dumps because both can expose sensitive configuration. Add targeted queries only when they answer a specific hypothesis.

9. Least-destructive repair order

1. Preserve first-failure evidence.
2. Confirm exact repository URL and intended immutable revision.
3. Confirm plugin/Git/agent baseline and network reachability.
4. Validate credential ID and protocol without exposing the secret.
5. Validate host identity for SSH/TLS.
6. Inspect refspec/branch selection and workspace state.
7. Correct one configuration cause.
8. Re-run the smallest safe scope and compare source evidence.
9. Only then clean old workspaces or retire duplicate trigger paths.

10. Security review checklist

• No real token/private key appears in logs, config exports, screenshots, or artifacts.
• Untrusted source does not gain release/deployment credentials.
• SSH host-key verification remains enabled.
• Webhook/notification access control remains enabled and tokens are rotated if exposed.
• Repository URLs are expected and do not redirect to untrusted hosts.
• Additional repositories have explicit directories and independent identities.
• Builds run on appropriate agents rather than the controller.

Knowledge check

Official references and version notes

• Jenkins LTS changelog — current LTS release and tested Java configurations.
• Git plugin — repositories, credentials, polling, push notifications, checkout behavior, environment variables, and plugin security notes.
• Git Client plugin — command-line/JGit implementations and SSH host-key verification strategies.
• scmGit Pipeline reference — explicit Git checkout configuration and supported checkout capabilities.
• Credentials security — limiting credential access and protecting secrets.
• Using credentials — credential kinds, stores, scope, and Jenkins usage.
• Controller Isolation — why routine checkout/build execution belongs on agents.
• Using a Jenkinsfile — Pipeline checkout and credential-handling patterns.