Chapter 23Lesson 04210–290 min

SBOMs, Component Provenance, Vulnerability Governance, Repository Firewall Concepts, and Supply-Chain Risk: Diagnostics, Failure Modes, Security, and Performance

Diagnose supply-chain failures by preserving artifact identity and evidence first. Do not “solve” a policy or scanner problem by deleting blobs, bypassing Nexus, disabling TLS, rewriting provenance, or treating one security signal as proof of everything.

DiagnosticsFailure modesBypass riskStale intelligencePerformance

Learning objectives

  • Apply the course diagnostic sequence to SBOM, provenance, vulnerability, policy, and repository-routing failures.
  • Diagnose claims that an SBOM proves provenance or a signature proves vulnerability safety.
  • Handle newly vulnerable components without destroying historical/rollback evidence impulsively.
  • Detect direct-upstream bypass and stale client/cache evidence that makes repository policy appear inconsistent.
  • Separate scanner/intelligence latency from Nexus cache, database, blob, network, and client-cache performance.
Version/product baseline (27 August 2026). The dated Nexus reference line is 3.95.2-01 with Java 21. Repository Firewall/IQ examples are architecture or fixture exercises unless the learner already has an authorized license. Re-check live release notes, feature matrix, and integration documentation before applying version-specific UI/API steps.
Evidence boundary. A checksum/digest can prove byte identity relative to an expected value; an SBOM inventories declared/discovered components; provenance describes build origin/process; a signature authenticates according to its key/trust model; vulnerability intelligence is time-varying; policy is a governance decision. None of these alone proves that software is safe.
Safety boundary. Use only synthetic artifacts, loopback/private endpoints, fake namespaces, and local fixtures. Do not fetch suspicious packages for demonstrations, publish employer SBOMs, expose provenance secrets, delete production artifacts because a CVE appears, or bypass governed Nexus endpoints to make a build pass.

1. Evidence-first diagnostic sequence

  1. Preserve the exact failing artifact coordinate, digest, client request, timestamp, and error/policy result.
  2. Confirm Nexus version/edition/runtime and any IQ/Firewall/scanner version/license involved.
  3. Inspect client URL/auth and prove traffic uses the intended Nexus hosted/proxy/group endpoint.
  4. Inspect repository type, group order, routing rules, and authorization.
  5. Inspect component/assets/metadata and independently hash the exact bytes.
  6. Inspect SBOM/provenance/signature evidence and confirm every subject/reference maps to the same digest.
  7. Inspect current vulnerability-intelligence timestamp/source and policy/waiver version.
  8. Inspect proxy cache/upstream behavior, then database/blob/disk only when evidence points there.
  9. Inspect logs/tasks/metrics and external scanner latency separately.
  10. Apply the least destructive correction and repeat one controlled request with a fresh client cache.

2. Failure: “the SBOM proves this binary came from our source”

Symptom: a release review accepts an SBOM as proof that a binary was built from approved commit abc123.

Diagnosis: inspect what the SBOM actually says. An inventory can list the application and dependencies and may contain pedigree/provenance-related metadata, but that does not automatically establish an authenticated build path for these exact bytes.

Correction: verify the artifact digest and a provenance/attestation statement whose subject digest matches it and whose builder/source claims satisfy your trust policy. Keep the SBOM for composition.

3. Failure: “the signature is valid, so the package is safe”

Symptom: a signed component with a valid cryptographic signature is exempted from vulnerability policy.

Diagnosis: signature verification and vulnerability analysis answer different questions. Verify the signing identity/trust chain, then independently evaluate current security/license/malware intelligence.

Correction: require both controls where appropriate instead of converting one into an all-purpose trust signal.

4. Failure: a new CVE triggers immediate repository deletion

Symptom: an operator deletes the only stored release and tries to compact the blob store to “remove the vulnerability.”

Why this is dangerous: deployed systems may still contain the bytes; deletion does not patch them. It can remove rollback, reproduction, customer, audit, or forensic evidence. Blob reclamation can make recovery harder.

Correction: first identify affected artifact digests and deployments, prevent new consumption when policy requires, plan remediation/upgrade, preserve required evidence, and delete only according to approved retention/security procedure with recovery context.

5. Failure: stale vulnerability data produces contradictory decisions

Symptom: one CI job allows the component while a security dashboard quarantines it.

Diagnosis: compare evaluation timestamps, intelligence source/version, policy version, application/repository context, waivers, and client cache. The bytes can be identical while the risk decision differs legitimately.

Correction: define freshness SLAs, preserve evaluation timestamps, centralize policy ownership where appropriate, and avoid using vulnerability fields embedded in old SBOMs as the current source of truth.

6. Intentionally broken example: fallback bypasses Nexus policy

# BROKEN design concept — do not copy into production
primary-index = https://repo.example.invalid/repository/pypi-group/simple
fallback-index = https://pypi.org/simple

Symptom: the Nexus/Firewall route denies a dependency, but the build still succeeds.

Diagnosis: the package manager can retrieve the same namespace directly from the public upstream. Depending on client semantics, fallback ordering can also create dependency-confusion risk.

Correction: make the governed endpoint authoritative where policy requires it; control network egress/client configuration; define internal namespace ownership; do not disable policy to match the bypass result.

7. Failure: provenance digest mismatch is “fixed” by editing JSON

Symptom: provenance names digest A but the downloaded release hashes to B. An operator edits the provenance file to B.

Diagnosis: the mismatch is evidence of an identity or chain-of-custody problem: wrong download, mutable coordinate, compromised publication, bad evidence generation, or incorrect build selection.

Correction: preserve both files, stop promotion/deployment, trace the original build and publication, and regenerate trustworthy evidence only from the authoritative build process—not by rewriting history.

8. Performance: identify the slow layer before tuning

Symptom Likely layer Evidence
First proxy download slow, later fast Upstream/proxy cache Nexus request logs, remote latency, cache state.
Every download fast but policy gate slow External intelligence/scanner/policy Evaluation timings, IQ/scanner connectivity.
Nexus browse/search slow Database/JVM/task load Metrics, DB latency, heap/direct memory, task history.
Only one CI agent reports old package Client cache Fresh isolated client home versus existing cache.
SBOM generation dominates build Build tooling/dependency graph Build step timing, scanner scope; not blob-store tuning.

Do not increase Nexus heap because an external scanner is slow, and do not disable vulnerability analysis because a package client cache is stale.

9. Security-sensitive evidence handling

  • Redact credentials from scanner logs, provenance parameters, support ZIPs, and webhook/CI artifacts.
  • Do not include signing private keys in build workspaces or evidence bundles.
  • Review SBOM/provenance before sharing externally; internal names and URLs can be sensitive.
  • Use least-privilege service identities for Nexus, scanner, and evidence stores separately.
  • Preserve original evidence read-only during investigation; derive redacted copies for sharing.

10. Mini broken lab: prove the cause without changing Nexus

Reuse Lesson 2. Copy provenance.intoto.json to broken-provenance.json and change the subject.digest.sha256 to 64 zeros. Run the verification assertion. It must fail. Restore by deleting the broken copy—not by modifying the artifact or original provenance. Then compare the day-1/day-30 vulnerability decisions and explain why those differences are valid while the digest mismatch is not.

11. Knowledge check

A package is signed by a trusted vendor but newly has a critical CVE. Which control wins?

Why is deleting a vulnerable artifact from Nexus not equivalent to remediation?

What should you compare when two systems disagree on vulnerability status for identical bytes?

What does a provenance subject digest mismatch require?

Why can direct public-registry fallback undermine Repository Firewall?

12. Summary and next step

Evidence-first diagnosis preserves the very facts you need to investigate. You now know how to distinguish valid time-varying policy changes from invalid identity mismatches, and how to correct bypass, stale intelligence, and unsafe deletion without weakening Nexus. Lesson 5 assembles three complete component evidence packets and proves immutable versus mutable facts in one checkpoint.

Official references and version notes

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0 Send only Ethereum/ERC-20 compatible assets to this address.