Chapter 20Lesson 04210–290 min

REST API, Repository Provisioning, Security Automation, Pagination, and Infrastructure-as-Code Patterns: Diagnostics, Failure Modes, Security, and Performance

Diagnose automation as a distributed state problem: preserve the original HTTP exchange without leaking secrets, verify the target and schema, distinguish authentication from authorization, recognize incomplete pagination and duplicate creation, handle conflicts deliberately, and recover from wrong-target or drift mistakes with the least destructive correction.

DiagnosticsHTTP failuresDriftSecurityRecovery

Learning objectives

  • Diagnose hard-coded/leaked credentials without propagating them into evidence.
  • Explain duplicate creation, dropped pagination, schema drift, 400/409, and wrong-base-URL failures.
  • Separate Nexus authorization failures from network, client, database, and blob symptoms.
  • Apply a least-destructive diagnostic sequence before retries or deletion.
  • Define safe recovery when humans and automation share ownership accidentally.

Version baseline (26 August 2026). These lessons use Sonatype Nexus Repository 3.95.2-01 as the dated self-hosted reference point, verified from Sonatype's public release repository, with Java 21 as the current runtime requirement. The mandatory lab assumes a small, disposable, loopback-only self-hosted Community Edition installation using H2 and a file blob store. The local instance's own /service/rest/swagger.json is authoritative for the exact endpoint schema you are about to call; API fields, repository recipes, security endpoints, and edition entitlements can change between releases.

Automation safety boundary. Do not put real passwords, user-token passcodes, Authorization headers, production base URLs, or employer namespaces into source files or lesson evidence. The examples use environment variables, synthetic names, loopback URLs, and disposable resources. User Tokens are a Pro capability in current self-hosted Nexus Repository; Community labs therefore use a disposable local account/password only for bootstrap and immediately reduce privileges for the service identity.

Pagination documentation note. Current Sonatype pages agree on the continuationToken contract but currently disagree on the documented default item count for components/assets. The robust automation rule is independent of page size: process every returned items array, pass the returned token unchanged, and stop only when continuationToken is null.

1. Diagnostic sequence: preserve → identify → compare → correct → verify

  1. Preserve timestamp, method, redacted URL, status, response body, request-log evidence, and automation version.
  2. Confirm Nexus version/edition/runtime and exact base URL.
  3. Inspect local Swagger path/schema.
  4. Check authentication without printing credentials; then required privileges.
  5. GET actual repository/security state and compare owned fields.
  6. For content symptoms, inspect repository type/group/routing, authorization, components/assets, proxy/cache/upstream, database/blob/disk, logs/tasks/metrics.
  7. Apply the smallest correction and rerun a controlled request.

2. Failure mode: a password or token is hard-coded

Once a real credential is committed or printed into CI logs, removing the line is insufficient. Rotate/revoke according to policy, prevent future logging, inspect where it was used, and review retained artifacts. Demonstrate only with password-FAKE_DO_NOT_USE.

# Broken pattern — intentionally fake.
AUTH = ("svc-learner-api", "password-FAKE_DO_NOT_USE")
# Better: runtime injection; never print the value.
AUTH = (os.environ["NEXUS_USER"], os.environ["NEXUS_PASSWORD"])

3. Failure mode: rerun tries to create the same resource

First run succeeds; second receives conflict/create failure. Root cause: action scripting rather than reconciliation. GET deterministic name, create if absent, normalize/compare if present, PUT only drift, then fresh GET verification. Do not append timestamps to resource names—that converts a bug into sprawl.

4. Failure mode: first-page success hides missing artifacts

HTTP can succeed while the algorithm fails:

# Broken
items = requests.get(url, params={"repository": repo}, auth=auth).json()["items"]
# Correct control condition
while True:
    body = requests.get(url, params=params, auth=auth).json()
    items.extend(body["items"])
    token = body.get("continuationToken")
    if token is None:
        break
    params["continuationToken"] = token

Compare complete results with a known fixture inventory or another trustworthy observation where practical.

5. Failure mode: field or endpoint changed after upgrade

400 validation errors, unknown/missing fields, or 404 require comparison of server version, automation/client version, current Swagger, redacted payload, and release notes. Do not delete fields until the request passes; understand the compatibility change.

6. Failure mode: 400 or 409 is ignored

Bad behavior Why unsafe Correct behavior
Continue after 400 Later steps assume configuration never applied Fail fast and inspect schema/response
Retry 409 forever Conflict is usually logical state GET actual state and decide ownership
Accept any 2xx without verification Functional behavior may still be wrong Fresh GET + data-plane/authorization test
Turn 403 into nx-admin grant Expands blast radius Identify exact privilege

7. Intentionally broken example: DELETE points at the wrong base URL

Do not send a destructive request to demonstrate this failure. Prove that a guard blocks execution before DELETE.

from urllib.parse import urlparse
import os
base = os.environ["NEXUS_URL"]
expected = "127.0.0.1"
host = urlparse(base).hostname
if host != expected:
    raise SystemExit(f"REFUSE destructive action: expected {expected}, got {host}")
name = "learner-api-hosted"
if not name.startswith("learner-"):
    raise SystemExit("REFUSE destructive action: unmanaged resource name")
print("preflight passed; destructive call would still require explicit --destroy")

A machine-verifiable allowlist, naming marker, dry-run plan, and separate destruction flag are stronger than a generic “Are you sure?” prompt.

8. Failure mode: automation and humans fight over one field

Stop automatic reconciliation for the disputed field, preserve current state and impact, agree on an owner, update desired state or restore the reviewed configuration once, then resume. This is an ownership failure, not a Nexus defect.

9. Authentication and authorization failures

401 means Nexus did not accept the presented identity; 403 means it accepted the identity but denied the action. Test a harmless endpoint the identity should be allowed to call, then inspect roles/privileges. Do not expose Authorization headers through verbose curl output in shared logs.

10. Network/TLS failure versus API failure

Connection timeout, DNS failure, TLS validation, proxy misconfiguration, or reverse-proxy context-path mismatch can occur before Nexus returns application-level HTTP. Reuse Chapter 17's network model and never globally disable TLS verification to make automation pass.

11. Performance: separate client fan-out from Nexus load

Measure automation request rate/concurrency, Nexus request latency/errors, database latency/pool, blob IO when content is involved, task overlap, retries/backoff, and upstream latency for proxy tests. Pagination is a correctness mechanism, not permission to launch unbounded parallel calls.

12. Evidence packet

  • Nexus version/edition/runtime and Swagger checksum.
  • Automation version and desired-state checksum.
  • Redacted method/path/status/response.
  • Resource ownership marker.
  • Before/after normalized GET state.
  • Complete pagination count and final null token.
  • Positive/negative authorization tests.
  • Relevant logs with credentials redacted.
  • Correction and final verification.

13. Knowledge check

A 403 becomes 200 after assigning nx-admin. Is it solved?

The first page returns the count your test expected but the token is non-null. What wins?

Why simulate wrong-target DELETE instead of sending it?

What should happen after 409 on create?

Why can API 200 coexist with a broken package build?

14. Summary and next step

You can diagnose automation at the correct layer: target, schema, identity, privilege, state diff, pagination, repository behavior, database/blob/log evidence, and client function. Lesson 5 integrates these skills into a complete disposable topology.

Official references and version notes

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0 Send only Ethereum/ERC-20 compatible assets to this address.