Logging, Metrics, Support ZIPs, Request Auditing, Performance Tuning, and Capacity Diagnostics: Guided Hands-On Workflow and Core Operations
The safest way to learn observability is to create a small amount of known traffic and predict the evidence before looking at dashboards. This lesson uses a disposable Nexus or faithful local fixtures to connect client timings, request/application logs, metrics, support-bundle review, cold/warm proxy behavior, and one synthetic slow request.
Learning objectives
- Generate controlled repository traffic without stressing public services.
- Correlate client latency with request, application, metrics, DB/blob, and upstream evidence.
- Create or simulate a Support ZIP and review/redact it before sharing.
- Explain cold versus warm proxy behavior without confusing client cache and Nexus cache.
- Produce a repeatable diagnostic packet from a single controlled request.
1. Lab boundary and assumptions
127.0.0.1 and synthetic repository/package
names. The mandatory path is a local fixture and does not send
traffic to Maven Central, npmjs, Docker Hub, PyPI, or any employer
registry.
Reference assumptions: Nexus 3.95.2-01, Java 21, Community-compatible single-node learning path, synthetic hosted/proxy evidence, and no production credentials. If your local instance differs, capture the exact difference rather than forcing the lab to imitate these values.
2. Predict the evidence before generating traffic
| Action | Prediction |
|---|---|
| Read a hosted artifact |
request.log entry; no upstream outbound request;
artifact digest unchanged.
|
| Cold proxy fetch | Client request + Nexus request + outbound upstream timing + new cache/blob state. |
| Warm proxy fetch | Client/request evidence remains, but upstream work should normally be reduced/absent depending on metadata freshness and request type. |
| Generate support ZIP | Support-bundle operation plus a file containing selected diagnostic categories; must be reviewed before sharing. |
3. A fixture that behaves like an observability pipeline
The following script is intentionally local. It generates synthetic request/application/database/blob records, calculates request percentiles, and correlates one slow request by request ID. No Nexus internals are edited.
from statistics import median
requests = [
{"id":"r1","path":"/repository/lab/raw/a.txt","ms":42,"status":200,"cache":"warm"},
{"id":"r2","path":"/repository/lab/raw/a.txt","ms":39,"status":200,"cache":"warm"},
{"id":"r3","path":"/repository/proxy/pkg","ms":1280,"status":200,"cache":"cold"},
{"id":"r4","path":"/repository/proxy/pkg","ms":73,"status":200,"cache":"warm"},
]
subsystem = {
"r3": {"db_ms":18,"blob_ms":32,"upstream_ms":1170,"cpu_pct":34,"heap_pct":51},
"r4": {"db_ms":17,"blob_ms":28,"upstream_ms":0,"cpu_pct":29,"heap_pct":50},
}
lat = sorted(r["ms"] for r in requests)
print("count", len(lat), "median_ms", median(lat), "max_ms", max(lat))
slow = max(requests, key=lambda r:r["ms"])
print("slow_request", slow)
print("correlated", subsystem[slow["id"]])
assert subsystem["r3"]["upstream_ms"] > subsystem["r3"]["db_ms"]
assert requests[2]["ms"] > requests[3]["ms"] * 10
Expected interpretation: the cold request is slow primarily because of synthetic upstream latency, not because CPU, heap, DB, or blob IO were saturated. Therefore “increase heap” would be an unsupported change.
4. Controlled live traffic (optional local Nexus)
Use a harmless synthetic asset that already exists in a disposable hosted repository. Time the same request repeatedly. Keep credentials out of the command line where possible; if authentication is required, use a temporary scoped identity and an isolated shell environment.
export NEXUS=http://127.0.0.1:8081
URL="$NEXUS/repository/learner-raw/observability/sample.txt"
# Capture status, bytes, and total transfer time without modifying repository state.
for i in 1 2 3; do
curl -fsS -o /tmp/nx-sample.txt \
-w 'run=%{num_connects} status=%{http_code} bytes=%{size_download} total=%{time_total}\n' \
"$URL"
done
A repeated hosted read mainly checks path stability. For a proxy-cache exercise, use a controlled/local upstream fixture rather than intentionally generating repeated requests against a public registry.
5. Request-log correlation
Current request.log records request timing and identity
fields. Find the exact request window and preserve a small excerpt.
Do not dump the entire log into a ticket.
# Read-only local example; adapt the path to your recorded $data-dir.
grep '/repository/learner-raw/observability/sample.txt' \
"$NEXUS_DATA/log/request.log" | tail -n 10
Interpret the HTTP status and total response time first. Then look for a matching application exception, an outbound request, task overlap, or DB/storage signal. Absence of an exception is also evidence: a slow successful request can be entirely explained by upstream or storage latency.
6. Cold versus warm proxy: four caches, not one
A package may be cached at the package-manager/client, reverse proxy/CDN, Nexus proxy repository, or underlying storage/network layers. “Warm” must name the layer. A useful cold/warm experiment isolates the client cache (temporary client home) and uses a disposable/local upstream fixture. Expected pattern:
- Cold Nexus proxy: inbound request + outbound upstream request + metadata/blob write + client response.
- Warm Nexus proxy: inbound request + local metadata/blob read; no equivalent full upstream body fetch unless freshness/metadata rules require remote validation.
- Warm client cache: Nexus may receive no request at all.
7. Capture metrics around the same window
# Current 3.81+ endpoints. Use a scoped read-only identity.
curl -fsS -u "$NEXUS_USER:$NEXUS_PASS" \
"$NEXUS/service/rest/metrics/prometheus" > before.prom
# run controlled request(s)
curl -fsS -u "$NEXUS_USER:$NEXUS_PASS" \
"$NEXUS/service/rest/metrics/prometheus" > after.prom
Compare only relevant series and state what they can prove. A scrape can show process/memory/request counters around the window; it cannot reconstruct an individual dependency download by itself.
8. Support ZIP: generate, inspect, then share
For a local disposable instance, the UI path is Settings → Support →
Support ZIP. Current support bundles can include system information,
thread dumps, metrics, configuration, security, logs, task logs,
audit logs, and JMX information. In a single-node self-hosted
instance, generated bundles are placed under
$data-dir/downloads; HA handles bundles per node/shared
storage.
Sonatype removes password-related sensitive information, but that is not a license to forward the archive blindly. Extract it to a protected temporary directory and review filenames/content for internal URLs, usernames, IP addresses, repository names, package coordinates, custom headers, certificates, security configuration, and business-sensitive metadata.
mkdir -p ./support-review
unzip -l supportzip.zip > support-review/file-list.txt
# Review the list first; extract only in an access-controlled disposable directory.
unzip -q supportzip.zip -d support-review/extracted
# Search for your synthetic lab namespace and known fake markers.
grep -R -n -E 'learner-example|repo\.example\.invalid|token-FAKE_DO_NOT_USE' \
support-review/extracted || true
9. Redaction is data classification, not search-and-delete
A safe evidence packet preserves diagnostic value while minimizing exposure. Rather than deleting arbitrary lines, classify each field:
| Field | Action |
|---|---|
| Passwords/tokens/Authorization | Never include. Remove entirely and rotate if exposure is suspected. |
| Internal IP/hostname/repository name | Keep only if needed for topology correlation; otherwise pseudonymize consistently. |
| Timestamps/status/duration | Preserve; they are essential for correlation. |
| Package coordinate | Use synthetic coordinate in labs; in production tickets disclose only as necessary. |
| Thread dump | Potentially sensitive; review class names, URLs, headers, paths, and environment-derived data. |
10. Correlate one slow request
- Client: record start/end, URL, status, downloaded bytes.
-
request.log: confirm Nexus saw the request and its server-side duration. -
outbound-request.log: if proxy, check remote request duration/status. - Metrics: inspect CPU/memory/request/error movement over the same window.
- DB: check slow queries, locks, connection pressure only if timing points there.
- Blob: inspect backend latency/capacity/soft-quota evidence.
- Tasks: identify IO-heavy cleanup/repository-size/maintenance overlap.
- Apply the smallest hypothesis-driven change, then repeat the exact request protocol.
11. Challenge: choose the next evidence, not the next knob
You observe request.log duration 4.2 s,
outbound-request.log remote duration 4.0 s, heap 48%,
no DB slow-query signal, and warm hosted reads at 40 ms. What should
you do next?
Answer: investigate upstream/network/cache freshness for that proxy request. Increasing JVM heap or PostgreSQL pool size is not supported by the current evidence.
12. Cleanup
Delete only the local extracted support-review directory and synthetic client artifacts you created. Do not delete Nexus logs, database rows, blob files, or production support evidence as “cleanup.” Preserve the small evidence packet if it is part of the chapter checkpoint.
Knowledge check
What is the first thing to predict in a controlled observability lab?
Which evidence should change for the action—request log, outbound request, cache/blob state, metrics, or audit/configuration state.
Why can a warm client cache invalidate a Nexus cache comparison?
The client may satisfy the request locally, so Nexus receives no equivalent request; the experiment is no longer comparing Nexus cold versus warm behavior.
Does Sonatype password redaction make a support ZIP safe to publish?
No. Bundles can still contain operationally sensitive URLs, usernames, IPs, repository/package metadata, configuration, thread dumps, and security context.
A proxy request is 4.2 s and its outbound request is 4.0 s. Heap and DB are normal. What is the best next hypothesis?
Upstream/network/cache freshness, because the remote request explains nearly all observed server-side latency.
Why preserve timestamps and durations during redaction?
They are the join keys that let client, Nexus, database, blob, and upstream evidence be correlated.
Summary and next step
The hands-on workflow tied a known request to request/outbound/application evidence, metrics, support-bundle review, cache state, DB/blob signals, and privacy-safe evidence handling.
Lesson 3 uses those observations to choose logging levels, memory budgets, cache behavior, database pools, scaling, retention, and capacity thresholds without tuning by folklore.
Official references and version notes
- Sonatype: Logging — current self-hosted log files, rotation concepts, Log Viewer behavior, request/outbound/audit evidence, and PostgreSQL logging boundary.
- Sonatype: Auditing — audit capability, JSON event records, daily rotation, and current 90-day maximum retention.
- Sonatype: Prometheus — current 3.81+ Prometheus endpoint and required metrics privilege.
- Sonatype: Service Metrics Data API — current component/request usage metrics endpoint, privilege requirement, and the fact that this API is not listed in instance Swagger.
- Sonatype: Status API — read/writable application state and explicit limits: these checks do not validate external DB or disk health.
- Sonatype: Support Features — Support ZIP creation, storage location, and HA per-node bundle behavior.
- Sonatype: Support API — support ZIP API payload categories including system information, thread dump, metrics, configuration, security, logs, audit logs, and JMX.
- Sonatype: Nexus Repository Memory Overview — heap, direct memory, host headroom, and memory-related JVM arguments.
- Sonatype: System Requirements — current workload profiles, H2 limits, PostgreSQL guidance, CPU/RAM/storage baselines, and low-latency DB requirements.
- Sonatype: PostgreSQL Installation/Configuration — slow-query logging guidance and connection-pool configuration.
- Sonatype: PostgreSQL Max Connections — sizing the server connection limit against Nexus node pool sizes and troubleshooting exhausted connections.
- Sonatype: Blob Stores — blob count/used-size status, soft quotas, and storage-health evidence.
- Sonatype: Storage Planning — locality/latency implications and object-store performance tradeoffs.
- Sonatype: Usage Metrics — current usage-center request/component measurements and edition differences.
- Sonatype: Self-Hosted Usage Guide — request-log fallback, per-node aggregation, and request-metric interpretation.
- Sonatype: Configuring the Runtime Environment — current runtime properties, including the 3.95 audit-log attribute-change control.
Keep the academy open
Support free, practical DevOps education.
Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0
Send only Ethereum/ERC-20 compatible assets to this
address.