Cleanup Policies, Retention, Component Age, Last-Downloaded Rules, Preview, and Storage Reclamation: Diagnostics, Failure Modes, Security, and Performance

1. Preserve evidence before “fixing” cleanup

Retention failures are unusually easy to make worse because the intuitive response to storage pressure is more deletion. If a release vanished, the first response should not be compaction. If disk space did not fall, the first response should not be rm. Preserve the policy definition, repository scope, preview output, task history, version/edition/database/blob-store information, logs, and client request evidence before changing anything.

2. Diagnostic sequence

flowchart TB
E[Preserve policy + preview + task evidence] --> V[Confirm Nexus version / edition / DB / blob store]
V --> R[Confirm repository format / type / policy attachment]
R --> C[Re-evaluate criteria and candidate semantics]
C --> U[Inspect download / age evidence and known issues]
U --> T[Inspect cleanup and unused-blob task status]
T --> S[Inspect repository visibility / search / client result]
S --> B[Inspect blob capacity and compaction status]
B --> L[Inspect logs / task logs / IO metrics]
L --> X[Apply smallest supported correction]
X --> Q[Verify with controlled requests]

The sequence intentionally defers mutation. The goal is to locate the layer that disagrees with expectation. A 404 after cleanup, a successful build from a client cache, and unchanged filesystem free space can all be simultaneously correct.

3. Symptom-to-layer matrix

4. Broken example: an asset matcher escaped its intended subtree

An operator intended to match learner-example/ci-expire/ but configured:

learner-example/.*expire.*

This can match paths whose names contain “expire” outside the intended directory. The safer H2/PostgreSQL rule includes the leading asset-path slash and anchors the path boundary:

^/learner-example/ci-expire/.*$

Do not repair this by deleting the policy after execution and hoping the content returns. If preview already shows unexpected candidates, stop before cleanup. If cleanup already happened, disable further cleanup/compaction, preserve the task evidence, identify exactly what disappeared, and move into a supported recovery path using soft-delete capability or validated backups as appropriate.

5. Failure mode: a needed release was deleted

Possible causes include a policy attached to the wrong repository, missing release-type protection, an over-broad asset matcher, a short age window, or a false assumption that “last downloaded” reflected business criticality. The recovery priority is to prevent hard deletion from reducing options.

1. Disable or detach the offending cleanup policy from the affected disposable/production repository according to change control.
2. Do not run compact-blob-store on the affected store.
3. Preserve task logs, policy config, preview evidence, and component coordinates/paths.
4. Determine whether the blob remains soft-deleted and whether a supported repair/recovery procedure applies to the exact version.
5. If recovery requires backup, use a validated restore procedure rather than copying random blob/database files.
6. After recovery, rebuild the policy from lifecycle intent and preview again.

Chapter 25 will cover full backup/restore. This lesson's responsibility is to keep the incident recoverable and avoid destructive improvisation.

6. Failure mode: download timestamps are not what the operator assumed

A component can look inactive because consumers use a local cache, because requests hit a different repository endpoint, because a group/proxy path changes which member serves the content, or because the format/version does not update the timestamp for the request method you expected. Diagnose from controlled client requests against a fresh cache and from version-specific documentation.

7. Failure mode: preview and execution disagree

Preview is a point-in-time evaluation. Between preview and task execution, a component can be downloaded, a new version can be uploaded, a policy can be edited, or repository scope can change. Large-repository previews can also have performance limits. A safe change process records a short preview-to-execution window, freezes policy changes during the change, and re-previews when the candidate set is safety-critical.

If the actual deletion count differs slightly from preview, do not immediately label it corruption. Compare timestamps and repository events. If the difference includes protected releases or a large unexplained expansion, halt compaction and investigate.

8. Failure mode: “cleanup worked, but disk is still full”

This is often a state-model error. Cleanup policies soft delete content. Unused-asset cleanup handles orphaned assets. Admin - Compact blob store performs physical deletion of eligible soft-deleted blobs. Object-storage backends can also have provider-specific lifecycle behavior. Therefore logical content disappearance and storage reclamation occur at different times.

Check free-space trend, blob-store status, compact-task configuration, Blobs Older Than grace period, last task execution, and task result. If the grace period is seven days, immediate reclamation is not expected. If compaction is failing, inspect its task log and storage backend health rather than bypassing Nexus.

9. Failure mode: compaction ran at the wrong time

Compaction can create IO load and permanently reduce recovery options. Running it during peak publishing, backup, migration, repository move, or other heavy maintenance can increase contention. Some Nexus tasks explicitly conflict with blob-store operations. A production runbook should list incompatible/high-cost jobs and an approved maintenance window.

Measure before tuning: task duration, blob-store latency, database latency, request error rate, CPU, heap/direct memory, and concurrent task activity. A slow compact task is not evidence that the JVM needs more heap.

10. Catastrophic shortcut: direct filesystem or database deletion

Deleting blob files with shell commands or deleting database rows with SQL destroys the invariant that Nexus maintains between component metadata, asset metadata, blob references, and binary content. The immediate symptom may be missing downloads, 500 errors, reconciliation reports, or search inconsistencies; the long-term symptom can appear during backup, migration, or recovery.

# DO NOT RUN — examples of unsupported cleanup shortcuts
# rm -rf "$NEXUS_DATA/blobs/default/content/vol-*"
# find "$NEXUS_DATA/blobs" -type f -mtime +90 -delete
# psql ... -c 'DELETE FROM asset WHERE ...'

The correct response to storage pressure is supported cleanup/compaction, additional capacity, or a documented migration/storage operation—not deleting internals behind Nexus's back.

11. Security: separate who defines policy, who attaches it, and who runs tasks

Current Sonatype cleanup documentation distinguishes administrative privilege to create policies, repository-admin edit privilege to attach them, and task-update/task-all privilege to modify tasks. That separation is useful: a repository owner may be allowed to apply an approved policy without being able to create arbitrary global cleanup logic or run unrelated administrative tasks.

Audit sensitive changes. Never put administrator passwords in cleanup scripts. The cleanup-policy REST management API is currently Pro-only, so Community automation should not depend on undocumented endpoints or browser automation.

12. Performance diagnosis: identify the bottleneck

13. Incident drill

A policy named cleanup-all-old is attached to maven-releases and maven-snapshots. At 02:00 the cleanup service completes. At 02:10 a rollback deployment fails because com.example:orders:4.8.2 is missing. The compact task is scheduled for 03:00.

Your first actions are: stop/detach the policy, disable the pending compaction task for the affected blob store, preserve task and policy evidence, verify the missing coordinate from a fresh client, identify whether the blob is still soft-deleted, and invoke the supported recovery path. You do not republish a newly rebuilt 4.8.2 under the same release identity because that would destroy artifact identity evidence.

14. Verification checklist after correction

• Exact protected coordinate/path is available again or formally declared unrecoverable with evidence.
• Cleanup policy scope and criteria match the documented retention SLA.
• Preview contains only intended candidates.
• Compaction is disabled until recovery/change acceptance is complete.
• Representative clients use a fresh cache and controlled Nexus endpoint.
• No direct DB/blob edits were performed.
• Task schedules avoid known high-contention windows.
• Current release notes were checked for format-specific cleanup issues.

15. Knowledge check

16. Summary and next step

Cleanup diagnosis is a state-localization exercise: criteria, scope, timestamps, task execution, soft deletion, blob reclamation, client cache, and recovery each have different evidence. Lesson 5 combines the chapter into a checkpoint: design a synthetic retention SLA, predict candidates, execute safe cleanup, prove logical versus physical state, and hand off a runbook to Chapter 19's broader scheduled-task operations.

Official references and version notes

• Sonatype: Cleanup Policies — current criteria, preview, system cleanup tasks, soft deletion, compact-blob-store reclamation, and format matrix.
• Sonatype: Nexus Repository 3.95.x Release Notes — 3.95 cleanup enhancements and current known issues.
• Sonatype: Tasks — current task types including Admin - Compact blob store.
• Sonatype: Keeping Disk Usage Low — supported cleanup/reclamation guidance and the separation between deletion and freed disk space.
• Sonatype: Data Repair Tasks — current supported consistency/recovery concepts; use only with exact-version procedure and evidence.