Secrets Management, External Secret Providers, Protected Data, Rotation, and Least-Privilege Patterns: Guided Hands-On Workflow and Core Operations

1. Scenario: two storage models, one safe evidence rule

Use a throwaway GitLab project or run the shell-only portions locally. The chapter compares:

• a synthetic project variable, optionally marked masked/protected, to show GitLab’s built-in variable controls;
• a local provider simulation that generates v1/v2 material outside the repository, applies a simple principal/version authorization policy, and returns a temporary file only when authorized.

The simulation is deliberately not presented as cryptographic security. Its purpose is to make the authorization/retrieval/rotation states observable without requiring Premium features, a cloud subscription, or a real Vault server.

2. Preflight: prove the project and source context before adding synthetic data

LAB_BRANCH="ch07/secrets-lab"
EVIDENCE_DIR="ch07-evidence"
mkdir -p "$EVIDENCE_DIR"

git status --short --branch
git rev-parse HEAD | tee "$EVIDENCE_DIR/head-before.txt"
git remote -v | tee "$EVIDENCE_DIR/remotes-before.txt"

# In a CI job, print only an allowlist:
printf 'source=%s\n' "$CI_PIPELINE_SOURCE"
printf 'ref=%s\n' "$CI_COMMIT_REF_NAME"
printf 'sha=%s\n' "$CI_COMMIT_SHA"

If you create a project variable for comparison, name it LAB_FAKE_CREDENTIAL and use a generated dummy value unrelated to any real system. Mark it masked; optionally protect it on an isolated protected lab branch. Record only its metadata and presence, never its value.

3. Build the local provider simulation outside the repository

PROVIDER_DIR="$(mktemp -d)"
trap 'rm -rf "$PROVIDER_DIR"' EXIT

mkdir -p "$PROVIDER_DIR/versions"
umask 077
openssl rand -hex 32 > "$PROVIDER_DIR/versions/v1"
openssl rand -hex 32 > "$PROVIDER_DIR/versions/v2"
printf 'v1\n' > "$PROVIDER_DIR/active"

provider_fetch() {
  principal="$1"
  requested="$2"
  output="$3"
  active="$(cat "$PROVIDER_DIR/active")"

  if [ "$principal" != 'project/ch07:deploy' ]; then
    printf 'provider_result=denied principal=%s\n' "$principal" >&2
    return 77
  fi
  if [ "$requested" != "$active" ]; then
    printf 'provider_result=revoked requested=%s active=%s\n' "$requested" "$active" >&2
    return 78
  fi
  cp "$PROVIDER_DIR/versions/$requested" "$output"
  chmod 600 "$output"
  printf 'provider_result=allowed version=%s\n' "$requested"
}

The generated bytes never enter Git, project settings, artifacts, or logs. The function models two decisions: “is this principal allowed?” and “is this requested version active?” A real provider replaces those shell comparisons with cryptographic authentication and authorization policy.

4. Authorized retrieval: prove use without disclosure

SECRET_FILE="$PROVIDER_DIR/retrieved"
provider_fetch 'project/ch07:deploy' 'v1' "$SECRET_FILE"

test -f "$SECRET_FILE"
printf 'secret_file_present=yes\n'
printf 'secret_bytes=%s\n' "$(wc -c < "$SECRET_FILE")"
printf 'secret_sha256=%s\n' "$(sha256sum "$SECRET_FILE" | awk '{print $1}')"
rm -f "$SECRET_FILE"

The digest is safe here only because the underlying value is a randomly generated synthetic lab secret. For a low-entropy real secret such as a short password, publishing a digest can aid guessing. Evidence policy must consider secret entropy and threat model.

5. Denied retrieval: authorization failure is successful evidence

set +e
provider_fetch 'fork/untrusted' 'v1' "$PROVIDER_DIR/denied" 2> "$EVIDENCE_DIR/denied.txt"
rc=$?
set -e

test "$rc" -eq 77
test ! -e "$PROVIDER_DIR/denied"
printf 'denied_job_assertion=passed rc=%s\n' "$rc"

A denial is not a broken pipeline if denial is the expected security outcome. The job can assert the provider returned the expected denial and still complete successfully, while preserving the non-secret denial reason.

6. Compare a masked/protected synthetic variable without printing it

if test -n "${LAB_FAKE_CREDENTIAL:-}"; then
  printf 'project_variable_present=yes\n'
  printf 'project_variable_length=%s\n' "${#LAB_FAKE_CREDENTIAL}"
else
  printf 'project_variable_present=no\n'
fi

7. Rotate v1 → v2, then prove v1 is revoked

# Activate v2.
printf 'v2\n' > "$PROVIDER_DIR/active"

# Old reference must fail.
set +e
provider_fetch 'project/ch07:deploy' 'v1' "$PROVIDER_DIR/old" 2> "$EVIDENCE_DIR/revoked-v1.txt"
old_rc=$?
set -e
test "$old_rc" -eq 78
test ! -e "$PROVIDER_DIR/old"

# New active reference succeeds.
provider_fetch 'project/ch07:deploy' 'v2' "$PROVIDER_DIR/new"
printf 'rotation_new_sha256=%s\n' "$(sha256sum "$PROVIDER_DIR/new" | awk '{print $1}')"
rm -f "$PROVIDER_DIR/new"
printf 'rotation_status=v2-active-v1-revoked\n' 

The key proof is not the v2 digest. It is the pair of outcomes: v2 retrieval succeeds for the authorized principal and v1 retrieval is denied after rotation.

8. Map each simulation state to a real provider integration

9. Optional architecture exercise: Vault-style request with OIDC

Do not execute this unless you own a disposable Vault/GitLab integration. The point is to identify the control surfaces:

vault_example:
  id_tokens:
    VAULT_ID_TOKEN:
      aud: https://vault.example.invalid
  secrets:
    LAB_PASSWORD:
      vault: lab/app/password@ops
      token: $VAULT_ID_TOKEN
  script:
    - test -f "$LAB_PASSWORD"
    - printf 'vault_secret_materialized=yes\n' 

Provider configuration must bind the audience and meaningful GitLab claims. A successful GitLab job token minting event does not prove Vault authorization; the provider policy owns that decision.

10. Challenge: choose the correct repair layer

An authorized job has the correct secret reference but receives a provider denial. Which layer should you inspect first?

1. Confirm source/ref/SHA and that the intended job ran.
2. Confirm the token/audience configuration without logging the token.
3. Confirm provider trust-policy claims and requested secret path/version.
4. Only then investigate network/runtime issues.

Do not “fix” a 403 by widening the provider policy to every project or every ref. Authorization failures should be repaired by reconciling intended identity with the narrow policy.

11. Cleanup and verification

• The trap removes the temporary provider directory automatically; verify it no longer exists.
• Remove only LAB_FAKE_CREDENTIAL and any protected lab branch/rule you created for this disposable project.
• Preserve only non-secret evidence: pipeline/job IDs, source/ref/SHA, provider/principal/version labels, result codes, synthetic digests, and cleanup confirmation.
• Never archive the generated secret files themselves.

Knowledge check

Official references and version notes

• Pipeline security — current guidance that CI/CD variables are less secure than dedicated secret-management providers and that sensitive values should use stronger secret-management controls where possible.
• CI/CD variables — masking, hiding, protection, fork/MR behavior, file variables, and explicit warnings that masking is not a defense against malicious job code.
• External secrets in CI/CD — current supported provider integrations and tier/offering requirements.
• OIDC authentication using ID tokens — job-scoped ID tokens, audiences, claims, and third-party trust boundaries.
• HashiCorp Vault secrets — id_tokens, secrets:vault, file materialization, and provider-side authorization.
• GitLab Secrets Manager — current limited/beta availability, permissions, branch/environment scoping, Runner requirements, rotation reminders, and file-by-default behavior.
• Secret detection — defense-in-depth for accidentally committed credentials; detection is not a substitute for rotation after exposure.