Chapter 34Lesson 05~230 minutes

Checkpoint Lab — Notifications, Checks, Commit Status, Pull Request Feedback, Chat Integrations, and Release Communication

Build a complete provider-free feedback lifecycle: publish failure, verify duplicate-safe retry, publish recovery tied to a later successful build, and prove every message points to the correct source/build identity.

checkpointfailurerecoverydedupevidence packetcleanup

Learning objectives

  • Publish success/failure/recovery events to a local mock endpoint.
  • Prove source SHA, job/build and destination identity for every event.
  • Demonstrate that a repeated delivery does not create a new logical event.
  • Retain request/response and sink-side evidence without secrets.
  • Explain exactly what the lab does and does not prove about real SCM/chat/email integrations.

1. Preflight and predictions

Before changing anything, record two predictions:

  1. A controlled failing Jenkins build will retain its original build result and also create exactly one logical failure feedback event tied to that build/source SHA.
  2. A later successful build will create a separate recovery event only when the checkpoint logic explicitly identifies a prior failure; retrying either event will keep the same event/delivery identity.

Use only a synthetic repository, disposable job/agent and loopback sink. No SCM token, SMTP account, Slack workspace or production channel is required.

set -euo pipefail
printf 'source=%s\n' "$(git rev-parse HEAD)"
printf 'java=%s\n' "$(java -version 2>&1 | head -n 1)"
printf 'python=%s\n' "$(python3 --version 2>&1)"
printf 'job=%s build=%s node=%s\n' \
  "${JOB_NAME:-local}" "${BUILD_NUMBER:-0}" "${NODE_NAME:-local}"
test "${NODE_NAME:-local}" != "built-in" || {
  echo 'Use a disposable agent, not the built-in node' >&2; exit 70;
}

2. Checkpoint files

Reuse the Lesson 2 sink and publisher, and add a small state-transition helper. The helper receives the previous completed build result from Jenkins as explicit input; it does not query an ambiguous “latest” endpoint.

# ci/select_feedback_state.py
import sys
current = sys.argv[1].upper()
previous = sys.argv[2].upper()
if current == "FAILURE":
    print("failure")
elif current == "SUCCESS" and previous == "FAILURE":
    print("recovery")
elif current == "SUCCESS":
    print("success")
else:
    raise SystemExit(f"unsupported current={current} previous={previous}")

3. Checkpoint Jenkinsfile

This Pipeline uses a parameter to make the prior-state input visible and reproducible in the lab. A production implementation can obtain prior completed state from a reviewed Jenkins API/library pattern, but it must guard the exact job/build identity and concurrency semantics.

pipeline {
  agent { label 'feedback-lab' }
  parameters {
    choice(name: 'SCENARIO', choices: ['success', 'failure'], description: 'Controlled current outcome')
    choice(name: 'PREVIOUS_RESULT', choices: ['SUCCESS', 'FAILURE'], description: 'Explicit prior completed result for lab')
  }
  options {
    timestamps()
    disableConcurrentBuilds()
    buildDiscarder(logRotator(numToKeepStr: '30'))
  }
  stages {
    stage('Preflight') {
      steps {
        checkout scm
        sh '''
          set -euo pipefail
          mkdir -p feedback-evidence
          git rev-parse HEAD | tee feedback-evidence/source-sha.txt
          printf 'job=%s\nbuild=%s\nnode=%s\nscenario=%s\nprevious=%s\n' \
            "$JOB_NAME" "$BUILD_NUMBER" "$NODE_NAME" "$SCENARIO" "$PREVIOUS_RESULT" \
            | tee feedback-evidence/build.txt
        '''
      }
    }
    stage('Synthetic delivery decision') {
      steps {
        script {
          if (params.SCENARIO == 'failure') {
            error('Controlled checkpoint failure')
          }
          sh 'printf "delivery=verified\\n" > feedback-evidence/delivery-state.txt'
        }
      }
    }
  }
  post {
    success {
      sh '''
        set -euo pipefail
        state="$(python3 ci/select_feedback_state.py SUCCESS "$PREVIOUS_RESULT")"
        python3 ci/build_feedback_event.py "$state"
        bash ci/publish_feedback.sh http://127.0.0.1:18080/events \
          feedback-evidence/event.json feedback-evidence/response.json
      '''
    }
    failure {
      sh '''
        set -euo pipefail
        python3 ci/build_feedback_event.py failure
        bash ci/publish_feedback.sh http://127.0.0.1:18080/events \
          feedback-evidence/event.json feedback-evidence/response.json
      '''
    }
    always {
      archiveArtifacts artifacts: 'feedback-evidence/**', allowEmptyArchive: true, fingerprint: true
    }
  }
}

In a real implementation, do not interpolate untrusted parameters into shell commands as shown generically in unsafe examples. Here the choice parameters are constrained to fixed values; production code should still prefer structured library logic and validation.

4. Required run sequence

Run SCENARIO PREVIOUS_RESULT Expected current build Expected feedback
A success SUCCESS SUCCESS success event for A.
B failure SUCCESS FAILURE failure event for B.
B retry test No rebuild No change Original B remains FAILURE Re-send B event file; same delivery ID, duplicate=true.
C success FAILURE SUCCESS recovery event for C, referencing C source/build identity.
Optional D success SUCCESS SUCCESS Normal success, not another recovery.

5. Duplicate-safe retry of the failed-build message

After Run B completes, download or access its archived event.json on the disposable lab agent/workspace only if still available, or reproduce the exact event from retained build metadata. Do not create a new build to test notification retry.

set -euo pipefail
# Run from the exact disposable lab workspace/evidence copy for Build B.
bash ci/publish_feedback.sh \
  http://127.0.0.1:18080/events \
  feedback-evidence/event.json \
  feedback-evidence/retry-response.json
python3 - <<'PY'
import json
x=json.load(open('feedback-evidence/retry-response.json'))
assert x['accepted'] is True
assert x['duplicate'] is True
print('same logical event delivery_id=', x['delivery_id'])
PY

6. Required evidence packet

Evidence Capture for each relevant run
Controller/runtime Jenkins LTS/core, Java, advisory-check date, installed notification plugin versions if any.
Job/build Full name, number, URL, cause, current result, explicit previous-result input.
Source Full SHA and PR/change ID if applicable.
Agent Node, label, executor/workspace, Python version.
Feedback identity Event ID, check/status name, state, destination route.
Payload safety Retained canonical JSON; confirm no token/password/auth header/raw environment dump.
Delivery HTTP code, application accepted, delivery ID, duplicate flag.
Sink state Stored event file named by delivery ID; one logical file after duplicate retry.
Transition Why C is recovery rather than ordinary success; identify B as prior controlled failure.
Limitations Local sink does not prove real GitHub/GitLab/SMTP/chat permission, rate or UI behavior.

7. Verification checklist

  • All builds execute on feedback-lab, not the built-in node.
  • Every event contains the exact source SHA of its own Jenkins build.
  • Run B remains a Jenkins failure even though its feedback publication succeeds.
  • Retrying Run B’s event produces the same delivery ID and duplicate=true.
  • Run C is a new Jenkins build with a new event ID and recovery state.
  • No secret or provider credential exists in the mandatory path.
  • Payload summaries contain links/identifiers rather than raw logs.
  • Sink request/response evidence is archived before cleanup.
  • Only exact lab paths/processes are removed.

8. Claims the checkpoint can and cannot support

Supported claim Not supported
Jenkins can publish secret-free result events tied to exact build/source identity. A real provider will accept the same payload/schema.
A stable event ID enables duplicate-safe retry against the lab sink. Every chat/SCM/email provider guarantees idempotency the same way.
Failure and recovery can be modeled as separate meaningful transitions. A human saw or acted on the notification.
The original Jenkins result remains distinct from feedback delivery success. A required SCM branch rule is configured correctly.
The evidence packet supports diagnosis of local transport/application state. External provider availability, permission and retention policies were tested.

9. Cleanup / rollback

set -euo pipefail
if [ -f feedback-evidence/sink.pid ]; then
  pid="$(cat feedback-evidence/sink.pid)"
  case "$pid" in (*[!0-9]*|'') echo 'invalid pid' >&2; exit 70;; esac
  kill "$pid" 2>/dev/null || true
fi
case "$PWD" in /|/home|/var|/tmp) echo 'unsafe cleanup location' >&2; exit 71;; esac
rm -rf -- feedback-sink feedback-evidence
# Keep committed lab source and Jenkins archived evidence until review is complete.

Delete the Jenkins lab job/agent only by exact name after evidence review. If you ran optional provider integrations, revoke/delete only the exact lab credential/app/webhook/channel resources you created.

10. What Chapter 34 adds to a production Jenkins model

A secure Jenkins platform now has an explicit feedback contract: stable source/build identity, stable check/status names, narrow publisher credentials, secret-free canonical events, bounded retries, deduplication and retained delivery evidence. These feedback side effects no longer blur with the build, artifact, deployment or human-decision states they describe.

Chapter 35 moves inward from external communication to controller resilience: queue recovery, agent loss, Pipeline durability, maintenance windows and failure modes. The same discipline applies—preserve exact state and first-failure evidence before retrying or restarting.

Next chapter

Chapter 35 — Controller Resilience, Queue Recovery, Agent Loss, Pipeline Durability, Maintenance Windows, and Failure Modes

Carry the verified evidence and operating discipline from this chapter into the next chapter.

Knowledge check

Answer before revealing the explanation.

1. Why is Run B not “fixed” just because its failure notification delivered successfully?

2. What proves the retry did not create a second logical event?

3. Why is Run C a recovery rather than generic success?

4. What must change when moving from the local sink to GitHub Checks?

5. What is the bridge to Chapter 35?

Official references and version notes

Notification plugins and provider APIs change independently. Verify current primary documentation before applying these patterns to a real organization.

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0Send only Ethereum/ERC-20 compatible assets to this address.