Job DSL, Seed Jobs, Folder Hierarchies, Programmatic Job Creation, and Jenkins API Automation: Guided Hands-On Workflow and Core Operations

1. Disposable scenario and guardrails

The lab uses a local Jenkins 2.568.3-lts-jdk21 controller, a trusted disposable agent labeled trusted-seed, and a synthetic Git repository named ch27-jobdsl. The controller has Job DSL, Script Security and Folders installed. Matrix Authorization and Authorize Project are recommended for the full permission exercise.

Preflight evidence:

mkdir -p ch27-jobdsl/{jobs,evidence}
cd ch27-jobdsl
git init
git config user.name 'CH27 Lab'
git config user.email 'ch27@example.invalid'
printf '%s\n' "$JENKINS_URL" > evidence/controller-url.txt

2. Pin the controller/plugin baseline

For a reproducible disposable image, preinstall the reviewed plugin set rather than clicking “latest” during the exercise:

FROM jenkins/jenkins:2.568.3-lts-jdk21
RUN jenkins-plugin-cli --plugins \
  job-dsl:3732.v9a_c49a_61a_313 \
  script-security:1422.v06869826dd9b_ \
  cloudbees-folder:6.1106.v3a_d9a_6d2465e \
  matrix-auth:3.3 \
  authorize-project:534.v2f208c45e11c

Record the built image ID/digest. Plugin installation is controller supply-chain state; the DSL repository is item desired state. Keep them separate.

3. Create the smallest useful DSL source

Create jobs/platform.groovy:

folder('apps') {
  description('[managed-by=job-dsl][owner=platform-lab]')
}

['api-ci', 'worker-ci'].each { name ->
  pipelineJob("apps/${name}") {
    description("[managed-by=job-dsl][owner=platform-lab][schema=v1]")
    disabled(false)
    definition {
      cps {
        sandbox(true)
        script("""pipeline {
  agent { label 'linux-ci' }
  stages {
    stage('Identity') {
      steps { echo 'synthetic ${name} build' }
    }
  }
}""".stripIndent())
      }
    }
  }
}

The DSL creates Jenkins items. The embedded Pipeline is intentionally tiny so the lesson remains about generation, not Pipeline syntax. In a real repository, prefer SCM-backed Jenkinsfiles for application jobs.

git add jobs/platform.groovy
git commit -m 'ch27: desired jobs schema v1'
git rev-parse HEAD | tee evidence/dsl-v1.sha
sha256sum jobs/platform.groovy | tee evidence/dsl-v1.sha256

4. Create the seed job

Create a folder platform-lab and a Pipeline job platform-lab/seed. Configure SCM to check out this synthetic repository on the trusted seed agent. The Pipeline should record source identity before calling Job DSL:

pipeline {
  agent { label 'trusted-seed' }
  options { timestamps() }
  stages {
    stage('Source identity') {
      steps {
        sh 'git rev-parse HEAD | tee seed-source.sha'
        sh 'sha256sum jobs/*.groovy | tee dsl-files.sha256'
      }
    }
    stage('Generate') {
      steps {
        jobDsl targets: 'jobs/**/*.groovy',
          sandbox: true,
          lookupStrategy: 'SEED_JOB',
          removedJobAction: 'IGNORE',
          removedViewAction: 'IGNORE'
      }
    }
  }
  post {
    always {
      archiveArtifacts artifacts: 'seed-source.sha,dsl-files.sha256', fingerprint: true
    }
  }
}

With SEED_JOB, apps/api-ci resolves relative to the seed folder, producing platform-lab/apps/api-ci. The seed’s folder and permissions are still the actual authorization boundary.

5. Run the seed with an intentional identity

For the strongest sandbox exercise, configure Authorize Project so the seed runs as a dedicated jobdsl-seed-bot identity. Grant it only the folder/item permissions needed under platform-lab. Do not grant Overall/Administer or Script Console power. A simplified all-admin disposable lab is acceptable only as a documented simulation of this authorization boundary.

Before the first run, predict:

1. platform-lab/apps, api-ci and worker-ci will appear.
2. The seed build will retain the exact Git SHA and generated-object evidence.
3. No application build is implied merely because the item was created.

6. First generation and before/after evidence

Trigger platform-lab/seed once. Inspect the console for “Added items”/“Generated items” information, then query the folder:

curl -fsS --user "$JENKINS_USER:$JENKINS_API_TOKEN" \
  "$JENKINS_URL/job/platform-lab/api/json?tree=jobs[fullName,displayName,color,_class]" \
  | tee evidence/items-after-run1.json

curl -fsS --user "$JENKINS_USER:$JENKINS_API_TOKEN" \
  "$JENKINS_URL/job/platform-lab/job/seed/lastBuild/api/json?tree=number,url,result,actions[causes[*]]" \
  | tee evidence/seed-run1.json

Expected persisted state: folder plus two Pipeline jobs. Their later build histories remain empty until triggered separately.

7. Prove idempotency

Run the seed again without changing Git. Record build 2 and query the same item set. Compare normalized configuration or API-visible metadata. The new seed build record is expected; unexplained generated-item changes are not.

git rev-parse HEAD > evidence/run2-source.sha
# Save API output after run 2, then compare item full names/descriptions.

Idempotency is not “the console says SUCCESS.” It means the same desired input converges to the same intended item configuration.

8. Make one controlled schema change

Change only api-ci to add [schema=v2] and a build-discarder policy. Commit it:

pipelineJob('apps/api-ci') {
  description('[managed-by=job-dsl][owner=platform-lab][schema=v2]')
  logRotator { numToKeep(10) }
  // existing definition unchanged
}

git add jobs/platform.groovy
git commit -m 'ch27: api-ci schema v2'
git rev-parse HEAD | tee evidence/dsl-v2.sha

After the seed run, verify that api-ci changed and worker-ci did not. This isolates generator causality.

9. Simulate a removed generated job safely

Remove the worker-ci definition from the DSL and change the Job DSL step to removedJobAction: 'DISABLE' for this exercise. Commit both changes together so policy and desired state are reviewable.

Run once. Expected evidence: platform-lab/apps/worker-ci remains present but disabled. Its prior history/configuration can still be inspected. Do not use DELETE in the mandatory lab.

10. API automation around the seed

The Remote API is useful for exact triggering and evidence, not for replacing the generator. With an API token stored outside the script:

# Read seed state
curl -fsS --user "$JENKINS_USER:$JENKINS_API_TOKEN" \
  "$JENKINS_URL/job/platform-lab/job/seed/api/json?tree=fullName,lastBuild[number,url,result]"

# Trigger the exact seed. API token authentication is preferred for scripted clients.
curl -fsS -X POST --user "$JENKINS_USER:$JENKINS_API_TOKEN" \
  "$JENKINS_URL/job/platform-lab/job/seed/build"

A successful POST may only mean the request was accepted. Correlate queue/build state and then inspect the resulting seed build.

11. Small challenge: choose the right layer

A team asks to change the global Jenkins URL, create two application Pipeline jobs, and trigger the seed after review. Which mechanisms should own each?

Expected reasoning: global URL belongs to JCasC/controller configuration; application job creation belongs to Job DSL; triggering/inspection can use the Remote API. Do not encode global controller state into Job DSL merely because Groovy can reach internal objects.

12. Verification and cleanup

• Retain DSL commits/hashes and seed build URLs.
• Verify generated full names are under platform-lab.
• Verify the removed worker is disabled, not deleted.
• Verify no credentials appear in console/archive/Git.
• Delete only the disposable controller/agent and synthetic repo created for the lab.

Never use a wildcard cleanup against a shared Jenkins instance.

Knowledge check

Answer before revealing the explanation.

Official references and version notes

• Job DSL plugin — current release, minimum core requirement, maintenance/security status and links to the API Viewer.
• Jenkins Pipeline Steps — Job DSL — current jobDsl parameters including sandbox, lookup strategy and removed-item actions.
• Job DSL upstream repository — getting started, Job DSL versus Pipeline DSL, and API Viewer guidance.
• Job DSL — Script Security — sandbox/approval behavior and build-identity implications.
• Script Security plugin — Groovy sandbox and signature-approval boundary.
• Folders plugin — folder hierarchy baseline used by the examples.
• Matrix Authorization Strategy — optional fine-grained folder/item permission model for the lab.
• Authorize Project plugin — optional explicit build identity used when exercising sandboxed Job DSL with access-control checks.
• Jenkins Remote Access API and CSRF protection — current API authentication/POST semantics.
• Jenkins LTS changelog and Java support policy.