Concurrency, resource_group, interruptible Jobs, Retry Policies, Timeouts, and Duplicate-Pipeline Control: Configuration, Design Choices, and Tradeoffs

1. Design from the mutable resource and failure contract

Concurrency controls should be selected from the thing that can be corrupted, not from whichever YAML keyword is familiar. Start with the external resource identity, ask whether operations commute, whether cancellation is safe, whether a retry can repeat the side effect, and what evidence proves the final state. Only then choose resource-group scope, process mode, interruption, retry, and timeout policy.

2. resource_group process-mode choices

Changing process mode is API-managed state for an existing resource group, not a property you should casually mutate in a shared production lab. The mandatory course path teaches the modes read-only and keeps the default unless the learner owns an isolated project.

3. Resource-group key granularity: broad enough for correctness, narrow enough for throughput

A key such as production is correct only if every operation truly conflicts on the same production resource. If two regions are independent, production/$REGION can preserve concurrency—but only when $REGION is validated and maps one-to-one to the actual resource. A key that includes the commit SHA is usually too narrow because every pipeline gets a different lock and conflicting deployments can overlap.

4. Cancel stale work versus preserve evidence

workflow:auto_cancel:on_new_commit: conservative is the current default. Choose interruptible when you deliberately want stale safe jobs canceled while non-interruptible jobs continue. Use none where preserving every pipeline is more important than saving compute.

5. Retries versus idempotency and compensation

Retry policy should answer two independent questions: is the failure plausibly transient? and can repeating this operation be safe? The first chooses retry:when or retry:exit_codes; the second belongs to the application/external API contract.

6. Timeout layers and evidence budget

A project timeout is a default policy. A job-level timeout specializes one job and can exceed the project default, but the runner maximum timeout is an upper bound. If you need failure artifacts or cleanup, set RUNNER_SCRIPT_TIMEOUT shorter than the job timeout and reserve enough time for after_script and uploads.

Do not use extremely short timeouts to simulate circuit breakers around remote deployments unless the deployment API has an explicit cancellation/reconciliation contract. A client timeout is often an unknown outcome, not a rollback.

7. Duplicate pipelines: allowlist creation rather than clean up later

Pipeline suppression is cheaper and easier to audit than canceling duplicate pipelines after jobs start. Prefer a positive workflow model that names accepted pipeline sources. When switching between branch and MR pipelines, use CI_OPEN_MERGE_REQUESTS together with CI_PIPELINE_SOURCE == "push" so API/trigger/downstream sources are not accidentally blocked.

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"'
      when: never
    - if: '$CI_COMMIT_BRANCH'
    - if: '$CI_COMMIT_TAG' 

Record CI_PIPELINE_SOURCE and CI_COMMIT_SHA in evidence jobs so reviewers can prove which rule path created the pipeline.

8. Resource groups with child/downstream pipelines

A resource group can guard a trigger job that starts a child/downstream deployment pipeline. When the resource must remain locked until the downstream work finishes, current GitLab guidance requires a waiting trigger strategy such as strategy: mirror. Review resource-group ordering carefully: oldest_first combined with parent/child pipelines that compete for the same key can deadlock if the parent waits on a child while an older parent job owns the next queue position.

Design principle: place the lock at the orchestration boundary that owns the entire side effect. Do not acquire the same resource independently in parent and child in a way that creates cyclic waiting.

9. Worked decision table

10. Performance and cost are downstream of correctness

Overly broad serialization raises wait time and increases stale-work pressure. Overly narrow serialization can corrupt shared state. Aggressive auto-cancel saves runner capacity but can discard evidence you intended to retain. Broad retry multiplies compute and can amplify outages. Tune these controls from measured queue time, retry rate, timeout frequency, and external reconciliation outcomes—not from anecdotal “pipelines feel slow.”

11. Review checklist for a production change

• What exact external resource does the resource-group key represent?
• Could two different keys still mutate the same target?
• Which jobs are safe to cancel at any instruction boundary?
• Which failure reasons are truly transient, and what is the retry maximum?
• What stable idempotency/reconciliation key proves whether the side effect already happened?
• What job/project/runner timeout actually applies?
• Which pipeline sources are allowed to exist, and can a branch+MR duplicate still be created?
• Are first failed/canceled attempts retained long enough for audit?

Knowledge check

Official references and version notes

Documentation verification date: 2026-09-12. Concurrency, auto-cancel, retry failure reasons, retry-count variables, and timeout behavior are version-sensitive. Re-check the GitLab and Runner versions used by production pipelines before applying the exact examples.

• Resource groups — serialization, process modes, downstream-pipeline locking, waiting-for-resource diagnostics, and deadlock guidance.
• CI/CD YAML syntax reference — resource_group, interruptible, retry, timeout, and workflow:auto_cancel.
• workflow keyword — pipeline creation, duplicate branch/MR prevention, and CI_OPEN_MERGE_REQUESTS patterns.
• Predefined variables — pipeline/job identity, job timeout, and current retry-attempt metadata.
• Configure runners — runner maximum job timeout and script/after-script timeout controls.
• Resource Groups API — reading/updating process mode for an existing resource group.

Current assumptions used in this chapter: mandatory examples use Free-tier CI/CD features and synthetic data. A resource group serializes one resource at a time. Current process modes are unordered (default), oldest_first, newest_first, and newest_ready_first; newest-first modes require idempotent jobs. workflow:auto_cancel:on_new_commit currently supports conservative (default), interruptible, and none. Job retry allows 0–2 retries; retry:exit_codes is generally available. Current GitLab 19.3 docs expose CI_JOB_RETRY_COUNT; older deployments need a different lab signal. Job-level timeout can override the project default but remains bounded by runner maximum timeout.