GitLab CI/CD: Architecting, Deploying, and Optimizing Pipelines
Architecture Core Concepts
Resource Groups
In GitLab CI/CD, pipelines run concurrently by default. When you need to serialize critical jobs—such as deployments—across all pipelines in a project, resource groups provide mutual exclusion. Only one job in a given group can run at a time; the rest queue until the resource is released.
To learn more, see the CI/CD YAML syntax reference for resource_group
.
Basic Workflow Example
The example below runs deploy-job
on the main
branch. Without a resource group, multiple pipelines could execute the deploy concurrently.
workflow:
name: Exploring GitLab CI Concepts
rules:
- if: $CI_COMMIT_BRANCH == 'main'
variables:
DEPLOY_VARIABLE: "PRODUCTION"
deploy-job:
stage: deploy
script:
- echo "Deploying application..."
- echo "Application successfully deployed to $DEPLOY_VARIABLE environment"
Enforcing Mutual Exclusion with Resource Groups
Add the resource_group
keyword to ensure that jobs sharing the same group never overlap:
deploy-to-production:
stage: deploy
script:
- deploy
resource_group: production
When several pipelines reach deploy-to-production
at the same time, only one job obtains the production
resource. The others wait in line.
Process Modes
Resource groups support three modes for dequeuing waiting jobs:
Process Mode | Description |
---|---|
unordered | Default; any waiting job may start next (no guarantee) |
oldest_first | Jobs run in the order they were queued |
newest_first | The most recently queued job starts first |
To update the process mode for an existing resource group, use the GitLab CI REST API.
Example: Build and Deploy Across Three Pipelines
This configuration triggers three successive pipelines. Each pipeline queues its deploy
job against production
:
build:
stage: build
script:
- echo "Building..."
deploy:
stage: deploy
script:
- echo "Deploying..."
environment: production
resource_group: production
- unordered: Any of deploy-1, deploy-2, or deploy-3 may run first; others wait.
- oldest_first: deploy-1 → deploy-2 → deploy-3.
- newest_first: deploy-3 → deploy-2 → deploy-1.
Demo: Simulating Long-Running Deployments
Add a sleep command to observe the queuing behavior in the GitLab UI:
name: Exploring GitLab CI Concepts
rules:
- if: '$CI_COMMIT_BRANCH == main'
variables:
DEPLOY_VARIABLE: "PRODUCTION"
deploy-job:
stage: deploy
resource_group: production
script:
- echo "Deploying application..."
- sleep 300
- echo "Application successfully deployed to $DEPLOY_VARIABLE environment"
- Commit to
main
to trigger the first pipeline. - Manually schedule a second pipeline on
main
.
Note
The GitLab UI pipelines list shows each pipeline’s status, name, and project navigation options.
Note
Since the production resource is occupied by the first deploy-job, the second deploy-job remains queued and waiting.
- Cancel the first pipeline (or just its deploy-job) to free the resource.
Note
Once the first deploy-job is canceled and the production resource is freed, the queued deploy-job automatically starts.
Resource groups let you control job concurrency precisely, ensuring critical jobs never overlap across pipelines.
Links and References
Watch Video
Watch video content