GitLab CI/CD: Architecting, Deploying, and Optimizing Pipelines
Optimization Security and Monitoring
Reference Tags Reuse Configuration Integration Testing
In this lesson, we’ll walk through how to leverage GitLab CI/CD reference tags (!reference
) and standard YAML anchors to share configuration snippets across multiple jobs. By doing so, you can DRY up your pipeline definitions—pulling in values like image
, before_script
, or script
from one job into another and avoiding duplication.
1. Basic YAML Anchors and !reference
Tags
GitLab CI/CD supports two primary mechanisms for reusing snippets:
Method | Syntax | Use Case |
---|---|---|
YAML Anchors | &anchor / *alias | Share simple lists or maps within the file |
GitLab !reference | !reference [job, section] | Import complete sections (script , etc.) |
# .gitlab-ci.yml
.default_scripts: &default_scripts
- ./default-script1.sh
- ./default-script2.sh
job1:
script:
- *default_scripts
Or use GitLab’s custom reference tag:
# setup.yml
.setup:
script:
- echo "creating environment"
# .gitlab-ci.yml
include:
- local: setup.yml
.teardown:
after_script:
- echo "deleting environment"
test:
script:
- !reference [.setup, script]
Note
YAML anchors work within the same file, while !reference
can pull from hidden jobs or included files.
2. Preparing the Kubernetes Environment
Let’s define two hidden jobs: one for Node.js and one for Kubernetes deployment. We’ll attach an anchor (&kubernetes_deploy_job
) to reuse the Kubernetes job’s image
later.
variables:
# Add global variables here…
.prepare_nodejs_environment:
image: node:14
script:
- npm install
.prepare_deployment_environment: &kubernetes_deploy_job
image:
name: alpine:3.7
dependencies: []
before_script:
- wget https://storage.googleapis.com/kubernetes-release/release/$(wget -q -O - https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
- chmod +x ./kubectl
- mv ./kubectl /usr/bin/kubectl
- apk add --no-cache gettext
- envsubst --version
Warning
Make sure hidden jobs (prefixed with .
) are not executed on their own—they only serve as references.
3. Two Almost-Identical Integration Tests
Consider two integration testing jobs—dev and staging. They share the same image
, identical before_script
, and script
blocks:
k8s_dev_deploy:
stage: dev-deploy
image: alpine:3.7
needs:
- k8s_stage_deploy
before_script:
- apk --no-cache add curl jq
script:
- echo "$INGRESS_URL"
- curl -s -k "https://$INGRESS_URL/live" | jq -r .status | grep -i live
- curl -s -k "https://$INGRESS_URL/ready" | jq -r .status | grep -i ready
k8s_stage_deploy:
stage: stage-deploy
image: alpine:3.7
needs:
- k8s_dev_deploy
before_script:
- apk --no-cache add curl jq
script:
- echo "$INGRESS_URL"
- curl -s -k "https://$INGRESS_URL/live" | jq -r .status | grep -i live
- curl -s -k "https://$INGRESS_URL/ready" | jq -r .status | grep -i ready
Rather than duplicating, let’s pull these blocks into new jobs via !reference
.
4. Reusing the image
Definition
We can reuse the image
from .prepare_deployment_environment
with:
k8s_dev_integration_testing:
stage: dev-deploy
image: !reference [.prepare_deployment_environment, image]
needs:
- k8s_dev_deploy
before_script:
- apk --no-cache add curl jq
script:
- echo "$INGRESS_URL"
- curl -s -k "https://$INGRESS_URL/live" | jq -r .status | grep -i live
- curl -s -k "https://$INGRESS_URL/ready" | jq -r .status | grep -i ready
k8s_stage_integration_testing:
stage: stage-deploy
image: !reference [.prepare_deployment_environment, image]
needs:
- k8s_stage_deploy
before_script:
- apk --no-cache add curl jq
script:
- echo "$INGRESS_URL"
- curl -s -k "https://$INGRESS_URL/live" | jq -r .status | grep -i live
- curl -s -k "https://$INGRESS_URL/ready" | jq -r .status | grep -i ready
Now, updating the base image in one place updates both jobs.
5. Reusing before_script
and script
Blocks
To avoid repeating before_script
or script
, reference the dev-integration job directly:
k8s_dev_integration_testing:
stage: dev-deploy
image: !reference [.prepare_deployment_environment, image]
needs:
- k8s_dev_deploy
before_script:
- apk --no-cache add curl jq
script:
- echo "$INGRESS_URL"
- curl -s -k "https://$INGRESS_URL/live" | jq -r .status | grep -i live
- curl -s -k "https://$INGRESS_URL/ready" | jq -r .status | grep -i ready
k8s_stage_integration_testing:
stage: stage-deploy
image: !reference [.prepare_deployment_environment, image]
needs:
- k8s_stage_deploy
before_script: !reference [k8s_dev_integration_testing, before_script]
script: !reference [k8s_dev_integration_testing, script]
This ensures both jobs stay in sync for setup and testing steps.
6. Full Snippet with Artifacts and Environment
For a consolidated pipeline, include artifacts and environment details alongside your reused blocks:
artifacts:
reports:
dotenv:
- app_ingress_url.env
environment:
name: staging
url: "https://$INGRESS_URL"
k8s_stage_integration_testing:
stage: stage-deploy
image: !reference [.prepare_deployment_environment, image]
needs:
- k8s_stage_deploy
before_script: !reference [k8s_dev_integration_testing, before_script]
script: !reference [k8s_dev_integration_testing, script]
Conclusion
By combining standard YAML anchors (&
/ *
) with GitLab’s custom !reference
tags, you can effectively DRY up your CI/CD definitions. Pull in common image
, before_script
, or script
sections from hidden or existing jobs—simplifying maintenance and reducing errors.
Links and References
- GitLab CI/CD Reference Tags
- GitLab CI/CD Anchors and Aliases
- Kubernetes Official Documentation
- jq JSON Processor
Watch Video
Watch video content