GitLab CI/CD: Architecting, Deploying, and Optimizing Pipelines
Optimization Security and Monitoring
Reference Tags Reuse Configuration Integration Testing
In this lesson, we’ll walk through how to leverage GitLab CI/CD reference tags (!reference) and standard YAML anchors to share configuration snippets across multiple jobs. By doing so, you can DRY up your pipeline definitions—pulling in values like image, before_script, or script from one job into another and avoiding duplication.
1. Basic YAML Anchors and !reference Tags
GitLab CI/CD supports two primary mechanisms for reusing snippets:
| Method | Syntax | Use Case |
|---|---|---|
| YAML Anchors | &anchor / *alias | Share simple lists or maps within the file |
GitLab !reference | !reference [job, section] | Import complete sections (script, etc.) |
# .gitlab-ci.yml
.default_scripts: &default_scripts
- ./default-script1.sh
- ./default-script2.sh
job1:
script:
- *default_scripts
Or use GitLab’s custom reference tag:
# setup.yml
.setup:
script:
- echo "creating environment"
# .gitlab-ci.yml
include:
- local: setup.yml
.teardown:
after_script:
- echo "deleting environment"
test:
script:
- !reference [.setup, script]
Note
YAML anchors work within the same file, while !reference can pull from hidden jobs or included files.
2. Preparing the Kubernetes Environment
Let’s define two hidden jobs: one for Node.js and one for Kubernetes deployment. We’ll attach an anchor (&kubernetes_deploy_job) to reuse the Kubernetes job’s image later.
variables:
# Add global variables here…
.prepare_nodejs_environment:
image: node:14
script:
- npm install
.prepare_deployment_environment: &kubernetes_deploy_job
image:
name: alpine:3.7
dependencies: []
before_script:
- wget https://storage.googleapis.com/kubernetes-release/release/$(wget -q -O - https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
- chmod +x ./kubectl
- mv ./kubectl /usr/bin/kubectl
- apk add --no-cache gettext
- envsubst --version
Warning
Make sure hidden jobs (prefixed with .) are not executed on their own—they only serve as references.
3. Two Almost-Identical Integration Tests
Consider two integration testing jobs—dev and staging. They share the same image, identical before_script, and script blocks:
k8s_dev_deploy:
stage: dev-deploy
image: alpine:3.7
needs:
- k8s_stage_deploy
before_script:
- apk --no-cache add curl jq
script:
- echo "$INGRESS_URL"
- curl -s -k "https://$INGRESS_URL/live" | jq -r .status | grep -i live
- curl -s -k "https://$INGRESS_URL/ready" | jq -r .status | grep -i ready
k8s_stage_deploy:
stage: stage-deploy
image: alpine:3.7
needs:
- k8s_dev_deploy
before_script:
- apk --no-cache add curl jq
script:
- echo "$INGRESS_URL"
- curl -s -k "https://$INGRESS_URL/live" | jq -r .status | grep -i live
- curl -s -k "https://$INGRESS_URL/ready" | jq -r .status | grep -i ready
Rather than duplicating, let’s pull these blocks into new jobs via !reference.
4. Reusing the image Definition
We can reuse the image from .prepare_deployment_environment with:
k8s_dev_integration_testing:
stage: dev-deploy
image: !reference [.prepare_deployment_environment, image]
needs:
- k8s_dev_deploy
before_script:
- apk --no-cache add curl jq
script:
- echo "$INGRESS_URL"
- curl -s -k "https://$INGRESS_URL/live" | jq -r .status | grep -i live
- curl -s -k "https://$INGRESS_URL/ready" | jq -r .status | grep -i ready
k8s_stage_integration_testing:
stage: stage-deploy
image: !reference [.prepare_deployment_environment, image]
needs:
- k8s_stage_deploy
before_script:
- apk --no-cache add curl jq
script:
- echo "$INGRESS_URL"
- curl -s -k "https://$INGRESS_URL/live" | jq -r .status | grep -i live
- curl -s -k "https://$INGRESS_URL/ready" | jq -r .status | grep -i ready
Now, updating the base image in one place updates both jobs.
5. Reusing before_script and script Blocks
To avoid repeating before_script or script, reference the dev-integration job directly:
k8s_dev_integration_testing:
stage: dev-deploy
image: !reference [.prepare_deployment_environment, image]
needs:
- k8s_dev_deploy
before_script:
- apk --no-cache add curl jq
script:
- echo "$INGRESS_URL"
- curl -s -k "https://$INGRESS_URL/live" | jq -r .status | grep -i live
- curl -s -k "https://$INGRESS_URL/ready" | jq -r .status | grep -i ready
k8s_stage_integration_testing:
stage: stage-deploy
image: !reference [.prepare_deployment_environment, image]
needs:
- k8s_stage_deploy
before_script: !reference [k8s_dev_integration_testing, before_script]
script: !reference [k8s_dev_integration_testing, script]
This ensures both jobs stay in sync for setup and testing steps.
6. Full Snippet with Artifacts and Environment
For a consolidated pipeline, include artifacts and environment details alongside your reused blocks:
artifacts:
reports:
dotenv:
- app_ingress_url.env
environment:
name: staging
url: "https://$INGRESS_URL"
k8s_stage_integration_testing:
stage: stage-deploy
image: !reference [.prepare_deployment_environment, image]
needs:
- k8s_stage_deploy
before_script: !reference [k8s_dev_integration_testing, before_script]
script: !reference [k8s_dev_integration_testing, script]
Conclusion
By combining standard YAML anchors (& / *) with GitLab’s custom !reference tags, you can effectively DRY up your CI/CD definitions. Pull in common image, before_script, or script sections from hidden or existing jobs—simplifying maintenance and reducing errors.
Links and References
- GitLab CI/CD Reference Tags
- GitLab CI/CD Anchors and Aliases
- Kubernetes Official Documentation
- jq JSON Processor
Watch Video
Watch video content