GitLab CI/CD: Architecting, Deploying, and Optimizing Pipelines

Architecture Core Concepts

Working with Variables at different levels

In this guide, you’ll learn how to define and manage variables at different scopes within a GitLab CI/CD pipeline. Proper use of variables helps you follow DRY principles, streamline maintenance, and reduce the risk of errors when updating image names, versions, or other configuration values.

Note

GitLab CI/CD supports both custom variables and predefined variables. Use them to parameterize your pipeline and avoid hard-coding values.


1. Pain Point: Hard-Coded Values in Every Job

Here’s a typical pipeline that builds, tests, and pushes a Docker image. Notice how the registry, username, image name, and version are repeated in each job:

docker_build:
  stage: docker
  needs:
    - build_file
  script:
    - echo "docker build -t docker.io/dockerUsername/imageName:version"
    - sleep 15s

docker_testing:
  stage: docker
  needs:
    - docker_build
  script:
    - echo "docker run -p 80:80 docker.io/dockerUsername/imageName:version"
    - sleep 10s
    - exit 1

docker_push:
  stage: docker
  needs:
    - docker_testing
  script:
    - echo "docker login --username=dockerUsername --password=s3cUrePaSsW0rd"
    - echo "docker push docker.io/dockerUsername/imageName:version"

Maintaining pipelines like this is error-prone. Every change to imageName or version requires edits in multiple places.


2. Variable Scopes: Global vs. Job-Level

GitLab CI/CD provides two scopes for custom variables:

ScopeDeclaration LocationEffective In
Globaltop-level variables: blockAll jobs
Jobvariables: inside a jobThat specific job
  • Global variables can be overridden by job-level variables with the same name.
  • Job-level variables are isolated to their job and not visible elsewhere.

3. Defining Global Variables

Move shared configuration into a global variables: block. This makes your pipeline DRY and easier to update.

variables:
  DEPLOY_SITE: "https://example.com/"

deploy_job:
  stage: deploy
  script:
    - deploy-script --url "$DEPLOY_SITE" --path "/"
  environment: production

deploy_review_job:
  stage: deploy
  variables:
    REVIEW_PATH: "/review"
  script:
    - deploy-review-script --url "$DEPLOY_SITE" --path "$REVIEW_PATH"
  environment: production
  • DEPLOY_SITE is available to both jobs.
  • REVIEW_PATH applies only to deploy_review_job.

4. Refactoring Docker Jobs with Shared Variables

First, here’s a version that still repeats variables at the job level:

docker_build:
  stage: docker
  needs:
    - build_file
  variables:
    USERNAME: dockerUsername
    REGISTRY: docker.io/$USERNAME
    IMAGE: ascii-artwork
    VERSION: latest
  script:
    - echo "docker build -t $REGISTRY/$IMAGE:$VERSION"

docker_testing:
  stage: docker
  needs:
    - docker_build
  variables:
    USERNAME: dockerUsername
    REGISTRY: docker.io/$USERNAME
    IMAGE: ascii-artwork
    VERSION: latest
  script:
    - echo "docker run -p 80:80 $REGISTRY/$IMAGE:$VERSION"

docker_push:
  stage: docker
  needs:
    - docker_testing
  variables:
    USERNAME: dockerUsername
    REGISTRY: docker.io/$USERNAME
    IMAGE: ascii-artwork
    VERSION: latest
    PASSWORD: s3cUrePaSsW0rd
  script:
    - echo "docker login --username=$USERNAME --password=$PASSWORD"
    - echo "docker push $REGISTRY/$IMAGE:$VERSION"

Each job redeclares USERNAME, REGISTRY, IMAGE, and VERSION—we can improve this.


5. Promoting Common Variables to Global Scope

Define all shared variables at the top level. Only sensitive or job-specific variables stay within the job.

workflow:
  name: Generate ASCII Artwork

stages:
  - build
  - test
  - docker
  - deploy

variables:
  USERNAME: dockerUsername
  REGISTRY: docker.io/$USERNAME
  IMAGE: ascii-artwork
  VERSION: latest

build_file:
  stage: build
  script: …

test_file:
  stage: test
  script: …

docker_build:
  stage: docker
  needs:
    - build_file
  script:
    - echo "docker build -t $REGISTRY/$IMAGE:$VERSION"

docker_testing:
  stage: docker
  needs:
    - docker_build
  script:
    - echo "docker run -p 80:80 $REGISTRY/$IMAGE:$VERSION"

docker_push:
  stage: docker
  needs:
    - docker_testing
  variables:
    PASSWORD: s3cUrePaSsW0rd
  script:
    - echo "docker login --username=$USERNAME --password=$PASSWORD"
    - echo "docker push $REGISTRY/$IMAGE:$VERSION"

Now only PASSWORD remains in docker_push, while USERNAME, REGISTRY, IMAGE, and VERSION are defined once.


6. Leveraging Predefined CI/CD Variables for Dynamic Tagging

Instead of a static latest tag, use $CI_PIPELINE_ID or $CI_COMMIT_SHA to uniquely tag each build:

variables:
  USERNAME: dockerUsername
  REGISTRY: docker.io/$USERNAME
  IMAGE: ascii-artwork
  VERSION: $CI_PIPELINE_ID
docker_build:
  stage: docker
  needs:
    - build_file
  script:
    - echo "docker build -t $REGISTRY/$IMAGE:$VERSION"

Every pipeline run now pushes ascii-artwork:<pipeline_id>, making image versions traceable.


7. Viewing Expanded Variables in Job Logs

When the pipeline runs, GitLab replaces variables with their values:

$ echo "docker login --username=$USERNAME --password=$PASSWORD"
docker login --username=dockerUsername --password=s3cUrePaSsW0rd

$ echo "docker push $REGISTRY/$IMAGE:$VERSION"
docker push docker.io/dockerUsername/ascii-artwork:1153576211

Here, $VERSION was replaced by the numeric pipeline ID.


8. Job-Level Variables Are Isolated

Job-specific variables do not carry over to other jobs:

deploy_ec2:
  stage: deploy
  script:
    - echo "Username: $USERNAME, Password: $PASSWORD"

Log output:

$ echo "Username: $USERNAME, Password: $PASSWORD"
Username: dockerUsername, Password:

$PASSWORD is empty because it was only defined in the docker_push job.

Warning

Avoid exposing sensitive variables in job logs. Use Masked Variables or Protected Variables to secure credentials and secrets.


9. Next Steps

Watch Video

Watch video content

Previous
Using needs Keyword