GitLab CI/CD: Architecting, Deploying, and Optimizing Pipelines

Architecture Core Concepts

GitLab CI CD Core Components

In this guide, we’ll dive into the four essential GitLab CI/CD building blocks—Pipeline, Stages, Jobs, and Scripts—and show you how they collaborate to automate testing, building, and deployment. By the end, you’ll have a clear understanding of how to configure a robust .gitlab-ci.yml and choose the right pipeline type for your workflow.

CI/CD Core Components Overview

ComponentDescription
PipelineAutomated process orchestrating jobs and stages
StageLogical group of jobs (e.g., build, test, deploy)
JobIndividual unit of work that runs on a runner
ScriptShell commands executed by a job

Pipeline

A pipeline is defined via a .gitlab-ci.yml file at the root of your repo and represents the blueprint for your CI/CD workflow.

Note

Ensure your .gitlab-ci.yml lives in the repository root; otherwise GitLab won’t detect it.
See GitLab CI/CD YAML reference.

Key points:

  • Definition: YAML file named .gitlab-ci.yml.
  • Naming: Use workflow:name to assign a custom name shown in the Pipelines tab.
  • Triggers: Pipelines start on push, merge_request, schedule, or manual actions.
  • Conditions: Use rules to control when pipelines run.

Example: Run pipeline only on commits to main

workflow:
  name: My Awesome App Pipeline
  rules:
    - if: $CI_COMMIT_BRANCH == 'main'

Jobs

A job is the fundamental execution unit in a pipeline:

  • Execution: Runs on a GitLab Runner (shared or self-hosted).
  • Runner selection: Use tags to pick specific runners.
  • Stage assignment: Each job belongs to a defined stage.

Example: Selecting a specific runner

unit_test_job:
  stage: test
  tags:
    - saas-linux-small-amd64

Warning

Keep sensitive data out of script blocks. Use CI/CD variables for secrets like tokens or credentials.

Scripts

The script section lists shell commands executed by the job:

  • Sequential execution: Commands run in order.
  • Common use cases: Install dependencies, build code, run tests, deploy artifacts, perform security scans.
  • Before/after hooks:
    • before_script: Setup steps (e.g., install Node.js, configure DB).
    • after_script: Cleanup tasks (e.g., remove temp files).
before_script:
  - echo "Setting up environment"
script:
  - npm install
  - npm test
after_script:
  - echo "Cleaning up"

Example .gitlab-ci.yml

Here’s a complete sample that defines two stages—test and deploy—and a workflow rule:

workflow:
  name: My Awesome App Pipeline
  rules:
    - if: $CI_COMMIT_BRANCH == 'main'

stages:
  - test
  - deploy

unit_test_job:
  stage: test
  tags:
    - saas-linux-small-amd64
  before_script:
    - echo "Install NodeJS"
  script:
    - npm install
    - npm test
  after_script:
    - echo "Tests complete"

deploy_job:
  stage: deploy
  script:
    - echo "Deploying to production..."

In this configuration:

  • The pipeline runs only on main.
  • The test stage executes unit_test_job with setup, tests, and teardown.
  • The deploy stage runs deploy_job after successful tests.

Stages

Stages define the sequence of execution in a pipeline. Jobs within the same stage run in parallel. Common stages include:

StagePurposeExample Commands
buildCompile or assemble codemvn package, docker build
testExecute automated testsnpm test, pytest
deployRelease artifacts to an environmentkubectl apply, aws s3 sync
securityRun vulnerability scanstrivy image, npm audit

Pipeline Types

The image lists different types of pipelines, including Basic Pipeline, DAG Pipeline, Merge Request Pipelines, Merged Results Pipelines, Merge Trains, Parent-Child Pipelines, and Multi-Project Pipelines. Each type is displayed in a colored box with an icon.

Choose the pipeline type that best suits your project complexity and team structure:

Pipeline TypeDescription
Basic PipelineExecutes all jobs in each stage concurrently, then proceeds sequentially.
DAG PipelineRuns jobs based on explicit dependencies for maximum parallelism and efficiency.
Merge Request PipelineValidates new changes in merge requests before merging.
Merged Results PipelineTests the merged result of a MR to catch conflicts early.
Merge TrainQueues MRs in sequence to streamline integration.
Parent-Child PipelinesSplits large pipelines into parent and child configs—ideal for monorepos.
Multi-Project PipelinesCoordinates pipelines across multiple repositories for cross-team workflows.

Watch Video

Watch video content

Previous
Create GitLab Account