🇬🇧GitLab CI – For CI/CD Pipelines & DevOps

Posted on 24 de October de 202424 de October de 2024 by Gustavo Santana

🇧🇷 – para ler este artigo em português clique aqui

In this article, I explain CI/CD in practice, specifically GitLab CI/CD. CI/CD is a development practice that facilitates and automates the process of building and delivering software. CI (Continuous Integration) ensures that developers frequently integrate their code, running automated tests to guarantee nothing breaks. CD (Continuous Delivery or Continuous Deployment) automates the deployment of new versions to production or other environments, making the process faster and more secure. This means quicker updates and fewer manual errors. We will talk about build, deploy, security, checks, Runners, and much more.

Let’s get started:

Creating an environment for us to work in:

Access my repository on GitHub:

https://github.com/GustavoSantanaData/Git-CI-CD

Installing GitLab locally:

In the file install ./04-install-gitlab.sh, change the IP address (this step is only necessary if you are using GitLab locally).

#!/bin/bash

sudo mkdir -p /storage/docker-homol/deploy/gitlab/{data,logs,config}

docker run -dit \

 -p "2222:22" \

 -p "8080:80" \

 -p "443:443" \

 --name gitlab \

 --hostname endereço-server \

 -v /storage/docker-homol/deploy/gitlab/data:/var/opt/gitlab \

 -v /storage/docker-homol/deploy/gitlab/logs:/var/log/gitlab \

 -v /storage/docker-homol/deploy/gitlab/config:/etc/gitlab \

 --shm-size 256m \

 gitlab/gitlab-ce:14.7.6-ce.0

Run the file in the terminal:

./04-install-gitlab.sh

Access:

localhost:8080

Username: root
Password: 123456789

At this point, your GitLab will already be available locally. Now, create a repository.

Runner:

Runners in GitLab are the “executors” of CI/CD pipeline jobs. They run on servers configured to build, test, and deploy projects. There are two types: shared runners, which are provided by GitLab and used by multiple projects, and specific runners, which you configure to meet the specific needs of your project, such as running on a machine with custom settings. Essentially, they are the ones that take the CI/CD tasks and make them happen.

Inside the repository, in the right-side menu, go to CI/CD and grab the registration token.

In the file 05-config-runners.sh, run the first block of code, adding the GitLab token to create the runner.

sudo gitlab-runner register -n \

 --url http://seu -ip:8080 \

 --registration-token GR1348941w5eCWTWhMTxorW_RB4Qu \

 --executor shell \

 --description "Runner Shell"

Run the file in your terminal.

Going back to the GitLab site, the runner will appear green. Click on “Edit” and leave it as is.

Run the commands:

sudo systemctl restart gitlab-runner
sudo systemctl enable gitlab-runner

Run the second part of the file to create the runner container.

docker run -dit \

 --name runner-docker \

 --restart always \

 -v /var/run/docker.sock:/var/run/docker.sock \

 -v /opt/gitlab-runner/config:/etc/gitlab-runner \

 gitlab/gitlab-runner:ubuntu-v14.9.1

Run the third code block, adjusted with the token and the URL with the IP address of the runner.

docker exec -it runner-docker \

gitlab-runner register -n \

 --url http://seu-ip:8080/ \

 --registration-token GR1348941w5eCWTWhMTxorW_RB4Qu \

 --clone-url http://seu-ip:8080/ \

 --executor docker \

 --docker-image "docker:latest" \

 --docker-privileged

All set! Your GitLab environment is configured and ready for us to start creating our CI/CD pipelines.

CI:

In the repository, create a .gitlab-ci.yml file.

The .gitlab-ci.yml file is where you define the CI/CD pipelines in GitLab. In it, you specify the pipeline stages, such as build, test, and deploy, and the commands to be executed at each stage. This file is located at the root of the project and is automatically read by GitLab to orchestrate the tasks. It is fully customizable, allowing you to define jobs, environment variables, dependencies, and even on which machines (runners) the jobs should run. Essentially, the .gitlab-ci.yml file is the brain that controls the entire automation process in GitLab.

Creating a job with a manual trigger to start.

job1:

 when: manual

 tags:

   - "shell"

 scripts:

   - echo "Executa somente manual"

This code defines a job called job1 in the GitLab CI/CD pipeline. Explanation:

job1: The name of the job, used to identify this step in the pipeline.
when: manual: Specifies that this job must be run manually, meaning it won’t be triggered automatically when the pipeline runs. Someone will need to click “play” in GitLab to execute it.
tags: Defines that this job should be run by a runner that has the “shell” tag. This is useful when you have different runners configured for different types of tasks.
script: A list of commands that will be executed in the job. In this case, the job will only run the command echo "Executa somente manual", which prints this message in the terminal.

Scheduling pipeline play.

job2:

 when: delayed

 start_in: 1 minute

 script: echo "execução em 1 muinuto"

This code defines a job called job2 that will be executed with a scheduled delay in the GitLab CI/CD pipeline. Explanation:

job2: The name of the job, used to identify this step in the pipeline.
when: delayed: Indicates that the job will be executed with a delay, meaning it won’t run immediately when the pipeline starts.
start_in: 1 minute: Defines the delay time before the job starts executing. In this case, the job will begin 1 minute after the pipeline starts.
script: Defines the command that will be executed. In this case, the job will run the command echo "execução em 1 minuto", which prints this message in the terminal.

Adding stages (tasks) to the pipeline:

If stage 1 executes successfully, it executes a script.

stages:

 - e1

 - e2

 - e3

Etapa1:

 stage: e1

 when: on_success

 script:

   - echo "Etapa1"

Etapa2:

 stage: e2

 when: on_failure

 script:

   - echo "Falha na etapa 1"

Etapa3:

 stage: e3

 when: always

 script:

   - echo "sempre sera executado"

result:

This code has three stages and three jobs, each with specific execution conditions.

Definition of the Stages

stages:

 - e1

 - e2

 - e3

Here, the pipeline is divided into three stages: e1, e2, and e3. GitLab executes the stages in the defined order. Each job is assigned to one of these stages.

Job Stage1

Stage1:

stage: e1
when: on_success
script:
  - echo "Stage1"

stage: e1: This job belongs to stage e1.
when: on_success: The Stage1 job will be executed only if the previous stage (if any) is successful. In this case, it will run automatically since it is the first stage.
script: The job executes the command echo "Stage1", printing this message in the terminal.

Job Stage2

Stage2:

stage: e2
when: on_failure
script:
  - echo "Failure in stage 1"

stage: e2: This job belongs to stage e2.
when: on_failure: The Stage2 job will only be executed if the previous job (Stage1) fails.
script: If executed, the job prints the message echo "Failure in stage 1".

Job Stage3

Stage3:

stage: e3
when: always
script:
  - echo "will always be executed"

stage: e3: This job belongs to stage e3.
when: always: This job will always be executed, regardless of the success or failure of the previous stages.
script: The job prints the message echo "will always be executed".

===========================================================

timeout:

The timeout defines the maximum time that a pipeline can wait for the script to run.

job3:

 timeout: 1m

 script:

   - sleep 20

   - echo "executa com timeout de 1 minuto"

before_script e after_script:

Executes something before starting the jobs, and after the jobs finish, it executes something else.

before_script:

 - echo "Antes do script"

after_script:

 - echo "Depois do script"

job4:

 script:

   - echo "executando script"

Only merge_requests, tags, and variables

Executes the jobs only in specific situations.

job6:

script:
  - echo "Executing when there is a merge"
only:
  - merge_request

job7:

only:
  - tags
script:
  - echo "Executing when the tag is created"
when: manual

job8:

only:
  variables:
    - $RELEASE == 'staging'
script:
  - echo "Executed because the RELEASE variable is set to staging"

Explanation of the code:

This code defines three jobs in the GitLab .gitlab-ci.yml file, each with specific execution conditions. Let’s detail each job:

Job job6

job6:
  script:
    - echo "Executing when there is a merge"
  only:
    - merge_request

script: This job executes the command echo "Executing when there is a merge", which prints the message in the terminal.
only: The job will be executed only in merge request situations. This means it will be activated only when there is a merge request, i.e., when a developer requests to merge one branch into another.

Job job7

job7:
  only:
    - tags
  script:
    - echo "Executing when the tag is created"
  when: manual

only: This job will be executed only when a tag is created in the repository. Tags are markers used to identify specific versions of the code.
script: When the job is triggered, it will execute echo "Executing when the tag is created", printing this message.
when: manual: This means that even if the tag is created, the execution of this job will need to be started manually. The user must click “play” for the job to run.

Job job8

job8:
  only:
    variables:
      - $RELEASE == 'staging'
  script:
    - echo "Executed because the RELEASE variable is set to staging"

only: This job is conditioned on an environment variable called RELEASE. It will be executed only if RELEASE has the value staging.
script: If the conditions are met, the job executes the command echo "Executed because the RELEASE variable is set to staging", printing the message that explains the reason for execution.

Execution of the jobs

Executing job8:

Go to pipeline > run pipeline and execute with the value staging; the pipeline will be created and executed.

Executing job7:

Go to repository > tags > new tag and create a tag. The pipeline will be created and executed.

Executing job6:

Create a new branch in the repository called developer.

Change a sample file (README) and start a merge request from the developer branch to the main branch. After that, clicking on merge will trigger job 6, which is set with the only merge request trigger.

Parallel:

Executes jobs in parallel.

Job8:

parallel: 3

script: echo "sleep 5"

File Cache:

Stores files from the repository in cache, and you can download them or run a series of commands for it.

Job18:

script:
   - echo "Example of file cache."

name: "artifact-example"

artifacts:
   expire_in: 2 minutes
   paths:
     - ./teste.txt

Using Docker and Shell runners and learning needs:

Build_a:

image: alpine

script:
   - echo "Running a build"
   - sleep 10

tags:
   - docker

Teste_a:

needs: [build_a]

script:
   - docker run -dit --name teste ubuntu

tags:
   - shell

The build_a job pulls the image in Alpine and prints an image, using the runner with the Docker tag.

The teste_a job uses needs to indicate that it must wait for build_a to finish executing before starting. It uses the script to run an Ubuntu image on my local machine, all while using the runner with the Shell tag.

Debug and display all variables:

Displays all the job variables and can be used for some kind of trigger.

Job12:

script:
   - export

Result:

CI/CD Variables:

Creating variables within GitLab to call them in the CI/CD code.

Go to Settings > CI/CD > Variables and create a new variable.

job12:

script:
  - echo $senha

This code displays the value of the variable.

Variables in the .gitlab-ci.yml file:

variables:
  VAR1: "teste1"
  VAR2: "teste2"

job12_a:

script:
  - echo $VAR1
  - echo $VAR2

Running a pipeline with User and branch conditions:

If the branch is main and the user is root, it starts a manual pipeline process. If the branch is developer, it does not execute any pipeline.

job13:

script:
  - echo "Pipeline executed only on branch main and executor user gustavo"
rules:
  - if: '$CI_COMMIT_BRANCH == "main" && $GITLAB_USER_LOGIN == "root" '
    when: manual
  - if: '$CI_COMMIT_BRANCH == "developer" '
    when: never

Running a pipeline with except:

Executes the pipeline except if there is a specific message.

job14:

script:
  - echo "Executes the pipeline except if there is a message push-teste"
except:
  variables:
    - $CI_COMMIT_MESSAGE =~ /push-test/

Anchor:

The anchor takes the commands from the job in which it was declared. In this case, job10 will use all the commands from job9 that are not declared in job10.

For example, in job10, we have a script command, so it does not take the script from job9, only the other commands.

job9: &anchor
  when: manual
  script:
    - echo "Executes the job"

job10:
  <<: *anchor
  script: echo "At least the script has been replaced"

Anchor only in only:

Job10 uses the anchor only to get the commands from only in job1.

job1:
  only:
    variables: &variable
      - $TRIGGER == 'value'
  script: echo "Anchor in a job session"

job10:
  only: *variable
  script: echo "At least the script has been replaced"

More examples of using anchors:

.before-script: &before
  - echo "Executes script before"

.script: &script
  - echo "Executes script second"

.after-script: &after
  - echo "Executes this script last"

job1:

before_script:
  - *before
script:
  - *script
  - echo "Execute only for this job"
after_script:
  - *after

Execute pipeline via API:

Executing by tag:

Add a new trigger in Settings > CI/CD > Pipeline Trigger.

Replace curl with the generated token and reference it:

curl -X POST \
      --fail \
      -F token=TOKEN \
      -F ref=0.0.1 \
      http://your-ip:your-port/api/v4/projects/2/trigger/pipeline

Create a new tag 0.0.1.

Run command in .gitlab-ci.yml

job1:
  script: echo "Executing via API"

Then, run the curl command in the terminal:

curl -X POST \
      --fail \
      -F token=TOKEN \
      -F ref=0.0.1 \
      http://your-ip:your-port/api/v4/projects/2/trigger/pipeline

Every time you run this command in the terminal, the pipeline will start automatically.

Executing with branch and variable check:

Include the following code:

job1:
  only:
    variables:
      - $TRIGGER == "value"
  script: echo "Executing via API with main branch and variable"

Run the following command in the terminal, passing a variable with the value “value” and referencing the main branch:

docker exec -it runner-docker \
gitlab-runner register -n \
  --url http://172.17.0.1:8080/ \
  --registration-token GR1348941ig4xM7hguPgAvLL5hkst \
  --clone-url http://172.17.0.1:8080/ \
  --executor docker \
  --docker-image "docker:latest" \
  --docker-privileged

Conclusion:

In this article, we saw how to create and configure GitLab locally and explored various examples of CI/CD commands. These examples help to understand how to automate the development workflow using pipelines to integrate and deliver software more efficiently. I hope this has clarified how to apply these concepts in your projects.

Leave a Reply Cancel reply