Passing Variables Through GitLab Pipelines
During working with GitLab multi-project pipelines and parent-child pipelines, I have encountered the problem how to pass variables through these pipelines. The GitLab documentation describes very well how to pass variables to a downstream pipeline. My challenge is how to pass variables from child to parent pipeline and how the parent pipeline can pass these variables to a downstream pipeline, that it describes in another GitLab project. Let's start, how to publish the variable that are defined in a child pipeline.
Publishing Variables of a Child Pipeline
Assume that we have a GitLab project with the following structure for the pipelines.
1. 2├── pipelines 3│ └── child-pipeline.yml 4└── .gitlab-ci.yml
The parent pipeline, defined in
.gitlab-ci.yml, triggers the child pipeline, that is defined in
1# .gitlab-ci.yml 2stages: 3 - a 4 5trigger-child-pipeline-job: 6 stage: a 7 trigger: 8 include: pipelines/child-pipeline.yml 9 strategy: depend
The child pipeline
pipelines/child-pipeline.yml defines the variables and publishes them via the report artifact dotenv.
1stages: 2 - define-env 3 4define-env-job: 5 stage: define-env 6 script: 7 - echo "MODULE_A_VERSION=1.0.0" >> .env 8 artifacts: 9 reports: 10 dotenv: .env
Dotenv is a standardized way to handle environment variables. Following the dotenv concept, the environment variables are stored in a file that have the following structure.
1# .env 2VARIABLE_NAME=variable-value 3MODULE_A_VERSION=1.0.0
For more information, please visit the dotenv homepage.
Let's go to the next step, how to consume this variable in the parent pipeline.
Consuming Variables From a Child Pipeline in a Parent Pipeline
The first challenge is how the parent pipeline can consume the variable, that is defined in the child pipeline (in our sample, it is the variable
The child pipeline publishes its variable via a report artifact.
This artifact can be used by the parent pipeline via the
Unfortunately, it is not enough to reference the job name of the child pipeline that creates the report artifact.
You also have to add a reference to the project that contains the parent and the child pipeline.
Now, the parent pipeline can use the variable that is stored in the report artifact.
1stages: 2 - a 3 - b 4 5trigger-child-pipeline-job: 6 stage: a 7 trigger: 8 include: pipelines/child-pipeline.yml 9 strategy: depend 10 11consume-env-from-child-pipeline-job: 12 stage: b 13 script: 14 - "echo $MODULE_A_VERSION" 15 needs: 16 - project: sparsick/gitlab-ci-passing-variable-pipeline 17 job: define-env-job 18 ref: main 19 artifacts: true
The next challenge is to consume this variable in a downstream pipeline that is defined in another project.
Consuming Variable From a Child Pipeline in a Downstream Pipeline of the Parent Pipeline
It exists two ways how a downstream pipeline can consume a variable from a child pipeline of its upstream pipeline.
The first way works similarly that I described in the above section. Assume, that we have the following parent pipeline that triggered a child pipeline and a downstream pipeline in another project.
1# .gitlab-ci.yaml 2stages: 3 - a 4 - b 5 6trigger-child-pipeline-job: 7 stage: a 8 trigger: 9 include: pipelines/child-pipeline.yml 10 strategy: depend 11 12trigger-another-pipeline-job: 13 stage: b 14 trigger: sparsick/gitlab-ci-passing-variable-downstream-pipeline
MODULE_A_VERSION is defined in the child pipeline like I described in the above section.
The variable can be consumed by the downstream pipeline in the same way as the parent pipeline, that I described in the above section.
1# .gitlab-ci.yaml of the downstream pipeline 2stages: 3 - print-env 4 5print-env-from-a-child-pipeline-of-the-upstream-job: 6 stage: print-env 7 script: 8 - echo $MODULE_A_VERSION 9 needs: 10 - project: sparsick/gitlab-ci-passing-variable-pipeline 11 job: define-env-job 12 ref: main 13 artifacts: true
This approach has a big disadvantage. If the job/variable/project/branch of the upstream pipeline changes its name, the downstream pipeline doesn't recognize this change automatically, and it couldn't work anymore as expected.
A second way solves this disadvantage. Here, the variable value is passed via a new variable to the downstream pipeline. Assume, that we have the following parent pipeline that triggered a child pipeline and a downstream pipeline in another project and pass a variable to the downstream pipeline.
1# .gitlab-ci.yaml 2stages: 3 - a 4 - b 5 6trigger-child-pipeline-job: 7 stage: a 8 trigger: 9 include: pipelines/child-pipeline.yml 10 strategy: depend 11 12trigger-another-pipeline-with-var-job: 13 stage: b 14 variables: 15 ARTIFACT_VERSION: "1.0.0" 16 trigger: sparsick/gitlab-ci-passing-variable-downstream-pipeline 17
The downstream pipeline can use the
ARTIFACT_VERSION variable in the common way.
1# .gitlab-ci.yaml of the downstream pipeline 2stages: 3 - print-env 4 5print-env-from-upstream-job: 6 stage: print-env 7 script: 8 - echo $ARTIFACT_VERSION
Now, I want, that the value of the variable
MODULE_A_VERSION of the child pipeline is pass to the downstream pipeline.
My first idea was to add with
needs a dependency like I used it above in the
But this is invalid because
needs with a reference to a project can't be used together in the same job.
1# invalid job definition 2trigger-another-pipeline-with-var-job: 3 stage: b 4 variables: 5 ARTIFACT_VERSION: "$MODULE_A_VERSION" 6 trigger: sparsick/gitlab-ci-test-downstream 7 needs: 8 - project: sparsick/gitlab-ci-test 9 job: define-env-job 10 ref: main 11 artifacts: true 12
Therefore, I have to take a detour via a new job that read the variable from the child and create a new dotenv report artifact.
1stages: 2 - a 3 - b 4 - c 5 6trigger-child-pipeline-job: 7 stage: a 8 trigger: 9 include: pipelines/child-pipeline.yml 10 strategy: depend 11 12consume-env-from-child-pipeline-job: 13 stage: b 14 script: 15 - echo "MODULE_A_VERSION=$MODULE_A_VERSION" >> .env 16 needs: 17 - project: sparsick/gitlab-ci-test 18 job: define-env-job 19 ref: main 20 artifacts: true 21 artifacts: 22 reports: 23 dotenv: .env 24 25trigger-another-pipeline-with-var-job: 26 stage: c 27 variables: 28 ARTIFACT_VERSION: "$MODULE_A_VERSION" 29 needs: 30 - job: consume-env-from-child-pipeline-job 31 artifacts: true 32 trigger: sparsick/gitlab-ci-passing-variable-downstream-pipeline
You can find the whole example on GitLab.
When you have another or better approach how to solve this described problem, let me know and please write a comment.