A Guide to Handling Merge Requests in Jenkins Pipelines (2023)

Introduction

About a month ago I have been to a Stuttgart DevOps meetup that was dedicated to Jenkins Pipelines. I was positively impressed, because it has always bothered me to manually configure multiple build steps of complex build jobs in Jenkins. Moreover, when you have multiple kinds of build jobs for the same project, you end up configuring exactly the same build steps again and again. Now there is a solution to this problem – Jenkins Pipelines to define a CI/CD pipeline as a code.

After the meetup I could not wait to introduce a Jenkinsfile to my current project to replace the three Jenkins build jobs that are currently used:

  • Multi-branch for main integration branches: develop, release, hotfix and master.
  • Merge request for automatic testing of GitLab merge requests.
  • Parameterized for on demand testing.

I mainly wanted to solve these problems:

  • Define the CI/CD pipeline as a code to make it self-documented, reproducible and versioned.
  • Have a single definition of build steps for any type of build job, be it multi-branch, merge requests or parameterized.
  • Get away from manual configuration of build steps.
  • Make the pipeline easily extendable. E.g., it should not be complicated to add a new static analysis tool report into all the configured build jobs.

And so I began. Below you will find a guide to setting up Jenkins Pipelineto handle merge requests for GitFlow workflow. I will only focus on checking out the source code from Git, since I find this the only non-trivial part.

Simple checkout

Building a single branch

The counter-intuitive part of Jenkins Pipeline is that Jenkins doesn’t checkout the source code for you and you have to do it yourself. However, if you tell Jenkins to take the Jenkinsfile from a Git repository, checking out the source code becomes trivial, because Jenkins provides our Groovy script with an object named scm. We can directly reuse this object to checkout the same repository that was configured as the source of the Jenkinsfile:



This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters

Show hidden characters

#!groovy
def handleCheckout = {
sh "echo 'Checking out given branch…'"
checkout scm
}
node() {
stage('setup') {
sh "env | sort"
handleCheckout()
sh "git branch -vv"
}
stage('test') {
sh "echo 'Throw in some tests here'"
}
}

With this Jenkinsfile in the root of your Git repository you can create a new Pipeline build job in Jenkins.

Building Integration Branches with Multibranch Pipeline

In many projects you will normally have several integration branches that should be tested nightly or on new commits. This can be done by using Jenkins Multibranch Pipeline. After you create a Multibranch Pipeline build job, Jenkins will scan the branches in the given Git repository and create a sub-project for each branch, which name matches the include/exclude patterns that you specify.

From the Jenkinsfile perspective, it makes no difference compared to checking out a single branch. Therefore you can use exactly the same Jenkinsfile as for the single branch.

Additional checkout behavior

Often you will need additional checkout behavior. For example, to reset files in the repository that were left from a previous build run and are not a part of the Git repository.

Remember the scm object provided to our Groovy script? We can still reuse it for our more complicated checkout behavior. This object is a Java object of type hudson.plugins.git.GitSCM that represents the git repository from which the Jenkinsfile was taken. Here is how it can be used to specify additional checkout behavior:



This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters

Show hidden characters

#!groovy
def handleCheckout = {
checkout ([
$class: 'GitSCM',
branches: scm.branches,
extensions: [
[$class: 'PruneStaleBranch'],
[$class: 'CleanCheckout']
],
userRemoteConfigs: scm.userRemoteConfigs
])
}
node() {
stage('setup') {
sh "env | sort"
handleCheckout()
sh "git branch -vv"
}
stage('test') {
sh "echo 'Throw in some tests here'"
}
}

view raw

gistfile1.txt

hosted with ❤ by GitHub

If you need to know what other properties are provided in scm object, do not hesitate to checkout the Git repository of Jenkins GitSCM plugin. It is plain old Java and all properties can be conveniently accessed from the Groovy script.

If you want to know what else you can do in the checkout step (or any other build step), open the Pipeline Snippet Generator by clicking the link “Pipeline Syntax” at the very bottom of the Configuration page of your Pipeline build job in Jenkins.

Building GitLab Merge Requests

Here comes the most complex part of the guide. We will let GitLab automatically trigger build jobs on Jenkins, and Jenkins will perform the merge of the given branches and run the tests.

Create a Jenkins Pipeline job for merge requests

Create a new build job of type “Pipeline” in Jenkins with the following parameters:

  1. Enable this checkbox in build triggers: “Build when a change is pushed to GitLab”. If it is not available, install Gitlab Hook Plugin in Jenkins.
  2. Copy the displayed “GitLab CI Service URL” (e.g., “https://my-jenkins/jenkins/project/my_project”), you will need it later for GitLab configuration.
  3. Check events that should trigger builds on Jenkins. I personally like these three events: “Merge Request”, “Push to source branch” and “Comment”.

Configure Web Hooks in your GitLab project

Open your project in GitLab and go to GitLab Web Hooks in project settings.

Add the following web hook:

  • URL: paste the “GitLab CI Service URL” that you copied from the build job configuration in Jenkins. (e.g., “https://my-jenkins/jenkins/project/my_project”)
  • Enable the following triggers: “Merge request events”, “Push events”, “Comments”

Jenkinsfile

Now comes the tricky part. When a build job is triggered by a GitLab Web Hook, there are several additional environment variables provided by the Gitlab Hook Plugin, such as gitlabSourceBranch and gitlabTargetBranch. What we need to do, is to use these additional variables to checkout and merge the required branches in Jenkinsfile.

The provided scm object contains only one repository to fetch the Jenkinsfile, therefore we need to configure two different repositories for two reasons:

  • The source and target branches of the merge request are not necessarily the branches that Jenkins used to fetch the Jenkinsfile. Remember, the scm object represents the repository of the Jenkinsfile for the build job.
  • The source branch of the merge request might originate from a different repository than the target branch.

Therefore, the only use of the scm object in this case is to obtain the Jenkins credentials ID for accessing the Git repositories.



This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters

Show hidden characters

#!groovy
def handleCheckout = {
sh "echo 'Checking out a merge request…'"
def credentialsId = scm.userRemoteConfigs[0].credentialsId
checkout ([
$class: 'GitSCM',
branches: [[name: "${env.gitlabSourceNamespace}/${env.gitlabSourceBranch}"]],
extensions: [
[$class: 'PruneStaleBranch'],
[$class: 'CleanCheckout'],
[
$class: 'PreBuildMerge',
options: [
fastForwardMode: 'NO_FF',
mergeRemote: env.gitlabTargetNamespace,
mergeTarget: env.gitlabTargetBranch
]
]
],
userRemoteConfigs: [
[
credentialsId: credentialsId,
name: env.gitlabTargetNamespace,
url: env.gitlabTargetRepoSshURL
],
[
credentialsId: credentialsId,
name: env.gitlabSourceNamespace,
url: env.gitlabSourceRepoSshURL
]
]
])
}
node() {
stage('setup') {
sh "env | sort"
handleCheckout()
sh "git branch -vv"
}
stage('test') {
sh "echo 'Throw in some tests here'"
}
}

view raw

gistfile1.txt

hosted with ❤ by GitHub

Putting it all together

Now that we have figured out how to build individual branches and how to build merge requests, let us put everything together. Here is a Jenkinsfile that can handle all the types of build jobs that I mentioned in the introduction:



This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters

Show hidden characters

#!groovy
def handleCheckout = {
if (env.gitlabMergeRequestId) {
sh "echo 'Merge request detected. Merging…'"
def credentialsId = scm.userRemoteConfigs[0].credentialsId
checkout ([
$class: 'GitSCM',
branches: [[name: "${env.gitlabSourceNamespace}/${env.gitlabSourceBranch}"]],
extensions: [
[$class: 'PruneStaleBranch'],
[$class: 'CleanCheckout'],
[
$class: 'PreBuildMerge',
options: [
fastForwardMode: 'NO_FF',
mergeRemote: env.gitlabTargetNamespace,
mergeTarget: env.gitlabTargetBranch
]
]
],
userRemoteConfigs: [
[
credentialsId: credentialsId,
name: env.gitlabTargetNamespace,
url: env.gitlabTargetRepoSshURL
],
[
credentialsId: credentialsId,
name: env.gitlabSourceNamespace,
url: env.gitlabSourceRepoSshURL
]
]
])
} else {
sh "echo 'No merge request detected. Checking out current branch'"
checkout ([
$class: 'GitSCM',
branches: scm.branches,
extensions: [
[$class: 'PruneStaleBranch'],
[$class: 'CleanCheckout']
],
userRemoteConfigs: scm.userRemoteConfigs
])
}
}
node() {
stage('setup') {
sh "env | sort"
handleCheckout()
sh "git branch -vv"
}
stage('test') {
sh "echo 'Throw in some tests here'"
}
}

view raw

Jenkinsfile

hosted with ❤ by GitHub

Conclusion

I hope that this guide was useful for you and other Jenkins users. As for me, Jenkins Pipeline turned out to be a good solution to the problems that I stated in the introduction.

I would be glad to hear any feedback and critique, positive or negative. Please, do not hesitate to leave a comment or drop me a message.

Advertisement

Top Articles
Latest Posts
Article information

Author: Domingo Moore

Last Updated: 05/20/2023

Views: 6180

Rating: 4.2 / 5 (53 voted)

Reviews: 84% of readers found this page helpful

Author information

Name: Domingo Moore

Birthday: 1997-05-20

Address: 6485 Kohler Route, Antonioton, VT 77375-0299

Phone: +3213869077934

Job: Sales Analyst

Hobby: Kayaking, Roller skating, Cabaret, Rugby, Homebrewing, Creative writing, amateur radio

Introduction: My name is Domingo Moore, I am a attractive, gorgeous, funny, jolly, spotless, nice, fantastic person who loves writing and wants to share my knowledge and understanding with you.